Multi-modal sensing and active risk suppression system and method for marine hoisting equipment

Abstract

The invention discloses a multi-mode sensing and active risk suppression system and method for marine hoisting equipment, and belongs to the technical field of intelligent control and safety protection of the marine hoisting equipment. The technical problems that an existing crane safety system is single in sensing dimension, passive and rough in response and low in man-machine interaction efficiency are solved. The system collects stability, structural health, steel wire rope state and spatial position information in real time through a multi-source sensor network; the risk fusion predictor accurately calculates overturning, overload, rope disorder and collision risk indexes based on the information; according to the hierarchical active risk suppression strategy, a three-level active risk suppression strategy from multi-mode early warning guidance, smooth capability limitation to directional safe shutdown is executed according to the risk level, and a clear risk state and operation guidance are provided for an operator through an integrated human-computer interaction interface. And through a multi-level and intelligent intervention strategy, the intrinsic safety and the operation reliability of the marine crane are greatly improved.

Description

Technical Field

[0001] This invention belongs to the technical field of marine lifting equipment, and relates to a multimodal perception and active risk mitigation system and method for marine lifting equipment. Background Technology

[0002] Marine cranes, as critical pieces of equipment for offshore operations, operate in harsh environments, frequently facing complex dynamic influences such as wind, waves, and ship swaying, posing significant challenges to their safe operation. Major safety risks include: the risk of overturning due to excessive torque or ship tilting; the risk of overload damage to metal structures (such as the boom) due to overload or impact loads; the risk of cable slippage, crushing, or even breakage due to disordered wire rope arrangement; and the risk of accidental collision between the hook assembly and the boom head.

[0003] Currently, safety protection for marine cranes primarily relies on torque limiters and their derivatives. These systems typically calculate the working torque by monitoring the lifting load and boom amplitude. When the torque exceeds a set threshold, an audible and visual alarm is triggered, and the movement in the relevant dangerous direction is stopped. However, this traditional method has significant drawbacks: 1. Focusing only on load and amplitude, there is a lack of direct and comprehensive monitoring of key safety factors such as outrigger stress, actual structural stress, wire rope morphology, and dynamic ship tilt angle, which makes it impossible to cope with complex risks caused by the coupling of multiple factors.

[0004] 2. Traditional methods typically employ a simple logic of triggering an alarm when the threshold is exceeded, followed by immediate shutdown. This sudden power cut-off can itself become a disturbance, causing the load to fluctuate violently, thus creating new dynamic loads and risks.

[0005] 3. It can only issue an alarm when the risk reaches a critical point, and cannot identify or guide intervention in the early stages of risk accumulation, nor does it have the ability to smooth and suppress risks when they are approaching.

[0006] Therefore, there is an urgent need in this field for a safety system that can comprehensively detect early signs of risk, perform intelligent classification, and actively smooth and suppress them, so as to improve the inherent safety level of marine cranes. Summary of the Invention

[0007] The purpose of this invention is to overcome the deficiencies in the prior art and provide a multimodal sensing and active risk mitigation system and method for marine lifting equipment.

[0008] To achieve the above objectives, the technical solution of the present invention is to design a multimodal sensing and active risk mitigation method for marine lifting equipment, comprising the following steps: Step S100: Real-time data collection of information on the stability, structural health, wire rope status, and collision avoidance of the marine crane via a multimodal sensing network; Step S200: Based on the collected information, calculate the overturning risk index, structural overload index, rope tangling risk index and collision risk index using the risk fusion predictor; Step S300: Compare the various risk indices with preset multi-level thresholds, and execute corresponding graded proactive risk mitigation strategies according to the risk level; the graded proactive risk mitigation strategies include: When any risk index exceeds the first-level threshold, an early warning and operational guidance strategy will be implemented. When any risk index exceeds the second-level threshold, an active capability restriction strategy is implemented. When any risk index exceeds the third-level threshold, a targeted safety shutdown strategy is executed.

[0009] Specifically, the information collected by the multimodal sensing network in step S100 includes: Real-time pressure values ​​of the outriggers are collected by pressure sensors located on each outrigger of the crane base. Structural microstrain data were collected using fiber optic strain gauges attached to key load-bearing structures of the crane boom and A-frame. Three-dimensional arrangement data of the wire rope were collected by a 3D line laser scanner installed near the hoisting drum and guide pulley. The real-time distance and approach speed between the hook assembly and the boom head are collected by a millimeter-wave radar installed at the boom head.

[0010] Specifically, the method for calculating the risk index by the risk fusion predictor in step S200 includes: The capsizing risk index is calculated based on the fusion of data on the pressure distribution of each outrigger, the ship's attitude angle, the lifting moment, and the wind speed and direction. The unevenness of the outrigger pressure and the depressurization trend are used as the core weighting factors. The structural overload index is based on fiber optic strain gauge data and combined with the material yield limit to calculate the stress safety margin at key structural points. The tangled rope risk index uses pattern recognition to determine whether there is a tendency for overlapping or skipping of the wire rope by reconstructing the three-dimensional shape of the steel wire rope using a 3D line laser scanner. The collision risk index is dynamically calculated based on the distance and approach speed measured by millimeter-wave radar, combined with the system response time.

[0011] Specifically, the active capability limitation strategy is as follows: When faced with the risk of overturning or structural overload, the system automatically and smoothly reduces the output torque or displacement of the hoisting and luffing mechanisms' drive systems, limiting the operating speed but allowing operation in a way that reduces the risk. When faced with the risk of rope tangling, the system automatically fine-tunes the rope-laying motor of the hoisting drum and the slewing mechanism of the crane to actively correct the deviation. When faced with a collision risk, the system automatically triggers the deceleration control of the hoisting mechanism, causing the hook assembly to approach the limit position in a buffered manner.

[0012] Specifically, the targeted safety shutdown strategy is an ordered, multi-step process: First, the automatic control of the slewing mechanism and luffing mechanism adjusts the boom to a predefined stable and safe position; Then, control the lifting mechanism to lower the load to the bearing surface at a safe and controllable speed; Finally, the various implementing agencies were identified.

[0013] This application also provides a multimodal perception and active risk mitigation system for marine lifting equipment, including: A multimodal sensing network is used to collect various status information of the crane in real time; A risk fusion predictor, connected to the multimodal sensing network signal, is used to receive information and calculate various risk indices; A graded active risk mitigation controller is connected to the risk fusion predictor and is used to execute corresponding risk mitigation strategies based on the risk index. The human-computer interaction interface is connected to the risk fusion predictor and the hierarchical active risk mitigation controller to display risk status and early warning information.

[0014] Specifically, the multimodal sensing network includes: The stability sensing unit includes the outrigger pressure sensor, the ship attitude sensor, the load sensor, and the boom angle sensor; The structural health sensing unit includes the fiber optic strain gauge and the corresponding demodulator; The wire rope status sensing unit includes the 3D line laser scanner; The collision avoidance sensing unit includes the millimeter-wave radar.

[0015] Specifically, the graded active risk mitigation controller implements active capacity limitation and directional safe shutdown strategies by controlling the electro-hydraulic proportional valve, servo drive motor, or power source of the crane's hydraulic system.

[0016] Specifically, the human-computer interaction interface includes: a high-resolution display screen for graphically displaying risk information, predicted trajectories, and system status; a zoned voice alarm module for playing graded and directional voice prompts; and a haptic feedback joystick with a built-in motor that can generate vibration and variable resistance.

[0017] The advantages and beneficial effects of this invention are as follows: 1. By introducing new sensing dimensions such as outrigger pressure, structural strain, and wire rope morphology, the system can accurately capture hidden and complex risks that traditional systems cannot detect, thus achieving "prevention before problems occur".

[0018] 2. The traditional approach of triggering an alarm and immediately shutting down the machine after exceeding the threshold has been replaced with a human-machine interface that guides the operator. At the same time, the system restricts the operator from continuing to operate in dangerous directions and automatically controls the equipment to shut down smoothly based on the actual situation. This ensures safety while respecting the operator's autonomy and avoiding new problems caused by improper system intervention.

[0019] 3. The targeted safety shutdown strategy fundamentally solves the dynamic risks during emergency shutdown through orderly energy release and attitude adjustment, achieving true intrinsic safety.

[0020] 4. Direct monitoring of structural stress and smooth suppression of impact effectively reduce the accumulation of internal damage in equipment and extend the service life of key structural components.

[0021] 5. The human-computer interaction interface uses visual, auditory, and tactile feedback commands to alert the operator to the approaching danger and guide the operator to take the correct action. This allows the operator to better understand the danger in all aspects and make a timely and correct response to prevent the situation from deteriorating further. Attached Figure Description

[0022] Figure 1 This is a flowchart of the active risk mitigation method of the present invention; Figure 2 This is a block diagram of the information collected by the multimodal sensing network of this invention; Figure 3 This is a block diagram of the method for calculating the risk index using the risk fusion predictor of the present invention; Figure 4 This is a block diagram of the active limitation strategy for the capabilities of this invention; Figure 5 This is a block diagram of the targeted safe shutdown strategy of the present invention; Figure 6 This is a block diagram of the active suppression system of the present invention. Detailed Implementation

[0023] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0024] according to Figures 1-5 As shown, this invention is a multimodal sensing and active risk mitigation method for marine lifting equipment, comprising the following steps: Step S100: Real-time data collection of information on the stability, structural health, wire rope status, and collision avoidance of the marine crane via a multimodal sensing network; Step S200: Based on the collected information, calculate the overturning risk index, structural overload index, rope tangling risk index and collision risk index using the risk fusion predictor; Step S300: Compare the various risk indices with preset multi-level thresholds, and execute corresponding graded proactive risk mitigation strategies according to the risk level; the graded proactive risk mitigation strategies include: When any risk index exceeds the first-level threshold, an early warning and operational guidance strategy will be implemented. When any risk index exceeds the second-level threshold, an active capability restriction strategy is implemented. When any risk index exceeds the third-level threshold, a targeted safety shutdown strategy is executed.

[0025] like Figure 2 As shown, the information collected by the multimodal sensing network in step S100 includes: Real-time pressure values ​​of the outriggers are collected by pressure sensors located on each outrigger of the crane base. Structural microstrain data were collected using fiber optic strain gauges attached to key load-bearing structures of the crane boom and A-frame. Three-dimensional arrangement data of the wire rope were collected by a 3D line laser scanner installed near the hoisting drum and guide pulley. The real-time distance and approach speed between the hook assembly and the boom head are collected by a millimeter-wave radar installed at the boom head.

[0026] like Figure 3 As shown, the method for calculating the risk index by the risk fusion predictor in step S200 includes: The capsizing risk index is calculated based on the fusion of data on the pressure distribution of each outrigger, the ship's attitude angle, the lifting moment, and the wind speed and direction. The unevenness of the outrigger pressure and the depressurization trend are used as the core weighting factors. Calculation of the capsizing risk index (TRI): The TRI index is a normalized value between [0,1], derived by weighted fusion of multiple stability-related parameters. Its calculation formula is as follows: TRI=α1*F_unbalance+α2*M_ratio+α3*θ_effect+α4*W_effect Where: F_unbalance is the outrigger pressure imbalance, calculated as (F_max - F_min) / F_avg, where F_max, F_min, and F_avg are the real-time maximum, minimum, and average pressures of all outriggers, respectively. This value directly reflects the foundation's instability trend.

[0027] M_ratio is the ratio of real-time torque to rated torque, which is the core parameter of a traditional torque limiter: (Load*Range) / M_rated.

[0028] θ_effect is the hull roll effect factor, calculated as |θ_roll| / θ_roll_max + |θ_pitch| / θ_pitch_max, where θ_roll and θ_pitch are the real-time roll and pitch angles, and θ_roll_max and θ_pitch_max are the maximum operating roll angles allowed according to the crane design. This factor quantifies the dynamic impact of hull sway on stability.

[0029] W_effect is the wind load influence factor, calculated as (0.5*ρ*A*V_wind²*C_d*H_cg) / M_stable, where ρ is air density, A is windward area, V_wind is wind speed, C_d is drag coefficient, H_cg is the height of wind pressure, and M_stable is the stabilizing moment. This factor quantifies the overturning moment caused by wind load.

[0030] α1, α2, α3, and α4 are weighting coefficients, and α1 + α2 + α3 + α4 = 1. Through experiments and simulations, α1 (outrigger pressure weight) was set to the maximum to reflect its most direct characterization effect on stability.

[0031] The structural overload index is based on fiber optic strain gauge data and combined with the material yield limit to calculate the stress safety margin at key structural points. Calculation of structural overload index: The SOI index is also normalized to [0,1], and it directly reflects the stress safety margin at critical structural points. Its calculation formula is: SOI=max(σ_measured_i / σ_allowable_i) in: σ_measured_i is the real-time stress value measured and calculated by the i-th fiber grating strain gauge, where σ = E * ε, E is the elastic modulus of the material, and ε is the measured micro-strain.

[0032] σ_allowable_i is the allowable stress of the material at the corresponding measurement point i (usually taken as 0.6-0.8 times the yield strength as a safety margin).

[0033] The max() function takes the maximum value among all monitoring points, meaning the system always focuses on the state of the most dangerous point.

[0034] The tangled rope risk index uses pattern recognition to determine whether there is a tendency for overlapping or skipping of the wire rope by reconstructing the three-dimensional shape of the steel wire rope using a 3D line laser scanner. Calculation of the tangled rope risk index: The WRI index is a binarization and trend judgment result based on image processing and pattern recognition, with a value range of {0, 0.5, 1}, representing "normal", "warning", and "high risk" respectively. "Warning" corresponds to the second-level threshold, and "high risk" corresponds to the third-level threshold.

[0035] Data source: Point cloud data of steel wire rope arrangement obtained by a 3D line laser scanner.

[0036] Calculation process: a. 3D Reconstruction and Feature Extraction: The point cloud data is processed to reconstruct the 3D shape of the wire rope on the drum and key features are extracted, such as the center distance between adjacent loops, the distance between the loop and the drum flange, and the height difference between the loops.

[0037] b. Pattern recognition and judgment: When all eigenvalues ​​are within the ideal tolerance range, WRI=0 (normal).

[0038] When an abnormally reduced center distance or a frequently increasing height difference is detected in any rope loop, indicating an "overlapping" trend, WRI=0.5 (warning).

[0039] When it is identified that the wire rope has jumped out of the rope groove or made physical contact with the flange, WRI=1 (high risk).

[0040] The collision risk index is dynamically calculated based on the distance and approach speed measured by millimeter-wave radar, combined with the system response time.

[0041] Calculation of the collision risk index: The CRI index is a dynamic value based on the Time to Collision (TTC) concept. Its calculation formula is as follows: CRI=(D_safe-D_current) / D_safe+β*(V_approach / V_max) in: D_current is the shortest distance between the hook assembly and the boom head, measured in real time by millimeter-wave radar.

[0042] D_safe is a preset safety distance, which typically includes mechanical buffer space and system response distance.

[0043] V_approach is the approach velocity (radial velocity) of the hook assembly toward the boom head.

[0044] V_max is the maximum approach speed allowed by the system.

[0045] β is the weighting coefficient for the velocity term.

[0046] When D_current≤0 (theoretically, a collision has already occurred) or V_approach is extremely large, CRI approaches 1, indicating an extremely high risk of collision. like Figure 4 As shown, the active capability limitation strategy is specifically as follows: When faced with the risk of overturning or structural overload, the system automatically and smoothly reduces the output torque or displacement of the hoisting and luffing mechanisms' drive systems, limiting the operating speed but allowing operation in a way that reduces the risk. When faced with the risk of rope tangling, the system automatically fine-tunes the rope-laying motor of the hoisting drum and the slewing mechanism of the crane to actively correct the deviation. When faced with a collision risk, the system automatically triggers the deceleration control of the hoisting mechanism, causing the hook assembly to approach the limit position in a buffered manner.

[0047] like Figure 5 As shown, the targeted safety shutdown strategy is specifically an ordered multi-step process: First, the automatic control of the slewing mechanism and luffing mechanism adjusts the boom to a predefined stable and safe position; Then, control the lifting mechanism to lower the load to the bearing surface at a safe and controllable speed; Finally, the various implementing agencies were identified.

[0048] like Figure 6 As shown, a multimodal perception and active risk mitigation system for marine crane equipment includes: a multimodal perception network for real-time acquisition of various status information of the crane; A risk fusion predictor, connected to the multimodal sensing network signal, is used to receive information and calculate various risk indices; A graded active risk mitigation controller is connected to the risk fusion predictor and is used to execute corresponding risk mitigation strategies based on the risk index. The human-computer interaction interface is connected to the risk fusion predictor and the hierarchical active risk mitigation controller to display risk status and early warning information.

[0049] The multimodal sensing network includes: The stability sensing unit includes the outrigger pressure sensor, the ship attitude sensor, the load sensor, and the boom angle sensor; The structural health sensing unit includes the fiber optic strain gauge and the corresponding demodulator; The wire rope status sensing unit includes the 3D line laser scanner; The collision avoidance sensing unit includes the millimeter-wave radar.

[0050] The graded active risk mitigation controller implements active capacity limitation and directional safe shutdown strategies by controlling the electro-hydraulic proportional valves, servo drive motors, or power sources of the crane's hydraulic system.

[0051] The human-machine interface includes: a high-resolution display screen for graphically displaying risk information, predicted trajectories, and system status; a zoned voice alarm module for playing graded and directional voice prompts; and a haptic feedback joystick with a built-in motor that can generate vibration and variable resistance.

[0052] To more intuitively demonstrate the workflow of this system, the following example of lifting operations in wind and waves is provided, based on the aforementioned risk index calculation: A marine crane is preparing to lift a heavy piece of equipment in rough seas. First, the operator starts the crane, and the system self-check passes. The human-machine interface displays a normal crane operating status model. The operator then rotates the boom to prepare to lift the equipment over a barge.

[0053] The millimeter-wave radar of the space sensing unit detected that the distance between the hook and the barge mast was rapidly decreasing.

[0054] The risk fusion predictor calculates the Collision Risk Index (CRI) in real time and detects that it exceeds the Level 1 threshold. The system immediately executes a warning and operational guidance strategy, which is displayed through the human-machine interface. On the display screen, the barge mast icon begins to flash yellow, and a red predicted collision trajectory line extends from the hook to the mast. The voice system announces: "Attention, obstacle approaching to the right front." The operator's control lever experiences a continuous slight vibration.

[0055] The operator failed to adjust in time, and the CRI index exceeded the Level 2 threshold due to excessive approach speed. The system automatically implemented a proactive limiting strategy: the system controlled the slewing mechanism to smoothly reduce output power. Simultaneously, when the operator attempted to continue operating in the dangerous direction (closer to the mast), the control lever generated significant reverse resistance, making it difficult to push. Forcing it forward resulted in a pulsating, jerky sensation. At the same time, a red prohibition symbol appeared on the slewing operation icon on the display screen.

[0056] Based on the multi-channel warning information, the operator immediately maneuvers in a safe direction (away from the mast). Once the direction is correct, the lever resistance disappears instantly. The hook trajectory deviates from the mast, the CRI index decreases, the system is deactivated, and the interface returns to normal.

[0057] In extreme scenarios, such as a strong gust of wind causing the stability sensing unit to detect a severe imbalance in outrigger pressure, with the TRI index instantly exceeding Level 3, the system decisively initiates a directional safety shutdown strategy. The voice system announces in a calm tone: "Instability risk detected, automatic safety procedure initiated." A clear flowchart appears on the display screen, highlighting the three steps sequentially: "Adjust boom orientation," "Lower load," and "System lock," each accompanied by a progress bar. The operator sees and hears the system automatically and systematically handling the emergency, preventing panic. No action is required; the operator simply monitors the system's completion.

[0058] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A multimodal sensing and active risk mitigation method for marine lifting equipment, characterized in that, Includes the following steps: Step S100: Real-time data collection of information on the stability, structural health, wire rope status, and collision avoidance of the marine crane via a multimodal sensing network; Step S200: Based on the collected information, calculate the overturning risk index, structural overload index, rope tangling risk index and collision risk index using the risk fusion predictor; Step S300: Compare the various risk indices with preset multi-level thresholds, and execute corresponding graded proactive risk mitigation strategies according to the risk level; The tiered proactive risk mitigation strategy includes: When any risk index exceeds the first-level threshold, an early warning and operational guidance strategy will be implemented. When any risk index exceeds the second-level threshold, an active capability restriction strategy is implemented. When any risk index exceeds the third-level threshold, a targeted safety shutdown strategy is executed.

2. The multimodal perception and active risk mitigation method for marine lifting equipment according to claim 1, characterized in that, The information collected by the multimodal sensing network in step S100 includes: Real-time pressure values ​​of the outriggers are collected by pressure sensors located on each outrigger of the crane base. Structural microstrain data were collected using fiber optic strain gauges attached to key load-bearing structures of the crane boom and A-frame. Three-dimensional arrangement data of the wire rope were collected by a 3D line laser scanner installed near the hoisting drum and guide pulley. The real-time distance and approach speed between the hook assembly and the boom head are collected by a millimeter-wave radar installed at the boom head.

3. The multimodal perception and active risk mitigation method for marine lifting equipment according to claim 1, characterized in that, The method for calculating the risk index by the risk fusion predictor in step S200 includes: The capsizing risk index is calculated based on the fusion of data on the pressure distribution of each outrigger, the ship's attitude angle, the lifting moment, and the wind speed and direction. The unevenness of the outrigger pressure and the depressurization trend are used as the core weighting factors. The structural overload index is based on fiber optic strain gauge data and combined with the material yield limit to calculate the stress safety margin at key structural points. The tangled rope risk index uses pattern recognition to determine whether there is a tendency for overlapping or skipping of the wire rope by reconstructing the three-dimensional shape of the steel wire rope using a 3D line laser scanner. The collision risk index is dynamically calculated based on the distance and approach speed measured by millimeter-wave radar, combined with the system response time.

4. The multimodal perception and active risk mitigation method for marine lifting equipment according to claim 1, characterized in that, The active capability restriction strategy is as follows: When faced with the risk of overturning or structural overload, the system automatically and smoothly reduces the output torque or displacement of the hoisting and luffing mechanisms' drive systems, limiting the operating speed but allowing operation in a way that reduces the risk. When faced with the risk of rope tangling, the system automatically fine-tunes the rope-laying motor of the hoisting drum and the slewing mechanism of the crane to actively correct the deviation. When faced with a collision risk, the system automatically triggers the deceleration control of the hoisting mechanism, causing the hook assembly to approach the limit position in a buffered manner.

5. The multimodal perception and active risk mitigation method for marine lifting equipment according to claim 4, characterized in that, The targeted safety shutdown strategy is specifically an ordered, multi-step process: First, the automatic control of the slewing mechanism and luffing mechanism adjusts the boom to a predefined stable and safe position; Then, control the lifting mechanism to lower the load to the bearing surface at a safe and controllable speed; Finally, the various implementing agencies were identified.

6. A multimodal sensing and active hazard mitigation system for marine lifting equipment used to implement the method of any one of claims 1-5, characterized in that, include: A multimodal sensing network is used to collect various status information of the crane in real time; A risk fusion predictor, connected to the multimodal sensing network signal, is used to receive information and calculate various risk indices; A graded active risk mitigation controller is connected to the risk fusion predictor and is used to execute corresponding risk mitigation strategies based on the risk index. The human-computer interaction interface is connected to the risk fusion predictor and the hierarchical active risk mitigation controller to display risk status and early warning information.

7. The system according to claim 6, characterized in that, The multimodal sensing network includes: The stability sensing unit includes outrigger pressure sensors, ship attitude sensors, load sensors, and boom angle sensors; The structural health sensing unit includes fiber optic strain gauges and corresponding demodulators; The wire rope status sensing unit includes a 3D line laser scanner; Collision avoidance sensing unit, including millimeter-wave radar.

8. The system according to claim 6, characterized in that, The graded active risk mitigation controller implements active capacity limitation and directional safe shutdown strategies by controlling the electro-hydraulic proportional valves, servo drive motors, or power sources of the crane's hydraulic system.

9. The system according to claim 6, characterized in that, The human-machine interface includes: a high-resolution display screen for graphically displaying risk information, predicted trajectories, and system status; a zoned voice alarm module for playing graded and directional voice prompts; and a haptic feedback joystick with a built-in motor that can generate vibration and variable resistance.

Images