Crane lifting risk early warning method and system based on image recognition

Abstract

The application relates to the field of image recognition, and discloses a crane hoisting risk early warning method and system based on image recognition. The crane hoisting risk early warning method and system independently quantize double-dimension risk coefficients by adopting a short plate model, the environmental risk and the personnel risk are each dominated by the most serious single item, subjective weight setting and experience constant stacking are eliminated, an independent linear overrun comparison channel of a fast environmental risk threshold and a fast artificial risk threshold is established, a one-vote veto alarm is directly triggered when a single dimension deteriorates, the covering of an emergency risk by a comprehensive operation delay is avoided, a double-dimension risk coefficient vector is synthesized to generate a comprehensive alarm coefficient, and a time sequence updating and coupling updating double mechanism is adopted to dynamically adjust an alarm threshold, so that the threshold is real-time adapted to working conditions and the change process is stable and controllable.

Description

Technical Field

[0001] This application relates to the field of image recognition technology, and in particular to a crane hoisting risk early warning method and system based on image recognition. Background Technology

[0002] As core equipment in engineering fields such as mining, ports, and construction, the safety of lifting operations directly relates to the safety of personnel and engineering property. With the development of lifting equipment towards larger and more intelligent models, the environmental factors faced by lifting operations are becoming increasingly complex. These include dynamic environmental loads such as sudden gusts of wind, ground subsidence, and sudden changes in visibility, as well as fluctuations in operator condition such as fatigue, differences in experience, and stress responses. All of these can lead to the failure of traditional static safety management models. Therefore, safety risk early warning technology during lifting operations has become a key research focus in the industry.

[0003] Existing safety risk early warning technologies for lifting machinery mainly focus on single-dimensional monitoring and alarms. Regarding environmental risk monitoring, Chinese patent CN116777296A proposes a method for objectively quantifying and predicting the energy consumption of lifting machinery during service. This method establishes an energy consumption evaluation formula based on parameters such as motor rated power, load rate, and lifting height, taking into account the transmission characteristics of the hoisting and running mechanisms. However, this technology does not consider the direct impact of environmental factors such as wind speed and road surface roughness on safety status, nor does it involve a dynamic adjustment mechanism for safety thresholds.

[0004] In the area of ​​personnel behavior monitoring, Chinese patent CN118183515A discloses a method and system for early warning and control of safety risks during hoisting operations. This method detects the operator's posture through infrared thermal imaging differential processing, extracts the arm region using the pixel distribution rate of the pixel color interval, and then calculates the operational difference value to assess the personnel's risk threat value. However, this technology has the following inherent defects: First, the infrared thermal imaging scheme relies on the assumption of temperature difference between the arm and torso in a mining environment. In actual working conditions, heat sources such as engine cooling, hydraulic system heating, and direct sunlight interfere with this assumption, making the robustness of arm region extraction extremely poor. Second, the method of indirectly deriving temperature distribution through pixel color values ​​to identify body parts involves a large number of subjectively set segmentation thresholds and morphological processing parameters, lacking clear physical quantity correspondences, thus limiting engineering feasibility. Third, the system adopts a hard-switching control strategy of "pause-retest-shutdown," directly suspending operation when high-risk behavior is detected, thus affecting the crane's performance. Pausing the hoist in a suspended state actually increases the risk of hoist swaying, and frequent alarm interference can easily cause operators to make stress-induced mistakes; fourth, it adopts fixed weight adaptive, selects basic parameters according to load status, and once the threshold is set, it simply switches with the load status, and cannot be dynamically updated according to real-time wind speed, temperature, visibility and personnel fatigue status, resulting in the threshold lagging far behind the actual working conditions; fifth, its environmental risk assessment and personnel risk assessment share the same comprehensive threshold, and it fails to establish independent alarm channels for environmental and personnel dimensions. When the environment deteriorates extremely or personnel are seriously abnormal, the delay in comprehensive calculation masks the need for rapid response to emergency risks. Summary of the Invention

[0005] This application proposes a crane hoisting risk early warning method and system based on image recognition. It possesses the advantages of independent assessment of environmental and personnel risks, dynamic adaptive alarm thresholds, and progressive alarm output. This addresses the problems of poor robustness of infrared thermal imaging recognition, static risk thresholds, separation of environmental and personnel risks, and crude alarm methods in existing technologies, leading to delayed safety warnings and secondary risks. By constructing dual datasets and employing a bottleneck model to achieve risk decoupling and quantification, an independent and rapid threshold channel is established to perform a veto comparison. Real-time threshold adaptation is achieved through vector synthesis and dual-mechanism dynamic updates. A three-level progressive audible and visual alarm replaces forced shutdown, improving early warning accuracy and avoiding secondary risks.

[0006] To achieve the above objectives, this application adopts the following technical solution: a crane hoisting risk early warning method based on image recognition, the specific steps of which are as follows: S1. Data Acquisition and Processing: Real-time image data of the crane hoisting operation site is acquired through an image acquisition device. Environmental feature parameters and personnel feature parameters are extracted based on the real-time image data, and the environmental feature parameters and personnel feature parameters are preprocessed to generate a first dataset and a second dataset. S2. Factor Extraction: Calculate the real-time environmental risk coefficient from the first and second datasets in step S1. and real-time human risk coefficient ; S3. Rapid Feedback: Extract and calculate the parameters from the first and second datasets generated in step S1 to generate a rapid environmental risk threshold. and rapid human risk threshold and will rapidly set environmental risk thresholds and rapid human risk threshold Respectively with real-time environmental risk coefficient and real-time human risk coefficient Compare the results to generate the first comparison result. Based on the first comparison result, issue a rapid alarm or proceed to the next step normally. S4. Comprehensive Analysis: Analyze the real-time environmental risk coefficients from step S2. and real-time human risk coefficient Perform integrated calculations to generate a comprehensive alarm coefficient. And set the alarm threshold Z, and combine the comprehensive alarm coefficient The alarm threshold Z is compared to generate a second comparison result. A regular alarm is triggered based on the comparison result, and update rules are set to generate a dynamically updated threshold in real time, updating the alarm threshold according to the real-time situation. .

[0007] Furthermore, in step S1, the specific steps for data acquisition and processing are as follows: S1.1. A first image acquisition device is installed at the end of the crane boom, and a second image acquisition device is installed on the top of the turntable. The first image acquisition device acquires images of the hoisting area and extracts the operating wind speed A, ground tilt angle B, current hoisting mass C, and current operating radius D. The gust coefficient E is calculated from the image sequence of the hoisting area. The second image acquisition device acquires panoramic images of the operation and extracts the visibility F and ambient temperature G. The factory-calibrated maximum allowable operating wind speed Q, rated hoisting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, allowable temperature deviation span V, and critical visibility are read. The system response delay β, the model correction coefficient γ, and the coupling update coefficient μ are organized into the first dataset. S1.2. A third image acquisition device is installed above and in front of the control room to acquire images of the operator's face and upper body, and extract the fatigue index O and field of view deviation angle P. The operation response delay L, joystick operation frequency M, and standard process execution rate N are obtained through joystick displacement sensor, hydraulic pressure sensor, and PLC logic comparison device. The maximum permissible response delay calibrated by the factory is read. Maximum allowed operation frequency and maximum permissible field of view deviation Organize the above parameters into a second dataset; S1.3. Denoise, normalize and time-align the real-time collected parameters in the first dataset and the second dataset. Calculate the historical trend slope α from the operating wind speed A in the first dataset. Incorporate the historical trend slope α into the first dataset as a derived calculation parameter. Write the factory calibration parameters directly into the corresponding dataset.

[0008] Furthermore, in step S2, the specific steps for factor extraction are as follows: S2.1 Environmental Factor Extraction: Extract multiple parameters from the first dataset and perform calculations to obtain the real-time environmental risk coefficient. ; S2.2 Human Factor Extraction: Extract multiple parameters from the second dataset and perform calculations to obtain the real-time human risk coefficient. .

[0009] Furthermore, in step S2.1, the specific steps for extracting environmental factors are as follows: S2.11. Extract the operating wind speed A, gust coefficient E, ground tilt angle B and maximum allowable operating wind speed Q from the first dataset, calculate the wind load risk index FL according to the formula, extract the current hoisting mass C, operating radius D, ambient temperature G, rated hoisting mass R, maximum operating radius S, optimal operating temperature U and temperature allowable deviation span V, and calculate the load risk index HL according to the formula. S2.12. Compare the wind load risk index FL and the load risk index HL, and take the larger value as the real-time environmental risk coefficient. .

[0010] Furthermore, in step S2.2, the specific steps for extracting human factors are as follows: S2.21. Extract the fatigue index O, visual field deviation angle P, and maximum permissible visual field deviation from the second dataset. The larger of O and P divided by Pmax is taken as the physiological state component PS. Operation response delay L, joystick operation frequency M, standard procedure execution rate N, and maximum permissible response delay are then extracted. and maximum allowed operation frequency Calculate the ratio of the three terms and take the larger value as the operation behavior component XO; S2.22. Compare the physiological state component PS with the operational behavior component XO, and take the larger value as the real-time human risk coefficient. .

[0011] Furthermore, in step S3, the specific steps for quick feedback are as follows: S3.1 Extract the following parameters from the first dataset: operating wind speed A, ground tilt angle B, current lifting mass C, current operating radius D, ambient temperature G, maximum allowable operating wind speed Q, rated lifting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, and allowable temperature deviation range V. Calculate the wind speed margin, tilt margin, load margin, and temperature margin. Multiply these margins together to generate a rapid environmental risk threshold. ; Extract the following parameters from the second dataset: operation response delay L, joystick operation frequency M, standard procedure execution rate N, fatigue index O, field of view deviation angle P, and maximum permissible response delay. Maximum allowed operation frequency and maximum permissible field of view deviation Calculate response margin, operation frequency margin, process execution rate, fatigue margin, and field of view margin, and multiply these components together to generate a rapid human risk threshold. ; S3.2, Rapidly set environmental risk thresholds and rapid human risk threshold The real-time environmental risk coefficients obtained in step S2 are respectively compared with those obtained in step S2. and real-time human risk coefficient A comparison was performed, generating the first comparison result, as follows: when At that time, it represents the real-time environmental risk coefficient. If there are no special abnormalities, there is no need to issue a rapid alarm; proceed normally to step S4. when At that time, it represents the real-time environmental risk coefficient. In the event of any special abnormality, a rapid alarm should be triggered to remind operators to stop operations and evacuate to a safe area as soon as possible; when Time represents the real-time human risk coefficient. If there are no special abnormalities, there is no need to issue a rapid alarm; proceed normally to step S4. when Time represents the real-time human risk coefficient. In the event of any unusual abnormality, a rapid alarm should be triggered to remind operators to stop operations and leave the operating area as soon as possible.

[0012] Furthermore, in step S4, the specific steps of the comprehensive analysis are as follows: S4.1 First Comparison Result Generation: Real-time environmental risk coefficient and real-time human risk coefficient A comprehensive alarm coefficient is generated after performing nonlinear coupling calculations. Establish alarm thresholds And will integrate alarm coefficients With alarm threshold Compare the results to generate the first comparison result; S4.2 Threshold Dynamic Update: Based on the parameter changes in the first and second datasets, determine the direction of threshold update, and shrink or widen the threshold while determining the shrinkage or widening range, generating a new alarm threshold. It replaces the old alarm threshold Z and the comprehensive alarm coefficient. By comparing the results and setting a recovery window period, when the threshold is updated using a shrinking approach, the threshold recovery range is established based on the parameter changes within the recovery window period.

[0013] Furthermore, in step S4.1, the specific steps for generating the first comparison result are as follows: S4.11, Real-time environmental risk coefficient output in step S2.1 Compared with the real-time human risk coefficient output in step S2.2 Perform vector synthesis calculations to generate the comprehensive alarm coefficient BJ; S4.12, Combine alarm coefficients The result is compared with the preset alarm threshold Z to generate the first comparison result, as follows: when When the time is right, it means there is no risk and no alarm needs to be triggered. when When this occurs, it indicates that there is a risk and an alarm needs to be triggered. when At this time, both the warning lights and sirens will provide a warning at an interval of 1 second, with the sirens volume set between 45% and 55%. when At this time, both the warning lights and sirens will be activated at 0.5-second intervals, with the sirens volume set between 55% and 75%. when At this time, both the warning lights and sirens will provide a warning at an interval of 0.3 seconds, with the sirens volume set between 75% and 135%.

[0014] Furthermore, in step S4.2, the specific steps for dynamically updating the threshold are as follows: S4.21. The time-series target value and coupled target value are calculated in parallel operation for each cycle. The trend slope α is calculated from the operating wind speed A, combined with the system response delay β, the maximum allowable operating wind speed Q, the model correction factor γ, the ground tilt angle B, the ambient temperature G, the visibility F, and the critical visibility. Operation timing target value The coupled target value is calculated by combining the comprehensive alarm coefficient BJ and the coupling update coefficient μ. , coupling target value When the threshold is low, contraction is prioritized; if no trigger is triggered, the direction is determined based on the timing results; if the condition is not met, the threshold is maintained. S4.22 When the threshold shrinks, it is gradually tightened according to the preset ratio based on the previous cycle value. When the threshold is widened, it is gradually released according to the preset ratio lower than the shrinkage rate after the stabilization recovery period of all parameters. The threshold is dynamically updated and output to the regular alarm comparison stage.

[0015] This application also provides a crane hoisting risk early warning system based on image recognition, including an image acquisition module, a risk extraction module, a rapid alarm module, a comprehensive analysis module, and a threshold update module: The image acquisition module is used to acquire images of the hoisting area, panoramic operation images, and operator images, extract environmental feature parameters and personnel feature parameters, and generate the first dataset and the second dataset. The risk extraction module is used to calculate real-time environmental risk coefficients based on the first and second datasets. and real-time human risk coefficient ; The rapid alarm module is used to quickly set environmental risk thresholds. and rapid human risk threshold and with real-time environmental risk coefficient and real-time human risk coefficient Compare and generate the first comparison result; The comprehensive analysis module is used to analyze real-time environmental risk coefficients. and real-time human risk coefficient Perform vector synthesis calculations to generate a comprehensive alarm coefficient. And compare it with the dynamic alarm threshold Z to generate a second comparison result; The threshold update module is used to dynamically update the alarm threshold Z based on environmental parameters, the maximum allowable operating wind speed Q, and the comprehensive alarm coefficient BJ.

[0016] The beneficial effects of this invention are as follows: This application provides a crane hoisting risk early warning method and system based on image recognition. It employs a short-board model to independently quantify dual-dimensional risk coefficients, allowing environmental and personnel risks to be independently dominated by the most severe single factor. This eliminates subjective weighting and the accumulation of empirical constants. An independent linear over-limit comparison channel is established for rapid environmental risk thresholds and rapid human risk thresholds. When a single dimension deteriorates, a veto alarm is directly triggered, avoiding the masking of emergency risks by comprehensive calculation delays. The dual-dimensional risk coefficients are vector-synthesized to generate a comprehensive alarm coefficient, and a dual mechanism of time-series update and coupled update is used to dynamically adjust the alarm threshold, ensuring that the threshold adapts to the working conditions in real time and that the change process is stable and controllable. A three-level progressive audible and visual alarm signal is output to replace traditional forced shutdown. The progressive increase in light frequency, sound interval, and volume visually reflects the approaching risk, providing operators with a progressive autonomous decision-making window and avoiding secondary risks caused by crane swaying due to forced shutdown. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort: Figure 1 This is a diagram illustrating the overall method steps of this application.

[0018] Figure 2 This is a step diagram of the extended method of step S2.1 of this application.

[0019] Figure 3 This is a step diagram of the extended method of step S2.2 of this application.

[0020] Figure 4 This is a step diagram of the extended method of step S4.1 of this application.

[0021] Figure 5 This is a step diagram of the extended method of step S4.2 of this application.

[0022] Figure 6 This is the overall system flowchart of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] Example 1, as Figure 1 A crane hoisting risk early warning method based on image recognition, the specific steps of which are as follows: S1. Data Acquisition and Processing: Real-time image data of the crane hoisting operation site is acquired through an image acquisition device. Environmental feature parameters and personnel feature parameters are extracted based on the real-time image data, and the environmental feature parameters and personnel feature parameters are preprocessed to generate a first dataset and a second dataset. S2. Factor Extraction: Calculate the real-time environmental risk coefficient from the first and second datasets in step S1. and real-time human risk coefficient ; S3. Rapid Feedback: Extract and calculate the parameters from the first and second datasets generated in step S1 to generate a rapid environmental risk threshold. and rapid human risk threshold and will rapidly set environmental risk thresholds and rapid human risk threshold Respectively with real-time environmental risk coefficient and real-time human risk coefficient Compare the results to generate the first comparison result. Based on the first comparison result, issue a rapid alarm or proceed to the next step normally. S4. Comprehensive Analysis: Analyze the real-time environmental risk coefficients from step S2. and real-time human risk coefficient Perform integrated calculations to generate a comprehensive alarm coefficient. And set the alarm threshold Z, and combine the comprehensive alarm coefficient The alarm threshold Z is compared to generate a second comparison result. A regular alarm is triggered based on the comparison result, and update rules are set to generate a dynamically updated threshold in real time, updating the alarm threshold according to the real-time situation. .

[0025] In this embodiment: Step S1 involves setting up first, second, and third image acquisition devices at the end of the crane boom, the top of the turntable, and the front and upper part of the operator's cab, respectively, to simultaneously acquire images of the hoisting area, panoramic operation, and the operator's face and upper body. By combining the real-time image data extracted by the sensors, environmental feature parameters and personnel feature parameters are extracted. The collected parameters are then organized into a first dataset and a second dataset, respectively. This achieves the digital representation and structured classification of all elements of the hoisting operation, providing an independent data foundation for subsequent hierarchical risk calculation. This effectively overcomes the defect of traditional infrared thermal imaging relying on the temperature difference assumption, which leads to recognition failure.

[0026] Step S2 generates real-time environmental risk coefficients by performing calculations on the parameters of the first and second datasets respectively. and real-time human risk coefficient This approach achieves decoupled quantitative assessment of environmental and personnel dimensions, accurately characterizing the risk level of a single dimension, and provides independent numerical basis for the hierarchical alarm mechanism, avoiding the obscuring of specific risk dimensions by the traditional single comprehensive threshold.

[0027] Step S3 involves extracting parameters from the first and second datasets respectively, and then using a cascaded residual model to generate a rapid environmental risk threshold. and rapid human risk threshold The two thresholds were then compared with the real-time environmental risk coefficient. and real-time human risk coefficient Independent comparisons are performed, enabling rapid response and veto protection in emergency situations. This allows for direct alarm triggering when a single-dimensional risk worsens without waiting for comprehensive calculations, thus forming independent rapid alarm channels for environmental and personnel risks.

[0028] Step S4 involves using the real-time environmental risk coefficient. and real-time human risk coefficient Perform vector synthesis calculations to generate comprehensive alarm coefficients. The system sets up a dual mechanism of time-series update and coupled update to dynamically adjust the alarm threshold. It compares the comprehensive alarm coefficient with the dynamic threshold to generate a three-level progressive alarm signal. This achieves coupled assessment of multi-dimensional risks and real-time adaptation of thresholds to working conditions. It achieves the goal of balancing progressive risk warning and operator autonomy decision-making window, avoiding missed alarms caused by static threshold lag and secondary risks caused by crane swing due to forced shutdown.

[0029] Compared with existing image recognition-based crane lifting risk warning methods, which rely on infrared thermal imaging temperature difference assumptions for personnel identification, use fixed-weight linear superposition to assess risks, and issue alarms through voice broadcasts or forced shutdowns, this method directly extracts multiple quantifiable parameters with clear physical meaning by fusing visible light and infrared images, thereby improving the robustness of identification and engineering feasibility under complex working conditions.

[0030] By constructing a hierarchical architecture of the first and second datasets, independent assessment and cross-protection of environmental and personnel risks are achieved. It can provide emergency alarms by rapidly comparing thresholds when a single dimension deteriorates, and it can also provide progressive routine early warnings by coupling the comprehensive alarm coefficient with the dynamic threshold. This effectively overcomes the masking and delay of emergency risks by traditional single comprehensive thresholds.

[0031] The alarm threshold is adjusted using a dual mechanism of time-series update and coupled update, which allows the threshold to contract or expand in real time with changes in environmental changes and fluctuations in personnel status. Combined with a three-level progressive audible and visual alarm, it provides operators with an autonomous decision-making window while avoiding secondary risks such as load swaying caused by forced shutdown, thereby improving the safety early warning accuracy and engineering applicability of the entire hoisting operation process.

[0032] Example 2, as Figure 1 In step S1, the specific steps for data acquisition and processing are as follows: S1.1. A first image acquisition device is installed at the end of the crane boom, and a second image acquisition device is installed on the top of the turntable. The first image acquisition device acquires images of the hoisting area and extracts the operating wind speed A, ground tilt angle B, current hoisting mass C, and current operating radius D. The gust coefficient E is calculated from the image sequence of the hoisting area. The second image acquisition device acquires panoramic images of the operation and extracts the visibility F and ambient temperature G. The factory-calibrated maximum allowable operating wind speed Q, rated hoisting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, allowable temperature deviation span V, and critical visibility are read. The system response delay β, the model correction coefficient γ, and the coupling update coefficient μ are organized into the first dataset. The gust coefficient E is obtained by extracting the time series data of the hoisting swing amplitude from the image sequence of the hoisting area using the optical flow method, and calculating the ratio of the statistical peak value to the mean value of the hoisting displacement change rate between adjacent frames. It represents the amplification effect of instantaneous wind load relative to average wind load and is a dimensionless parameter. S1.2. A third image acquisition device is installed above and in front of the control room to acquire images of the operator's face and upper body, and extract the fatigue index O and field of view deviation angle P. The operation response delay L, joystick operation frequency M, and standard process execution rate N are obtained through joystick displacement sensor, hydraulic pressure sensor, and PLC logic comparison device. The maximum permissible response delay calibrated by the factory is read. Maximum allowed operation frequency and maximum permissible field of view deviation Organize the above parameters into a second dataset; S1.3. Denoise, normalize and time-align the real-time collected parameters in the first dataset and the second dataset. Calculate the historical trend slope α from the operating wind speed A in the first dataset. Incorporate the historical trend slope α into the first dataset as a derived calculation parameter. Write the factory calibration parameters directly into the corresponding dataset.

[0033] In this embodiment: Step S1.1 involves setting a first image acquisition device at the end of the crane boom and a second image acquisition device at the top of the turntable. The operating wind speed A, ground tilt angle B, current lifting mass C, and current operating radius D are extracted from the images of the lifting area. The gust coefficient E is calculated from the image sequence. The visibility F and ambient temperature G are extracted from the panoramic operating images. The factory-calibrated maximum allowable operating wind speed Q, rated lifting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, allowable temperature deviation span V, critical visibility F0, system response delay β, model correction coefficient γ, and coupling update coefficient μ are read. This achieves the image reconstruction of the lifting operation environment and the fusion and collection of structural benchmark parameters, completing the hierarchical construction task of the first dataset and achieving the goal of providing an independent data source for subsequent environmental risk quantification.

[0034] Step S1.2: By installing a third image acquisition device above and in front of the control room, images of the operator's face and upper body are acquired, and the fatigue index O and field of view deviation angle P are extracted. The operation response delay L, the joystick operation frequency M, and the standard process execution rate N are obtained through the joystick displacement sensor, hydraulic pressure sensor, and PLC logic comparison device. The maximum allowable response delay calibrated by the factory is read. Maximum allowed operation frequency and maximum permissible field of view deviation This achievement enabled the fusion and aggregation of image-based perception of personnel behavior and operational sensor data, completed the hierarchical construction of the second dataset, and achieved the goal of providing an independent data source for subsequent personnel risk quantification.

[0035] Step S1 establishes an independent data acquisition and dual dataset architecture for environmental and personnel dimensions, fundamentally improving the traditional single-sensor acquisition and hybrid data processing mode. This hierarchical architecture effectively overcomes the defects of traditional infrared thermal imaging temperature difference assumption recognition failure and chaotic data coupling, providing a physically clear and boundary-clear data foundation for subsequent hierarchical alarms, and improving the independence and engineering feasibility of the risk warning data foundation.

[0036] Example 3, as Figure 1 In step S2, the specific steps for factor extraction are as follows: S2.1 Environmental Factor Extraction: Extract multiple parameters from the first dataset and perform calculations to obtain the real-time environmental risk coefficient. ; S2.2 Human Factor Extraction: Extract multiple parameters from the second dataset and perform calculations to obtain the real-time human risk coefficient. .

[0037] In this embodiment: Step S2.1 extracts multiple parameters from the first dataset and calculates the real-time environmental risk coefficient. This method overcomes the limitations of traditional single-sensor acquisition. It directly obtains quantifiable parameters with clear physical meaning, such as load swing angle and wind speed estimation, through image recognition. It avoids subjective weight setting and accumulation of empirical constants. The physical mechanism is clear and interpretable, providing an independent environmental dimension data foundation for subsequent hierarchical alarms.

[0038] Step S2.2 Extract multiple parameters from the second dataset and calculate the real-time human risk coefficient. This method overcomes the recognition failure problem caused by the reliance on temperature difference assumptions in traditional infrared thermal imaging. It recognizes facial orientation, eye closure, head posture, and gesture trajectory by fusing visible light and infrared images. Combined with operational sensor data, it avoids the subjective threshold and fixed weighting of pixel color linear weighting, improves the engineering feasibility of personnel risk quantification, and provides an independent personnel dimension data foundation for subsequent hierarchical alarms.

[0039] Compared to existing technologies that rely on infrared thermal imaging temperature difference assumptions for personnel pose recognition, use fixed-weight linear superposition to assess risk, and directly shut down operations through hard-switching control strategies, step S2 establishes an independent and decoupled assessment system for environmental and personnel dimensions. Environmental and human factor extraction respectively adopt a bottleneck model instead of traditional linear weighting, eliminating subjective weight coefficients and empirical constants. This ensures that environmental and personnel risks are each dominated by the most severe single factor, with clear physical meaning and consistent dimensions. Quantifiable parameters with clear physical meaning, such as load swing angle, wind speed estimation, personnel fatigue, and field-of-view deviation, are directly extracted through image recognition, improving the robustness and engineering feasibility of identification under complex working conditions. This hierarchical assessment architecture avoids the masking of specific risk dimensions by a single comprehensive threshold, giving environmental and personnel risks independent computational boundaries and quantification paths. This provides clear data support and computational foundation for subsequent rapid independent threshold comparison and dynamic updating of comprehensive thresholds, improving the accuracy and real-time performance of risk quantification in crane lifting operations and reducing the probability of false alarms and missed alarms due to improper data coupling.

[0040] Example 4, as Figure 2 In step S2.1, the specific steps for extracting environmental factors are as follows: S2.11. Extract the operating wind speed A, gust coefficient E, ground tilt angle B and maximum allowable operating wind speed Q from the first dataset, calculate the wind load risk index FL according to the formula, extract the current hoisting mass C, operating radius D, ambient temperature G, rated hoisting mass R, maximum operating radius S, optimal operating temperature U and temperature allowable deviation span V, and calculate the load risk index HL according to the formula. The specific calculation formulas for the wind load risk index FL and the load risk index HL are as follows:

[0041]

[0042] Where: A is the operating wind speed, E is the gust coefficient, B is the ground tilt angle, Q is the maximum allowable operating wind speed, C is the current lifting mass, D is the operating radius, G is the ambient temperature, R is the rated lifting mass, S is the maximum operating radius, U is the optimal operating temperature, and V is the allowable temperature deviation range. S2.12. Compare the wind load risk index FL and the load risk index HL, and take the larger value as the real-time environmental risk coefficient. ; Real-time environmental risk coefficient The specific calculation formula is as follows:

[0043] Wherein: FL is the wind load risk index, and HL is the load risk index.

[0044] In this embodiment: Step S2.11 extracts the operating wind speed A, gust coefficient E, ground tilt angle B, maximum allowable operating wind speed Q, current hoisting mass C, operating radius D, ambient temperature G, and the factory-calibrated rated hoisting mass R, maximum operating radius S, optimal operating temperature U, and temperature allowable deviation span V from the first dataset, and calculates the wind load risk index FL and load risk index HL respectively. This realizes the sub-item quantification and independent verification of environmental risks, completes the physical calculation task of risk pressure in the two mechanical dimensions of wind load and load, and achieves the purpose of providing a traceable mechanical calculation basis for the subsequent short-board model.

[0045] Step S2.12 compares the wind load risk index FL and the load risk index HL and takes the larger value as the real-time environmental risk coefficient SHJ. This realizes the determination of the environmental risk weakness dominated by the most severe mechanical single factor, completes the engineering application of the reliability principle of the series system in environmental risk assessment, and achieves the goal of ensuring that the environmental risk coefficient is always controlled by the weakest link and improving the fault tolerance of the project under complex working conditions.

[0046] Step S2.1 establishes an environmental risk quantification system with independent sub-items of wind load and load, and a short-board-dominated synthesis. This fundamentally improves upon the traditional infrared thermal imaging-dependent temperature difference assumption identification and fixed-weight linear superposition evaluation mode. It completes the technical reconstruction task of making environmental risk dominated by the most severe mechanical component rather than mathematical patchwork. This achieves the goal of improving the feasibility and resistance to extreme conditions of crane hoisting operation environmental risk quantification project. It provides an independent, stable and physically clear environmental dimension data foundation for subsequent hierarchical alarms, and reduces the probability of false alarms and missed alarms caused by improper data coupling.

[0047] Example 5, as Figure 3In step S2.2, the specific steps for extracting human factors are as follows: S2.21. Extract the fatigue index O, visual field deviation angle P, and maximum permissible visual field deviation from the second dataset. The fatigue index O and the visual field deviation angle P are divided by the maximum permissible visual field deviation. The larger value is used as the physiological state component PS, and the operation response delay L, joystick operation frequency M, standard procedure execution rate N, and maximum permissible response delay are extracted. and maximum allowed operation frequency Calculate the ratio of the three terms and take the larger value as the operation behavior component XO; The specific calculation method is as follows:

[0048]

[0049] Where: O is the fatigue index, P is the field of view deviation angle, L is the operation response delay, M is the joystick operation frequency, and N is the standard procedure execution rate. To the maximum allowable response latency, To the maximum allowed operation frequency, Maximum permissible field of view deviation; S2.22. Compare the physiological state component PS with the operational behavior component XO, and take the larger value as the real-time human risk coefficient. ; Real-time human risk coefficient The specific calculation formula is as follows:

[0050] In the formula, PS represents the physiological state component, and XO represents the operational behavior component.

[0051] In this embodiment: Step S2.21 extracts the fatigue index O, visual field deviation angle P, and maximum permissible visual field deviation Pmax from the second dataset, and takes the larger of the ratios obtained by dividing O and P by Pmax as the physiological state component PS. It also extracts the operation response delay L, joystick operation frequency M, standard process execution rate N, maximum permissible response delay Lmax, and maximum permissible operation frequency Mmax, calculates the ratios of the three items, and takes the larger value as the operation behavior component XO. This realizes the sub-item quantification and independent verification of personnel status, completes the physical calculation task of risk pressure in the physiological and operational dimensions, and achieves the purpose of providing a traceable personnel status calculation basis for the subsequent short-board model.

[0052] Step S2.22 compares the physiological state component PS with the operational behavior component XO and takes the larger value as the real-time human risk coefficient SRW. This realizes the determination of human risk shortcomings dominated by the worst personnel state, completes the engineering application of the reliability principle of the serial system in personnel risk assessment, and achieves the goal of ensuring that the personnel risk coefficient is always controlled by the weakest link and improving the fault tolerance of personnel state under complex working conditions.

[0053] Step S2.2 establishes an artificial risk quantification system that integrates physiological state and operational behavior as independent sub-items and is dominated by the weakest link. This fundamentally improves upon the traditional infrared thermal imaging-dependent temperature difference assumption identification and fixed-weight linear superposition evaluation mode. It completes the technical reconstruction task of making personnel risk dominated by the worst-case scenario rather than mathematically pieced together, thereby achieving the goal of improving the feasibility and resilience of crane lifting personnel risk quantification project.

[0054] Example 6, as Figure 1 In step S3, the specific steps for quick feedback are as follows: S3.1 Extract the following parameters from the first dataset: operating wind speed A, ground tilt angle B, current lifting mass C, current operating radius D, ambient temperature G, maximum allowable operating wind speed Q, rated lifting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, and allowable temperature deviation range V. Calculate the wind speed margin, tilt margin, load margin, and temperature margin. Multiply these margins together to generate a rapid environmental risk threshold. ; Rapid Environmental Risk Threshold The specific calculation formula is as follows:

[0055] Where: A is the operating wind speed, B is the ground tilt angle, C is the current lifting mass, D is the current operating radius, G is the ambient temperature, Q is the maximum allowable operating wind speed, R is the rated lifting mass, S is the maximum operating radius, T is the maximum allowable tilt angle, U is the optimal operating temperature, and V is the allowable temperature deviation range. Extract the following parameters from the second dataset: operation response delay L, joystick operation frequency M, standard procedure execution rate N, fatigue index O, field of view deviation angle P, and maximum permissible response delay. Maximum allowed operation frequency and maximum permissible field of view deviation Calculate response margin, operation frequency margin, process execution rate, fatigue margin, and field of view margin, and multiply these components together to generate a rapid human risk threshold. ; Rapid human risk threshold The specific calculation formula is as follows:

[0056] Where: L is the operation response delay, M is the joystick operation frequency, N is the standard procedure execution rate, O is the fatigue index, and P is the field of view deviation angle. To the maximum allowable response latency, To the maximum allowed operation frequency, Maximum permissible field of view deviation; S3.2, Rapidly set environmental risk thresholds and rapid human risk threshold The real-time environmental risk coefficients obtained in step S2 are respectively compared with those obtained in step S2. and real-time human risk coefficient A comparison was performed, generating the first comparison result, as follows: when At that time, it represents the real-time environmental risk coefficient. If there are no special abnormalities, there is no need to issue a rapid alarm; proceed normally to step S4. when At that time, it represents the real-time environmental risk coefficient. In the event of any special abnormality, a rapid alarm should be triggered to remind operators to stop operations and evacuate to a safe area as soon as possible; when Time represents the real-time human risk coefficient. If there are no special abnormalities, there is no need to issue a rapid alarm; proceed normally to step S4. when Time represents the real-time human risk coefficient. In the event of any unusual abnormality, a rapid alarm should be triggered to remind operators to stop operations and leave the operating area as soon as possible.

[0057] In this embodiment: Step S3.1 extracts the operating wind speed A, gust coefficient E, current lifting mass C, operating radius D, and ambient temperature G from the first dataset. Combined with the factory-calibrated maximum allowable operating wind speed Q, rated lifting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, and temperature allowable deviation range V, the wind speed margin, tilt margin, load margin, and temperature margin are calculated and multiplied together to generate a rapid environmental risk threshold. Simultaneously, the operation response delay L, joystick operation frequency M, standard procedure execution rate N, fatigue index O, and field of view deviation angle P are extracted from the second dataset, combined with the maximum permissible response delay specified at the factory. Maximum allowed operation frequency and maximum permissible field of view deviation Calculate response margin, operation frequency margin, process execution rate, fatigue margin, and field of view margin, and multiply them together to generate a rapid human risk threshold. It realizes the parallel construction of independent threshold channels for both environment and personnel, completes the task of rapid threshold generation based on the serial margin model, and achieves the purpose of providing warning benchmarks for each dimension for subsequent independent linear over-limit comparisons.

[0058] Rapid Environmental Risk Threshold The model is constructed using a series of margin multiplication. Its engineering basis is that the safety constraints of crane hoisting operations have the characteristics of series failure—exceeding the safety boundary of any single factor such as wind speed exceeding the limit, ground tilt exceeding the limit, load exceeding the limit, or temperature exceeding the limit may lead to structural instability or uncontrolled hoisting. Therefore, the multiplication of each margin factor is used to represent the comprehensive safety margin. When any factor approaches zero, the overall threshold approaches zero, triggering a veto alarm. To avoid the threshold collapse caused by a single factor abnormally returning to zero, each margin factor is set with a flexible lower limit protection. When the calculated value is lower than 0.05, it is taken as 0.05 to ensure that other normal dimensions still retain basic warning capabilities. Step S3.2 involves obtaining the real-time environmental risk coefficient in step S2. With rapid environmental risk threshold Direct numerical comparison will be performed to determine the real-time human risk coefficient. With rapid human risk threshold Direct numerical comparison is performed to generate the first comparison result. When any single item in the environmental or personnel dimension exceeds the limit, a rapid alarm is triggered. If the limit is not exceeded, the comprehensive analysis step is entered normally. This realizes a direct response and veto mechanism when a single-dimensional risk deteriorates. It completes the emergency alarm triggering task independent of the comprehensive calculation, and achieves the goal of avoiding the delay of the comprehensive calculation from masking the emergency risk and shortening the response time of the emergency.

[0059] Compared to existing technologies that include environmental and personnel risks in the same comprehensive threshold for unified comparison, which leads to delays in emergency response, step S3 enables alarms to be triggered directly in the event of extreme environmental deterioration or severe personnel malfunction without waiting for comprehensive calculations. This forms a dual rapid protection mechanism with environmental and personnel veto power, avoiding the masking of emergency risks by comprehensive calculation delays and improving the real-time safety response and engineering reliability of the entire hoisting operation process.

[0060] Example 7, as Figure 1 In step S4, the specific steps of the comprehensive analysis are as follows: S4.1 First Comparison Result Generation: Real-time environmental risk coefficient and real-time human risk coefficient A comprehensive alarm coefficient is generated after performing nonlinear coupling calculations. Establish alarm thresholds And will integrate alarm coefficients With alarm threshold Compare the results to generate the first comparison result; S4.2 Threshold Dynamic Update: Based on the parameter changes in the first and second datasets, determine the direction of threshold update, and shrink or widen the threshold while determining the shrinkage or widening range, generating a new alarm threshold. It replaces the old alarm threshold Z and the comprehensive alarm coefficient. Compare them.

[0061] In this embodiment: Step S4.1 involves reading the real-time environmental risk coefficient output in step S2. With real-time human risk coefficient The method involves vector synthesis of two orthogonal and independent risk dimensions to generate a comprehensive alarm coefficient BJ, establishing an alarm threshold Z determined by the smaller of the time-series updated target value and the coupled updated target value, and comparing the comprehensive alarm coefficient BJ with the alarm threshold Z to generate a second comparison result. This achieves the physical quantification of the coupling amplification effect of environmental risk and personnel risk, completes the task of constructing a benchmark for nonlinear superposition and dynamic comparison of comprehensive risks, and achieves the goal of providing coupled risk values ​​with clear mechanical and statistical correlations for progressive conventional alarms.

[0062] Step S4.2 involves real-time monitoring of parameter change trends in the first and second datasets, parallel computation of time-series update target values ​​and coupled update target values ​​to determine the threshold update direction. When performing threshold contraction, the threshold is gradually tightened based on the previous cycle value at a preset ratio. When performing threshold relaxation, after confirmation within the recovery window period, the threshold is gradually released at a preset ratio lower than the contraction rate, generating a new alarm threshold. It replaces the previous cycle threshold, realizing a dual mechanism for real-time adaptation of alarm thresholds as the environment gradually deteriorates and the overall risk increases sharply. It completes the closed-loop control task of dynamic threshold updates and smooth transitions, achieving the goal of avoiding false alarms and missed alarms caused by static threshold lag or jumps.

[0063] Compared to existing technologies that use static thresholds or simple weighting, which lead to threshold rigidity and alarm oscillations, step S4 reflects the coupled amplification effect of dual-dimensional risks through vector synthesis, captures the gradual deterioration trend of the environment through time-series updates, and responds to the impact of sudden increases in comprehensive risks through coupled updates. This ensures that the alarm threshold is always dominated by the most dangerous mechanism and that the change process is stable and controllable, reducing the probability of false alarms and missed alarms caused by threshold rigidity or sudden changes, and improving the engineering reliability of risk early warning throughout the entire process.

[0064] Example 8, as Figure 4 In step S4.1, the specific steps for generating the first comparison result are as follows: S4.11, Real-time environmental risk coefficient output in step S2.1 Compared with the real-time human risk coefficient output in step S2.2 Perform vector synthesis calculations to generate the comprehensive alarm coefficient BJ; Comprehensive alarm coefficient The specific calculation formula is as follows:

[0065] in: For real-time environmental risk coefficient, Real-time human risk factor.

[0066] S4.12, Combine alarm coefficients The result is compared with the preset alarm threshold Z to generate the first comparison result, as follows: when When the time is right, it means there is no risk and no alarm needs to be triggered. when When this occurs, it indicates that there is a risk and an alarm needs to be triggered. when At this time, both the warning lights and sirens will provide a warning at an interval of 1 second, with the sirens volume set between 45% and 55%. when At this time, both the warning lights and sirens will be activated at 0.5-second intervals, with the sirens volume set between 55% and 75%. when At this time, both the warning lights and sirens will provide a warning at an interval of 0.3 seconds, with the sirens volume set between 75% and 135%.

[0067] In this embodiment: Step S4.11 reads the real-time environmental risk coefficient SHJ output in step S2.1 and the real-time human risk coefficient SRW output in step S2.2, and performs vector synthesis operation on the two orthogonal and independent risk dimensions to generate a comprehensive alarm coefficient BJ. This realizes the geometric superposition and quantification of environmental risk and human risk, completes the physical calculation task of the dual-dimensional risk coupling amplification effect, and achieves the purpose of providing a comprehensive risk value with clear input-output correlation and mechanical causal relationship for progressive alarm.

[0068] Step S4.12 compares the comprehensive alarm coefficient BJ with the preset alarm threshold Z and outputs a three-level progressive alarm signal based on the ratio. The progressively increasing light flashing frequency, sound interval, and volume reflect the degree of risk approach, thus replacing the traditional voice broadcast or forced shutdown alarm method. This completes the task of continuously quantifying the degree of risk approach, providing operators with an autonomous decision-making window and avoiding secondary risks caused by emergency braking.

[0069] Compared to existing discrete alarm methods that rely on voice broadcasts or forced shutdowns, step S4.1 establishes a collaborative architecture combining comprehensive risk vector synthesis and a three-level progressive alarm system. This architecture uses vector synthesis to geometrically superimpose environmental and personnel risks as orthogonal independent components, ensuring that the rate of comprehensive risk growth when both dimensions deteriorate simultaneously exceeds linear prediction. Furthermore, the three-level progressive audible and visual alarm system visually reflects the approaching risk through continuous changes in light frequency, sound intervals, and volume. This improvement replaces the traditional hard-switching control strategy, avoiding secondary risks such as load swaying caused by forced shutdowns, providing operators with a progressive, autonomous decision-making window, and enhancing the continuity and reliability of risk warnings for crane lifting operations.

[0070] Example 9, as Figure 5 In step S4.2, the specific steps for dynamically updating the threshold are as follows: S4.21. The time-series target value and coupled target value are calculated in parallel operation for each cycle. The trend slope α is calculated from the operating wind speed A, combined with the system response delay β, the maximum allowable operating wind speed Q, the model correction factor γ, the ground tilt angle B, the ambient temperature G, the visibility F, and the critical visibility. Operation timing target value The coupled target value is calculated by combining the comprehensive alarm coefficient BJ and the coupling update coefficient μ. ; Time series target value The specific calculation formula is as follows: ; Coupled target value The specific calculation formula is as follows: ; Update threshold The specific calculation formula is as follows:

[0071] Where: Z is the alarm threshold, A is the operating wind speed, α is the trend slope, β is the combined system response delay, Q is the maximum allowable operating wind speed, γ is the model correction factor, B is the ground tilt angle, G is the ambient temperature, and F is the visibility. Where BJ is the critical visibility level, μ is the comprehensive alarm coefficient, and μ is the coupling update coefficient. S4.22 When the threshold shrinks, it is gradually tightened according to the preset ratio based on the previous cycle value. When the threshold is widened, it is gradually released according to the preset ratio lower than the shrinkage rate after the stabilization and recovery period of all parameters. The threshold is dynamically updated and output to the regular alarm comparison stage. Previous alarm threshold The determination method is as follows:

[0072] During contraction:

[0073]

[0074] When relaxed:

[0075]

[0076] in: To update the threshold.

[0077] In this embodiment: Step S4.21 calculates the trend slope α based on the operating wind speed A in the first dataset for each cycle, and combines it with the system response delay β, the maximum allowable operating wind speed Q, the model correction coefficient γ, the ground tilt angle B, the ambient temperature G, the visibility F, and the time-series target value of the critical visibility F0. Simultaneously, the comprehensive alarm coefficient BJ and the coupled update coefficient μ are read to calculate the coupled target value. Take the smaller of the two values ​​as the update threshold. If the alarm threshold Z is not triggered within a cycle, the time-series target value will be used. Update the alarm threshold Z to generate an updated threshold. If the alarm threshold Z is triggered within a cycle, the target value is determined according to the coupling. Update the alarm threshold Z to generate an updated threshold. The system records nodes and relaxes thresholds in the future. Through the above scheme, it achieves parallel prediction of both the gradual trend of the environment and the sudden increase of comprehensive risk. It completes the conservative judgment task of the threshold direction dominated by the more dangerous mechanism and achieves the goal of ensuring that the alarm threshold always responds to the severe risk situation.

[0078] The The relative degree of wind speed deterioration during the system response lag period is represented by α, where α is the historical trend slope in meters per second squared, β is the system response delay in seconds, and α × β in meters per second represents the equivalent change in wind speed trend accumulated during the system response delay time. After being divided by the reference wind speed Q, it is normalized to a dimensionless ratio. The larger the ratio, the closer the wind speed is to the dangerous level within the time window when the system cannot respond in time, thereby driving the threshold contraction to provide early warning. The historical trend slope α is obtained by linear fitting of the operating wind speed A time series data in the most recent 30 seconds using the least squares method. The sliding window step size is synchronized with the system sampling period.

[0079] Step S4.22 Read the alarm threshold from the previous cycle. The first cycle uses the preset alarm threshold Z, and subsequent cycles use the updated threshold output from the previous cycle. The threshold will be updated. Compared with the alarm threshold of the previous cycle In comparison, during contraction, the contraction is gradually tightened by negative changes, while during relaxation, after confirmation of a stable recovery period for all parameters, the contraction is gradually released by positive changes. When the target coupling value is reached... Update the alarm threshold Z to generate an updated threshold. After the event, output the shrinkage value. Or relax the value Replace the updated threshold The above scheme achieves rate constraint and smooth transition of threshold change, completes the closed-loop control task of avoiding alarm system oscillation caused by threshold jump, and achieves the purpose of providing a continuous and stable dynamic benchmark for routine alarm comparison.

[0080] Compared to existing technologies where the threshold is fixed or only changes with a single parameter, resulting in severe lag, step S4.2 uses a dual-mechanism parallel operation to capture gradual environmental deterioration and sudden surges. It ensures that the threshold contraction and relaxation process is smooth and controllable through rate of change constraints, and avoids repeated oscillations caused by environmental fluctuations through recovery period confirmation. This effectively reduces the defect that static thresholds cannot reflect real-time working conditions and improves the dynamic adaptability and overall stability of risk warning for hoisting operations.

[0081] This application also provides a crane lifting risk early warning system based on image recognition; please refer to [link / reference]. Figure 2 It includes an image acquisition module, a risk extraction module, a rapid alarm module, a comprehensive analysis module, and a threshold update module: The image acquisition module is used to acquire images of the hoisting area, panoramic operation images, and operator images, extract environmental feature parameters and personnel feature parameters, and generate the first dataset and the second dataset. The risk extraction module is used to calculate real-time environmental risk coefficients based on the first and second datasets. and real-time human risk coefficient ; The rapid alarm module is used to quickly set environmental risk thresholds. and rapid human risk threshold and with real-time environmental risk coefficient and real-time human risk coefficient Compare and generate the first comparison result; The comprehensive analysis module is used to analyze real-time environmental risk coefficients. and real-time human risk coefficient Perform vector synthesis calculations to generate a comprehensive alarm coefficient. And compare it with the dynamic alarm threshold Z to generate a second comparison result; The threshold update module is used to dynamically update the alarm threshold Z based on environmental parameters, the maximum allowable operating wind speed Q, and the comprehensive alarm coefficient BJ.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A crane hoisting risk early warning method based on image recognition, characterized in that, The specific steps are as follows: S1. Data Acquisition and Processing: Real-time image data of the crane hoisting operation site is acquired through an image acquisition device. Environmental feature parameters and personnel feature parameters are extracted based on the real-time image data, and the environmental feature parameters and personnel feature parameters are preprocessed to generate a first dataset and a second dataset. S2. Factor Extraction: Calculate the real-time environmental risk coefficient from the first and second datasets in step S1. and real-time human risk coefficient ; S3. Rapid Feedback: Extract and calculate the parameters from the first and second datasets generated in step S1 to generate a rapid environmental risk threshold. and rapid human risk threshold and will rapidly set environmental risk thresholds and rapid human risk threshold Respectively with real-time environmental risk coefficient and real-time human risk coefficient Compare the results to generate the first comparison result. Based on the first comparison result, issue a rapid alarm or proceed to the next step normally. S4. Comprehensive Analysis: Analyze the real-time environmental risk coefficients from step S2. and real-time human risk coefficient Perform integrated calculations to generate a comprehensive alarm coefficient. And set the alarm threshold Z, and combine the comprehensive alarm coefficient The alarm threshold Z is compared to generate a second comparison result. A regular alarm is triggered based on the comparison result, and update rules are set to generate a dynamically updated threshold in real time, updating the alarm threshold according to the real-time situation. .

2. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S1, the specific steps for data acquisition and processing are as follows: S1.

1. A first image acquisition device is installed at the end of the crane boom, and a second image acquisition device is installed on the top of the turntable. The first image acquisition device acquires images of the hoisting area and extracts the operating wind speed A, ground tilt angle B, current hoisting mass C, and current operating radius D. The gust coefficient E is calculated from the image sequence of the hoisting area. The second image acquisition device acquires panoramic images of the operation and extracts the visibility F and ambient temperature G. The factory-calibrated maximum allowable operating wind speed Q, rated hoisting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, allowable temperature deviation span V, and critical visibility are read. The system response delay β, the model correction coefficient γ, and the coupling update coefficient μ are organized into the first dataset. S1.

2. A third image acquisition device is installed above and in front of the control room to acquire images of the operator's face and upper body, and extract the fatigue index O and field of view deviation angle P. The operation response delay L, the joystick operation frequency M, and the standard process execution rate N are obtained through the joystick displacement sensor, hydraulic pressure sensor, and PLC logic comparison device. The maximum permissible response delay calibrated by the factory is read. Maximum allowed operation frequency and maximum permissible field of view deviation Organize the above parameters into a second dataset; S1.

3. Denoise, normalize and time-align the real-time collected parameters in the first dataset and the second dataset. Calculate the historical trend slope α from the operating wind speed A in the first dataset. Incorporate the historical trend slope α into the first dataset as a derived calculation parameter. Write the factory calibration parameters directly into the corresponding dataset.

3. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S2, the specific steps for factor extraction are as follows: S2.1 Environmental Factor Extraction: Extract multiple parameters from the first dataset and perform calculations to obtain the real-time environmental risk coefficient. ; S2.2 Human Factor Extraction: Extract multiple parameters from the second dataset and perform calculations to obtain the real-time human risk coefficient. .

4. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S2.1, the specific steps for extracting environmental factors are as follows: S2.

11. Extract the operating wind speed A, gust coefficient E, ground tilt angle B and maximum allowable operating wind speed Q from the first dataset, calculate the wind load risk index FL according to the formula, extract the current hoisting mass C, operating radius D, ambient temperature G, rated hoisting mass R, maximum operating radius S, optimal operating temperature U and temperature allowable deviation span V, and calculate the load risk index HL according to the formula. S2.

12. Compare the wind load risk index FL and the load risk index HL, and take the larger value as the real-time environmental risk coefficient. .

5. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S2.2, the specific steps for extracting human factors are as follows: S2.

21. Extract the fatigue index O, visual field deviation angle P, and maximum permissible visual field deviation from the second dataset. Take the fatigue index O and the visual field deviation angle P, and divide by The larger value in the equation is used as the physiological state component PS. Operation response delay L, joystick operation frequency M, standard procedure execution rate N, and maximum permissible response delay are extracted from these components. and maximum allowed operation frequency Calculate the ratio of the three terms and take the larger value as the operation behavior component XO; S2.

22. Compare the physiological state component PS with the operational behavior component XO, and take the larger value as the real-time human risk coefficient. .

6. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S3, the specific steps for quick feedback are as follows: S3.1 Extract the following parameters from the first dataset: operating wind speed A, ground tilt angle B, current lifting mass C, current operating radius D, ambient temperature G, maximum allowable operating wind speed Q, rated lifting mass R, maximum operating radius S, maximum allowable tilt angle T, optimal operating temperature U, and allowable temperature deviation range V. Calculate the wind speed margin, tilt margin, load margin, and temperature margin. Multiply these margins together to generate a rapid environmental risk threshold. ; Extract the following parameters from the second dataset: operation response delay L, joystick operation frequency M, standard procedure execution rate N, fatigue index O, field of view deviation angle P, and maximum permissible response delay. Maximum allowed operation frequency and maximum permissible field of view deviation Calculate response margin, operation frequency margin, process execution rate, fatigue margin, and field of view margin, and multiply these components together to generate a rapid human risk threshold. ; S3.2, Rapidly set environmental risk thresholds and rapid human risk threshold The real-time environmental risk coefficients obtained in step S2 are respectively compared with those obtained in step S2. and real-time human risk coefficient A comparison was performed, generating the first comparison result, as follows: when At that time, it represents the real-time environmental risk coefficient. If there are no special abnormalities, there is no need to issue a rapid alarm; proceed normally to step S4. when At that time, it represents the real-time environmental risk coefficient. In the event of any special abnormality, a rapid alarm should be triggered to remind operators to stop operations and evacuate to a safe area as soon as possible; when Time represents the real-time human risk coefficient. If there are no special abnormalities, there is no need to issue a rapid alarm; proceed normally to step S4. when Time represents the real-time human risk coefficient. In the event of any unusual abnormality, a rapid alarm should be triggered to remind operators to stop operations and leave the operating area as soon as possible.

7. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S4, the specific steps of the comprehensive analysis are as follows: S4.1 First Comparison Result Generation: Real-time environmental risk coefficient and real-time human risk coefficient A comprehensive alarm coefficient is generated after performing nonlinear coupling calculations. Establish alarm thresholds And will integrate alarm coefficients With alarm threshold Compare the results to generate the first comparison result; S4.2 Threshold Dynamic Update: Based on the parameter changes in the first and second datasets, determine the direction of threshold update, and shrink or widen the threshold while determining the shrinkage or widening range, generating a new alarm threshold. It replaces the old alarm threshold Z and the comprehensive alarm coefficient. Compare them.

8. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S4.1, the specific steps for generating the first comparison result are as follows: S4.11, Real-time environmental risk coefficient output in step S2.1 Compared with the real-time human risk coefficient output in step S2.2 Perform vector synthesis calculations to generate the comprehensive alarm coefficient BJ; S4.12, Combine alarm coefficients The result is compared with the preset alarm threshold Z to generate the first comparison result, as follows: when When the time is right, it means there is no risk and no alarm needs to be triggered. when When this occurs, it indicates that there is a risk and an alarm needs to be triggered. when At this time, both the warning lights and sirens will provide a warning at an interval of 1 second, with the sirens volume set between 45% and 55%. when At this time, both the warning lights and sirens will be activated at 0.5-second intervals, with the sirens volume set between 55% and 75%. when At this time, both the warning lights and sirens will provide a warning at an interval of 0.3 seconds, with the sirens volume set between 75% and 135%.

9. The crane hoisting risk early warning method based on image recognition according to claim 1, characterized in that, In step S4.2, the specific steps for dynamically updating the threshold are as follows: S4.

21. The time-series target value and coupled target value are calculated in parallel operation for each cycle. The trend slope α is calculated from the operating wind speed A, combined with the system response delay β, the maximum allowable operating wind speed Q, the model correction factor γ, the ground tilt angle B, the ambient temperature G, the visibility F, and the critical visibility. Operation timing target value The coupled target value is calculated by combining the comprehensive alarm coefficient BJ and the coupling update coefficient μ. ; S4.22 When the threshold shrinks, it is gradually tightened according to the preset ratio based on the previous cycle value. When the threshold is widened, it is gradually released according to the preset ratio lower than the shrinkage rate after the stabilization recovery period of all parameters. The threshold is dynamically updated and output to the regular alarm comparison stage.

10. A crane hoisting risk early warning system based on image recognition, characterized in that, This image recognition-based crane lifting risk warning system is used to execute the image recognition-based crane lifting risk warning method described in any one of claims 1 to 9. The system includes an image acquisition module, a risk extraction module, a rapid alarm module, a comprehensive analysis module, and a threshold update module. The image acquisition module is used to acquire images of the hoisting area, panoramic operation images, and operator images, extract environmental feature parameters and personnel feature parameters, and generate the first dataset and the second dataset. The risk extraction module is used to calculate real-time environmental risk coefficients based on the first and second datasets. and real-time human risk coefficient ; The rapid alarm module is used to quickly set environmental risk thresholds. and rapid human risk threshold and with real-time environmental risk coefficient and real-time human risk coefficient Compare and generate the first comparison result; The comprehensive analysis module is used to analyze real-time environmental risk coefficients. and real-time human risk coefficient Perform vector synthesis calculations to generate a comprehensive alarm coefficient. And compare it with the dynamic alarm threshold Z to generate a second comparison result; The threshold update module is used to dynamically update the alarm threshold Z based on environmental parameters, the maximum allowable operating wind speed Q, and the comprehensive alarm coefficient BJ.

Images