A crane hoist area intruder recognition method and system

Abstract

The present application provides a kind of crane hoist area personnel intrusion identification method, comprising the following steps: S1, video image is collected;S2, video image is preprocessed;S3, the division of dangerous area is completed;S4, extract foreground moving target in video frame;S5, target detection is carried out to video frame;S6, measure real-time relative distance;S7, confirm effective personnel target;S8, confirm risk level;S9, execute hierarchical early warning linkage control.The present application also provides a kind of crane hoist area personnel intrusion identification system, based on the crane hoist area personnel intrusion identification method described above, comprising video image acquisition module, AI image processing module, edge computing module, early warning linkage module and cloud management module.The present application solves the technical problems that the existing crane hoist area safety protection mode is difficult to adapt to complex hoisting conditions and has low precision, and produces the technical effects of improving the response capability to complex operating environment and improving the detection precision.

Description

Technical Field

[0001] This invention relates to the field of crane safety monitoring technology, and in particular to a method for identifying unauthorized personnel entering a crane lifting area and a crane lifting area unauthorized personnel identification system based on the method. Background Technology

[0002] Crane lifting operations are a core production process in industries such as metallurgy, manufacturing, construction, ports, and energy, but they are also high-risk special operations. During crane lifting operations, personnel illegally entering high-risk areas of the lifting zone or manually supporting the load can easily lead to accidents such as falling objects striking, crushing, or collisions, resulting in injuries, fatalities, and business shutdowns.

[0003] Currently, safety protection in crane lifting areas typically employs a combination of monitoring equipment and manual control. However, traditional monitoring equipment suffers from poor environmental adaptability, high false alarm rates under complex conditions, and insufficient robustness of machine vision algorithms, resulting in weak adaptability to personnel targets and a tendency to generate false alarms or missed alarms. Furthermore, the lack of accurate three-dimensional spatial perception capabilities leads to distorted risk assessments. Manual control relies on on-site inspections and visual observations in the lifting area, which suffers from issues such as personnel fatigue and blind spots, and cannot achieve 24-hour, comprehensive control. Therefore, existing safety protection methods for crane lifting areas are ill-suited to complex lifting conditions and suffer from low accuracy. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method for identifying unauthorized personnel entering a crane lifting area. This method solves the problems of existing crane lifting area safety protection methods being difficult to adapt to complex lifting conditions and having low accuracy.

[0005] According to an embodiment of the present invention, a method for identifying unauthorized personnel entering a crane lifting area includes the following steps:

[0006] S1. Use a binocular camera to capture left and right eye video images of the crane's lifting area;

[0007] S2. Perform preprocessing operations on the acquired left and right eye video images, including grayscale transformation, Gaussian filtering for noise reduction, adaptive image enhancement, and distortion correction.

[0008] S3. Hough transform is used to identify and calibrate the dangerous areas in the crane lifting area under static conditions, and the target detection algorithm is used to identify targets such as hooks and loads in the crane lifting area in real time, generating a following dangerous area that follows the target, thus completing the division of dangerous areas;

[0009] S4. Using Gaussian mixture background modeling and background subtraction, foreground moving targets are extracted from video frames of left and right eye video images, and suspected abnormal targets are screened.

[0010] S5. Using the YOLOv8 deep learning algorithm, target detection is performed on the video frames of the left and right eye video images to identify suspected abnormal targets such as personnel targets, hook targets, and suspended objects, and the target category, bounding box and detection confidence are output.

[0011] S6. Based on binocular stereo vision technology, a disparity map is generated through stereo matching, the three-dimensional spatial coordinates of the identified personnel target are calculated, and the real-time relative distance between the personnel target and the hook target, the suspended object target, and the dangerous area is measured.

[0012] S7. The DeepSORT multi-target tracking algorithm is used to continuously track personnel targets, obtain the movement trajectory of personnel targets, and combine the detection confidence and trajectory confidence to complete the false alarm removal and confirm the valid personnel targets.

[0013] S8. Combine the overlap between personnel targets and hazardous areas, three-dimensional spatial distance, and crane operating conditions to determine the risk level and confirm the risk level.

[0014] S9. Execute corresponding graded early warning and linkage control according to the risk level, triggering audible and visual warnings, voice warnings, or emergency shutdown and action prohibition of the crane.

[0015] On the other hand, according to an embodiment of the present invention, a crane hoisting area personnel intrusion identification system is also provided. Based on the above-mentioned crane hoisting area personnel intrusion identification method, it includes a video image acquisition module, an AI image processing module, an edge computing module, an early warning linkage module, and a cloud management module. The AI ​​image processing module and the early warning linkage module are electrically connected to the edge computing module, and the video image acquisition module and the cloud management module are communicatively connected to the edge computing module.

[0016] Compared to existing technologies, this invention offers the following advantages: By combining the YOLOv8 algorithm with binocular stereo vision technology, dynamic real-time tracking of the lifting area, multi-dimensional false alarm suppression, multi-target tracking, and hierarchical early warning and control, it achieves high detection accuracy and real-time performance, reducing false alarm and missed alarm rates. It can also cope with various harsh industrial environments, adapt to various cranes and full-lifting conditions, and utilizes binocular stereo vision technology to achieve precise three-dimensional positioning and ranging, making risk assessment more scientific and preventing crane-related accidents from the source. Furthermore, by replacing manual inspections with machines, it achieves 24 / 7 comprehensive control, effectively avoiding accident losses. It solves the technical problems of existing crane lifting area safety protection methods being difficult to adapt to complex lifting conditions and having low accuracy. It combines the efficiency of traditional machine vision with the high accuracy of deep learning algorithms, while achieving precise spatial perception through binocular stereo vision, balancing real-time detection, accuracy, and reliability, thus improving the ability to cope with complex operating environments and enhancing detection accuracy. Attached Figure Description

[0017] Figure 1 This is a step diagram of a crane hoisting area intrusion identification method according to an embodiment of the present invention;

[0018] Figure 2 This is an architecture diagram of a crane hoisting area personnel intrusion identification system according to an embodiment of the present invention.

[0019] In the above attached diagram: 1. Video image acquisition module; 2. AI image processing module; 3. Edge computing module; 4. Early warning linkage module; 5. Cloud management module. Detailed Implementation

[0020] The technical solutions of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] like Figure 1 As shown in the figure, this invention proposes a method for identifying unauthorized personnel entering a crane lifting area, comprising the following steps:

[0022] S1. Use a binocular camera to capture left and right eye video images of the crane's lifting area.

[0023] In step S1, industrial-grade binocular high-definition cameras are installed in the crane lifting area to collect left and right eye video images of the crane lifting area in real time.

[0024] S2. Perform preprocessing operations on the acquired left and right eye video images, including grayscale transformation, Gaussian filtering for noise reduction, adaptive image enhancement, and distortion correction.

[0025] In step S2, due to the harsh conditions at the work site, such as dust, rain, fog, strong backlight, low light, and metal reflection, preprocessing operations including grayscale transformation, Gaussian filtering for noise reduction, adaptive image enhancement, and distortion correction are performed on the acquired left and right eye video images to eliminate environmental noise and interference, improve image quality, and facilitate subsequent detection.

[0026] S3. Hough transform is used to identify and calibrate the dangerous areas in the crane lifting area under static conditions. Target detection algorithm is used to identify targets such as hooks and loads in the crane lifting area in real time, generating a following dangerous area that follows the target, thus completing the division of dangerous areas.

[0027] In step S3, for statically fixed hazardous areas of the crane, such as the triangular area of ​​the guide pulley steel rope, around the fast rope, around the column, and the track running section, Hough transform is used to detect the boundary straight lines of the hazardous areas. Combined with manually calibrated area masks, the static hazardous areas are delineated. For dynamic high-risk areas that move with the hook and load, such as under the load, under the boom, and in the stress area of ​​the inclined hook, target detection algorithms are used to detect targets such as the hook and load in real time. Combined with the safety regulations for lifting operations, a following hazardous area is dynamically generated with the hook / load as the center. The range of the following hazardous area is adjusted according to the lifting height, load size, and working conditions.

[0028] S4. Using Gaussian mixture background modeling and background subtraction, foreground moving targets are extracted from the video frames of the left and right eye video images, and suspected abnormal targets are screened.

[0029] In step S4, Gaussian mixture background modeling (MOG2) combined with background subtraction is used to quickly extract foreground targets from video frames of the left and right eye video images, thereby performing preliminary screening of abnormal intrusion targets, narrowing the scope of subsequent detection, improving algorithm processing efficiency, and reducing computing power consumption.

[0030] S5. Using the YOLOv8 deep learning algorithm, target detection is performed on the video frames of the left and right eye video images to identify suspected abnormal targets such as personnel targets, hook targets, and suspended objects, and the target category, bounding box and detection confidence are output.

[0031] In step S5, the YOLOv8 deep learning target detection model is used to achieve accurate and rapid identification of personnel targets, while also being compatible with simultaneous detection of multiple targets such as hook targets and suspended objects.

[0032] S6. Based on binocular stereo vision technology, a disparity map is generated through stereo matching, the three-dimensional spatial coordinates of the identified personnel target are calculated, and the real-time relative distance between the personnel target and the hook target, the suspended object target, and the dangerous area is measured.

[0033] In step S6, based on binocular stereo vision technology, a disparity map is generated by stereo matching of the left and right eye images to complete the three-dimensional spatial coordinate modeling of the target. This allows for the accurate calculation of the three-dimensional spatial distance between the personnel target, the suspended object target, and the dangerous area, providing accurate spatial data support for subsequent risk level determination.

[0034] S7. The DeepSORT multi-target tracking algorithm is used to continuously track personnel targets, obtain their movement trajectories, and combine detection confidence and trajectory confidence to remove false alarms and confirm valid personnel targets.

[0035] In step S7, the movement trajectory of personnel targets is monitored by DeepSORT multi-target tracking algorithm, and false alarm removal algorithm based on detection confidence and trajectory confidence is combined to reduce false alarm rate and ensure the accuracy and reliability of risk assessment.

[0036] S8. Combine the overlap between personnel targets and hazardous areas, three-dimensional spatial distance, and crane operating conditions to determine the risk level and confirm the risk level.

[0037] In step S8, the overlap between the personnel target and the designated static and following hazard areas is calculated to determine whether the personnel target has crossed the boundary into the hazard area in a plane. Based on the three-dimensional spatial distance between the personnel target, the suspended object target, and the hazard area, the risk level of the personnel target is confirmed.

[0038] S9. Execute corresponding graded early warning and linkage control according to the risk level, triggering audible and visual warnings, voice warnings, or emergency shutdown and action prohibition of the crane.

[0039] In step S9, when a person is detected entering a dangerous area and the risk level of the person is determined, the system triggers an audible and visual warning and a voice warning to alert the person based on the preset graded early warning linkage control corresponding to the risk level, or controls the crane to take emergency shutdown and action prohibition measures to prevent safety accidents.

[0040] Furthermore, in step S2, the RGB color image is converted to a grayscale image, and the weighted grayscale transformation formula is as follows:

[0041] ,

[0042] In the formula I R I G I B For the pixel values ​​corresponding to the RGB three channels, I gray This is the grayscale image output after preprocessing. Converting an RGB color image to grayscale reduces the computational load of subsequent algorithms, thereby improving processing efficiency, while preserving the core contours and texture features of the person target.

[0043] Furthermore, in step S2, the image is smoothed using a Gaussian convolution kernel, the Gaussian convolution kernel function being...

[0044] ,

[0045] In the formula, G(x,y) is the two-dimensional Gaussian convolution kernel function, σ is the filter standard deviation, and (x,y) are the image pixel coordinates. Smoothing the image using a Gaussian convolution kernel can eliminate Gaussian noise and salt-and-pepper noise, thereby reducing the impact of dust, particles, and other interference on subsequent detection. In this embodiment, the filter standard deviation is used to control the smoothing intensity, with a default value of 1.5, and can be adaptively adjusted according to environmental noise.

[0046] Furthermore, in step S2, for strong light, backlight, and low-light scenes, a Limiting Contrast Adaptive Histogram Equalization (CLAHE) algorithm is used to enhance local image contrast and highlight the features of people and other targets, thus solving the problems of targets in dark areas being unclear and targets in bright areas being overexposed. For rain, fog, and dust scenes, a dark channel prior dehazing algorithm is used to eliminate the occlusion of fog and dust on the image and restore clear image details.

[0047] Furthermore, in step S3, the Hough transform line detection formula is as follows:

[0048] ,

[0049] In the formula, ρ is the vertical distance from the origin of the image coordinate system to the straight line of the hanging area boundary, θ is the angle between the perpendicular line and the x-axis of the image, and x and y are the pixel coordinates of the image, respectively.

[0050] Furthermore, in step S4, the background subtraction method formula is:

[0051]

[0052] In the formula, F(x,y) is the foreground target mask, and I is the target mask. t B is the video frame image at the current moment, B is the adaptively updated background modeling image, and T is... bg The background difference threshold is used. Gaussian mixture background modeling enables real-time adaptive updates of the background model, adapting to dynamic background changes such as gradual lighting changes, wind-blown debris swaying, and slight equipment vibrations. This reduces the extraction of invalid foreground targets, thereby lowering the probability of subsequent false detections.

[0053] Furthermore, in step S5, the YOLOv8 deep learning object detection model is optimized. First, a lightweight YOLOv8n backbone network is adopted and GhostConv is introduced to reduce the number of model parameters and computational cost. Simultaneously, model pruning and INT8 quantization significantly reduce edge computing power consumption, enabling real-time edge detection with a single-frame processing latency of ≤20ms. Second, the multi-scale feature fusion network (PAN-FPN) is optimized by adding a small target detection layer to strengthen the fusion of shallow features and deep semantic features, improving the detection accuracy of distant, small-sized personnel targets. The CBAM convolutional attention mechanism is also introduced to enhance target feature extraction capabilities, suppress background interference, and address the issue of missed detections in scenarios with personnel occlusion and partial overlap. Finally, the CIoU bounding box regression loss function is adopted to improve the regression accuracy of target bounding boxes and accelerate model convergence. The FocalLoss loss function is introduced to address the imbalance between positive and negative samples, improving object detection accuracy in complex scenes. The CIoU bounding box regression loss function is...

[0054] ,

[0055] In the formula, IoU is the intersection-union ratio of the predicted box and the ground truth box, ρ is the Euclidean distance between the center point of the predicted box and the center point of the ground truth box, c is the diagonal length of the minimum bounding rectangle of the predicted box and the ground truth box, v is the aspect ratio consistency parameter of the predicted box and the ground truth box, and α is the weight coefficient, which is used to balance the aspect ratio loss.

[0056] Furthermore, in step S5, the YOLOv8 detection confidence calculation formula is as follows:

[0057] ,

[0058] In the formula S conf The final detection confidence level and S for the target cls Predict the probability of the target category, S obj The target has a confidence level.

[0059] Furthermore, in step S5, a dedicated dataset for crane hoisting operations covering the entire scenario is constructed, including on-site images of different crane models, working conditions, lighting conditions, weather conditions, and degrees of occlusion. Data augmentation methods such as mosaic enhancement, hybrid enhancement, random occlusion, lighting transformation, and scale transformation are used to expand the dataset size and improve the model's generalization ability. A transfer learning strategy is adopted, based on publicly available pre-trained weights, and fine-tuned training is performed using the dedicated dataset for crane hoisting operations. This enables the YOLOv8 deep learning object detection model to quickly adapt to the target features of crane hoisting operation scenarios, thereby improving detection accuracy.

[0060] Furthermore, in step S6, the Zhang Zhengyou calibration method is used to complete the calibration of the intrinsic and extrinsic parameters of the binocular camera, and key parameters such as camera focal length, principal point coordinates, distortion coefficient, and binocular baseline distance are obtained. The left and right eye images are corrected to the same horizontal line through epipolar correction to eliminate vertical parallax, thereby significantly reducing the computational complexity of subsequent stereo matching and improving matching accuracy.

[0061] Furthermore, in step S6, a semi-global stereo matching algorithm (SGBM) is adopted to improve matching accuracy and edge processing efficiency, thereby generating a high-precision disparity map. Through left-right consistency detection, sub-pixel interpolation, and disparity map filtering optimization, mismatched points are eliminated to improve disparity calculation accuracy and prevent metal reflections and texture loss at the crane hoisting site from affecting matching.

[0062] Furthermore, in step S6, the formula for calculating the three-dimensional spatial distance is as follows:

[0063] ,

[0064] In the formula, Z represents the depth distance between the target and the binocular camera, f is the equivalent focal length of the binocular camera, B is the baseline distance of the binocular camera, and d is the parallax value of the target in the left and right eye images. By transforming the target's three-dimensional coordinates, the real-time relative distance between personnel targets, suspended objects, and hazardous areas can be calculated, facilitating the classification of different risk levels and enabling graded early warning and control.

[0065] Furthermore, in step S7, the DeepSORT multi-target tracking algorithm is used to acquire the motion trajectory, direction, and speed of the personnel target. Combined with trajectory features, multi-dimensional false alarm suppression is achieved, eliminating false detections caused by swaying metal components, dust, and changes in light and shadow in crane lifting operations. The confidence-weighted fusion formula for false alarm removal is as follows:

[0066] ,

[0067] In the formula S final For the final effective confidence level of the target, S det The target detection confidence score and S output by the YOLOv8 algorithm track λ represents the target trajectory tracking confidence score, and λ is the weighting coefficient. A target is considered a valid human target only if its final effective confidence score is greater than 0.5. The target trajectory tracking confidence score is calculated based on trajectory continuity and motion rationality. The weighting coefficient defaults to 0.7 and can be adjusted according to the scenario. Furthermore, non-human targets are further filtered out using geometric and semantic features such as target size, aspect ratio, and texture features, achieving multiple false alarm suppression.

[0068] Furthermore, in step S8, the formula for calculating the overlap between personnel and the hazardous area is as follows:

[0069] ,

[0070] In the formula, IoUpz represents the overlap between the personnel target and the restricted area of ​​the hoisting zone, Area(person∩zone) represents the area of ​​the overlapping region between the personnel target and the restricted area, Area(zone) represents the total area of ​​the restricted area of ​​the hoisting zone, and T represents the intrusion judgment threshold. The intrusion judgment threshold is set to 0.1 by default and can be adjusted according to the operation scenario. If the overlap is greater than the threshold, it is judged as a planar boundary violation.

[0071] Furthermore, in step S8, based on the three-dimensional spatial distance calculation results, the spatial distance between the personnel target and the suspended object / dangerous area is divided into a safe zone, a warning zone, and a braking zone. The safe zone is defined as a spatial distance greater than 2m between the personnel target and the suspended object / dangerous area; when the personnel target is within the safe zone, there is no risk, and no warning action is triggered. The warning zone is defined as a spatial distance greater than 1m and less than or equal to 2m between the personnel target and the suspended object / dangerous area; when the personnel target is within the warning zone, it is considered low-risk, triggering an audible and visual warning and a voice warning to remind personnel to evacuate, without triggering equipment shutdown. The braking zone is defined as a spatial distance less than or equal to 1m between the personnel target and the suspended object / dangerous area; when the personnel target is within the braking zone, it is considered high-risk, triggering an audible and visual warning and a voice warning, while simultaneously triggering an emergency stop and action prohibition on the crane.

[0072] Furthermore, in step S8, the intrusion judgment threshold is dynamically adjusted according to the crane's operating status, such as stationary, lifting, lowering, trolley travel, and trolley travel. When the crane is in the lifting operation state, the threshold is tightened to improve the protection level, and when the crane is in the stationary maintenance state, the threshold is relaxed to reduce interference with normal maintenance operations.

[0073] Furthermore, after step S9, step S10 is also included: archiving the video, screenshots, detection data, and handling records of the early warning event to form a handling file, storing the handling file locally while simultaneously uploading it to the cloud management platform, and generating a security report.

[0074] like Figure 2 As shown, on the other hand, this embodiment of the invention also provides a crane hoisting area personnel intrusion identification system. Based on the above-mentioned crane hoisting area personnel intrusion identification method, it includes a video image acquisition module 1, an AI image processing module 2, an edge computing module 3, an early warning linkage module 4, and a cloud management module 5. The AI ​​image processing module 2 and the early warning linkage module 4 are electrically connected to the edge computing module 3, and the video image acquisition module 1 and the cloud management module 5 are communicatively connected to the edge computing module 3.

[0075] Furthermore, the video image acquisition module 1 includes multiple binocular high-definition cameras for real-time video image acquisition, encoding, and transmission in the crane's lifting area. In this embodiment, three industrial-grade vibration-resistant binocular high-definition cameras are deployed at both ends of the crane's trolley frame, the end of the boom, and below the cab. The cameras are installed at a downward tilt of 30°–45° to ensure coverage of the entire hook travel range, high-risk areas under the load, the steel cable triangle area, and the crane's running track area, eliminating blind spots. Adjacent binocular high-definition cameras maintain a 10% overlap in their monitoring range to ensure comprehensive coverage. Depending on the ambient lighting conditions, LED supplementary lighting can also be installed to ensure image quality at night or in low-light scenarios. The cables for the binocular high-definition cameras are shielded and protected with flexible metal conduits to prevent electromagnetic interference and mechanical damage.

[0076] Furthermore, the edge computing module 3 adopts an industrial-grade wide-temperature and vibration-resistant edge computing industrial control computer, which is installed in the control cabinet of the crane operator's cab. The industrial control computer is equipped with an embedded industrial operating system and has built-in computing power unit, local storage unit, communication interface unit, and local storage unit to provide computing power support for the operation of the AI ​​image processing module 2. It is also responsible for local data processing, logic control, and issuing equipment linkage instructions. All core algorithms and security control logic are run locally at the edge. Network outages do not affect local security protection functions, ensuring high reliability and low latency of the system. Moreover, the warning data and video are stored in the local storage unit for no less than 30 days.

[0077] Furthermore, the AI ​​image processing module 2 is built into the edge computing module 3, including an image preprocessing unit, a hanging area intelligent division unit, a foreground target extraction unit, a YOLOv8 target detection unit, a binocular stereo vision 3D positioning unit, an intrusion risk judgment unit, and a multi-target tracking and false alarm removal unit, which are used to fully realize the whole process algorithm processing from image input to risk judgment, and output accurate detection results and risk instructions.

[0078] Furthermore, the early warning linkage module 4 includes an audible and visual alarm, a voice deactivation unit, and a crane PLC safety interlocking unit. After receiving the risk command issued by the edge computing module 3, the early warning linkage module 4 executes graded early warning and linkage control. The audible and visual alarm is installed at both ends of the crane trolley and outside the driver's cab. The audible and visual alarm and the voice deactivation unit are used to realize on-site warnings. The PLC safety interlocking unit is connected to the crane main control system through industrial protocol and hard-wiring dual redundancy to realize active safety control such as emergency stop and action prohibition, and build a closed-loop protection.

[0079] Furthermore, the cloud management module 5 is deployed on an enterprise private cloud or public cloud platform, and includes a cloud server, database, web management platform, and mobile application. It is used to realize functions such as multi-device cluster management, real-time monitoring and remote viewing, historical early warning and video tracing, automatic generation of safety reports, user permission management, remote model update and distribution, and big data analysis of hidden dangers.

[0080] Furthermore, the technical architecture of the crane hoisting area personnel intrusion identification system is divided into three layers: support layer, platform layer, and application layer. The support layer provides hardware and data acquisition support, including industrial-grade binocular high-definition cameras, video storage units, edge computing industrial control computers, audible and visual early warning devices, crane PLC control systems, cloud servers, and mobile terminal devices. It also covers the underlying operating system, drivers, and communication protocol stack, ensuring the stable operation of the system hardware and underlying software. The platform layer is the core middleware of the system, enabling data interoperability and supporting public service capabilities. It includes map services, data services, AI computing services, device access and integration bus, access control services, database services, message push services, and industrial protocol parsing services. The platform layer achieves data interconnection and provides standardized service interfaces for upper-layer applications, while also supporting integration with third-party platforms such as MES, ERP, and safety production management systems. The application layer is the system's business function implementation layer, used to complete safety monitoring and early warning functions. It includes real-time video monitoring, personnel intrusion and foreign object intrusion warning, skewed and oblique hanging violation warning, no safety helmet detection, hierarchical early warning and handling, equipment life cycle management, personnel access control, safety reports and data analysis, remote operation and maintenance management and other functions, covering the needs of all scenarios of on-site operation, safety management and remote control.

[0081] Furthermore, after the crane hoisting area personnel intrusion identification system is powered on, it completes the delineation and marking of the crane's static danger zones through the Web configuration interface, including fixed areas such as the steel rope triangle area, the area around the fast rope, and the track section, and generates a static no-go zone mask. At the same time, it configures the parameters of the following danger zones, including the warning zone and braking zone range centered on the hook / load, to adapt to the safety requirements of on-site hoisting operations.

[0082] Furthermore, after the industrial control computer is installed, the pre-trained YOLOv8 detection model is imported, and the model inference parameters are configured, including the detection confidence threshold and NMS threshold. At the same time, the algorithm parameters are configured, including the intrusion overlap threshold, the binocular ranging risk level threshold, the weighting coefficient, the background difference threshold, etc. The above parameters can be fine-tuned and optimized according to the actual working conditions on site.

[0083] Furthermore, after the early warning linkage module 4 is installed, the linkage rules for graded early warning are configured, including the duration of the audible and visual alarms, voice content, shutdown trigger conditions, and early warning information push targets corresponding to different risk levels. At the same time, the crane operating condition linkage rules are configured, the operating status data of the crane PLC is connected, and the thresholds are adaptively adjusted according to the different operating conditions of the crane.

[0084] Furthermore, after the cloud management module 5 is connected to the edge computing module 3, it configures local data storage rules, including the storage duration and storage path of warning videos and screenshots, and configures cloud communication parameters, including platform address, MQTT communication protocol parameters, and data upload rules, to realize real-time uploading of device status and warning data. It also needs to configure user permissions, dividing the operation permissions of different roles such as administrators, safety officers, and drivers to ensure the safety of personnel intrusion identification system in the crane lifting area.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for identifying unauthorized personnel entering a crane lifting area, characterized in that, Includes the following steps: S1. Use a binocular camera to capture left and right eye video images of the crane's lifting area; S2. Perform preprocessing operations on the acquired left and right eye video images, including grayscale transformation, Gaussian filtering for noise reduction, adaptive image enhancement, and distortion correction. S3. Hough transform is used to identify and calibrate the dangerous areas in the crane lifting area under static conditions, and the target detection algorithm is used to identify targets such as hooks and loads in the crane lifting area in real time, generating a following dangerous area that follows the target, thus completing the division of dangerous areas; S4. Using Gaussian mixture background modeling and background subtraction, foreground moving targets are extracted from video frames of left and right eye video images, and suspected abnormal targets are screened. S5. Using the YOLOv8 deep learning algorithm, target detection is performed on the video frames of the left and right eye video images to identify suspected abnormal targets such as personnel targets, hook targets, and suspended objects, and the target category, bounding box and detection confidence are output. S6. Based on binocular stereo vision technology, a disparity map is generated through stereo matching, the three-dimensional spatial coordinates of the identified personnel target are calculated, and the real-time relative distance between the personnel target and the hook target, the suspended object target, and the dangerous area is measured. S7. The DeepSORT multi-target tracking algorithm is used to continuously track personnel targets, obtain the movement trajectory of personnel targets, and combine the detection confidence and trajectory confidence to complete the false alarm removal and confirm the valid personnel targets. S8. Combine the overlap between personnel targets and hazardous areas, three-dimensional spatial distance, and crane operating conditions to determine the risk level and confirm the risk level. S9. Execute corresponding graded early warning and linkage control according to the risk level, triggering audible and visual warnings, voice warnings, or emergency shutdown and action prohibition of the crane.

2. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S2, the RGB color image is converted to a grayscale image. The weighted grayscale transformation formula is as follows: ， In the formula I R I G I B For the pixel values ​​corresponding to the RGB three channels, I gray This is the grayscale image output after preprocessing.

3. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S2, the image is smoothed using a Gaussian convolution kernel. The Gaussian convolution kernel function is: ， In the formula, G(x,y) is the two-dimensional Gaussian convolution kernel function, σ is the standard deviation of the filter, and (x,y) are the pixel coordinates of the image.

4. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S3, the Hough transform line detection formula is as follows: ， In the formula, ρ is the vertical distance from the origin of the image coordinate system to the straight line of the hanging area boundary, θ is the angle between the perpendicular line and the x-axis of the image, and x and y are the pixel coordinates of the image, respectively.

5. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S4, the formula for the background subtraction method is: In the formula, F(x,y) is the foreground target mask, and I is the target mask. t B is the video frame image at the current moment, B is the adaptively updated background modeling image, and T is... bg This is the background difference threshold.

6. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S5, the YOLOv8 detection confidence score is calculated using the following formula: ， In the formula S conf The final detection confidence level and S for the target cls Predict the probability of the target category, S obj The target has a confidence level.

7. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S6, the formula for calculating the three-dimensional spatial distance is: ， In the formula, Z is the depth distance between the target and the stereo camera, f is the equivalent focal length of the stereo camera, B is the baseline distance of the stereo camera, and d is the disparity value of the target in the left and right eye images.

8. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: In step S7, the confidence-weighted fusion formula for false alarm removal is: ， In the formula S final For the final effective confidence level of the target, S det The target detection confidence score and S output by the YOLOv8 algorithm track λ represents the confidence level for target trajectory tracking, and λ is the weighting coefficient.

9. The method for identifying unauthorized personnel entering a crane lifting area as described in claim 1, characterized in that: Step S9 is followed by step S10, which involves archiving the videos, screenshots, detection data, and handling records of the early warning event to form a handling file. The handling file is stored locally and simultaneously uploaded to the cloud management platform to generate a security report.

10. A crane hoisting area personnel intrusion identification system, characterized in that: The crane hoisting area personnel intrusion identification method based on any one of claims 1-9 includes a video image acquisition module, an AI image processing module, an edge computing module, an early warning linkage module, and a cloud management module. The AI ​​image processing module and the early warning linkage module are electrically connected to the edge computing module, and the video image acquisition module and the cloud management module are communicatively connected to the edge computing module.

Images