Detection and alarm method for hand-held hoisting object in hoisting operation

Abstract

The invention discloses a detection and alarm method for a hand-held hoisting object in hoisting operation. The detection and alarm method comprises the following steps: S100, detecting and tracking the hoisting object; s200, detecting key points of a target human body; s300, performing safety assessment and alarm based on the spatial position relationship between the hanging object and the key points of the human body; the method has the advantages of 1) accuracy: ensuring that a target hanging object can be stably tracked under a complex background and a dynamic scene, and avoiding losing or misrecognition; 2) real-time performance: a video stream can be processed at an enough frame rate, the position and the state of a hanging object can be updated in real time, and the method adapts to rapidly changing environment and target motion; 3) robustness: the method has certain anti-interference capability on interference factors such as illumination variation, shielding and target deformation, and can still reliably track the hanging object under various complex conditions; and 4) long-time continuity: continuous tracking of the target can be kept in a long-time video sequence without being influenced by accumulative errors, and the stability and accuracy of tracking are kept.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a detection and alarm method for hand-held hoisted objects in hoisting operations. Background Art

[0002] In hoisting operations, it is necessary to ensure a safe distance between the operator and the hoisted object to ensure the safety of the operator. Therefore, the images of the work site must be processed effectively, quickly and accurately to achieve this. The existing single algorithm cannot complete the overall task. It is necessary to effectively identify the spatial position relationship between the hoisted object and the worker. Accurate identification of the position and boundary of the hoisted object requires the use of target detection algorithms, tracking algorithms and segmentation algorithms. The identification of the position of the hand of the person requires the use of key point recognition algorithms. The accurate judgment of the relative position of the two requires the introduction of depth estimation algorithms. The above five algorithms can be used to determine whether the position and work of the worker meet safety standards.

[0003] Existing target detection algorithms include EfficientDet, which achieves a new balance between accuracy and speed through multi-scale feature fusion and efficient network design. It is particularly effective in high-resolution images and small target detection. However, the high complexity of the model leads to relatively long training and inference time.

[0004] Existing segmentation methods include the SAM algorithm, which can provide high-precision results in image segmentation tasks, especially in complex scenes and edge detail processing. However, the complexity of the SAM model leads to a large consumption of computing resources, especially when processing high-resolution images, it will take up a lot of memory and computing time.

[0005] Among the tracking algorithms, there is a KCF tracker, which can utilize overlapping positive samples in the MIL tracker and has better speed and accuracy than the MIL and BOOSTING trackers, but it cannot recover from complete occlusion.

[0006] Common key point detection algorithms include SimpleBaseline, which has obvious advantages in computational efficiency and model size, but its relatively simple network structure may result in key point detection accuracy in complex scenes that is not as good as some more complex models, and SimpleBaseline has limited ability to handle occlusion and complex backgrounds.

[0007] A common depth estimation algorithm is DFW, which is an algorithm for estimating depth from video. It uses the timing information and motion clues in the video sequence to improve the accuracy of depth estimation. The advantage is that it can use the timing information in the video to improve the stability and accuracy of depth estimation. The disadvantage is that it relies on the stability and motion consistency of the video sequence, and the computational complexity is relatively high.

[0008] Therefore, it is necessary to use appropriate algorithms in each step to design the detection and alarm methods of hand-held hoisted objects in hoisting operations. Summary of the invention

[0009] The purpose of the present invention is to provide a detection and alarm method for hand-held hanging objects in lifting operations to solve the problems raised in the above-mentioned background technology.

[0010] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0011] The detection and alarm method of hand-held hanging objects in hoisting operations includes the following steps:

[0012] S100, detecting and tracking the hanging objects;

[0013] S200, detecting key points of the target human body;

[0014] S300: Perform safety assessment and alarm based on the spatial position relationship between the hanging object and key points of the human body.

[0015] According to the above technical solution, the method for detecting the hanging object in S100 is as follows:

[0016] S111, construct the suspended object data set and enrich the data set during the operation;

[0017] S112, the resolution of the data collection image is 1920×1080, and the image size is 500kb±10kb;

[0018] S113. Through the yolov8 training model, the accuracy, recall rate and average precision of the seat belt recognition model performance on the test set in the hanging object data set are tested, so that the average precision, average accuracy and average recall rate reach the indicator requirement of 80%.

[0019] According to the above technical solution, the method for tracking the hanging object in S100 is as follows:

[0020] The CSRT single target tracking algorithm is used to track the hanging object and output an accurate guide frame. The FastSAM image segmentation algorithm is used to segment the hanging object tracking image and output a covered mask to accurately determine the real-time position of the hanging object.

[0021] The FastSAM image segmentation algorithm consists of two stages:

[0022] S121. Use YOLOv8-seg to segment all objects or regions in the image, and use various prompts to identify specific objects of interest, which mainly involve point prompts, box prompts, and text prompts.

[0023] S122. The tracking frame obtained by CSRT single target tracking is used as the guide frame of the second stage of FastSAM, so as to accurately detect the edge information of the hanging object with the idea of ​​segmentation, and draw a mask to stably realize the tracking and detection of the hanging object.

[0024] According to the above technical solution, the specific method of detecting the key points of the target human body in S200 is:

[0025] The YOLOv8_pose network structure is used to detect the key points of the human body, including but not limited to shoulders, elbows and knees. In the process of detecting the key points of the human body, the hand key points to be used are extracted for subsequent logical judgment.

[0026] According to the above technical solution, the specific method of performing safety assessment and alarm based on spatial position relationship in S300 is as follows:

[0027] The captured image content is used to determine the position through the output of the hanging mask of FastSAM and the key points of the hands of the on-site staff detected by yolov8_pose. At the same time, in order to determine the depth information between images, Monodepth2 is introduced to analyze the two consecutive frames of the video taken by the monocular camera to obtain the changes in the angle and position of the monocular camera relative to the scene, and then obtain the overall position depth relationship;

[0028] The maximum and minimum values ​​of the depth of the lifting range and the maximum and minimum values ​​of the depth of the entire image are obtained through the overall position relationship, and an estimated threshold is generated. If the hand position depth is within this threshold, an alarm is issued, otherwise no alarm is issued.

[0029] Compared with the prior art, the beneficial effects achieved by the present invention are:

[0030] 1) Accuracy: Ensure that the target object can be stably tracked in complex backgrounds and dynamic scenes to avoid loss or misidentification;

[0031] 2) Real-time: Ability to process video streams at a sufficient frame rate, update the position and status of the suspended object in real time, and adapt to rapidly changing environments and target movements;

[0032] 3) Robustness: It has a certain ability to resist interference factors such as lighting changes, occlusion, target deformation, etc., ensuring that the suspended objects can be reliably tracked under various complex conditions;

[0033] 4) Long-term persistence: It can keep tracking the target continuously in a long video sequence without being affected by cumulative errors, and maintain tracking stability and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a flowchart of the present invention;

[0035] Figure 2 It is a structural flowchart of the tracking and detection algorithm of the hanging object of the present invention;

[0036] Figure 3 It is a structural flowchart of the FastSAM algorithm process of the present invention;

[0037] Figure 4 This is an example diagram of key point detection of a human body according to the present invention;

[0038] Figure 5 It is the alarm logic diagram of the present invention;

[0039] Figure 6 Schematic diagram of Monodepth2 feature extraction and encoding and decoding of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0041] like Figure 1-6 As shown, the present invention provides a technical solution: a method for detecting and alarming hand-held hanging objects in a hoisting operation, comprising the following steps:

[0042] S100, detecting and tracking the hanging objects;

[0043] S200, detecting key points of the target human body;

[0044] S300: Perform safety assessment and alarm based on the spatial position relationship between the hanging object and key points of the human body.

[0045] The method for detecting the hanging object in S100 is as follows:

[0046] S111, construct the suspended object data set and enrich the data set during the operation;

[0047] S112, the resolution of the data collection image is 1920×1080, and the image size is 500kb±10kb;

[0048] S113. Through the yolov8 training model, the accuracy, recall rate and average precision of the seat belt recognition model performance on the test set in the hanging object data set are tested, so that the average precision, average accuracy and average recall rate reach the indicator requirement of 80%.

[0049] The method for tracking the hanging object in S100 is as follows:

[0050] The CSRT single target tracking algorithm is used to track the hanging object and output an accurate guide frame. The FastSAM image segmentation algorithm is used to segment the hanging object tracking image and output a covered mask to accurately determine the real-time position of the hanging object.

[0051] The FastSAM image segmentation algorithm consists of two stages:

[0052] S121. Use YOLOv8-seg to segment all objects or regions in the image, and use various prompts to identify specific objects of interest, which mainly involve point prompts, box prompts, and text prompts.

[0053] S122. The tracking frame obtained by CSRT single target tracking is used as the guide frame of the second stage of FastSAM, so as to accurately detect the edge information of the hanging object with the idea of ​​segmentation, and draw a mask to stably realize the tracking and detection of the hanging object.

[0054] The specific method of detecting the key points of the target human body in S200 is:

[0055] The YOLOv8_pose network structure is used to detect the key points of the human body, including but not limited to shoulders, elbows and knees. In the process of detecting the key points of the human body, the hand key points to be used are extracted for subsequent logical judgment.

[0056] The specific method of performing safety assessment and alarm based on spatial position relationship in S300 is as follows:

[0057] The captured image content is used to determine the position through the output of the hanging mask of FastSAM and the key points of the hands of the on-site staff detected by yolov8_pose. At the same time, in order to determine the depth information between images, Monodepth2 is introduced to analyze the two consecutive frames of the video taken by the monocular camera to obtain the changes in the angle and position of the monocular camera relative to the scene, and then obtain the overall position depth relationship;

[0058] The maximum and minimum values ​​of the depth of the lifting range and the maximum and minimum values ​​of the depth of the entire image are obtained through the overall position relationship, and an estimated threshold is generated. If the hand position depth is within this threshold, an alarm is issued, otherwise no alarm is issued.

[0059] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0060] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting and alarming hand-held objects during lifting operations, characterized in that: The steps include: S100, detecting and tracking the hanging objects; S200, detecting key points of the target human body; S300: Perform safety assessment and alarm based on the spatial position relationship between the hanging object and key points of the human body.

2. The method for detecting and alarming hand-held objects in lifting operations according to claim 1 is characterized in that: The method for detecting the hanging object in S100 is as follows: S111, construct the suspended object data set and enrich the data set during the operation; S112, the resolution of the data collection image is 1920×1080, and the image size is 500kb±10kb; S113. Through the yolov8 training model, the accuracy, recall rate and average precision of the seat belt recognition model performance on the test set in the hanging object data set are tested, so that the average precision, average accuracy and average recall rate reach the indicator requirement of 80%.

3. The method for detecting and alarming hand-held objects in lifting operations according to claim 1 is characterized in that: The method for tracking the hanging object in S100 is as follows: The CSRT single target tracking algorithm is used to track the hanging object and output an accurate guide frame. The FastSAM image segmentation algorithm is used to segment the hanging object tracking image and output a covered mask to accurately determine the real-time position of the hanging object. The FastSAM image segmentation algorithm consists of two stages: S121. Use YOLOv8-seg to segment all objects or regions in the image, and use various prompts to identify specific objects of interest, which mainly involve point prompts, box prompts, and text prompts. S122. The tracking frame obtained by CSRT single target tracking is used as the guide frame of the second stage of FastSAM, so as to accurately detect the edge information of the hanging object with the idea of ​​segmentation, and draw a mask to stably realize the tracking and detection of the hanging object.

4. The method for detecting and alarming hand-held objects in lifting operations according to claim 1 is characterized in that: The specific method of detecting the key points of the target human body in S200 is: The YOLOv8_pose network structure is used to detect the key points of the human body, including but not limited to shoulders, elbows and knees. In the process of detecting the key points of the human body, the hand key points to be used are extracted for subsequent logical judgment.

5. The method for detecting and alarming hand-held objects in lifting operations according to claim 1 is characterized in that: The specific method of performing safety assessment and alarm based on spatial position relationship in S300 is as follows: The captured image content is used to determine the position through the output of the hanging mask of FastSAM and the key points of the hands of the on-site staff detected by yolov8_pose. At the same time, in order to determine the depth information between images, Monodepth2 is introduced to analyze the two consecutive frames of the video taken by the monocular camera to obtain the changes in the angle and position of the monocular camera relative to the scene, and then obtain the overall position depth relationship; The maximum and minimum values ​​of the depth of the lifting range and the maximum and minimum values ​​of the depth of the entire image are obtained through the overall position relationship, and an estimated threshold is generated. If the hand position depth is within this threshold, an alarm is issued, otherwise no alarm is issued.

Images