A method for detecting dangerous behaviors in lifting operation scenarios

Abstract

The present invention relates to a method for detecting dangerous behaviors in a hoisting operation scenario, and belongs to the field of target detection. The dangerous area detection module of the present invention mainly determines whether the operator enters the designated dangerous area in the hoisting scenario, such as crossing over the hoisted objects or drilling under the hoisted objects. An alarm will be issued for the operator who violates the rules. The number detection module mainly counts the number of people entering the work area in the hoisting scenario, and issues an alarm for dangerous work scenarios where the number of safety workers exceeds the limit. The present invention only requires a monocular camera to complete the identification and alarm of dangerous behaviors and the counting and alarm of the number of operators in the hoisting operation scenario. It requires low hardware and computing resources and is easy to deploy. It can detect dangerous behaviors in real time and make real-time alarm responses, and store key frames containing dangerous behaviors.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and in particular relates to a method for detecting dangerous behaviors in a lifting operation scenario. Background Art

[0002] Major accidents in lifting operations are often caused by workers failing to follow operating procedures and regulations. Therefore, monitoring and alerting for dangerous behaviors during lifting operations is crucial. Currently, the primary approach to addressing these issues is manual real-time review or playback of surveillance video, combined with verbal or written guidance and education. However, this approach is inefficient, labor-intensive, and prone to lags, and can also miss difficult-to-identify dangerous behaviors.

[0003] In recent years, intelligent monitoring systems based on computer vision for identifying dangerous behaviors have gradually gained widespread application. This method offers the advantages of simplicity and high real-time performance, enabling timely alerts for dangerous behaviors. A straightforward approach for identifying personnel crossing over objects or entering hazardous areas during lifting operations involves using computer vision to separately identify the objects, personnel, and hazardous areas, then calculating the spatial relationship between the three to determine if there is any overlap. However, this method is complex and requires a high level of computational effort.

[0004] For this specific scenario, this solution proposes a method that relies solely on monocular vision recognition to complete the identification and reminder of two dangerous behaviors: crossing or drilling through suspended objects and people in dangerous areas, as well as the identification and reminder of the number of operators not meeting the requirements. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is how to provide a method for detecting dangerous behaviors in a lifting operation scenario, so as to solve the problem of efficient and effective detection and alarm of two dangerous behaviors in a lifting production scenario.

[0007] (2) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes a method for detecting dangerous behaviors in a lifting operation scenario, which includes the following steps:

[0009] S11. Collect a video of personnel behavior in a lifting operation scene and segment the video into a series of images;

[0010] S12. Use the labelImg labeling software to label the image dataset in step S11, and select YOLOv5 as the labeling mode; wherein, labelImg software is used to label “people” and “human heads” in the dataset;

[0011] S13. Use the YOLOv5 model to train the datasets of the dangerous crossing detection module and the people counting module respectively. Both modules recognize people and share the same weight.

[0012] S14: Package the model weights and their corresponding inference modules trained in step S13 into the front-end framework, naming them "dangerous area detection" and "people detection" respectively;

[0013] S15, build an area overlap recognition algorithm module, input the dangerous area and the person recognition circumscribed rectangle obtained by S14 model inference into this module, output an alarm if they overlap, and insert this part into the front-end framework of S14;

[0014] S16. Place a camera with a bird's-eye view on the lifting rail, vertically above the hook. Click "Danger Area Detection" or "People Detection" in the front-end framework described in step S14, and select the corresponding video folder to demonstrate the inference effect online.

[0015] Furthermore, the dangerous area detection module mainly determines whether the operator enters the designated dangerous area in the lifting scenario, and issues an alarm to the operator who violates the rules. At the same time, the problem frames with identified safety hazards are stored locally.

[0016] Furthermore, entering the designated danger zone includes: crossing over suspended objects and crawling under suspended objects.

[0017] Furthermore, the dangerous area detection module uses the YOLOv5 target detection model to perform labeling training on workers and heads in lifting scenarios, with a training set to validation set ratio of 4:1. A non-maximum suppression module is constructed, which filters the multiple target box candidate sets generated by YOLOv5 to obtain a single target detection box, ensuring that a single target has only one detection box.

[0018] Furthermore, in the overhead camera image, a certain part of the central area is set as a dangerous area. When a person or head recognized by YOLOv5 enters the dangerous area, it is determined to be out of bounds.

[0019] Furthermore, the danger zone and the target person are abstracted into two rectangular boxes, and then it is determined whether the two rectangular boxes have overlapping parts. The determination of area overlap includes:

[0020] Assume that (x1, y1) (x2, y2) are the two points at the lower left and upper right corners of the first rectangle; Assume that (x3, y3) (x4, y4) are the two points at the lower left and upper right corners of the second rectangle;

[0021] If two rectangles intersect, then

[0022] (a) On the X-axis, max(x1,x3) <= min(x2,x4) must be satisfied.

[0023] (b) On the Y-axis, max(y1,y3)<=min(y2,y4) must be satisfied.

[0024] Furthermore, the number detection module counts the number of people entering the work area in the lifting scenario, and issues an alarm for dangerous work scenarios where the number of safe workers exceeds the limit. At the same time, it stores the problem frames with identified safety hazards locally to facilitate subsequent review of safety hazards.

[0025] Furthermore, the people detection module uses the YOLOv5 target detection model to perform labeling training on workers and heads in the lifting scene, with a training set and validation set ratio of 4:1; a non-maximum suppression module is constructed, which filters the multiple target box candidate sets generated by YOLOv5 to obtain a single target detection box, ensuring that a single target has only one rectangular box.

[0026] Furthermore, the number of people detection module simultaneously counts the number of people with the classification label "heads" and the number of "people". If the two are equal, the module outputs either number as the number of people. If the two are not equal, the module outputs the larger number. If the number of "heads" and "people" is equal, the module proves that the output result has a high confidence level.

[0027] A method for detecting dangerous behaviors in a lifting operation scenario, the method comprising the following steps:

[0028] S21. Collect a video of personnel behavior in a lifting operation scene and segment the video into a series of images;

[0029] S22. Use labelImg to label the image in step S21. Use YOLOv5 as the labeling model. The two functional modules in this scenario are the dangerous crossing detection module and the people counting module. The subjects to be identified are all "people", so only "people" and "heads" in the dataset need to be labeled using labelImg.

[0030] S23. Use the YOLOv5 model to train the training data sets of the above two modules respectively. Both modules recognize people and share the same weight.

[0031] S24, building a model inference module, which receives the weights trained in S23 and obtains the recognition rectangles of "people" and "human heads" in the image through inference code;

[0032] S25. Use multithreading to build a voice alarm module. This module is a sub-thread that is always open and continuously monitors a message queue to determine whether the parameters passed in by the message queue are consistent with the default parameters of this module. If they are inconsistent, a voice alarm is played, and if they are consistent, the module continues to monitor.

[0033] S26. Construct a danger zone detection module. The danger zone detection module needs to identify when someone enters a danger zone, which is a designated danger zone centered on the hanging object. If the above dangerous behavior is identified, a voice alarm is triggered.

[0034] The specific steps for building the S26 module are:

[0035] S261, constructing a region overlap identification algorithm module, the module input is the parameters of two rectangular boxes, wherein the parameters of each rectangular box input are the coordinates of the lower left corner and the upper right corner; the output is to determine whether the two rectangular boxes have an overlap intersection, if so, output True, otherwise output False;

[0036] S262: Receive the "person" or "head" detection rectangle output by the inference module in S24, where the danger zone rectangle is a default rectangle b centered on the hanging object and remains unchanged; call the region overlap recognition algorithm module in S261 to determine whether the detection rectangle overlaps with the default rectangle b, and output a list containing the results, recording all the judgment results;

[0037] S263, calling the voice alarm module of S25, setting the default parameter to "no overlap", the message queue in S25 continues to receive the result list obtained in S262, following the logic of the S25 module, and alarming if anyone enters the danger zone;

[0038] S27, constructing a number detection module; wherein the number detection module needs to calculate the number of people within the target range, and if it is detected that the number of people exceeds the limit, a voice alarm is triggered;

[0039] The specific steps for building the S27 module are:

[0040] S271, receiving the "person" or "head" detection result output by the inference module in S24 and counting them respectively, and outputting the larger value as the number of people;

[0041] S272. Call the voice alarm module in S25 and set the default parameters to the upper limit M and lower limit N of the maximum number of operators in this place. The message queue in S25 continuously receives the number K of operators in each frame of the real-time video obtained in S271. If M>K>N, continue to monitor. Otherwise, if it is detected that the number of operators exceeds the limit, a voice alarm is issued;

[0042] S28. The model weights trained in step S23, the voice alarm module in S25, the dangerous area detection module in S26, and the number of people detection module in S27 are packaged together into the front-end framework. The two interactive modules on the front end are the above two inference modules, named "dangerous area detection" and "number of people detection" respectively. After interaction, the inference results are output;

[0043] S29. Place a camera with a bird's-eye view on the hoisting rail, vertically above the hook. At this time, the hoisted object is in the center of the video, so set the danger zone to a rectangular box of a certain size in the center of the screen. Click "Danger Zone Detection" or "People Detection" in the front-end framework described in step S28, select the corresponding video folder, and the inference effect can be demonstrated online.

[0044] (3) Beneficial effects

[0045] The present invention proposes a method for detecting dangerous behaviors in a lifting operation scenario. Compared with the prior art, the present invention has the following advantages:

[0046] The present invention only requires a monocular camera to complete the identification and alarm of dangerous behaviors and the counting and alarm of the number of operators in the lifting operation scenario. It requires low hardware and computing resources and is easy to deploy. It can detect dangerous behaviors in real time and make real-time alarm responses, and store key frames containing dangerous behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the overall process framework diagram of the present invention;

[0048] Figure 2 Schematic diagram of the geometric relationship algorithm of the present invention;

[0049] Figure 3 This is a schematic diagram of the detection of the dangerous crossing detection module in the lifting operation scenario of the present invention;

[0050] Figure 4 This is an example of the detection module for detecting the number of operators in a lifting operation scenario of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0052] The technical problem to be solved by the present invention is how to provide a method for detecting dangerous behaviors in hoisting production operation scenarios, so as to solve the problem of efficient and effective detection and alarm of two dangerous behaviors in hoisting production scenarios.

[0053] In order to solve the above technical problems, the present invention proposes a method for detecting dangerous behaviors in a lifting operation scenario, which includes the following steps:

[0054] S11. Collect a video of personnel behavior in a lifting operation scene and segment the video into a series of images;

[0055] S12. Use the labeling software labelImg to label the image dataset in step S11, and select YOLOv5 as the labeling mode. The two functional modules in this scenario are the dangerous crossing detection module and the people counting module. The subjects to be identified are all "people", so only "people" and "human heads" in the dataset need to be labeled using the labelImg software.

[0056] S13. Use the YOLOv5 model to train the datasets of the dangerous crossing detection module and the people counting module respectively. Both modules recognize people and share the same weight.

[0057] S14: Package the model weights and their corresponding inference modules trained in step S13 into the front-end framework, naming them "dangerous area detection" and "people detection" respectively;

[0058] S15: Build an area overlap recognition algorithm module. Input the dangerous area and the person recognition bounding rectangle derived from the S14 model into this module. If there is overlap, an alarm will be output. This part will be inserted into the front-end framework of S14.

[0059] S16. Place a camera with a bird's-eye view on the lifting rail, vertically above the hook. Click "Danger Area Detection" or "People Detection" in the front-end framework described in step S14, and select the corresponding video folder to demonstrate the inference effect online.

[0060] Example 1:

[0061] The purpose of this invention is to provide a method for detecting dangerous behaviors in lifting operation scenarios. It focuses on new safety monitoring needs, closely follows the characteristics of cutting-edge artificial intelligence technology, focuses on the real-time detection and application of unsafe behaviors in the production process, and improves the level of intelligent production safety supervision.

[0062] The invention is divided into two submodules, namely the dangerous area detection module and the number of people detection module. Figure 1 shown.

[0063] In lifting scenarios, the Danger Zone Detection module determines whether operators enter designated danger zones, such as crossing over or under a hoisted object. Violators are alerted and identified frames with potential safety hazards are stored locally for subsequent safety hazard review.

[0064] The dangerous area detection module uses the YOLOv5 object detection model to annotate workers and heads in lifting scenarios, with a training set to validation set ratio of 4:1. A non-maximum suppression module is constructed. Because YOLOv5 generates multiple object detection boxes for the same target, this module filters the multiple object box candidates generated in the first step of YOLOv5 to obtain a single object detection box, ensuring that a single target has only one detection box.

[0065] The difficulty of the dangerous area detection module is that the overhead camera cannot accurately identify the person directly below. Therefore, the dangerous area detection module recognizes both the head and the person. When the person is directly below, although the person cannot be accurately identified, the head can be identified. In the overhead camera image, a part of the central area is set as a dangerous area. When the person or head identified by YOLOv5 enters the dangerous area, it is determined to be out of bounds. When the specific algorithm is implemented, the dangerous area and the target person can be abstracted into two rectangular boxes, and then it is determined whether the two rectangular boxes have overlapping parts. The algorithm for determining area overlap is as follows:

[0066] like Figure 2 As shown, assume that (x1, y1) (x2, y2) are the two points at the lower left corner and upper right corner of the first rectangle; assume that (x3, y3) (x4, y4) are the two points at the lower left corner and upper right corner of the second rectangle.

[0067] If two rectangles intersect, then

[0068] (a) On the X-axis, max(x1,x3) <= min(x2,x4) must be satisfied.

[0069] (b) On the Y axis, max(y1,y3)<=min(y2,y4) must be satisfied.

[0070] In the lifting scenario, the number of people detection module mainly counts the number of people entering the work area, and alarms for dangerous work scenarios where the number of safe personnel exceeds that of the safe personnel. At the same time, it stores the problem frames with identified safety hazards locally to facilitate the subsequent review of safety hazards.

[0071] The people detection module uses the YOLOv5 object detection model to annotate workers and heads in hoisting scenes, with a training set to validation set ratio of 4:1. A non-maximum suppression module is constructed. Because YOLOv5 generates multiple object detection boxes for the same object, this module filters the multiple object box candidates generated in the first step of YOLOv5 to obtain a single object detection box, ensuring that each object has only one rectangular box.

[0072] The headcount detection module simultaneously counts the number of "heads" and "people" classified as equal. If the two are equal, it outputs the larger number as the headcount. This approach effectively reduces missed detections, and if the number of "heads" and "people" is equal, it also proves that the output result has a high degree of confidence.

[0073] A method for detecting dangerous behaviors in a lifting operation scenario includes the following steps:

[0074] S21. Collect a video of personnel behavior in a lifting operation scene and segment the video into a series of images;

[0075] S22. Use the labeling software labelImg to label the image in step S21, and select YOLOv5 as the labeling model. The two functional modules in this scenario are the dangerous crossing detection module and the people counting module. The subjects to be identified are all "people", so only "people" and "heads" in the dataset need to be labeled using the labelImg software.

[0076] S23. Use the YOLOv5 model to train the training data sets of the above two modules respectively. Both modules recognize people and can share the same weight.

[0077] S24. Construct a model inference module. This module receives the weights trained in S23 and obtains the recognition rectangles of "people" and "human heads" in the image through the inference code.

[0078] S25. Use multithreading to build a voice alarm module. This module is a sub-thread that is always open and continuously monitors a message queue to determine whether the parameters passed in by the message queue are consistent with the default parameters of this module. If they are inconsistent, a voice alarm is played, and if they are consistent, the module continues to monitor.

[0079] S26. Build a danger zone detection module. This module identifies when someone enters a danger zone, which is a defined area centered around the suspended object. This includes situations such as a person stepping over the suspended object, a person approaching the suspended object, or a person passing under the suspended object. If any of these dangerous behaviors are detected, a voice alarm is triggered.

[0080] The specific steps for building the S26 module are:

[0081] S261. Construct an overlap region identification algorithm module. This module takes as input the parameters of two rectangular boxes, where the parameters for each rectangular box are the coordinates of the lower left corner and the upper right corner. The module outputs whether the two rectangular boxes overlap. If so, the output is True; otherwise, the output is False.

[0082] S262: Receive the "person" or "head" detection rectangles a1, a2, a3, etc. output by the inference module in S24. The danger zone rectangle remains unchanged, being the default rectangle b centered on the suspended object. Call the region overlap identification algorithm module in S261 to determine whether the detection rectangles a1, a2, a3, etc. overlap with the default rectangle b. Output a list containing the results, recording all the judgment results.

[0083] S263, call the voice alarm module of S25, set the default parameter to "no overlap", and the message queue in S25 continues to receive the result list obtained in S262, following the logic of the S25 module, and alarms if anyone enters the dangerous area.

[0084] S27, constructing a number detection module. The number detection module needs to calculate the number of people within the target range, and if it is detected that the number of people exceeds the limit, a voice alarm will be issued.

[0085] The specific steps for building the S27 module are:

[0086] S271. Accept the "person" or "head" detection result output by the inference module in S24 and count them respectively, and output the larger value of the two as the number of people.

[0087] S272. Call the voice alarm module in S25 and set the default parameters to the upper limit M and lower limit N of the maximum number of operators in this place. The message queue in S25 continuously receives the number K of operators in each frame of the real-time video obtained in S271. If M>K>N, continue to monitor. Otherwise, it is detected that the number of operators exceeds the limit and a voice alarm is issued.

[0088] S28. The model weights trained in step S23, the voice alarm module in S25, the dangerous area detection module in S26, and the number of people detection module in S27 are packaged together into the front-end framework. The two interactive modules on the front end are the above two inference modules, named "dangerous area detection" and "number of people detection" respectively. After interaction, the inference results can be output;

[0089] S29. Place a camera with a bird's-eye view on the lifting rail, vertically above the hook. At this time, the hoisted object is in the center of the video, so set the danger zone to a rectangular box of a certain size in the center of the picture. Click "Danger Zone Detection" or "People Detection" in the front-end framework described in step S28, select the corresponding video folder, and the inference effect can be demonstrated online.

[0090] The advantages of the present invention compared with the prior art are:

[0091] The present invention proposes a method for detecting dangerous behaviors in a lifting operation scenario. Compared with the prior art, the present invention has the following advantages:

[0092] The present invention only requires a monocular camera to complete the identification and alarm of dangerous behaviors and the counting and alarm of the number of operators in the lifting operation scenario. It requires low hardware and computing resources and is easy to deploy. It can detect dangerous behaviors in real time and make real-time alarm responses, and store key frames containing dangerous behaviors.

[0093] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting dangerous behaviors in a lifting operation scenario, characterized in that: The method comprises the following steps: S11. Collect a video of personnel behavior in a lifting operation scene and segment the video into a series of images; S12. Use labelImg to label the image dataset in step S11, and select YOLOv5 as the labeling mode; labelImg is used to label “person” and “head” in the dataset. S13. Use the YOLOv5 model to train the datasets of the dangerous crossing detection module and the people counting module respectively. Both modules recognize people and share the same weight. S14. Package the model weights and their corresponding inference modules trained in step S13 into the front-end framework, naming them "dangerous area detection" and "people detection" respectively. S15. Use multithreading to build a voice alarm module. This module is a sub-thread that is always open and continuously monitors a message queue to determine whether the parameters passed in by the message queue are consistent with the default parameters of this module. If they are inconsistent, a voice alarm is played; otherwise, the module continues to monitor. S16. Construct a danger zone detection module. The danger zone detection module needs to identify people entering the danger zone, which is a well-defined danger zone centered on the hanging object. If the above dangerous behavior is identified, a voice alarm is triggered. The specific steps for building the S16 module are: S161. Construct an area overlap recognition algorithm module. The module inputs the parameters of two rectangular boxes, where the parameters of each rectangular box are the coordinates of the lower left corner and the upper right corner. The module outputs whether the two rectangular boxes have an overlap or intersection. If so, it outputs True; otherwise, it outputs False. S162: Receive the "person" or "head" detection rectangle output by the inference module in S14. The danger zone rectangle is a default rectangle b centered on the hanging object and remains unchanged. Call the region overlap recognition algorithm module in S161 to determine whether the detection rectangle overlaps with the default rectangle b. Output a list containing the results, recording all the judgment results. S163: Call the voice alarm module of S15 and set the default parameter to "no overlap". The message queue in S15 continues to receive the result list obtained in S162 and follows the logic of the S15 module. If anyone enters the danger zone, an alarm is triggered. S17, constructing a number detection module; wherein the number detection module needs to calculate the number of people within the target range, and if it is detected that the number of people exceeds the limit, a voice alarm is issued; The specific steps for building the S17 module are: S171, receiving the "person" or "head" detection results output by the inference module in S14 and counting them respectively, and outputting the larger value of the two as the number of people; S172. Call the voice alarm module in S15 and set the default parameters to the upper limit M and lower limit N of the maximum number of operators in this place. The message queue in S15 continuously receives the number K of operators in each frame of the real-time video obtained in S171. If M>K>N, continue to monitor. Otherwise, if it is detected that the number of operators exceeds the limit, a voice alarm is issued; S18. Place a camera with a bird's-eye view on the lifting rail, vertically above the hook. Click "Danger Area Detection" or "People Detection" in the front-end framework described in step S14, and select the corresponding video folder to demonstrate the inference effect online.

2. The method for detecting dangerous behaviors in a lifting operation scenario according to claim 1, characterized in that: In the lifting scenario, the dangerous area detection module mainly determines whether the operator has entered the designated dangerous area, issues an alarm for the operator who violates the rules, and stores the problem frames with identified safety hazards locally.

3. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 2, wherein: Entering the designated danger zone includes: crossing over suspended objects and crawling under suspended objects.

4. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 2, wherein: The dangerous area detection module uses the YOLOv5 target detection model to annotate workers and heads in lifting scenarios, with a training set to validation set ratio of 4:

1. A non-maximum suppression module is constructed to filter multiple target box candidate sets generated by YOLOv5 to obtain a single target detection box, ensuring that a single target has only one detection box.

5. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 4, characterized in that: In the overhead camera image, a certain part of the central area is set as a danger zone. When a person or head recognized by YOLOv5 enters the danger zone, it is judged as out of bounds.

6. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 5, characterized in that: The danger zone and the target person are abstracted into two rectangular boxes, and then the two rectangular boxes are judged to have overlapping parts. The judgment of area overlap includes: Assume (x1, y1) (x2, y2) are the two points at the lower left and upper right corners of the first rectangle; Assume (x3, y3) (x4, y4) are the two points at the lower left and upper right corners of the second rectangle; If two rectangles intersect, then (a) On the X-axis, max(x1,x3) <= min(x2,x4) must be satisfied. (b) On the Y-axis, max(y1, y3) <= min(y2, y4) must be satisfied.

7. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 1, wherein: The number detection module counts the number of people entering the work area in the lifting scenario, and issues an alarm for dangerous work scenarios where the number of safe personnel exceeds the limit. At the same time, it stores the problem frames with identified safety hazards locally to facilitate subsequent review of safety hazards.

8. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 7, wherein: The people detection module uses the YOLOv5 target detection model to perform labeling training on workers and heads in hoisting scenarios, with a training set to validation set ratio of 4:

1. A non-maximum suppression module is constructed to filter multiple target box candidate sets generated by YOLOv5 to obtain a single target detection box, ensuring that a single target has only one rectangular box.

9. The method for detecting dangerous behaviors in a hoisting operation scenario according to claim 8, wherein: The number of people detection module simultaneously counts the number of "heads" and "people" with the classification label. If the two are equal, the module outputs the larger number as the number of people. If the two are not equal, the module outputs the larger number. If the number of "heads" and "people" is equal, the module proves that the output result has a high confidence level.

10. A method for detecting dangerous behaviors in a lifting operation scenario, characterized in that: The method comprises the following steps: S21. Collect a video of personnel behavior in a lifting operation scene and segment the video into a series of images; S22. Use labelImg to label the image in step S21. Use YOLOv5 as the labeling model. The two functional modules in this scenario are the dangerous crossing detection module and the people counting module. The subjects to be identified are both "people", so only "people" and "heads" in the dataset need to be labeled using labelImg. S23. Use the YOLOv5 model to train the training data sets of the above two modules respectively. Both modules recognize people and share the same weight. S24: Build a model inference module, which receives the weights trained in S23 and obtains the recognition rectangles of "people" and "human heads" in the image through inference code; S25. Use multithreading to build a voice alarm module. This module is a sub-thread that is always open and continuously monitors a message queue to determine whether the parameters passed in by the message queue are consistent with the default parameters of this module. If they are inconsistent, a voice alarm is played, and if they are consistent, the module continues to monitor. S26. Construct a danger zone detection module. The danger zone detection module needs to identify when someone enters a danger zone, which is a designated danger zone centered on the hanging object. If the above dangerous behavior is identified, a voice alarm is triggered. The specific steps for building the S26 module are: S261, constructing a region overlap identification algorithm module, the module input is the parameters of two rectangular boxes, wherein the parameters of each rectangular box input are the coordinates of the lower left corner and the upper right corner; the output is to determine whether the two rectangular boxes have an overlap intersection, if so, output True, otherwise output False; S262: Receive the "person" or "head" detection rectangle output by the inference module in S24, where the danger zone rectangle is a default rectangle b centered on the hanging object and remains unchanged; call the region overlap recognition algorithm module in S261 to determine whether the detection rectangle overlaps with the default rectangle b, and output a list containing the results, recording all the judgment results; S263: Call the voice alarm module of S25 and set the default parameter to "no overlap". The message queue in S25 continues to receive the result list obtained in S262 and follows the logic of the S25 module. If anyone enters the danger zone, an alarm is triggered. S27, constructing a number detection module; wherein the number detection module needs to calculate the number of people within the target range, and if it is detected that the number of people exceeds the limit, a voice alarm is triggered; The specific steps for building the S27 module are: S271, receiving the "person" or "head" detection result output by the inference module in S24, counting them respectively, and outputting the larger value as the number of people; S272. Call the voice alarm module in S25 and set the default parameters to the upper limit M and lower limit N of the maximum number of operators in this place. The message queue in S25 continuously receives the number K of operators in each frame of the real-time video obtained in S271. If M>K>N, continue to monitor. Otherwise, if it is detected that the number of operators exceeds the limit, a voice alarm is issued; S28. Package the model weights trained in step S23, the voice alarm module in S25, the dangerous area detection module in S26, and the number of people detection module in S27 into the front-end framework. The two interactive modules on the front end are the two inference modules mentioned above, named "dangerous area detection" and "number of people detection" respectively. After interaction, the inference results are output; S29. Place a camera with a bird's-eye view on the lifting rail, vertically above the hook. The object is centered in the video, so set the danger zone to a rectangular box of a certain size in the center of the image. Click "Danger Zone Detection" or "People Detection" in the front-end framework described in step S28 and select the corresponding video folder to demonstrate the inference effect online.

Images