Method, device, storage medium and electronic device for detecting high-altitude climbing operation behavior

Abstract

An embodiment of the present invention provides a method, device, storage medium, and electronic device for detecting high-altitude climbing operation behaviors. The method for detecting high-altitude climbing operation behaviors includes: obtaining video stream data when a target camera is in a first state as first video stream data, performing a first-level detection on the first video stream data, when the first detection result meets a first condition, converting the target camera from the first state to a second state, and when the target camera is in the second state, trajectory tracking of a target object can be performed, obtaining video stream data when the target camera is in the second state as second video stream data, performing a second-level detection on the second video stream data, and when the second detection result meets a second condition, outputting a target detection result. In the solution of the embodiment of the present invention, by performing multi-level detection on the video stream data, the computing power overhead is reduced, the detection process is optimized, and the detection efficiency is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer technology, and in particular, to a method, device, storage medium, and electronic device for detecting high-altitude climbing operation behaviors. Background Art

[0002] In many scenarios such as construction site operations, power grid circuit maintenance, high-rise building exterior cleaning, and advertisement installation, manual high-altitude operations are required. And in many scenarios, high-altitude operation platforms cannot be used, and people need to climb for operations. The risk of high-altitude climbing operations is relatively high. Therefore, in addition to enhancing protective measures, it is also necessary to conduct safety monitoring on high-altitude climbing operation behaviors.

[0003] It should be noted that there is currently no safety monitoring algorithm solution for high-altitude operations. For detections similar to climbing, basically key points are used for detection. And the detection based on key points often requires image analysis of many objects around the target object. Therefore, the execution of this solution will become extremely slow when there are multiple detection targets, and the practicality is relatively low.

[0004] Therefore, how to effectively and timely conduct safety monitoring on high-altitude climbing operation behaviors is one of the important problems that need to be solved currently. Summary of the Invention

[0005] In view of this, the embodiments of the present invention provide a method, device, storage medium, and electronic device for detecting high-altitude climbing operation behaviors to at least partially solve the above problems.

[0006] According to the first aspect of the embodiments of the present invention, a method for detecting high-altitude climbing operation behaviors is provided, including:

[0007] Obtaining video stream data when the target camera is in the first state as the first video stream data;

[0008] Performing a first-level detection on the first video stream data. When the first detection result meets the first condition, the target camera is switched from the first state to the second state, where the focal length of the target camera in the second state is greater than the focal length of the target camera in the first state, and when the target camera is in the second state, trajectory tracking of the target object can be performed;

[0009] Obtaining video stream data when the target camera is in the second state as the second video stream data;

[0010] Performing a second-level detection on the second video stream data. When the second detection result meets the second condition, outputting a target detection result.

[0011] In another example, when performing a first-level detection on the first video stream data, if the first detection result meets the first condition, then converting the target camera from the first state to the second state includes:

[0012] Performing human detection and target tracking on the first video stream data through a target deep learning network to obtain the first detection result;

[0013] According to the first detection result, if the target object is a human and the target object performs a climbing behavior, then converting the target camera from the first state to the second state.

[0014] In another example, performing human detection and target tracking on the first video stream data through a target deep learning network to obtain the first detection result further includes:

[0015] Performing human detection and target tracking on the first video stream data through a pre-trained DeepSort network to obtain the first detection result.

[0016] In another example, obtaining the video stream data when the target camera is in the second state as the second video stream data includes:

[0017] Controlling the rotation of the target camera through Kalman filtering and PID algorithm to obtain the video stream data when the target camera is in the second state;

[0018] In another example, when performing a second-level detection on the second video stream data, if the second detection result meets the second condition, outputting a target detection result includes:

[0019] Performing safety wearing behavior detection on the second video stream data to obtain the second detection result;

[0020] According to the second detection result, when the target object does not perform a safety wearing behavior, outputting a target detection result.

[0021] In another example, when performing a second-level detection on the second video stream data, if the second detection result meets the second condition, outputting a target detection result further includes:

[0022] According to the second video stream data, performing a secondary confirmation detection on the target object. If the secondary confirmation detection result of the target object is true, then perform the next step.

[0023] According to the second aspect of the embodiments of the present invention, there is provided a high-altitude climbing operation behavior detection device, including:

[0024] A first acquisition module, configured to acquire video stream data when the target camera is in a first state as first video stream data;

[0025] A first-level detection module, configured to perform a first-level detection on the first video stream data. When a first detection result meets a first condition, the target camera is switched from the first state to a second state, where the focal length of the target camera in the second state is greater than that in the first state, and when the target camera is in the second state, trajectory tracking of a target object can be performed;

[0026] A second acquisition module, configured to acquire video stream data when the target camera is in the second state as second video stream data;

[0027] A second-level detection module, configured to perform a second-level detection on the second video stream data. When a second detection result meets a second condition, a target detection result is output.

[0028] In some other examples, the first-level detection module includes:

[0029] A first detection unit, configured to perform human detection and target tracking on the first video stream data through a target deep learning network to obtain the first detection result;

[0030] A conversion unit, configured to, according to the first detection result, if the target object is a human body and the target object performs a climbing behavior, switch the target camera from the first state to the second state.

[0031] According to a third aspect of an embodiment of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, where the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0032] According to a fourth aspect of an embodiment of the present invention, there is also provided an electronic device, including a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0033] In the solution of the embodiment of the present invention, by obtaining the video stream data when the target camera is in the first state as the first video stream data, performing the first-level detection on the first video stream data, when the first detection result meets the first condition, the target camera is converted from the first state to the second state, where the focal length of the target camera when it is in the second state is greater than the focal length of the target camera when it is in the first state, and when the target camera is in the second state, trajectory tracking can be performed on the target object, obtaining the video stream data when the target camera is in the second state as the second video stream data, performing the second-level detection on the second video stream data, when the second detection result meets the second condition, outputting the target detection result, performing multi-level detection on the video stream data reduces the computing power overhead, optimizes the detection process, and improves the detection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings.

[0035] Figure 1 is a hardware structure block diagram of a mobile terminal for a method for detecting high-altitude climbing operation behaviors according to an embodiment of the present invention;

[0036] Figure 2 is a flowchart of a method for detecting high-altitude climbing operation behaviors according to an embodiment of the present invention;

[0037] Figure 3 is a structure block diagram of a device for detecting high-altitude climbing operation behaviors according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0038] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the embodiments of the present invention, rather than all embodiments. Based on the embodiments in the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art shall fall within the protection scope of the embodiments of the present invention.

[0039] The embodiments of the present invention will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0040] It should be noted that the terms "first", "second", etc. in the description, claims and the above-mentioned drawings of the present invention are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0041] The method embodiments provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 is a hardware structure block diagram of a mobile terminal for a high-altitude climbing operation behavior detection method according to an embodiment of the present invention. As Figure 1 shown, the mobile terminal may include one or more ( Figure 1 only one is shown in Figure 1 a processor 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above-mentioned mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may further include more or fewer components than Figure 1 shown, or have a different configuration from

[0042] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to a high-altitude climbing operation behavior detection method in an embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above-mentioned method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely provided with respect to the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network and their combinations.

[0043] The transmission device 106 is used to receive or send data via a network. Specific examples of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (abbreviated as RF) module, which is used to communicate with the Internet wirelessly.

[0044] In order to better solve the problems raised in the above-mentioned background art, the present invention discloses a method, device, storage medium and electronic device for detecting high-altitude climbing operation behaviors, which will be described in detail one by one in the following embodiments.

[0045] See Figure 2 , Figure 2 which shows a flowchart of a method for detecting high-altitude climbing operation behaviors provided according to an embodiment of this specification, specifically including the following steps:

[0046] S202, obtain video stream data when the target camera is in the first state as the first video stream data;

[0047] According to an embodiment of the present invention, preferably, the target camera is a PTZ camera, and the PTZ camera can rotate the position (shooting angle) through the pan-tilt head and change the focal length. It is worth mentioning that other cameras that can achieve the same functions and effects as the PTZ camera can also be applicable to the present invention. The PTZ camera in this specification is for example but not a limitation.

[0048] According to an embodiment of the present invention, obtaining video stream data when the target camera is in the first state as the first video stream data includes: when the target camera is in the first state, the target camera is in a stationary state.

[0049] Specifically, when the target camera is in the first state, the target camera performs video shooting at the original preset position. Through the video shooting of the camera in the first state, the first video stream data can be obtained.

[0050] It is worth mentioning that PTZ cameras are installed in many scenarios, especially on construction sites with a lot of high-altitude climbing operations. Therefore, a method for detecting high-altitude climbing operation behaviors provided by the present invention can be applicable to many scenarios and the original equipment in these scenarios, without the need to spend a lot of equipment costs to update the original cameras.

[0051] S204, perform a first-level detection on the first video stream data. When the first detection result meets the first condition, the target camera is converted from the first state to the second state, where the focal length of the target camera when it is in the second state is greater than the focal length of the target camera when it is in the first state, and when the target camera is in the second state, it can track the trajectory of the target object;

[0052] According to an embodiment of the present invention, performing a first-level detection on the first video stream data. When the first detection result meets the first condition, converting the target camera from the first state to the second state includes:

[0053] S2042, perform human detection and object tracking on the first video stream data through a target deep learning network to obtain the first detection result;

[0054] S2044, according to the first detection result, if the target object is a human body and the target object has performed a climbing behavior, then convert the target camera from the first state to the second state.

[0055] Wherein, according to an embodiment of the present invention, when the camera is in the second state, its focal length is greater than the focal length of the target camera when it is in the first state, and the camera switches from shooting in the long shot to shooting in the close shot. And when the target camera is in the second state, the shooting angle can be changed according to the target object for trajectory tracking so that the target object is at the center of the shooting screen.

[0056] According to an embodiment of the present invention, according to the first detection result, if it does not meet the condition that the target object is a human body and the target object has performed a climbing behavior, then the target camera remains in the first state. Specifically, for example, if the target object in the picture is clothes or a floating object, etc., the target camera remains stationary, avoiding false detection, and can specifically detect the human target object in the subsequent detection of safety behaviors, rather than detecting all objects in the picture, saving computing power and further improving the detection efficiency.

[0057] According to an embodiment of the present invention, step S2042, performing human detection and object tracking on the first video stream data through a target deep learning network to obtain the first detection result, includes:

[0058] Perform human detection and object tracking on the first video stream data through a pre-trained DeepSort network to obtain the first detection result.

[0059] Specifically, according to an embodiment of the present invention, through a pre-trained DeepSort network, first perform human detection on all objects in the first video stream data, that is, first determine whether the object is a human body. If the condition of human detection is met, then perform object tracking on the human target object, that is, detect whether the human target object has performed a climbing behavior.

[0060] According to an embodiment of the present invention, preferably, during the process of detecting the human target object, the detector can be replaced with yoloV5 to improve the detection rate of small targets.

[0061] Wherein, performing object tracking on the human target object, that is, detecting whether the human object has performed a climbing behavior, includes:

[0062] Perform trajectory analysis on each human target object. Specifically, include the target object moving slowly from bottom to top in the list of suspected climbing objects, so that key point detection can be performed only on a limited number of target objects, ensuring real-time performance. Then, perform key point detection on each suspected climbing target object, and use the variation law of the key points of the legs of the human target object to determine the real climbing behavior of the target object. When the human target object performs a climbing behavior, the target camera switches from the first state to the second state. At this time, the camera focal length becomes larger, and the camera switches from the long-distance shooting state to the close-up shooting state. Therefore, the captured image including the target object can be clearer.

[0063] In the prior art, for detections similar to climbing, although key point detection is used, key point detection needs to be performed on all objects in the image. Therefore, when there are multiple detection targets, the algorithm becomes extremely slow and has low practicality. The high-altitude climbing operation behavior detection method provided by the present invention screens humanoid targets and confirms climbing behaviors for the target objects in the image first, narrowing the range of target objects that need to be detected for safety wearing behaviors subsequently, reducing the computing power overhead, optimizing the detection process, and improving the detection efficiency.

[0064] S206, obtain the video stream data when the target camera is in the second state as the second video stream data;

[0065] According to an embodiment of the present invention, after the human target object performing a climbing behavior is confirmed, the camera switches to the close-up shooting state for this human target object. Through the video shooting of the target camera in the second state, the second video stream data can be obtained, that is, when the target camera is in the second state, clearer video stream data including the human target object can be obtained. At this time, the focus of the target camera is on the human target object, and the target camera can rotate and shoot according to the human target object.

[0066] According to an embodiment of the present invention, obtaining the video stream data when the target camera is in the second state as the second video stream data further includes: controlling the rotation of the target camera through the Kalman filter and the PID algorithm to obtain the video stream data when the target camera is in the second state.

[0067] Furthermore, first, quickly track the target object through the KCF tracking algorithm, and then control the pan-tilt head through the Kalman filter and the PID algorithm, thereby further controlling the rotation of the target camera to obtain the video stream data when the target camera is in the second state.

[0068] Specifically, through the KCF tracking algorithm, the target object is controlled to be in the middle of the camera's captured image. Features of the target object are extracted from the previous frame image, and the position of the target object is regressed in the next frame image. Then, the offset of the target object from the center point of the camera's capture and the size change are calculated, which are then converted into camera movement parameters. The pan-tilt camera (target camera) is gently controlled to move to track the target object through the Kalman filter algorithm and the PID control algorithm, thereby obtaining the video stream data when the target camera is in the second state. It can be known that since the second video stream data is obtained by tracking the target object when the target camera is in the close-up shooting state, the second video stream data is clearer and more useful than the first video stream data.

[0069] S208. Perform a second-level detection on the second video stream data. When the second detection result meets the second condition, output the target detection result.

[0070] According to an embodiment of the present invention, common safety specifications for high-altitude climbing operations include wearing safety helmets, safety belts, etc. In the embodiment provided by the present invention, the second-level detection mainly detects the safety wearing behavior of the target object.

[0071] According to an embodiment of the present invention, step S208, performing a second-level detection on the second video stream data, and when the second detection result meets the second condition, outputting the target detection result, includes:

[0072] S2082. Perform a safety wearing behavior detection on the second video stream data to obtain the second detection result;

[0073] S2084. According to the second detection result, when the target object does not perform a safety wearing behavior, output the target detection result.

[0074] Specifically, first, intercept the target detection area, expand the detection box of the human target object by 1.5 times to obtain the safety wearing behavior detection area; use the deep learning detection algorithm to detect the safety wearing device within the safety wearing behavior detection area. For example, use the YOLO detection algorithm to detect safety wearing devices such as safety helmets and safety ropes. If no safety wearing devices such as safety helmets or safety ropes are detected, output the detection result; if the target safety wearing device is detected within the safety wearing behavior detection area, then combine the physical logic relationship between the safety rope, safety helmet and the human body and the state classification algorithm to determine whether the safety wearing has been carried out. Specifically, judge by the IOU overlap between the safety helmet and safety rope and the person. The state classification algorithm is to classify whether the person's head wears a safety helmet and whether the body wears a safety rope using the deep learning classification algorithm (such as the EfficientNet algorithm). If the detection result is output. Through the detection result, relevant management personnel can take relevant warning measures in time to warn the high-altitude climbing operators who do not meet the safety specifications to wear safety equipment in time.

[0075] According to another embodiment of the present invention, perform a second-level detection on the second video stream data, and when the second detection result meets the second condition, output the target detection result, including:

[0076] S2082a, intercept the target detection area based on the second video stream data;

[0077] S2084a, perform target safety wearing device detection based on the target detection area. If the target safety wearing device is not detected, output the target detection result; if the target safety wearing device is detected, then determine whether the target object has carried out safety wearing on the target safety wearing device. If not, output the target detection result.

[0078] According to another embodiment of the present invention, the step S208 further includes a step:

[0079] Perform a secondary confirmation detection on the target object according to the second video stream data. If the secondary confirmation detection result of the target object is true, then perform the next step.

[0080] It is worth mentioning that since the first video stream data is obtained when the target camera takes a long-distance shot, the clarity of the image is not high. Therefore, when performing the first-level detection, misdetection may occur. For example, the fluttering clothes drying on the tree are easily misdetected as a person in the long-distance view, and when the target object shakes, it is very likely to be judged that a climbing behavior has occurred, and this situation will further lead to misdetection of whether the target object has performed a safe wearing behavior. Therefore, after the target camera switches to the second state, that is, when the target camera is in the close-range shooting state, perform a human detection and climbing behavior judgment again based on the new video stream data. If the result is true, continue with the subsequent safe wearing behavior detection. If the result is false, control the target camera to switch back to the long-distance shooting state and continue with the human and climbing behavior detection.

[0081] According to a method for detecting high-altitude climbing operation behaviors proposed in the embodiments provided by the present invention, only one dome camera is required to complete multiple tasks such as long-distance detection, close-range analysis, and judgment of the state of the safety wearing device, improving the versatility in multiple scenarios.

[0082] It is worth mentioning that according to the embodiments provided by the present invention, the complex task of detecting the safe wearing behavior of high-altitude climbing operators is solved through multi-level detection. Compared with the prior art, it has strong portability and small resource occupation. And it can further reduce the disability rate of high-altitude operations caused by the failure to perform standardized safe wearing behaviors due to human carelessness. In today's era of advocating safe and intelligent construction sites, it has important practical significance.

[0083] The method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0084] In this embodiment, a high-altitude climbing work behavior detection device is also provided. The device is used to implement the corresponding high-altitude climbing work behavior detection method in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here. In addition, the functional implementation of each module of the data processing device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be repeated here. As used below, the term "module" can implement a combination of software and / or hardware that has a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0085] According to another embodiment of the present invention, Figure 3 , provides a high-altitude climbing operation behavior detection device, including:

[0086] A first acquisition module 30 is configured to acquire video stream data when the target camera is in a first state as first video stream data;

[0087] According to one embodiment of the present invention, preferably, the target camera is a dome camera, which can rotate its position (shooting angle) and change its focal length through the pan / tilt head. It is worth mentioning that other cameras that can achieve the same functions and effects as the dome camera can also be applicable to the present invention. The dome camera in this specification is an example but not a limitation.

[0088] According to an embodiment of the present invention, the first acquisition module 30 may be configured such that when the target camera is in the first state, the target camera is in a stationary state.

[0089] Specifically, when the target camera is in the first state, the target camera performs video shooting at an original preset position, and the first video stream data can be obtained through the video shooting of the camera in the first state.

[0090] It is worth mentioning that dome cameras are installed in many scenarios, especially on construction sites where there are many high-altitude climbing operations. Therefore, the high-altitude climbing operation behavior detection method provided by the present invention can be applied to more scenarios and the original equipment of these scenarios, without the need to spend more equipment costs to update the original cameras.

[0091] a first-level detection module 40 configured to perform a first-level detection on the first video stream data, and when a first detection result satisfies a first condition, switch the target camera from a first state to a second state, wherein a focal length of the target camera in the second state is greater than a focal length of the target camera in the first state, and when the target camera is in the second state, a trajectory of a target object can be tracked;

[0092] According to an embodiment of the present invention, the first-level detection module 40 includes:

[0093] A first detection unit 42, configured to perform human detection and object tracking on the first video stream data through a target deep learning network to obtain the first detection result;

[0094] A conversion unit 44, configured to, according to the first detection result, if the target object is a human body and the target object has performed a climbing behavior, convert the target camera from the first state to the second state.

[0095] Wherein, according to an embodiment of the present invention, when the camera is in the second state, its focal length is greater than the focal length of the target camera when it is in the first state, and the camera switches from long-shot shooting to close-shot shooting. And when the target camera is in the second state, the shooting angle can be changed according to the target object for trajectory tracking so that the target object is at the center of the shooting screen.

[0096] According to an embodiment of the present invention, according to the first detection result, if it does not meet the condition that the target object is a human body and the target object has performed a climbing behavior, the target camera remains in the first state. Specifically, for example, if the target object in the picture is clothes or a floating object, etc., the target camera remains stationary, avoiding false detection, and can specifically detect the human target object in the subsequent detection of safety behaviors, rather than detecting all objects in the picture, saving computing power and further improving the detection efficiency.

[0097] According to an embodiment of the present invention, the first detection unit 42 may be configured to: perform human detection and object tracking on the first video stream data through a pre-trained DeepSort network to obtain the first detection result.

[0098] Specifically, according to an embodiment of the present invention, through the pre-trained DeepSort network, first perform human detection on all objects in the first video stream data, that is, first determine whether the object is a human body. If the condition of human detection is met, then perform object tracking on the human target object, that is, detect whether the human target object has performed a climbing behavior.

[0099] According to an embodiment of the present invention, preferably, during the detection of the human target object, the detector can be replaced with yoloV5 to improve the detection rate of small targets.

[0100] Wherein, performing object tracking on the human target object, that is, detecting whether the human object has performed a climbing behavior, includes:

[0101] Perform trajectory analysis on each human target object. Specifically, include the target object that moves slowly from bottom to top in the list of suspected climbing objects. This way, key point detection can be performed only on a limited number of target objects, ensuring real-time performance. Then, perform key point detection on each suspected climbing target object, and use the change rule of the key points of the legs of the human target object to determine the true climbing behavior of the target object. When the human target object performs a climbing behavior, the target camera switches from the first state to the second state. At this time, the camera focal length becomes larger, and the camera switches from the long-distance shooting state to the close-up shooting state. Therefore, the captured image including the target object can be clearer.

[0102] In the prior art, for detections similar to climbing, although key point detection is used, key point detection needs to be performed on all objects in the image. Therefore, when there are multiple detection targets, the algorithm becomes extremely slow and has low practicality. The high-altitude climbing operation behavior detection device provided by the present invention first screens humanoid targets and confirms climbing behaviors for the target objects in the image, narrowing the range of target objects that need to be detected for safety wearing behaviors in the subsequent process, reducing the computing power overhead, optimizing the detection process, and improving the detection efficiency.

[0103] A second acquisition module 50 is configured to acquire the video stream data when the target camera is in the second state as the second video stream data.

[0104] According to an embodiment of the present invention, after the human target object performing a climbing behavior is confirmed, the camera switches to the close-up shooting state for this human target object. Through the video shooting of the target camera in the second state, the second video stream data can be acquired, that is, when the target camera is in the second state, clearer video stream data including the human target object can be obtained. At this time, the focus of the target camera is on the human target object, and the target camera can rotate and shoot according to the human target object.

[0105] According to an embodiment of the present invention, the second acquisition module 50 can be configured to control the rotation of the target camera through the Kalman filter and the PID algorithm to acquire the video stream data when the target camera is in the second state.

[0106] Further, first, quickly track the target object through the KCF tracking algorithm, and then control the pan-tilt through the Kalman filter and the PID algorithm, thereby further controlling the rotation of the target camera to acquire the video stream data when the target camera is in the second state.

[0107] Specifically, through the KCF tracking algorithm, the target object is controlled to be in the middle of the camera shooting screen. The features of the target object are extracted from the previous frame image, and the position of the target object is regressed in the next frame image. Then, the offset and size change of the target object at the center point of the camera shooting are calculated, and thus converted into camera movement parameters. The ball machine (target camera) is gently controlled to move to track the target object through the Kalman filter algorithm and the PID control algorithm, so as to obtain the video stream data when the target camera is in the second state. It can be known that the second video stream data is relatively clearer and has higher data usability compared with the first video stream data because it is the tracking shooting of the target object when the target camera is in the close-up shooting state.

[0108] The second-level detection module 60 is configured to perform second-level detection on the second video stream data, and output a target detection result when the second detection result meets the second condition.

[0109] According to an embodiment of the present invention, common safety specifications for high-altitude climbing operations include wearing a safety helmet, wearing a safety belt, etc. In the embodiment provided by the present invention, the second-level detection is mainly to detect the safety wearing behavior of the target object.

[0110] According to an embodiment of the present invention, the second-level detection module 60 includes:

[0111] The second detection unit 62 is configured to perform safety wearing behavior detection on the second video stream data to obtain the second detection result;

[0112] The output unit 64 is configured to output a target detection result according to the second detection result when the target object does not perform a safety wearing behavior.

[0113] Specifically, first, intercept the target detection area, expand the detection box of the human target object by 1.5 times to obtain the safety wearing behavior detection area; use the deep learning detection algorithm to detect the safety wearing device within the safety wearing behavior detection area. For example, use the YOLO detection algorithm to detect safety wearing devices such as safety helmets and safety ropes. If no safety wearing devices such as safety helmets or safety ropes are detected, output the detection result; if the target safety wearing device is detected within the safety wearing behavior detection area, then combine the physical logic relationship between the safety rope, safety helmet and the human body and the state classification algorithm to determine whether the safety wearing has been carried out. Specifically, it is judged by the IOU overlap between the safety helmet and safety rope and the person. The state classification algorithm uses the deep learning classification algorithm (such as the EfficientNet algorithm) to classify whether the head wears a safety helmet and whether the body wears a safety rope. If the detection result is output. Through the detection result, relevant management personnel can take relevant warning measures in time to warn the high-altitude climbing operators who do not meet the safety specifications to carry out safety wearing in time.

[0114] According to another embodiment of the present invention, the second-level detection module 60 includes:

[0115] The second detection unit 62a is used to intercept the target detection area based on the second video stream data;

[0116] The output unit 64a is used to detect the target safety wearing device based on the target detection area. If the target safety wearing device is not detected, output the target detection result; if the target safety wearing device is detected, then judge whether the target object has carried out safety wearing on the target safety wearing device. If not, output the target detection result.

[0117] According to another embodiment of the present invention, the second-level detection module 60 further includes a secondary confirmation unit 61: used to perform secondary confirmation detection of the target object according to the second video stream data. If the secondary confirmation detection result of the target object is true, execute the next step.

[0118] It is worth mentioning that since the first video stream data is obtained when the target camera takes a long-shot, the clarity of the image is not high. Therefore, when performing the first-level detection, misdetection may occur. For example, the floating clothes drying on the tree are likely to be misdetected as a person in the long-shot, and when the target object shakes, it is very likely to be judged that a climbing behavior has occurred, and this situation will further lead to misdetection of whether the target object has performed a safe wearing behavior. Therefore, after the target camera switches to the second state, that is, when the target camera is in the close-shot state, perform a human detection and a climbing behavior judgment again based on the new video stream data. If the result is true, continue with the subsequent safe wearing behavior detection. If the result is false, control the target camera to switch back to the long-shot state and continue with the human and climbing behavior detection.

[0119] According to an aerial climbing operation behavior detection device proposed in the embodiments provided by the present invention, only one dome camera is required to complete multiple tasks such as long-shot detection, close-shot analysis, and safe wearing device state judgment, improving the versatility in multiple scenarios.

[0120] It is worth mentioning that according to the embodiments provided by the present invention, the complex task of detecting the safe wearing behavior of aerial climbing operation personnel is solved through multi-level detection. Compared with the prior art, it has strong portability and small resource occupancy. And it can further reduce the disability rate of aerial operations caused by the lack of standardized safe wearing behavior due to human carelessness. In today's era of advocating safe and intelligent construction sites, it has important practical significance.

[0121] The embodiments of the present invention also provide a computer-readable storage medium, in which a computer program is stored. Among them, the computer program is configured to execute the steps in any one of the above method embodiments when running.

[0122] In an exemplary embodiment, the above computer-readable storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM for short), random access memory (RAM for short), mobile hard disk, magnetic disk or optical disc and other various media that can store computer programs.

[0123] The embodiments of the present invention also provide an electronic device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0124] In an exemplary embodiment, the above electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0125] Specific examples in this embodiment may refer to the examples described in the above embodiments and exemplary implementation manners, and will not be elaborated herein again.

[0126] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a sequence different from that here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0127] The foregoing is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for detecting high-altitude climbing operation behaviors, characterized in that, Including: Obtain the video stream data of the target camera when it is in the first state as the first video stream data. When the target camera is in the first state, the target camera is in a stationary state; Perform a first-level detection on the first video stream data. When the first detection result meets the first condition, convert the target camera from the first state to the second state. When the target camera is in the second state, the focal length is greater than that when the target camera is in the first state, and when the target camera is in the second state, trajectory tracking of the target object can be performed; Obtain the video stream data of the target camera when it is in the second state as the second video stream data; Perform a second-level detection on the second video stream data. When the second detection result meets the second condition, output the target detection result; Among them, performing a first-level detection on the first video stream data. When the first detection result meets the first condition, converting the target camera from the first state to the second state includes: first performing human detection on the first video stream data through a pre-trained DeepSort network to determine whether the target object is a human; in the case of determining that the target object is the human, add the target object moving from bottom to top to the climbing object list; perform key point detection on the target object in the climbing object list, and use the change rule of the key points of the target object's legs to obtain the first detection result; according to the first detection result, if the target object performs the climbing behavior, convert the target camera from the first state to the second state.

2. The method according to claim 1, wherein Obtain the video stream data of the target camera when it is in the second state as the second video stream data, including: Control the rotation of the target camera through Kalman filtering and PID algorithm to obtain the video stream data of the target camera when it is in the second state.

3. The method according to claim 1, wherein Performing a second-level detection on the second video stream data. When the second detection result meets the second condition, outputting the target detection result includes: Perform safety wearing behavior detection on the second video stream data to obtain the second detection result; According to the second detection result, when the target object does not perform the safety wearing behavior, output the target detection result.

4. The method according to claim 3, wherein Performing a second-level detection on the second video stream data. When the second detection result meets the second condition, outputting the target detection result further includes: According to the second video stream data, perform secondary confirmation detection on the target object. If the secondary confirmation detection result of the target object is true, perform the next step.

5. An apparatus for detecting high-altitude climbing operation behaviors, characterized in that, Including: A first acquisition module for obtaining the video stream data of the target camera when it is in the first state as the first video stream data. When the target camera is in the first state, the target camera is in a stationary state; The first-level detection module is used to perform first-level detection on the first video stream data. When the first detection result meets the first condition, the target camera is converted from the first state to the second state, where the focal length of the target camera in the second state is greater than that in the first state, and when the target camera is in the second state, trajectory tracking of the target object can be performed; The second acquisition module is used to acquire the video stream data when the target camera is in the second state as the second video stream data; The second-level detection module is used to perform second-level detection on the second video stream data. When the second detection result meets the second condition, the target detection result is output; The first-level detection module is further used to: first perform human detection on the first video stream data through a pre-trained DeepSort network to determine whether the target object is a human body; in the case of determining that the target object is the human body, add the target object moving from bottom to top to the climbing object list; perform key point detection on the target object in the climbing object list, and use the change rule of the key points of the target object's legs to obtain the first detection result; according to the first detection result, if the target object has performed the climbing behavior, convert the target camera from the first state to the second state.

6. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, wherein the computer program is set to execute the method described in any one of claims 1 to 4 when running.

7. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is set to run the computer program to execute the method described in any one of claims 1 to 4.

Images