A method, system, electronic device and storage medium for real-time monitoring of site safety

Abstract

The present invention relates to the field of security monitoring technology, and specifically, to a method, system, electronic device and storage medium for real-time monitoring of site security, wherein the method comprises the following steps: dividing the site into regions, and achieving full coverage of the regions through multiple cameras; obtaining a global image of each region and preprocessing it, and determining the target position in the global image through a self-attention target recognition model; performing multi-target visual tracking based on the target position to obtain a local image; and classifying the targets based on the local image, and outputting an early warning interaction after correlation comparison and / or security detection of the classified targets. The present invention can realize full-scene, precise, and unmanned security monitoring without blind spots in complex environmental backgrounds, accurately identify and track targets, so that the monitoring results are not affected by the external environment, and ensure the high accuracy of the monitoring results.

Description

Technical Field

[0001] The present invention relates to the field of safety monitoring technology, and in particular to a method, system, electronic equipment and storage medium for real-time monitoring of site safety. Background Art

[0002] Safety inspection of construction sites is a basic work of safety management of construction sites. Inspection methods can be divided into manual inspection, on-site camera inspection, manned helicopter inspection and drone inspection. Although manual inspection is the most commonly used inspection method, it has always been inefficient; and on-site camera inspections are mostly based on fixed-position imaging of a single traditional camera, which is not only affected by factors such as weather, light intensity, and dust, but also has blind spots in vision and too far target distance. As a result, it is easy to lose targets, miss detections, and have low accuracy during real-time monitoring. It is impossible to achieve full-scene, accurate, and no-dead-angle safety monitoring, and it is difficult to meet the requirements of construction site safety management. Summary of the invention

[0003] The object of the present invention is to provide a method, system, electronic device and storage medium for real-time monitoring of site safety to solve the problems pointed out in the background technology.

[0004] The embodiment of the present invention is implemented by the following technical solution: A method for real-time monitoring of site safety comprises the following steps:

[0005] Divide the site into areas and achieve full coverage of the area through multiple cameras;

[0006] Obtain and preprocess the global image of each region, and determine the target position in the global image through the self-attention target recognition model;

[0007] Perform multi-target visual tracking based on the target positions to obtain a local image;

[0008] Target classification is performed based on the local image, and after the classified targets are subjected to correlation comparison and / or safety detection, an early warning interaction is output.

[0009] According to a preferred embodiment, the global image of each region is acquired through a wide-angle lens.

[0010] According to a preferred embodiment, the local image is acquired through a telephoto lens.

[0011] According to a preferred embodiment, a deep-pupil human eye camera is used to acquire the global image and the local image.

[0012] According to a preferred embodiment, the preprocessing includes but is not limited to video frame extraction, image enhancement and image filtering.

[0013] According to a preferred embodiment, after the classified targets are subjected to correlation comparison and / or safety detection, outputting warning interaction includes:

[0014] When the target is a person, firstly, the similarity is judged. When the target is an outsider, the first warning information is sent to the alarm.

[0015] The classified targets are subjected to security monitoring, and when the security monitoring results are abnormal, a second warning message is sent to the targets.

[0016] The present invention also provides a site safety real-time monitoring system, which is applied to the above-mentioned method, comprising:

[0017] The area division module divides the site into areas and achieves full coverage of the area through multiple cameras;

[0018] The overall target recognition module is used to obtain and preprocess the global image of each region and determine the target position in the global image through the self-attention target recognition model;

[0019] A single target recognition module, used to perform multi-target visual tracking based on the target position and obtain a local image;

[0020] The detection module is used to classify targets based on the local image, and output a warning interaction after the classified targets are subjected to correlation comparison and / or safety detection.

[0021] The present invention also provides an electronic device, comprising:

[0022] A memory storing execution instructions;

[0023] and a processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the method described above.

[0024] The present invention also provides a readable storage medium, wherein the readable storage medium stores execution instructions, and the execution instructions are used to implement the method described above when executed by a processor.

[0025] The technical solution of the embodiments of the present invention has at least the following advantages and beneficial effects: a method, system, electronic device and storage medium for real-time monitoring of site safety provided by the present invention determines the target position within the visual range based on a wide-angle lens, and then obtains a clearer local image through a telephoto lens, and obtains the target position and target type by identifying and processing the local image, and outputs corresponding warning interactions by processing it, thereby realizing unmanned supervision, and can achieve full-scene, precise, and no-dead-angle safety monitoring under complex environmental backgrounds, accurately identify and track targets, so that the monitoring results are not affected by the external environment, and the high accuracy of the monitoring results is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 A schematic diagram of a flow chart of a method for real-time monitoring of site safety provided in Example 1 of the present invention;

[0027] Figure 2 A schematic diagram of the structure of a real-time site safety monitoring system provided in Example 1 of the present invention;

[0028] Figure 3 A logic flow chart of a method for real-time monitoring of site safety provided in Example 1 of the present invention;

[0029] Figure 4 A schematic diagram of the principle of the self-attention target recognition model provided in Example 1 of the present invention;

[0030] Figure 5 A schematic diagram of the classification model principle provided in Example 1 of the present invention;

[0031] Figure 6 A schematic diagram of area division provided in Example 1 of the present invention;

[0032] Figure 7 A schematic diagram of the global image acquisition principle provided in Embodiment 1 of the present invention; DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0034] Example 1

[0035] refer to Figure 1 As shown, Figure 1 A schematic flow chart of a method for real-time monitoring of site safety provided by an embodiment of the present invention.

[0036] A method for real-time monitoring of site safety provided by an embodiment of the present invention comprises the following steps:

[0037] 1), reference Figure 7 As shown, the global image is obtained and preprocessed, and the target position in the global image is determined by a multi-eye self-attention target recognition model; the specific steps are as follows:

[0038] 1-1), reference Figure 6As shown, the venue is divided into areas, and multiple cameras are used in each area. Each area is responsible for a wide-angle lens + a telephoto lens to capture all the details of the entire venue and achieve full coverage of the area;

[0039] 1-2) Use a wide-angle lens to obtain a global image of the monitored area, and further perform image preprocessing operations such as enhancement and filtering on the obtained global image. If the wide-angle lens obtains a video stream, the video is divided frame by frame through video extraction to obtain the image of each frame and then perform image preprocessing.

[0040] 1-3) See Figure 4 As shown, the preprocessed image is input into the pre-trained self-attention target recognition model, thereby determining the position of the target in the global image, wherein the pre-trained self-attention target recognition model is obtained based on the following method: based on a hybrid model of Swin Transformer and YoloV4, the image is downsampled by 4 times, 8 times and 16 times using the hierarchical components of Swin Transformer, and a target detection network is constructed on this basis. Based on W-MSA, the feature map is divided into multiple non-overlapping areas during the downsampling process, wherein MSA is only performed in each window. Finally, SW-MSA is used to allow information to be transmitted in adjacent windows to improve the network's receptive field, thereby completing the construction of the model, and the global image is input into the target recognition model to obtain the target recognition result and its position in the monitoring area of ​​the wide-angle lens; the YoloV4 network built using Swin Transformer has good performance.

[0041] 2) Perform multi-target visual tracking based on the target position to obtain a clearer local image; specifically in this embodiment, a telephoto lens is used to perform single-target visual tracking.

[0042] In another implementation of this embodiment, a deep-pupil human eye camera is used to acquire the global image and the local image. The deep-pupil human eye camera can realize the functions of a wide-angle lens and a telephoto lens. A single camera can realize the acquisition of the global image and the local image. The specific implementation process will not be described in detail.

[0043] 3) Classifying targets based on the local images, and outputting warning interactions after correlation comparison and / or safety detection of the classified targets; the specific steps are as follows:

[0044] 3-1) Classify the target using the classification model. Specifically in this embodiment, the classification model is a hybrid model based on MobilNet v2 and Transformer. This embodiment combines traditional CNN feature extraction with Transformer, and uses MobilNet v2 as a feature extractor to construct the classification model. It should be noted that the convolutional layer of MobilNet v2 in this embodiment uses StdConv2d instead of the traditional Conv2d. In addition, all BatchNorm layers are replaced with GroupNorm layers, thereby reducing the number of network parameters and saving classification time. Principle reference Figure 5 As shown, in this embodiment, the local image is input into the classification model to achieve accurate classification.

[0045] 3-2) After the classified target has been subjected to correlation comparison and / or safety detection, the output warning interaction includes: when the target is a person, face recognition is performed through the pre-trained Haar cascade model, and the face recognition result is judged for similarity with the data in the library. When the target is an outsider, the PC sends the first warning information to the alarm through the PLC, and the data is uploaded to the cloud for backup through the WiFi module; the specific face recognition implementation process is as follows: load the classifier, load the classified target image in grayscale format, call the DetectMulitiScale(image, scaleFactor=1.2, minNeighbors=3, minSize=(32,32)) function to detect the face, where scaleFactor represents the ratio of each image size reduction, minNeighbors represents the number of detections required to determine it as a real target, and minSize represents the minimum size of the target. If a face is detected in the image, the coordinate value of the rectangular box of the area where the face is located is returned.

[0046] Furthermore, a safety detection model based on a hybrid neural network of Swin Transformer and YoloV4 is used to monitor the classified targets. When the safety monitoring result is abnormal, refer to Figure 2, for example, when the target is a person, the staff member does not meet the norms, such as not wearing a safety rope at high altitude, not wearing a safety helmet as required, using a mobile phone or smoking in violation of regulations; when the target is an object, for example, the equipment status is abnormal; when the target is an area, for example, an open fire occurs in the area and there is a safety hazard, then the PC sends a second warning message to the target or the corresponding manager of the target through the information sending module to achieve unmanned supervision. The following is an example of the above-mentioned safety monitoring. The process of smoking detection through the safety detection model is as follows: obtain the coordinates of the rectangular frame of the face area output by the Haar cascade model, obtain the target area image by adjusting the coordinates, input the safety detection model after preprocessing the target area image, output the judgment result and prompt. Specifically, after obtaining the face coordinates by face detection, this embodiment adjusts the size of the target window based on the coordinates so that the target window contains image information related to smoking. Specifically in this embodiment, the target area is extended from the face area to the position above the neck of the face, and it is appropriately expanded and adjusted to include image information such as the facial area, smoking gestures, and cigarette targets. Furthermore, the adjusted target area is dynamically changed. According to experimental tests, when the width of the face area is increased by 0.9 times and the height is increased by 1.1 times with respect to the center of the image, a better target area is obtained. In one embodiment, assuming that (top, right, bottom, left) is the detected face coordinate, the adjusted coordinates are as follows:

[0047]

[0048] Furthermore, the image with adjusted coordinates is input into the safety detection model for safety inspection, thereby improving the detection accuracy.

[0049] To sum up, the method for real-time monitoring of site safety provided by the present invention determines the target position within the visual range based on a wide-angle lens, and then obtains a clearer local image through a telephoto lens, and obtains the target position and target type by identifying and processing the local image, and outputs corresponding warning interaction through processing, so as to achieve unmanned supervision, and can realize full-scene, precise, and no-dead-angle safety monitoring under complex environmental backgrounds, accurately identify and track targets, so that the monitoring results are not affected by the external environment, and the high accuracy of the monitoring results is guaranteed.

[0050] See also Figure 2As shown, an embodiment of the present invention also provides a real-time monitoring system for site safety, which is applied to the method as described above, including: a region division module, which divides the site into regions and achieves full coverage of the regions through multiple cameras; an overall target recognition module, which is used to obtain a global image of each region and pre-process it, and determine the target position in the global image through a self-attention target recognition model; a single target recognition module, which is used to perform multi-target visual tracking based on the target position to obtain a local image; a detection module, which is used to classify targets based on the local image, and output a warning interaction after the classified targets are subjected to correlation comparison and / or safety detection.

[0051] An embodiment of the present invention further provides an electronic device, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the method described above.

[0052] An embodiment of the present invention further provides a readable storage medium, wherein the readable storage medium stores execution instructions, and the execution instructions are used to implement the method described above when executed by a processor.

[0053] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for real-time monitoring of site safety, characterized in that: The steps include: The venue is divided into zones and fully covered by multiple cameras, with each zone equipped with a wide-angle lens and a telephoto lens; The global image of each area is obtained through a wide-angle lens and preprocessed, and the target position in the global image is determined through a self-attention target recognition model; Perform multi-target visual tracking based on the target positions and obtain local images through a telephoto lens; Classifying the target based on the local image, and outputting a warning interaction after the classified target undergoes correlation comparison and / or safety detection, specifically comprising: classifying the target using a classification model, wherein the classification model is a hybrid model based on MobilNet v2 and Transformer, and the convolution layer of MobilNet v2 adopts StdConv2d; The self-attention target recognition model is obtained in the following way: based on a hybrid model of Swin Transformer and YoloV4, the hierarchical components of Swin Transformer are used to downsample the image by 4 times, 8 times and 16 times, and a target detection network is constructed on this basis.

2. The method for real-time monitoring of site safety according to claim 1, characterized in that: A deep pupil human eye camera is used to acquire the global image and the local image.

3. The method for real-time monitoring of site safety according to claim 1 or 2, characterized in that: The preprocessing includes but is not limited to video frame extraction, image enhancement and image filtering.

4. The method for real-time monitoring of site safety according to claim 3, characterized in that: After the classified targets are subjected to correlation comparison and / or safety detection, the output of warning interaction includes: When the target is a person, firstly, the similarity is judged. When the target is an outsider, the first warning information is sent to the alarm. The classified targets are subjected to security monitoring, and when the security monitoring results are abnormal, a second warning message is sent to the targets.

5. A site safety real-time monitoring system, applied to the method according to any one of claims 1 to 4, characterized in that: include: The area division module divides the site into areas and achieves full coverage of the area through multiple cameras; The overall target recognition module is used to obtain and preprocess the global image of each region and determine the target position in the global image through the self-attention target recognition model; A single target recognition module, used to perform multi-target visual tracking based on the target position and obtain a local image; The detection module is used to classify targets based on the local image, and output a warning interaction after the classified targets are subjected to correlation comparison and / or safety detection.

6. An electronic device, characterized in that: include: A memory storing execution instructions; and a processor, wherein the processor executes the execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1 to 4.

7. A readable storage medium, characterized in that: The readable storage medium stores execution instructions, which are used to implement the method according to any one of claims 1 to 4 when executed by a processor.

Images