A real-time high-precision reflective vest wearing state recognition method and device

Abstract

In order to solve the problem of whether the construction worker wears a reflective vest in the prior art, the application provides a real-time high-precision reflective vest wearing state recognition method and device. The method first acquires a construction site image, then inputs the construction site image into a target detection network structure model, and finally obtains a target detection prediction result image of the construction worker and the reflective vest category. Whether the construction worker wears the reflective vest is determined by calculating the coincidence degree of the anchor frame of the reflective vest and the construction worker, so as to realize the identification of whether the construction worker of high-altitude operation or heavy equipment wears the reflective vest. The application is not only suitable for reflective vest detection in multiple scenes, but also has strong robustness. In addition, it can be applied to low-cost and lightweight operation equipment in the construction site with small parameter operation amount, so as to promote the realization of the intelligent construction site.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a real-time, high-precision method and apparatus for recognizing the wearing status of reflective clothing. Background Technology

[0002] Object detection technology is a type of computer technology currently widely used in smart cities and smart construction sites. This technology uses computers to identify different categories of objects in an image using anchor boxes. Image object detection technology, after improvements to the YOLOv1 network structure designed by Redmon et al., can now accurately locate different categories of objects in an image using anchor boxes. From the creation of YOLOv1 to the present, through YOLOv4 designed by Bochkovskiy, YOLOv5 designed by Ultralytics et al., and the latest YOLOv7 series designed by Wang et al., object detection network structures have performed excellently in intelligent object detection systems, achieving an accuracy of 87% in multi-sample image datasets. However, certain shortcomings still exist in practical applications. For example, Huang et al. pointed out in PP-YOLO that the YOLO series has an excessively large number of parameters, resulting in high memory and GPU memory consumption on mobile devices, making it unsuitable for the low-cost, lightweight computing chips used extensively in construction scenarios.

[0003] Safety at construction sites is paramount. Reflective vests are essential for preventing accidents when heavy equipment or work at heights are involved. Currently, construction sites rely on safety officers to patrol and inspect workers. This method is not only resource-intensive but also lacks real-time monitoring and early warning capabilities. There is a significant gap in the recognition of reflective vests, requiring further research. Patent document CN113920469A proposes a method to detect whether workers in construction areas are wearing safety helmets, specifically for identifying the helmet status of electrical workers. However, this method requires initialization of the bounding boxes using K-means clustering, followed by detection using a YOLOv3 object detection network model. Finally, non-maximum suppression is applied to the predicted bounding boxes to eliminate duplicate detections and achieve the final detection result. This method utilizes a large amount of YOLOv3 memory and has a long inference speed. The patent document CN114005089A proposes a method for detecting reflective clothing based on a feature extraction network and an attention network module. This method can extract features through an attention mechanism, obtaining rich image features through multi-level feature extraction, thereby enabling the identification of occluded targets and targets with low brightness. The drawback of this method is that the feature extraction network has a large number of parameters, leading to an increased computational load on the target detection network model. While increasing model accuracy, it sacrifices real-time performance, resulting in excessive memory usage for the target detection network module, making it difficult to use on low-cost, lightweight computing chips. Summary of the Invention

[0004] To address the current lack of robust, real-time, and accurate target detection for recognizing reflective vests used in construction operations, this invention provides a real-time, high-precision method and apparatus for recognizing the wearing status of reflective vests.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows:

[0006] In a first aspect, the present invention provides a real-time, high-precision method for recognizing the wearing status of reflective clothing, the method comprising:

[0007] Acquire images of the construction site;

[0008] The acquired construction site images are input into the target detection network structure model. The backbone network of the target detection network structure model is based on the ResNet50 network structure. Image feature information is extracted from the input images, and then the extracted image feature information is aggregated using a pixel aggregation network to obtain an image feature information map. The image feature information map is then processed by an attention network module for feature information extraction. The feature information extracted by the attention network module is processed by the YOLO algorithm target detection head to obtain a target detection result map with predicted bounding boxes.

[0009] Based on the target detection result image, calculate whether the pixel overlap between the construction worker and the predicted frame of the reflective vest is lower than the set overlap threshold to determine whether the construction worker has violated the safety operation regulations.

[0010] Furthermore, the construction site image input into the target detection network structure model has a resolution of 768x768.

[0011] Furthermore, the batch size parameter of the target detection network structure model is set to a maximum of 12.

[0012] Furthermore, the backbone network of the target detection network structure model is based on the ResNet50 network structure, and the image feature information extraction of the input image includes:

[0013] ResNet50 is used to replace DarkNet-53 of the YOLO series as the backbone network for image feature extraction. The ACON activation function and DCN convolutional layer are used to replace the ReLU activation function and the last three 3x3 convolutional layers of the ResNet50 network structure.

[0014] Furthermore, the output of the DCN convolutional layer is used as the input to the pixel aggregation network.

[0015] Furthermore, the attention network module is connected through a residual network structure, and uses global pooling and fully convolutional layers to extract feature information from the image feature map, resulting in three feature maps of different sizes.

[0016] Furthermore, the target detection head using the YOLO algorithm outputs three feature maps of different sizes obtained from the attention network module. After normalization, three fused maps are obtained. Finally, the optimal bounding box is retained by using non-maximum values ​​to obtain the detection result map of the reflective clothing and construction workers with predicted bounding boxes.

[0017] Furthermore, when the pixel overlap between the construction worker's image and the predicted frame of the reflective vest is lower than the set overlap threshold, the image at this time is stored and an alarm message is sent.

[0018] Secondly, the present invention provides a real-time high-precision reflective clothing wearing status recognition device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the methods described above.

[0019] Thirdly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the methods described above.

[0020] Compared with the prior art, the advantages of this invention are as follows:

[0021] This invention provides a real-time, high-precision method for detecting the wearing status of reflective clothing. This method uses a lightweight target detection neural network to solve the problem of reflective clothing recognition in complex and changing natural environments. This method is aimed at scenarios with high safety requirements, such as the construction of elevated railways and the operation of heavy railway equipment, where construction workers wear reflective clothing. It can effectively improve supervision efficiency and reduce the burden of manual labor. The lightweight reflective clothing target detection neural network can also reduce the cost of purchasing high-computing equipment and promote the realization of smart construction sites. Attached Figure Description

[0022] Figure 1 This is an overall structural diagram of the target detection network structure model provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a schematic diagram of the components of the real-time high-precision reflective clothing wearing status recognition device provided in Embodiment 2 of the present invention. Detailed Implementation

[0024] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0025] Example 1:

[0026] Currently, most methods for identifying reflective vests worn by construction workers in construction areas employ object detection. This involves using monitoring equipment and deep learning neural networks for object detection to identify workers entering the construction area using bounding boxes. The overlap between the predicted bounding box of a worker and the predicted bounding box of a reflective vest is then used to determine whether the worker is wearing reflective vests and to issue an alert. However, this method has drawbacks. The original YOLO series object detection algorithms were primarily developed and applied for training on large-scale datasets, and the training platform was GPU computing, with insufficient attention paid to memory usage. Therefore, for the task of identifying whether workers are wearing reflective vests in construction environments, the following two problems arise: 1. The computing power required for deep learning operations in construction operations needs to be primarily low-cost and lightweight. 2. Collecting a large number of reflective vest samples is difficult, making it hard to form a large-scale dataset.

[0027] To address the problems existing in current technologies regarding the detection of reflective vests worn by construction workers, this invention provides a method for detecting reflective vest wear under complex and changing natural environmental conditions. The method first deploys monitoring equipment in the construction area, then segments the video stream collected by the monitoring equipment at a certain frequency. The segmented images are input into a target detection neural network based on an improved YOLO series, as proposed in this invention. Finally, target detection prediction results for construction workers and reflective vest categories are obtained. The overlap between the anchor frames of reflective vests and construction workers is calculated to determine whether the construction worker is wearing a reflective vest. If the overlap between the anchor frames of the construction worker category and the reflective vest category in the same image is less than a threshold set by safety personnel, it is determined that the construction worker is not wearing a reflective vest. This method enables the identification of whether construction workers working at heights or on heavy equipment are wearing reflective vests. This method is not only applicable to reflective vest detection in multiple scenarios but also has strong robustness. Furthermore, it can be applied to low-cost, lightweight computing devices on construction sites with relatively small parameter computations, thereby promoting the realization of smart construction sites.

[0028] Specifically, the overall structure of the object detection network model used in this application is as follows: Figure 1 As shown, this paper improves upon the YOLO series network structure by first increasing the size of the image input to the neural network model, then replacing the DarkNet network used in the YOLO series with an improved ResNet-vd as the backbone network, replacing the spatial feature pyramid network of the YOLO series with a pixel aggregation network, adding an attention network module, and finally using the YOLO object detection head to output the final object detection result image. The specific steps include the following:

[0029] 1. Video footage from the construction site is transmitted to the backend management system via video surveillance equipment. The backend management system extracts images from the construction site video at a certain frequency and transmits the processed images at a resolution of 768x768 to the target detection network structure model. Increasing the image input size helps preserve the pixel information of objects in the image, thus improving the accuracy of the target detection network. However, using large images as input increases memory usage; therefore, the maximum batch size parameter setting for the target detection network structure model in this invention cannot exceed 12.

[0030] 2. After the image is input into the object detection network structure model with a uniform size, ResNet50-vd is used instead of DarkNet-53 from the YOLO series as the backbone network to extract image feature information. While this reduces network depth and computational cost compared to DarkNet-53, it also leads to a decrease in accuracy. Therefore, the ACON activation function and Deformable Convolutional Networks (DCN) are used to replace the ReLU activation function and the last three 3x3 convolutional layers in the ResNet50-vd network structure. The outputs of these last three convolutional layers serve as input to the Pixel Aggregation Network (PAN) to transfer image information.

[0031] 3. After the image feature information of the input image is extracted by the backbone network, it is processed by the pixel aggregation network. By using PAN to aggregate image feature information instead of the spatial feature pyramid used by the YOLO series, image feature information can be aggregated from top to bottom to obtain an image feature information map. For construction scenarios in complex and changing natural environments, improving image feature information is beneficial to increasing the detection accuracy of the neural network model, and the computational load is similar to that of the spatial feature pyramid method of the YOLO series.

[0032] 4. After the image feature information passes through the pixel aggregation network, an attention network module is added to further extract contextual feature information from the image feature map. This attention network module is connected through a residual network structure and uses global pooling and fully convolutional layers to extract feature information from the image feature map. Compared with the fully connected layer method used by YOLO, this not only reduces the number of parameters and computational cost, but also has almost no impact on the network's detection accuracy.

[0033] 5. After passing through the attention network module, the image feature information yields three feature maps of different sizes. Finally, the YOLO target detection head outputs the feature map, and a normalization operation is used to obtain three fused maps. Finally, the optimal bounding box is retained by using non-maximum values ​​to obtain the detection result map of the reflective clothing and construction workers with predicted bounding boxes.

[0034] 6. After obtaining the detection result images of the reflective vest and the construction worker with the prediction frame, calculate the pixel overlap of the prediction frames of the reflective vest and the construction worker, set the overlap threshold, and send an alarm message to the background management system when the overlap is lower than the set threshold.

[0035] Therefore, it can be seen that the target neural network model structure of this invention uses skip connections extensively to replace the connection methods of the YOLO series because this method not only does not lose image feature information, but also preserves image feature semantic information and better extracts image semantic context information.

[0036] The invention will be further illustrated below with an application scenario example:

[0037] At high-speed railway station construction sites, the installation of overhead power lines is necessary, especially for live-line work. Construction workers must complete the overhead power line construction without interrupting power. According to safety requirements, all workers must wear reflective vests. Visible light cameras are deployed in a rectangular area of ​​the construction site, and video surveillance can be obtained through video PTZ cameras and drone aerial photography. The acquired video is then sent at a frequency of 18 frames per second to the target detection neural network based on the YOLO series, as described in this paper, for processing. The processed video uses confidence boxes to detect, identify, and label workers and reflective vests entering the construction area. When the overlap between the confidence boxes for worker and reflective vest categories falls below the pixel overlap threshold set by safety management personnel, this information is transmitted to the backend management system, which then issues an alarm.

[0038] In summary, this invention improves upon the YOLO series object detection neural network model. First, the backbone network is modified by replacing the original DarkNet53 network structure with an improved ResNet50. Compared to the original DarkNet53 network structure, this method reduces the number of layers by three and replaces the ReLU activation function with the ACON activation function, reducing the computation of invalid neurons and thus lowering the computational load of the original DarkNet53 network structure. Furthermore, a DCN convolutional module replaces the last three 3x3 convolutional layers of ResNet50, effectively improving ResNet50's image feature extraction capabilities. Secondly, a top-down pixel aggregation network structure module is added to make the object detection neural network more accurate in recognizing objects in images. Finally, an attention network module is added to further extract pixel feature information from the image, improving the recognition accuracy of the object detection neural network model. This invention can achieve real-time monitoring of construction workers using visible light cameras deployed at construction sites, effectively reducing the number of computational parameters in the neural network model and improving the inference speed of images, which is beneficial for the selection of low-cost, lightweight computing equipment at construction sites. The method of this invention is not only applicable to the detection of reflective clothing in multiple scenarios, but also has strong robustness. It can also be applied to low-cost and lightweight computing devices on construction sites with a small amount of parameter computation, thereby promoting the realization of smart construction sites.

[0039] Example 2:

[0040] See Figure 2 As shown, the real-time high-precision reflective clothing wearing status recognition device provided in this embodiment includes a processor 21, a memory 22, and a computer program 23 stored in the memory 22 and executable on the processor 21, such as a real-time high-precision reflective clothing wearing status recognition program. When the processor 21 executes the computer program 23, it implements the steps of Embodiment 1 described above.

[0041] For example, the computer program 23 can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 23 in the real-time high-precision reflective clothing wearing status recognition device.

[0042] The real-time, high-precision reflective clothing wearing status recognition device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The real-time, high-precision reflective clothing wearing status recognition device may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that... Figure 2This is merely an example of a real-time, high-precision reflective clothing wearing status recognition device and does not constitute a limitation of the real-time, high-precision reflective clothing wearing status recognition device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the real-time, high-precision reflective clothing wearing status recognition device may also include input / output devices, network access devices, buses, etc.

[0043] The processor 21 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0044] The memory 22 can be an internal storage unit of the real-time high-precision reflective clothing wear status recognition device, such as a hard drive or memory of the device. The memory 22 can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device. Furthermore, the memory 22 can include both internal storage units and external storage devices. The memory 22 is used to store the computer program and other programs and data required by the device. The memory 22 can also be used to temporarily store data that has been output or will be output.

[0045] Example 3:

[0046] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Embodiment 1.

[0047] The computer-readable medium shown can be any means that can contain, store, communicate, propagate, or transmit a program for use in or in conjunction with an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Additionally, the computer-readable medium can even be paper or other suitable media on which the program can be printed, for example, by optically scanning the paper or other medium, then editing, interpreting, or otherwise processing it as necessary to obtain the program electronically, and then storing it in computer memory.

[0048] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A real-time, high-precision method for recognizing the wearing status of reflective clothing, characterized in that, The method includes: Acquire images of the construction site; The acquired construction site images are input into the target detection network structure model. The backbone network of the target detection network structure model is based on the ResNet50 network structure. Image feature information is extracted from the input images, and then the extracted image feature information is aggregated using a pixel aggregation network to obtain an image feature information map. The image feature information map is then processed by an attention network module for feature information extraction. The feature information extracted by the attention network module is processed by the YOLO algorithm target detection head to obtain a target detection result map with predicted bounding boxes. Based on the target detection result image, calculate whether the pixel overlap between the construction worker and the predicted frame of the reflective vest is lower than the set overlap threshold, in order to determine whether the construction worker has violated the safety operation regulations. The backbone network of the target detection network structure model is based on the ResNet50 network structure, and the image feature information extraction of the input image includes: ResNet50 is used to replace DarkNet-53 of the YOLO series as the backbone network for image feature extraction. The ACON activation function and DCN convolutional layer are used to replace the ReLU activation function and the last three 3x3 convolutional layers of the ResNet50 network structure. The attention network module is connected through a residual network structure, and uses global pooling and fully convolutional layers to extract feature information from the image feature map, resulting in three feature maps of different sizes.

2. The real-time high-precision reflective clothing wearing status recognition method as described in claim 1, characterized in that, The construction site image input into the target detection network structure model has a resolution of 768x768.

3. The real-time high-precision reflective clothing wearing status recognition method as described in claim 2, characterized in that, The target detection network structure model has a maximum batch size parameter setting of 12.

4. The real-time high-precision reflective clothing wearing status recognition method as described in claim 1, characterized in that, The output of the DCN convolutional layer is used as the input to the pixel aggregation network.

5. The real-time high-precision reflective clothing wearing status recognition method as described in claim 1, characterized in that, The target detection head using the YOLO algorithm outputs three feature maps of different sizes obtained from the attention network module. After normalization, three fused maps are obtained. Finally, the optimal bounding box is retained by using non-maximum values ​​to obtain the detection result map of reflective clothing and construction workers with predicted bounding boxes.

6. The real-time high-precision reflective clothing wearing status recognition method as described in claim 1, characterized in that, When the pixel overlap between the construction worker's image and the predicted frame of the reflective vest is lower than the set overlap threshold, the image at this time is saved and an alarm message is sent.

7. A real-time, high-precision reflective clothing wearing status recognition device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 6.

Images