A method and device for detecting whether a safety helmet is worn, and a robot

Abstract

The application provides a safety helmet wearing detection method and device and a robot, and the method comprises the following steps: S1, acquiring an image collected by the robot; S2, dividing the image into N blocks to obtain a divided image block, wherein N is a positive integer greater than 2; S3, inputting the image block into an improved VDSR super-resolution network to reconstruct to obtain a reconstructed image group, wherein the convolution kernel size of the backbone network of a YOLOV5 backbone network is all 3*3, the sampling interval stride is all 1, the convolution network for learning the high-frequency residual between the high-resolution image and the low-resolution image in the VDSR is replaced to form the improved VDSR super-resolution network; and S4, inputting the reconstructed image group into a binary classification YOLOV5 network to detect the safety helmet wearing condition. The application divides the image into blocks and performs super-resolution reconstruction, thereby ensuring the accuracy and the speed.

Description

Technical Field

[0001] This invention relates to the field of robotics, and more specifically, to a method, apparatus, and robot for detecting the wearing of a safety helmet. Background Technology

[0002] Safety helmets are an important measure to protect construction workers. In construction sites and other scenarios where workers are required to wear safety helmets, it is generally mandatory for workers to wear them. However, there are still cases of people not wearing safety helmets. Therefore, it is necessary to promptly identify people who are not wearing safety helmets and issue an alarm.

[0003] The existing technical solutions for detecting whether a safety helmet is being worn mainly include the following two:

[0004] (1) Using a deep learning object detection network and a deep learning classification network to determine whether a safety helmet is being worn: The scheme first acquires the original image from a fixed camera, then performs preprocessing operations on the original image (scaling it to a specific scale and then subtracting the mean and dividing the variance), and then inputs the preprocessed image into the object detection network to detect human heads. After detecting human heads, the human head image is then input into the classification network to classify human heads wearing safety helmets and human heads not wearing safety helmets.

[0005] (2) Using a combined deep learning keypoint detection network and a deep learning object detection network to determine whether a safety helmet is being worn: This scheme first acquires images from a fixed camera, then preprocesses the original images (scaling them to a specific scale and subtracting the mean and dividing by the variance), and then simultaneously inputs the preprocessed images into the keypoint detection network and the object detection network. The object detection network outputs the head position, while the keypoint detection network outputs the helmet position. If the keypoint is on the head, it indicates that a safety helmet is being worn; otherwise, it is considered that no safety helmet is being worn.

[0006] In scenarios like construction sites, cameras are typically installed at a high position, far from workers. This results in heads and safety helmets appearing small in the captured images. In this context, detecting heads first and then classifying them as either those wearing safety helmets or not can lead to missed heads and misclassifications. Conversely, simultaneously using a keypoint network to locate safety helmets and an object detection network to detect heads can result in keypoint localization errors and missed heads.

[0007] The background description provided herein is for the purpose of generally presenting the context of this disclosure. Unless otherwise indicated herein, the material described in this section is not prior art to the claims of this application and should not be acknowledged as prior art by virtue of its inclusion in this section. Summary of the Invention

[0008] To address the aforementioned technical problems in related technologies, this invention proposes a method for detecting the wearing of a safety helmet, comprising the following steps:

[0009] S1, acquire the images collected by the robot;

[0010] S2, divide the image into N blocks to obtain the divided image blocks, where N is a positive integer greater than 2;

[0011] S3, the image block is input into the improved VDSR super-resolution network for reconstruction to obtain the reconstructed image group, wherein the YOLOV5 backbone network with all convolution kernels of 3*3 and all sampling interval stride of 1 is used to replace the convolutional network in VDSR that learns the high-frequency residual between high-resolution and low-resolution images to form the improved VDSR super-resolution network.

[0012] S4, input the reconstructed image group into a binary classification YOLOv5 network to detect whether a helmet is being worn.

[0013] Specifically, step S2 involves: downsampling the image to obtain a downsampled image, and then dividing the downsampled image into N blocks to obtain the divided image blocks.

[0014] Specifically, step S2 further includes: upsampling the segmented image blocks to obtain a first segmented image group.

[0015] Specifically, it also includes step S5, which involves deduplicating the target boxes detected by the reconstructed image group.

[0016] Specifically, step S5 is as follows:

[0017] S51, multiply the coordinates of all detected target boxes by 0.5 to obtain the coordinates of the second target box;

[0018] S52, modify the coordinates of the target boxes obtained from the first image block, the second image block, the third image block, and the fourth image block respectively;

[0019] S53. Traverse all target boxes in the fifth image block and perform the following operations for each target box: Calculate the absolute value of the area difference and the intersection area between the target box and all target boxes in the first, second, third, and fourth image blocks. If the absolute value of the area difference is less than a first preset multiple of the target area and the intersection area is greater than a second preset multiple of the target area, then keep the box; otherwise, delete it. The target boxes in the first, second, third, and fourth image blocks, as well as the remaining target boxes in the fifth image block, are the finally detected target boxes.

[0020] Secondly, another embodiment of the present invention discloses a helmet-wearing detection device, which includes the following units:

[0021] The image acquisition unit is used to acquire images collected by the robot.

[0022] An image segmentation unit is used to divide the image into N blocks to obtain segmented image blocks, where N is a positive integer greater than 2;

[0023] The super-resolution reconstruction unit is used to input the image patch into the improved VDSR super-resolution network for reconstruction to obtain the reconstructed image group. The improved VDSR super-resolution network is constructed by replacing the convolutional network in VDSR that learns the high-frequency residual between high-resolution and low-resolution images with a YOLOV5 backbone network whose kernel size is 3*3 and whose sampling interval stride is 1.

[0024] The helmet detection unit is used to input the reconstructed image group into a binary classification YOLOv5 network to detect whether a helmet is being worn.

[0025] Specifically, the image segmentation unit is configured to: downsample the image to obtain a downsampled image, and then divide the downsampled image into N blocks to obtain segmented image blocks.

[0026] Specifically, the image segmentation unit further includes: upsampling the segmented image blocks to obtain a first segmented image group.

[0027] Specifically, it also includes a target bounding box deduplication unit, used to deduplicate the target bounding boxes detected by the reconstructed image group.

[0028] Thirdly, another embodiment of the present invention discloses a robot, the robot comprising: a processing module, a chassis, a storage module, a lidar, and a camera, wherein the storage module stores instructions, which, when executed, are used to implement the above-described method for detecting the wearing of a safety helmet.

[0029] Fourthly, another embodiment of the present invention provides a non-volatile memory storing instructions that, when executed by a processor, are used to implement the above-described method for detecting the wearing of a safety helmet.

[0030] This invention divides an image into five small blocks, upsamples all of these blocks to the image size required for object detection inference, and then inputs the five upsampled images into an improved VDSR for super-resolution reconstruction. After reconstruction, the five reconstructed images are then input into the object detection network all at once. Upsampling the image blocks magnifies small objects, while super-resolution reconstruction of the upsampled images eliminates the jagged edges and blurring caused by upsampling. Therefore, this invention ensures the accuracy of small object detection. Due to the characteristics of convolutional neural networks, inferring multiple images at once is not only not slower but may even be faster than inferring one image at a time. Therefore, compared to solutions that use a single object detection network, a single classification network, or a single keypoint detection network and a single object detection network to determine whether a helmet is being worn, this invention actually guarantees both accuracy and speed.

[0031] This invention discloses a method for detecting the wearing of safety helmets. A robot carrying a single, inexpensive, 360-degree rotating camera on its head inspects construction sites and other similar locations along a fixed route. The robot uses an improved VDSR super-resolution network to reconstruct clear images from blurry and jagged images, and then employs a two-class YOLOv5 algorithm to detect whether workers are wearing safety helmets. In this embodiment, all convolutional kernels in the YOLOv5 backbone are changed to 3x3 with a stride of 1 (the size of the input image and the output feature map of the modified YOLOv5 backbone are the same). This modified YOLOv5 backbone then replaces the convolutional network in the VDSR that learns the high-frequency residuals between high-resolution and low-resolution images. Furthermore, this invention uses a two-class YOLOv5 algorithm to determine whether workers in construction sites and similar locations are wearing safety helmets. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0033] Figure 1 This is a flowchart of a method for detecting the wearing of a safety helmet provided in an embodiment of the present invention;

[0034] Figure 2This is a schematic diagram of image segmentation provided in an embodiment of the present invention;

[0035] Figure 3 This is the image before super-resolution reconstruction provided in the embodiments of the present invention;

[0036] Figure 4 This is the image before super-resolution reconstruction provided in the embodiments of the present invention;

[0037] Figure 5 This is a schematic diagram of the target width detection result of image0 provided in an embodiment of the present invention;

[0038] Figure 6 This is a schematic diagram of the target width detection result of image1 provided in an embodiment of the present invention;

[0039] Figure 7 This is a schematic diagram of the target width detection result of image2 provided in an embodiment of the present invention;

[0040] Figure 8 This is a schematic diagram of the target width detection result of image3 provided in an embodiment of the present invention;

[0041] Figure 9 This is a schematic diagram of the target width detection result of image4 provided in an embodiment of the present invention;

[0042] Figure 10 This is a schematic diagram of a helmet-wearing detection device provided in an embodiment of the present invention;

[0043] Figure 11 This is a schematic diagram of a helmet-wearing detection device provided in an embodiment of the present invention. Detailed Implementation

[0044] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0045] Example 1

[0046] refer to Figure 1 This embodiment provides a method for detecting the wearing of a safety helmet, which includes the following steps:

[0047] S1, acquire the images collected by the robot;

[0048] The robot in this embodiment is equipped with a camera on its head, which is used to capture images. This camera can rotate 360 ​​degrees. It continuously patrols a fixed route within the construction site, collecting real-time images of the site and other surrounding scenes.

[0049] The robot in this embodiment includes a lidar unit, which is used to acquire laser point cloud data for navigation and positioning. The lidar units are all located on the robot's head.

[0050] When robots use LiDAR for navigation, a map of the target area usually needs to be created beforehand. This embodiment uses a construction site as an example to detect the wearing of safety helmets; therefore, the target area in this embodiment is the construction site. Thus, it is necessary to create a map of the construction site, such as a 3D map. Specifically, mapping methods such as SLAM can be used. In this embodiment, SLAM mapping is used as follows: the robot is manually controlled to travel along the road in the target scene, and the point cloud data collected in each frame is uploaded to the backend. A 3D point cloud map of the target scene is then created using a certain matching and alignment algorithm.

[0051] The robot in this embodiment can transmit images of construction sites and other scenes to a backend server via wireless communication methods such as 5G communication for detecting whether safety helmets are being worn.

[0052] In another embodiment, the robot of this example can perform helmet detection locally. Specifically, the robot can be configured with a processing unit, such as a central processing unit or a graphics processing unit, to perform helmet detection.

[0053] S2, divide the image into N blocks to obtain the divided image blocks, where N is a positive integer greater than 2;

[0054] In this embodiment, the robot's camera captures original images of construction sites and other scenes at a size of 1920*1080. Since the inference input image size for the binary YOLOv5 object detection network used in this embodiment is 640*480, this embodiment first downsamples the original 1920*1080 image to 640*480, and then... Figure 2 The image shown is divided into five smaller blocks, image0 to image4, each with a size of 320*240. After being divided into blocks, the images from image0 to image4 are resized to 640*480 and then grouped together into an image group.

[0055] Specifically, in this embodiment, the image is first downsampled to obtain a downsampled image, and the downsampled image is then divided into N blocks to obtain image blocks.

[0056] Specifically, N in this embodiment is 5. The specific method of segmentation is as follows: the downsampled image is divided into 4 equal parts to obtain a first image block image0, a second image block image1, a third image block image2, and a fourth image block image3; a fifth image block image4 is obtained from the center of the downsampled image, wherein the size of the fifth image block is the same as the size of the first image block.

[0057] Specifically, in this embodiment, the size of the first image block is 1 / 4 of the downsampled image, the length of the first image block is 1 / 2 of the length of the downsampled image, and the width of the first image block is 1 / 2 of the width of the downsampled image.

[0058] Specifically, in this embodiment, the segmented image blocks are upsampled to obtain the first segmented image group.

[0059] S3, the image block is input into the improved VDSR super-resolution network for reconstruction to obtain the reconstructed image group, wherein the YOLOV5 backbone network with all convolution kernels of 3*3 and all sampling interval stride of 1 is used to replace the convolutional network in VDSR that learns the high-frequency residual between high-resolution and low-resolution images to form the improved VDSR super-resolution network.

[0060] In this embodiment, super-resolution reconstruction is performed on the first image block image0, the second image block image1, the third image block image2, the fourth image block image3, and the fourth image block image4 respectively to obtain the reconstructed first image block image0, the second image block image1, the third image block image2, the fourth image block image3, and the fourth image block image4. The reconstructed first image block image0, the second image block image1, the third image block image2, the fourth image block image3, and the fourth image block image4 are then combined to form a reconstructed image group.

[0061] Since images 0 to 4 become blurry and jagged after being upsampled from 320*240 to 640*480, super-resolution technology is needed to reconstruct them into clear, jagged images. This embodiment uses a VDSR network for super-resolution reconstruction. To enhance the reconstruction effect, this embodiment improves the VDSR. Specifically, it replaces the convolutional network in the VDSR that learns the high-frequency residuals between high-resolution and low-resolution images with a modified YOLOv5 backbone. All convolutional kernels in the modified YOLOv5 backbone are 3*3 in size and have a stride of 1, meaning the input and output sizes of the modified YOLOv5 backbone are the same. Therefore, the modified YOLOv5 backbone can be embedded into the VDSR. The training method still uses the VDSR training method. During inference, the five upsampled images are input into the trained and converged modified VDSR model at once.

[0062] refer to Figure 3 as well as Figure 4 ,in Figure 3 It is a pre-reconstruction image extracted from image0 to images4 after the above processing. Figure 4 This is the reconstructed image from this embodiment. As you can see, the image has become much clearer.

[0063] S4, input the reconstructed image group into a binary classification YOLOv5 network to detect the wearing of a safety helmet;

[0064] This embodiment categorizes the targets to be detected into heads wearing helmets and heads not wearing helmets, and uses a two-class YOLOv5 algorithm for detection. The network structure, training method, and YOLOv5 remain unchanged. During inference, five images are fed into the trained and converged two-class YOLOv5 model at a time. The output bounding box format is [x1, y1, x2, y2, id], where x1 and y1 represent the coordinates of the top-left corner of the box, x2 and y2 represent the coordinates of the bottom-right corner of the box, and id represents the category of the target in the box; id = 0 indicates a head wearing a helmet, and id = 1 indicates a head not wearing a helmet.

[0065] Figure 5-9 These are the detection results for the reconstructed images 0 to 4, where the yellow boxes represent heads without safety helmets and the red boxes represent heads with safety helmets.

[0066] This embodiment divides the image into five small blocks, then upsamples all of these blocks to the image size required for object detection inference. The five upsampled images are then input into an improved VDSR for super-resolution reconstruction. After reconstruction, the five reconstructed images are input into the object detection network all at once. Upsampling the image blocks magnifies small objects, while super-resolution reconstruction eliminates the jagged edges and blurring caused by upsampling. Therefore, this embodiment ensures the accuracy of small object detection. Due to the characteristics of convolutional neural networks, inferring multiple images at once is not only no slower than inferring one image at a time, but may even increase the speed. Therefore, compared to schemes that use a single object detection network, a single classification network, or a single keypoint detection network and a single object detection network to determine whether a helmet is being worn, this embodiment ensures both accuracy and speed.

[0067] This embodiment describes a method for detecting whether workers are wearing safety helmets. A robot carrying a single, inexpensive, 360-degree rotating camera on its head inspects construction sites and other similar locations along a fixed route. The robot uses an improved VDSR super-resolution network to reconstruct clear images from blurry, jagged images. Then, a two-class YOLOv5 algorithm is used to detect whether workers are wearing safety helmets. In this embodiment, all convolutional kernels in the YOLOv5 backbone are changed to 3x3 with a stride of 1 (the input image size and output feature map size are the same after the modification). This modified YOLOv5 backbone then replaces the convolutional network in the VDSR that learns the high-frequency residuals between high-resolution and low-resolution images. Furthermore, this embodiment uses a two-class YOLOv5 algorithm to determine whether workers in construction sites and similar locations are wearing safety helmets. Unlike existing technologies that first use an object detection network to detect heads and then use a classification network to classify heads as wearing or not wearing safety helmets, this embodiment views the problem as detecting heads wearing or not wearing safety helmets from an image. In this way, a binary object detection network YOLOv5 can be used to determine whether workers in construction sites and other scenarios are wearing safety helmets.

[0068] Furthermore, this embodiment also includes step S5, which involves deduplicating the target boxes detected by the reconstructed image group;

[0069] Specifically, N0, N1, N2, N3, and N4 target boxes (the number of boxes is greater than or equal to 0) were detected in the five images image0 to image4, respectively. Among them, the target boxes in image4 and image0 to image3 overlapped, so deduplication was required.

[0070] Specifically, step S5 includes:

[0071] S51, multiply the coordinates of all detected target boxes by 0.5 to obtain the coordinates of the second target box;

[0072] The coordinates of all bounding boxes detected in images0 through 4 are multiplied by 0.5, i.e., from [x1, y1, x2, y2, id] to [x1*0.5, y1*0.5, x2*0.5, y2*0.5, id].

[0073] S52, modify the coordinates of the target boxes obtained from the first image block, the second image block, the third image block, and the fourth image block respectively.

[0074] Specifically, in this embodiment, the N1 coordinates in image1 are modified to [x1*0.5+320,y1*0.5,x2*0.5+320,y2*0.5,id]; the N2 coordinates in image2 are modified to [x1*0.5,y1*0.5+240,x2*0.5,y2*0.5+240,id]; the N3 coordinates in image3 are modified to [x1*0.5+320,y1*0.5+240,x2*0.5+320,y2*0.5+240,id]; and the N4 coordinates in image4 are modified to [x1*0.5+180,y1*0.5+120,x2*0.5+180,y2*0.5+120,id].

[0075] S53. Traverse all target boxes in the fifth image block, and perform the following operations for each target box: Calculate the absolute value of the area difference and the intersection area between the target box and all target boxes in the first image block, the second image block, the third image block, and the fourth image block. If the absolute value of the area difference is less than a first preset multiple of the target area and the intersection area is greater than a second preset multiple of the target area, then keep the box; otherwise, delete it. The target boxes in the first image block, the second image block, the third image block, the fourth image block, and the remaining target boxes in the fifth image block are the finally detected target boxes.

[0076] Specifically, in this embodiment, the N4 target boxes of image4 are traversed, and the following operations are performed on each target box:

[0077] ① Calculate the absolute value of the area difference and the intersection area between the target box and the N0, N1, N2, and N3 target boxes in sequence.

[0078] ② If the absolute value of the area difference is less than 0.35 times the target area and the intersecting area is greater than 0.65 times the target area, then keep the box; otherwise, delete it.

[0079] The remaining bounding boxes in image4, along with the N0, N1, N2, and N3 bounding boxes from image0 to image3, constitute the final detected bounding boxes.

[0080] In this embodiment, duplicate target boxes are removed to avoid repetition.

[0081] Example 2

[0082] refer to Figure 9 This embodiment discloses a helmet-wearing detection device, which includes the following units:

[0083] The image acquisition unit is used to acquire images collected by the robot.

[0084] The robot in this embodiment is equipped with a camera on its head, which is used to capture images. This camera can rotate 360 ​​degrees. It continuously patrols a fixed route within the construction site, collecting real-time images of the site and other surrounding scenes.

[0085] The robot in this embodiment includes a lidar unit, which is used to acquire laser point cloud data for navigation and positioning. The lidar units are all located on the robot's head.

[0086] When robots use LiDAR for navigation, a map of the target area usually needs to be created beforehand. This embodiment uses a construction site as an example to detect the wearing of safety helmets; therefore, the target area in this embodiment is the construction site. Thus, it is necessary to create a map of the construction site, such as a 3D map. Specifically, mapping methods such as SLAM can be used. In this embodiment, SLAM mapping is used as follows: the robot is manually controlled to travel along the road in the target scene, and the point cloud data collected in each frame is uploaded to the backend. A 3D point cloud map of the target scene is then created using a certain matching and alignment algorithm.

[0087] The robot in this embodiment can transmit images of construction sites and other scenes to a backend server via wireless communication methods such as 5G communication for detecting whether safety helmets are being worn.

[0088] In another embodiment, the robot of this example can perform helmet detection locally. Specifically, the robot can be configured with a processing unit, such as a central processing unit or a graphics processing unit, to perform helmet detection.

[0089] An image segmentation unit is used to divide the image into N blocks to obtain segmented image blocks, where N is a positive integer greater than 2;

[0090] In this embodiment, the robot's camera captures original images of construction sites and other scenes at a size of 1920*1080. Since the inference input image size for the binary YOLOv5 object detection network used in this embodiment is 640*480, this embodiment first downsamples the original 1920*1080 image to 640*480, and then... Figure 2 The image shown is divided into five smaller blocks, image0 to image4, each with a size of 320*240. After being divided into blocks, the images from image0 to image4 are resized to 640*480 and then grouped together into an image group.

[0091] Specifically, in this embodiment, the image is first downsampled to obtain a downsampled image, and the downsampled image is then divided into N blocks to obtain image blocks.

[0092] Specifically, N in this embodiment is 5. The specific method of segmentation is as follows: the downsampled image is divided into 4 equal parts to obtain a first image block image0, a second image block image1, a third image block image2, and a fourth image block image3; a fifth image block image4 is obtained from the center of the downsampled image, wherein the size of the fifth image block is the same as the size of the first image block.

[0093] Specifically, in this embodiment, the size of the first image block is 1 / 4 of the downsampled image, the length of the first image block is 1 / 2 of the length of the downsampled image, and the width of the first image block is 1 / 2 of the width of the downsampled image.

[0094] Specifically, in this embodiment, the segmented image blocks are upsampled to obtain the first segmented image group.

[0095] The super-resolution reconstruction unit is used to input the image patch into the improved VDSR super-resolution network for reconstruction to obtain the reconstructed image group. The improved VDSR super-resolution network is constructed by replacing the convolutional network in VDSR that learns the high-frequency residual between high-resolution and low-resolution images with a YOLOV5 backbone network whose kernel size is 3*3 and whose sampling interval stride is 1.

[0096] In this embodiment, super-resolution reconstruction is performed on the first image block image0, the second image block image1, the third image block image2, the fourth image block image3, and the fourth image block image4 respectively to obtain the reconstructed first image block image0, the second image block image1, the third image block image2, the fourth image block image3, and the fourth image block image4. The reconstructed first image block image0, the second image block image1, the third image block image2, the fourth image block image3, and the fourth image block image4 are then combined to form a reconstructed image group.

[0097] Since images 0 to 4 become blurry and jagged after being upsampled from 320*240 to 640*480, super-resolution technology is needed to reconstruct them into clear, jagged images. This embodiment uses a VDSR network for super-resolution reconstruction. To enhance the reconstruction effect, this embodiment improves the VDSR. Specifically, it replaces the convolutional network in the VDSR that learns the high-frequency residuals between high-resolution and low-resolution images with a modified YOLOv5 backbone. All convolutional kernels in the modified YOLOv5 backbone are 3*3 in size and have a stride of 1, meaning the input and output sizes of the modified YOLOv5 backbone are the same. Therefore, the modified YOLOv5 backbone can be embedded into the VDSR. The training method still uses the VDSR training method. During inference, the five upsampled images are input into the trained and converged modified VDSR model at once.

[0098] refer to Figure 3 as well as Figure 4 ,in Figure 3 It is a pre-reconstruction image extracted from image0 to images4 after the above processing. Figure 4 This is the reconstructed image from this embodiment. As you can see, the image has become much clearer.

[0099] The helmet detection unit is used to input the reconstructed image group into a binary classification YOLOv5 network to detect whether a helmet is being worn.

[0100] This embodiment categorizes the targets to be detected into heads wearing helmets and heads not wearing helmets, and uses a two-class YOLOv5 algorithm for detection. The network structure, training method, and YOLOv5 remain unchanged. During inference, five images are fed into the trained and converged two-class YOLOv5 model at a time. The output bounding box format is [x1, y1, x2, y2, id], where x1 and y1 represent the coordinates of the top-left corner of the box, x2 and y2 represent the coordinates of the bottom-right corner of the box, and id represents the category of the target in the box; id = 0 indicates a head wearing a helmet, and id = 1 indicates a head not wearing a helmet.

[0101] Figure 5-9 These are the detection results for the reconstructed images 0 to 4, where the yellow boxes represent heads without safety helmets and the red boxes represent heads with safety helmets.

[0102] This embodiment divides the image into five small blocks, then upsamples all of these blocks to the image size required for object detection inference. The five upsampled images are then input into an improved VDSR for super-resolution reconstruction. After reconstruction, the five reconstructed images are input into the object detection network all at once. Upsampling the image blocks magnifies small objects, while super-resolution reconstruction eliminates the jagged edges and blurring caused by upsampling. Therefore, this embodiment ensures the accuracy of small object detection. Due to the characteristics of convolutional neural networks, inferring multiple images at once is not only no slower than inferring one image at a time, but may even increase the speed. Therefore, compared to schemes that use a single object detection network, a single classification network, or a single keypoint detection network and a single object detection network to determine whether a helmet is being worn, this embodiment ensures both accuracy and speed.

[0103] This embodiment describes a method for detecting whether workers are wearing safety helmets. A robot carrying a single, inexpensive, 360-degree rotating camera on its head inspects construction sites and other similar locations along a fixed route. The robot uses an improved VDSR super-resolution network to reconstruct clear images from blurry, jagged images. Then, a two-class YOLOv5 algorithm is used to detect whether workers are wearing safety helmets. In this embodiment, all convolutional kernels in the YOLOv5 backbone are changed to 3x3 with a stride of 1 (the input image size and output feature map size are the same after the modification). This modified YOLOv5 backbone then replaces the convolutional network in the VDSR that learns the high-frequency residuals between high-resolution and low-resolution images. Furthermore, this embodiment uses a two-class YOLOv5 algorithm to determine whether workers in construction sites and similar locations are wearing safety helmets. Unlike existing technologies that first use an object detection network to detect heads and then use a classification network to classify heads as wearing or not wearing safety helmets, this embodiment views the problem as detecting heads wearing or not wearing safety helmets from an image. In this way, a binary object detection network YOLOv5 can be used to determine whether workers in construction sites and other scenarios are wearing safety helmets.

[0104] Furthermore, this embodiment also includes a target bounding box deduplication unit, used to deduplicate the target bounding boxes detected by the reconstructed image group;

[0105] Specifically, N0, N1, N2, N3, and N4 target boxes (the number of boxes is greater than or equal to 0) were detected in the five images image0 to image4, respectively. Among them, the target boxes in image4 and image0 to image3 overlapped, so deduplication was required.

[0106] Specifically, the target box deduplication unit includes:

[0107] The first coordinate acquisition unit is used to multiply the coordinates of all detected target boxes by 0.5 to obtain the coordinates of the second target box;

[0108] The coordinates of all bounding boxes detected in images0 through 4 are multiplied by 0.5, i.e., from [x1, y1, x2, y2, id] to [x1*0.5, y1*0.5, x2*0.5, y2*0.5, id].

[0109] The second coordinate modification unit is used to modify the coordinates of the target boxes obtained from the first image block, the second image block, the third image block, and the fourth image block, respectively.

[0110] Specifically, in this embodiment, the N1 coordinates in image1 are modified to [x1*0.5+320,y1*0.5,x2*0.5+320,y2*0.5,id]; the N2 coordinates in image2 are modified to [x1*0.5,y1*0.5+240,x2*0.5,y2*0.5+240,id]; the N3 coordinates in image3 are modified to [x1*0.5+320,y1*0.5+240,x2*0.5+320,y2*0.5+240,id]; and the N4 coordinates in image4 are modified to [x1*0.5+180,y1*0.5+120,x2*0.5+180,y2*0.5+120,id].

[0111] The deduplication unit iterates through all target boxes in the fifth image block and performs the following operations for each target box: it sequentially calculates the absolute value of the area difference and the intersection area between the target box and all target boxes in the first, second, third, and fourth image blocks. If the absolute value of the area difference is less than a first preset multiple of the target area and the intersection area is greater than a second preset multiple of the target area, the box is retained; otherwise, it is deleted. The target boxes in the first, second, third, and fourth image blocks, as well as the remaining target boxes in the fifth image block, are the finally detected target boxes.

[0112] Specifically, in this embodiment, the N4 target boxes of image4 are traversed, and the following operations are performed on each target box:

[0113] ① Calculate the absolute value of the area difference and the intersection area between the target box and the N0, N1, N2, and N3 target boxes in sequence.

[0114] ② If the absolute value of the area difference is less than 0.35 times the target area and the intersecting area is greater than 0.65 times the target area, then keep the box; otherwise, delete it.

[0115] The remaining bounding boxes in image4, along with the N0, N1, N2, and N3 bounding boxes from image0 to image3, constitute the final detected bounding boxes.

[0116] In this embodiment, duplicate target boxes are removed to avoid repetition.

[0117] Example 3

[0118] This embodiment discloses a robot, which includes: a processing module, a chassis, a storage module, a lidar, and a camera. The storage module stores instructions, which, when executed, are used to implement the helmet-wearing detection method as described in Embodiment 1.

[0119] In another embodiment, a robot includes: a processing module, a chassis, a storage module, a lidar, a camera, and a helmet detection device as described in Embodiment 2.

[0120] Example 4

[0121] refer to Figure 11 , Figure 11 This is a schematic diagram of the structure of a helmet-wearing detection device according to this embodiment. The helmet-wearing detection device 20 of this embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, it implements the steps in the above method embodiments. Alternatively, when the processor 21 executes the computer program, it implements the functions of each module / unit in the above device embodiments.

[0122] For example, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the helmet-wearing detection device 20. For example, the computer program can be divided into the modules shown in Embodiment 2. The specific functions of each module are described in the working process of the device described in the above embodiments, and will not be repeated here.

[0123] The helmet-wearing detection device 20 may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will understand that the schematic diagram is merely an example of the helmet-wearing detection device 20 and does not constitute a limitation on the device. It may include more or fewer components than illustrated, or combine certain components, or use different components. For example, the helmet-wearing detection device 20 may also include input / output devices, network access devices, buses, etc.

[0124] The processor 21 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the helmet-wearing detection device 20, connecting all parts of the device through various interfaces and lines.

[0125] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements various functions of the helmet detection device 20 by running or executing the computer programs and / or modules stored in the memory 22 and calling the data stored in the memory 22. The memory 22 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 22 may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0126] If the integrated module / unit of the helmet-wearing detection device 20 is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by the processor 21, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content contained in the computer-readable medium may be appropriately added to or subtracted from the content as required by the legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium may not include electrical carrier signals and telecommunication signals.

[0127] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Furthermore, in the accompanying drawings of the device embodiments provided by this invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without any creative effort.

[0128] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting helmet wearing, comprising the following steps: S1, acquire the images collected by the robot; S2, divide the image into N blocks to obtain the divided image blocks, where N is a positive integer greater than 2; S3, the image block is input into the improved VDSR super-resolution network for reconstruction to obtain the reconstructed image group, wherein the YOLOV5 backbone network with all convolution kernels of 3*3 and all sampling interval stride of 1 is used to replace the convolutional network in VDSR that learns the high-frequency residual between high-resolution and low-resolution images to form the improved VDSR super-resolution network. S4, input the reconstructed image group into a binary classification YOLOv5 network to detect whether a helmet is being worn.

2. The method according to claim 1, wherein step S2 specifically comprises: downsampling the image to obtain a downsampled image, and dividing the downsampled image into N blocks to obtain the divided image blocks.

3. The method according to claim 2, wherein step S2 further comprises: Upsampling is performed on the segmented image blocks to obtain the first segmented image group.

4. The method according to claim 3 further includes step S5, deduplicating the target boxes detected by the reconstructed image group.

5. The method according to claim 4, wherein step S5 specifically comprises: S51, multiply the coordinates of all detected target boxes by 0.5 to obtain the coordinates of the second target box; S52, modify the coordinates of the target boxes obtained from the first image block, the second image block, the third image block, and the fourth image block respectively; S53. Traverse all target boxes in the fifth image block and perform the following operations for each target box: Calculate the absolute value of the area difference and the intersection area between the target box and all target boxes in the first, second, third, and fourth image blocks. If the absolute value of the area difference is less than a first preset multiple of the area of ​​the target box and the intersection area is greater than a second preset multiple of the area of ​​the target box, then keep the box; otherwise, delete it. The target boxes in the first, second, third, and fourth image blocks, as well as the remaining target boxes in the fifth image block, are the finally detected target boxes.

6. A helmet-wearing detection device, comprising the following units: The image acquisition unit is used to acquire images collected by the robot. An image segmentation unit is used to divide the image into N blocks to obtain segmented image blocks, where N is a positive integer greater than 2; The super-resolution reconstruction unit is used to input the image patch into the improved VDSR super-resolution network for reconstruction to obtain the reconstructed image group. The improved VDSR super-resolution network is constructed by replacing the convolutional network in VDSR that learns the high-frequency residual between high-resolution and low-resolution images with a YOLOV5 backbone network whose kernel size is 3*3 and whose sampling interval stride is 1. The helmet detection unit is used to input the reconstructed image group into a binary classification YOLOv5 network to detect whether a helmet is being worn.

7. The apparatus according to claim 6, wherein the image segmentation unit specifically comprises: downsampling the image to obtain a downsampled image, and dividing the downsampled image into N blocks to obtain segmented image blocks.

8. The apparatus according to claim 7, wherein the image segmentation unit further comprises: Upsampling is performed on the segmented image blocks to obtain the first segmented image group.

9. The apparatus of claim 8 further includes a target bounding box deduplication unit for deduplicating target bounding boxes detected from the reconstructed image group.

10. A robot, the robot comprising: The system includes a processing module, a chassis, a storage module, a lidar, and a camera. The storage module stores instructions, which, when executed, are used to implement the helmet-wearing detection method as described in any one of claims 1-5.

Images