Rapid safety helmet detection method

Abstract

The invention belongs to the field of target detection, and particularly relates to a rapid safety helmet detection method. Comprising the following steps: a data processing step: acquiring image information by using a multi-modal sensor, constructing a diversified training data set, labeling data, and converting the labeled data into standard YOLO format data; a model optimization step: calculating data features by using SPD-Cov, constructing a safety helmet wearing recognition model by using GhostNetV2, training the model by using the training set, testing the trained safety helmet wearing recognition model by using the test set, and optimizing the safety helmet wearing recognition model according to a test result; the model reasoning step is used for deploying the safety helmet wearing recognition model to the mobile terminal equipment, and collecting and detecting the safety helmet wearing condition of the workshop worker through the mobile terminal equipment. According to the scheme, the problems of low detection accuracy and low detection speed when whether the workshop worker wears the safety helmet or not is detected are solved.

Description

Technical Field

[0001] The present invention belongs to the field of target detection, and specifically relates to a rapid helmet detection method. Background Art

[0002] In a digital workshop, it is very important whether the workers wear safety helmets. If they do not wear safety helmets, it will cause safety hazards. Therefore, safety helmets must be worn in digital workshops. The main method used to check whether workers are wearing safety helmets is manual inspection. Inspectors are deployed at each entrance of the workshop. This method is not all-weather and can easily lead to missed inspections, which brings many safety hazards to the workshop.

[0003] In recent years, artificial intelligence technology has been applied to various industries. From the current actual results, deep learning algorithms are superior to existing manual or traditional machine learning methods. In the field of object detection based on deep learning, there are roughly two models, namely two-stage and one-stage models. The detection process of the former is divided into two steps: first, the algorithm generates several candidate boxes, and then classifies the candidate boxes through CNN; the latter directly regresses the category probability and position coordinates of the target, which has a relative loss of accuracy, but is faster than the two-stage algorithm.

[0004] The YOLO (You Only Look Once) series of algorithms belong to the one-stage target detection algorithm mode, while YOLOv5 (one of the YOLO series of algorithms) has problems such as low detection accuracy and poor real-time detection during the detection process. In order to provide a high real-time and high-accuracy detection method when detecting whether workshop workers are wearing helmets, Chinese patent CN115995093A proposes a helmet detection method based on the improved YOLOv5: the output layer adds a 9*9 feature scale, uses the softpool method as the pooling method, and uses GIOU_Loss as the loss function. However, the backbone network of the method proposed by Chinese patent CN115995093A still uses DarkNet53, resulting in insufficient feature extraction, and does not consider lightweight design, resulting in slow detection speed. Although Chinese patent CN115995093A adds a feature scale (a 9*9 feature scale is added to the output layer), which can improve small target detection to a certain extent, the softpool method is only a simple pooling operation, and does not extract features, which will introduce noise, resulting in inaccurate detection results. The model used in Chinese patent CN115995093A has a large amount of calculation and a slow operation speed on mobile devices. Therefore, when detecting whether workshop workers are wearing safety helmets, there are still problems of low detection accuracy and slow detection speed. Summary of the invention

[0005] The purpose of this scheme is to provide a rapid safety helmet detection method to solve the problems of low detection accuracy and slow detection speed when detecting whether workshop workers are wearing safety helmets.

[0006] In order to achieve the above object, this scheme provides a rapid helmet detection method, comprising the following steps: The data processing step is used to collect image information using multimodal sensors, build a diverse training data set, annotate the data, and convert the annotated data into standard YOLO format data; The model optimization step is used to replace the original pooling layer with SPD-Cov, and use GhostNetV2 to build a helmet wearing recognition model, then train it with the training set, and test the trained helmet wearing recognition model with the test set, and then optimize the helmet wearing recognition model according to the test results; The model reasoning step is used to deploy the helmet wearing recognition model to the mobile device, and collect and detect the helmet wearing status of the workshop workers through the mobile device.

[0007] The principle and effect of this scheme are: the multimodal sensor used in this scheme can adapt to different lighting conditions and collect more accurate image data. SPD-Cov can model data on symmetric positive definite manifolds. Compared with traditional pooling layers, it can capture more complex structures and relationships of data. At the same time, SPD-Cov can better describe the relationship between different regions in the image, thereby improving the ability to distinguish different categories. This scheme uses SPD-Cov to replace the original pooling layer, which can greatly improve the processing accuracy of the helmet wearing recognition model. GhostNetV2 is a lightweight convolutional neural network (CNN) designed for mobile applications, aiming to provide faster reasoning speed. GhosNeIV2 introduces a hardware-friendly attention mechanism called DFC attention. This attention mechanism is built on a gold connection layer. It is designed to be fast on general hardware and can capture long-distance dependencies between observations. The introduction of DFC attention helps GhostNeV2 better capture global information without significantly reducing the reasoning speed. At the same time, GhosNetV2 adopts a new network architecture for mobile applications. The basic building block of GhosINeTV2 is Ghost Block, which is composed of two stacked Ghost modules. This GhostBlock adopts a reverse bottleneck structure. The first Ghost module acts as an expansion layer to increase the output channel efficiency, and the second Ghost module reduces the number of channels to match the shortcut path. This structure helps to improve the abstraction ability and representation quality of features. The GhostNetV2 architecture is specially designed for mobile terminals to provide faster inference speed and higher computing efficiency. It adopts a lightweight design and can run efficiently on mobile devices with limited computing resources.

[0008] In summary: This solution improves the data collection accuracy in the data processing step, and replaces the original pooling layer with SPD-Cov in the model optimization step, which improves the data processing accuracy, thereby improving the detection accuracy of this solution; this solution uses the lightweight module GhostNetV2 in the backbone network part to implement a high-performance lightweight image classification solution on mobile devices. By introducing DFC attention and optimized GhostBlock, the GhostNeV2 architecture successfully improves the performance and efficiency of the model. It can be seen that this solution solves the problem of low detection accuracy and slow detection speed when detecting whether workshop workers are wearing safety helmets.

[0009] Furthermore, the multimodal sensor includes visible light and infrared cameras.

[0010] Visible light cameras can capture very clear image details; infrared cameras do not need to rely on visible light, which makes infrared cameras have better imaging effects at night or in low light environments. At the same time, infrared rays have a certain penetrating ability and can penetrate some thin materials such as smoke, fog, plastic film, etc., which enables infrared cameras to capture clear images under harsh environmental conditions. The combination of visible light and infrared cameras can enable multimodal sensors to collect more accurate image data during the day, at night, in special weather conditions, and other environments.

[0011] Furthermore, the training data set covers helmets of various colors and shapes.

[0012] By including data on helmets of various colors and shapes, the model can learn the features corresponding to different colors and shapes, so that it can more accurately identify various types of helmets. Rich data can help the model better understand the feature boundaries of helmets and reduce the probability of misidentifying other objects as helmets or missing real helmets. For example, avoid mistaking a red bucket for a red helmet, or miss a partially obscured helmet in a complex background.

[0013] Furthermore, after collecting the image information, deep learning technology is used to perform background segmentation and foreground detection on the image information.

[0014] Deep learning technology can accurately separate foreground objects from the background in an image, which makes the analysis of specific targets more accurate. Object recognition, behavior analysis, image retrieval and other tasks can benefit from accurate foreground detection. Through background segmentation, the scene structure of the image can be better understood. Understanding the characteristics and patterns of the background helps to distinguish different scene types, providing richer contextual information for subsequent image analysis and processing, and is more conducive to accurately annotating the torso features and motion features of human motion.

[0015] Furthermore, when using SPD-Cov to calculate data features, the original convolution is improved as follows: Consider any intermediate feature map X XX of size S × S × C 1 S × S × C1S×S×C1, and cut out a series of sub-feature maps as follows: These sub-feature maps are then connected along the channel dimension to obtain a feature map X ′ X'X′, whose spatial dimension is reduced by a scaling factor and the channel dimension is increased by a scaling factor 2 22. That is, SPD converts the feature map X ( S , S ,C 1 ) X(S,S,C_1)X(S,S,C1) into an intermediate feature map X ′ ( S / sacle , S / sacle, scale 2 C 1 )X'(S / sacle,S / sacle,scale^2C_1)X′(S / sacle,S / sacle,scale2C1).

[0016] By splitting and connecting sub-feature graphs, the new feature graph X′ can incorporate more local information, thereby enhancing the ability to express data features, which helps improve the performance of the model in subsequent tasks. This improvement makes the structure of the feature graph more suitable for SPD-Cov calculations. SPD-Cov models data on symmetric positive definite manifolds and requires a specific feature graph structure to effectively calculate the covariance matrix. The adjustment of the improved feature graph X′ in spatial and channel dimensions can better meet the calculation requirements of SPD-Cov and improve the accuracy and efficiency of the calculation.

[0017] Furthermore, when annotating data, the features of the helmet, the features of the head and the torso are annotated. When the helmet is above the head, the number of pixels occupied by the helmet and the head are collected respectively, and the ratio of the number of pixels of the helmet to the number of pixels of the head is calculated. When the ratio is within the preset range of the occupancy ratio, the standard wearing is taken as the wearing result of the helmet; when annotating data, the annotated torso features are collected, and the torso features are combined with the torso structure of the human body to obtain the human body structure corresponding to the torso features, and it is determined whether the human body structure is complete. If the human body structure is complete, the human body weight is inferred through the human body structure, and the volume of the head is inferred through the weight of the human body, and the volume of the head and the size of the helmet are combined. Combined with the above, the occupancy ratio of the human head when wearing a safety helmet is inferred as the preset occupancy ratio, and whether the ratio of the number of pixels of the safety helmet to the number of pixels of the human head is within the preset occupancy ratio. If so, standard wearing is taken as the wearing result of the safety helmet; if not, the human body structure and torso features are combined to determine whether the hands in the features are in contact with the head or the safety helmet. If not, irregular wearing is taken as the wearing result of the safety helmet; if so, the judgment time is extended, and continuous monitoring is performed to determine whether the hands are in contact with the head or the safety helmet. If the hands are in contact with the head or the safety helmet for a continuous contact time that exceeds the preset prescribed time, avoiding wearing of the safety helmet is taken as the wearing result.

[0018] This solution makes a comprehensive judgment by combining the pixel ratio of the helmet and the head and the estimated ratio calculated based on the human body structure information, and no longer relies on a single judgment method. The pixel ratio can be analyzed from the intuitive coverage of the helmet and the head presented in the image, while the estimated ratio is inferred from deeper human characteristics related factors such as human body structure, weight, and volume. This multi-dimensional consideration makes the judgment of the helmet wearing result more comprehensive and accurate, reducing the possibility of misjudgment. At the same time, in the judgment process, further combining the torso characteristics and whether the hand is in contact with the head or the helmet, it can better adapt to various wearing states that may appear in actual scenes. For example, when the pixel ratio and the estimated ratio judgment results are inconsistent, by observing the contact between the hand and the head or the helmet, it can be distinguished whether it is due to improper wearing (such as wearing the helmet crookedly but the hand is not in contact for a long time to adjust) or there is a behavior of evading wearing (such as deliberately covering the helmet with the hand or fiddling with the helmet for a long time without wearing it properly), so as to more accurately determine the wearing result. This plan also clearly stipulates the classification of helmet wearing results in different situations, such as standard wearing, irregular wearing, evasion wearing, etc., so that there can be clear and definite definition standards for helmet wearing during the supervision process, which is convenient for taking corresponding measures later. When there is a suspected evasion of wearing (that is, the continuous contact time between the hand and the head or the helmet exceeds the preset time), the method of extending the judgment time and continuous monitoring can more accurately identify the true evasion of wearing the helmet, avoid misjudgment due to some short-term normal actions (such as occasionally raising the hand to adjust the hair and touching the helmet), thereby ensuring the fairness and rationality of the implementation of supervision measures.

[0019] Furthermore, when the wearing result is not a standard wearing of a safety helmet, if the pixel positions and quantity change ratios occupied by the facial features in the head features of the image information collected by the multimodal sensor do not exceed the preset facial features change ratio, the shooting frequency of the multimodal sensor is increased, and the time span of image acquisition is increased; the pixel positions and quantity change ratios occupied by the facial features in the image information are obtained again as the correction ratio. If the correction ratio and the change ratio obtained for the first time do not exceed the preset facial features change ratio, and the similarity between the correction ratio and the change ratio obtained for the first time is higher than the preset similarity, the object that does not wear it in a standard manner is judged as non-human, and the wearing result is ignored.

[0020] When it is initially determined that the helmet is worn in a standard manner, by further paying attention to the change ratio of the pixel position and number occupied by the facial features in the head features, a more detailed screening can be performed on the seemingly normal wearing situation. Even if the helmet wearing performance meets the standards, the pixel performance of the facial features of a person will have a certain dynamic change law during normal activities. If this change ratio does not exceed the preset value, further in-depth monitoring will be carried out, which will help to discover some abnormal situations that may be covered by the initial standard wearing judgment, such as the situation where a non-human object such as a fixed puppet model is used to pretend to wear a helmet correctly. At the same time, this scheme increases the shooting frequency of the multimodal sensor and the time span of image acquisition to obtain the correction ratio and compare it with the change ratio obtained for the first time. If both do not exceed the preset facial features change ratio and the similarity is high, this further strengthens the non-human suspicion judgment. This multi-round, multi-angle comparative analysis greatly improves the accuracy of recognition, avoids misjudging non-human objects as human individuals wearing helmets normally due to a single standard wearing judgment, and effectively eliminates the interference of such abnormal situations on subsequent analysis and supervision.

[0021] Furthermore, when annotating data, if no helmet feature is identified above the head feature identified in the image information, the edge contour of the head feature is identified to determine whether the edge contour of the head feature is circular. If it is circular, not wearing a helmet is taken as the wearing result; if it is not circular, the image information is used as reference data, the shooting frequency of the multimodal sensor is increased, and the time span of image acquisition is increased. The acquired image information is used as corrected image information, and the pixel position and number of the head feature and torso feature in the corrected image information are compared with the reference data to determine whether the pixel position and number of the head feature and torso feature in the corrected image are changed. The edge contour of the changed head feature is determined to be circular. If it is circular, it is determined whether there is a helmet feature above the head feature. If not, not wearing a helmet is taken as the wearing result; if there is a helmet feature, the head feature, the helmet feature and the torso feature are combined for processing, and the helmet wearing result is output.

[0022] When the head feature is recognized but the helmet feature is not recognized, this solution determines whether the edge contour of the head feature is circular. If it is circular, it can be known that the head feature is completely recognized and collected. Combined with the fact that the helmet feature is not recognized above the head, it can be concluded that the person is not wearing a helmet. When the recognized edge contour is not circular, the head feature is not completely collected (for example, there is an obstruction between the head and the multimodal sensor, resulting in the incomplete collection of the head feature). At this time, it is impossible to judge the wearing result. Increasing the shooting frequency of the multimodal sensor and increasing the time span of image collection can collect more image data of this person. When this person moves (when the position in the image information changes, or when the movement is small, the corresponding number of pixels will also change), the head feature is collected and judged. When the head feature with a circular edge contour is collected (the head feature is completely collected when the person moves), the wearing result corresponding to the head feature is judged again. This solution verifies the head features (whether they are fully collected) through the edge contour information in the head features, avoiding misjudgment of wearing results caused by incomplete head feature collection. At the same time, after increasing the shooting frequency of the multimodal sensor and the time span of image collection, this solution determines whether the person is moving by whether the pixel positions of the head features and torso features of the image information change, and collects the complete head features of the person during the movement of the person, so as to make a more accurate judgment on the person's wearing results. It can be seen that this solution reduces misjudgments while improving the accuracy of the judgment results. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 The figure is a flow chart related to a rapid helmet detection method.

[0024] Figure 2 This is a schematic diagram of the process of using SPD-Cov to calculate data features in this embodiment.

[0025] Figure 3 Schematic diagram of the blocks in GhostNetV1 and GhostNetV2 in the embodiment of this scheme. DETAILED DESCRIPTION

[0026] The following will clearly and completely describe the concept and technical effects of the present invention in combination with the embodiments, so as to fully understand the purpose, features and effects of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative work are all within the scope of protection of the present invention: A fast helmet detection method, such as Figure 1 As shown, the following steps are included: 1) Data processing step: used to collect image information using multimodal sensors (including visible light and infrared cameras) (i.e. Figure 1 ), build a diverse training dataset (covering helmets of various colors and shapes), and annotate the data (i.e. Figure 1 The labeled data is converted into standard YOLO format data.

[0027] After collecting image information, deep learning technology is used to perform background segmentation and foreground detection on the image information.

[0028] 2) Model optimization step: used to calculate data features using SPD-Cov (i.e. Figure 1 The accuracy of the yolov8 model in is improved, and GhostNetV2 is used to build a helmet wearing recognition model (i.e. Figure 1 The lightweight improvement of yolov8 model in is then used to train it, and the trained helmet wearing recognition model is tested with the test set, and then the helmet wearing recognition model is optimized according to the test results.

[0029] Among them, SPD-Cov is used to calculate data features, such as Figure 2 As shown, the original convolution is improved as follows: Consider any intermediate feature map X XX of size S × S × C 1 S × S × C1S×S×C1, and cut out a series of sub-feature maps as follows: These sub-feature maps are then connected along the channel dimension to obtain a feature map X ′ X'X′, whose spatial dimension is reduced by a scaling factor and the channel dimension is increased by a scaling factor 2 22. That is, SPD converts the feature map X ( S , S ,C 1 ) X(S,S,C_1)X(S,S,C1) into an intermediate feature map X ′ ( S / sacle , S / sacle, scale 2 C 1 )X'(S / sacle,S / sacle,scale^2C_1)X′(S / sacle,S / sacle,scale2C1).

[0030] GhostNetV2 used in building the helmet wearing recognition model is a lightweight convolutional neural network (CNN) designed for mobile applications, aiming to provide faster inference speed. It includes the following key points: 1. Hardware-friendly DFC attention mechanism: GhostNeV2 introduces a hardware-friendly attention mechanism called DFC attention. This attention mechanism is built on a gold connection layer. It is designed to be fast on general hardware and can capture long-range dependencies between observations. The introduction of DFC attention helps GhostNeV2 better capture global information without significantly reducing the inference speed. 2. GhostNetV2 Architecture: GhosNetV2 uses a new network architecture for mobile applications.

[0031] The main features of the GhostNetV2 architecture include the following points: 1. Ghost Block: The basic building block of GhosINeTV2 is Ghost Block, which is composed of two stacked Ghost modules. This Ghost Block adopts a reverse bottleneck structure. The first Ghost module acts as an expansion layer to increase the output channel efficiency, and the second Ghost module reduces the number of channels to match the shortcut path. This structure helps to improve the abstraction ability and representation quality of features.

[0032] 2. The GhostNetV2 architecture is specifically designed for mobile devices to provide faster inference speed and higher computing efficiency. It adopts a lightweight design and can run efficiently on mobile devices with limited computing resources.

[0033] Figure 3 Schematic diagram of the blocks in GhostNetV1 and GhostNetV2, including the reverse residual bottleneck of two Ghost modules, where DFC attention enhances the expanded features to improve expressiveness.

[0034] Figure (a): A block of GhostNet is built by stacking two Ghost modules. Similar to MobileNetV2, it is also a reverse bottleneck, that is, the first Ghost module acts as an expansion layer to increase the number of output channels, while the second Ghost module reduces the number of channels to match the fast path. Figure (b): Schematic diagram showing the bottleneck of GhostV2. DFC attention is used in parallel with the first Ghost module to enhance the extended features. The enhanced features are then sent to the second Ghost module to produce output features, which capture the long-range dependencies between pixels in different spatial locations and enhance the expressiveness of the model. This structure helps improve the performance and feature representation of the horizontal model.

[0035] 3) Model reasoning step: used to deploy the helmet wearing recognition model to mobile devices, and collect and detect the helmet wearing status of workshop workers through mobile devices.

[0036] When annotating data, the features of the helmet, head and torso are annotated. When the helmet is above the head, the number of pixels occupied by the helmet and the head are collected respectively, and the ratio of the number of pixels of the helmet to the number of pixels of the head is calculated. When the ratio is within the preset occupancy ratio range, the standard wearing is regarded as the wearing result of the helmet. When annotating data, the annotated torso features are collected, and the torso features are combined with the human torso structure to obtain the human body structure corresponding to the torso features, and whether the human body structure is complete is judged. If the human body structure is complete, the human body weight is inferred through the human body structure, and the head volume is inferred through the human body weight. The head volume and the size of the safety helmet are combined to infer the occupancy ratio of the head when wearing the safety helmet as the preset occupancy ratio, and it is judged whether the ratio of the number of pixels of the safety helmet to the number of pixels of the head is within the preset occupancy ratio. If so, standard wearing is taken as the wearing result of the safety helmet; if not, the human body structure and torso features are combined to judge whether the hand in the feature is in contact with the head or the safety helmet. If not, the irregular wearing is taken as the wearing result of the safety helmet; if so, the judgment time is extended, and the hand is continuously monitored to judge whether it is in contact with the head or the safety helmet. If the hand is in contact with the head or the safety helmet for a continuous contact time exceeding the preset prescribed time, the avoidance of wearing the safety helmet is taken as the wearing result.

[0037] When the wearing result is not a standard wearing of a safety helmet, if the pixel positions and quantity change ratios occupied by the facial features in the head features of the image information collected by the multimodal sensor do not exceed the preset facial features change ratio, the shooting frequency of the multimodal sensor is increased, and the time span of image acquisition is increased; the pixel positions and quantity change ratios occupied by the facial features in the image information are obtained again as the correction ratio. If the correction ratio and the change ratio obtained for the first time do not exceed the preset facial features change ratio, and the similarity between the correction ratio and the change ratio obtained for the first time is higher than the preset similarity, the object that does not wear it in a standard manner is judged as non-human, and the wearing result is ignored.

[0038] When annotating data, if no helmet feature is identified above the head feature identified in the image information, the edge contour of the head feature is identified to determine whether the edge contour of the head feature is circular. If it is circular, not wearing a helmet is taken as the wearing result; if it is not circular, the image information is used as reference data, the shooting frequency of the multimodal sensor is increased, and the time span of image acquisition is increased. The acquired image information is used as corrected image information, and the pixel position and number of the head feature and torso feature in the corrected image information are compared with the reference data to determine whether the pixel position and number of the head feature and torso feature in the corrected image have changed. The edge contour of the changed head feature is determined to be circular. If it is circular, it is determined whether there is a helmet feature above the head feature. If not, not wearing a helmet is taken as the wearing result; if there is a helmet feature, the head feature, the helmet feature and the torso feature are combined for processing, and the helmet wearing result is output.

[0039] In the specific implementation, a multimodal sensor combining visible light and infrared cameras is first installed and deployed in the digital workshop. The multimodal sensor is used to collect image information of workers passing through the deployment location, and the workers' head features and safety helmet features in the image information are annotated, and the safety helmet wearing recognition model is used for judgment.

[0040] After the deployment is completed, the visible light and infrared cameras collect image information and annotate the image information, and the helmet wearing recognition model extracts the head features and helmet features in the image information.

[0041] The safety helmet wearing recognition model recognizes that worker A is wearing a safety helmet on his head, and determines that worker A is wearing a safety helmet. The ratio of the number of pixels in the head portrait features of worker A to the number of pixels in the safety helmet is 40%, which is within the preset occupancy ratio range (35%-50%). Therefore, the wearing result of worker A is judged to be wearing a safety helmet in a standard manner.

[0042] The safety helmet wearing recognition model recognizes that worker F is wearing a safety helmet on his head, but the ratio of the number of pixels in the head features of worker F to the number of pixels in the safety helmet is 52%, which is not within the preset occupancy ratio range (35%-50%). Combining the torso features and human body structure of worker F, it is inferred that the weight of worker F is between 230-250 kilograms, and the ratio of the volume of worker F's head when wearing a safety helmet is between (45%-60%). The actual measured ratio is 52%, so it is judged that worker F is wearing a safety helmet in a standard manner.

[0043] The helmet wearing recognition model recognizes that worker G is wearing a helmet on his head, but the proportion of the number of pixels in worker G's head features to the number of pixels in the helmet is 60%, which is not within the preset occupancy ratio range (35%-50%). The torso features and human body structure of worker F are combined to infer that the weight of the main worker F is between 140-160 catties, and the proportion of worker G's head volume when wearing a helmet is between (35%-50%). The actual measured proportion is 60%, which does not meet the proportion of wearing a helmet in accordance with the regulations. The torso features and human body structure of worker G are combined to determine that worker G's hands are in contact with the helmet, and the contact time exceeds the preset prescribed time (10 seconds), and it is determined that worker G evades wearing a helmet. When worker G holds the helmet, the helmet does not fit his head, resulting in the proportion of pixels between the helmet and the head features not meeting the wearing specifications. Worker G insists on holding the helmet for 10 seconds. In actual work, the worker's hands will not be in contact with the normally worn helmet for too long. It can be judged from the behavior of worker G that contradicts common sense that worker G evades wearing a helmet. When the worker G's helmet is too large, it does not meet the wearing specifications. At this time, the pixel ratio of the helmet and the head features of worker G may not be within the preset occupancy ratio range. However, the size of the helmet is limited when worn, and there will not be a large gap between the helmet and the head of worker G. Worker G can still wear it firmly without holding the helmet with his hands.

[0044] The helmet wearing recognition model recognizes that worker H is not wearing a helmet, so the shooting rate of the multimodal sensor is increased, and the time span of image acquisition is increased, so that image information of worker H is collected more frequently and for a longer time. Through comparison, it is found that worker H's expression remains unchanged during the 15 seconds when passing by the camera, and he does not even blink. A normal person will not have such an abnormal situation. It can be judged that worker H is not a human being. Worker H may be an image of a human head or a model similar to a human.

[0045] The safety helmet wearing recognition model recognizes that worker J is not wearing a safety helmet on his head, so the collected image of worker J's head contour is reviewed. It is verified that the edge of worker J's head is not round, and it is judged that worker J's head is blocked, so the shooting rate of the multimodal sensor is increased, and the time span of image acquisition is increased, so that the image information of worker H is collected more frequently and for a longer time. Through comparison, it is found that when worker J is moving, a certain image information collects the head features of worker J with a circular edge contour, and it is judged that there is a safety helmet feature above the head of worker J in this image information. After judgment, the head features and safety helmet features of worker J are consistent with the standard wearing, and the standard wearing is taken as the wearing result of worker J.

[0046] The safety helmet wearing recognition model recognizes that there is worker B behind worker A at the current moment, but worker B's head is blocked by worker A, so it is impossible to determine whether worker B is wearing a safety helmet. Therefore, this time is used as the tracing time node, and the image information collected within the past 5 seconds (within the preset fixed time) is pulled. Based on the torso features and motion features of worker B recognized at this time, the missing torso features and motion features of worker B are searched in the image information, and the torso features and motion features of worker B are completed based on the found information. Based on the complete torso features of worker B, the safety helmet wearing recognition model determines that worker B is wearing a safety helmet.

[0047] The safety helmet wearing recognition model recognizes that worker C is also blocked by worker A. The safety helmet wearing recognition model infers the position of worker C's head based on the collected torso features of worker C, and infers the position of the safety helmet when it is properly worn and the appearance of the safety helmet based on the head position. The inferred safety helmet position is then used to search for a safety helmet with the same color and similar appearance in the image information. After inference and verification by the safety helmet wearing recognition model, it is determined that worker C is wearing a safety helmet.

[0048] The helmet wearing recognition model identified that workers D and E were not wearing helmets, so the time when workers D and E were not wearing helmets was used as the tracing time node to pull the image information collected within 5 seconds. After re-testing, it was found that the torso characteristics of worker D were similar to those of the human body, but worker D was stationary, and the motion characteristics did not conform to the torso characteristics of the human body. In addition, the surface temperature of the torso collected by the infrared camera was lower than the temperature of the human body, and there was no obvious difference in the surface temperature of each part of the torso. Therefore, worker D was determined to be a mechanical tool in the vehicle space. Worker E was found to have similar torso characteristics of the human body. Worker D was stationary, but the surface temperature of the torso collected by the infrared camera was very similar to the temperature of the human body, and the surface temperature difference of each part of the torso was similar to that of the human body. Therefore, worker E was continuously tested. After 2 minutes, it was found that the motion characteristics and torso characteristics of worker E were consistent with the characteristics of the human body, so it was determined that worker E was not wearing a helmet, and relevant information was displayed, and a sound prompt was issued at the same time.

[0049] The helmet wearing recognition model recognized that workers A and C were wearing helmets. Two seconds later, it recognized that workers A and C were not wearing helmets. Therefore, the action features, helmet features, and torso features of workers A and C in the two judgment intervals were judged. After re-testing, other parts of worker A's torso did not come into contact with the helmet. During the separation of the helmet from the head, the torso and head did not come into contact with the helmet. It was determined that worker A did not wear the helmet properly, so a reminder sound was issued to wear the helmet properly. During the separation of the helmet from the head, the torso of worker C came into contact with the helmet. It was determined that worker C took the initiative to take off the helmet, so a reminder sound was issued to wear the helmet as required.

[0050] The above is only an embodiment of the present invention, and the common knowledge such as the known specific structure and characteristics in the scheme is not described in detail here. It should be pointed out that for those skilled in the art, several deformations and improvements can be made without departing from the structure of the present invention, which should also be regarded as the protection scope of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A rapid helmet detection method, characterized in that: The following steps are involved: The data processing step is used to collect image information using multimodal sensors, build a diverse training data set, annotate the data, and convert the annotated data into standard YOLO format data; The model optimization step is used to calculate data features using SPD-Cov, and use GhostNetV2 to build a helmet wearing recognition model, then train it using the training set, and use the test set to test the trained helmet wearing recognition model, and then optimize the helmet wearing recognition model based on the test results; The model reasoning step is used to deploy the helmet wearing recognition model to the mobile device, and collect and detect the helmet wearing status of the workshop workers through the mobile device.

2. A rapid helmet detection method according to claim 1, characterized in that: The multimodal sensor includes visible light and infrared cameras.

3. A rapid helmet detection method according to claim 2, characterized in that: The training dataset covers helmets of various colors and shapes.

4. A rapid helmet detection method according to claim 3, characterized in that: After collecting image information, deep learning technology is used to perform background segmentation and foreground detection on the image information.

5. A rapid helmet detection method according to claim 4, characterized in that: When using SPD-Cov to calculate data features, the original convolution is improved as follows: Consider any intermediate feature map XXX of size S×S×C1S×S×C1S×S×C1, and cut out a series of sub-feature maps as follows: These sub-feature maps are then connected along the channel dimension to obtain a feature map X'X'X', whose spatial dimension is reduced by a scaling factor and whose channel dimension is increased by a scaling factor of 222; that is, SPD converts the feature map X(S,S,C1)X(S,S,C_1)X(S,S,C1) into an intermediate feature map X'(S / sac le,S / sacle,scale2C1)X'(S / sac le,S / sacle,scale^2C_1)X'(S / sac le,S / sacle,scale2C1).

6. A rapid helmet detection method according to claim 5, characterized in that: When annotating data, the features of the helmet, head and torso are annotated. When the helmet is above the head, the number of pixels occupied by the helmet and the head are collected respectively, and the ratio of the number of pixels of the helmet to the number of pixels of the head is calculated. When the ratio is within the preset occupancy ratio range, the standard wearing is regarded as the wearing result of the helmet. When annotating data, collect annotated torso features, combine the torso features with the human torso structure to obtain the human body structure corresponding to the torso features, and judge whether the human body structure is complete. If the human body structure is complete, infer the human body weight through the human body structure, infer the head volume through the human body weight, combine the head volume with the size of the helmet, infer the occupancy ratio of the head when wearing the helmet as the preset occupancy ratio, and judge whether the ratio of the number of pixels of the helmet to the number of pixels of the head is within the preset occupancy ratio. If so, use the standard wearing as the result of wearing the helmet; If not, the human body structure and torso features are combined to determine whether the hand is in contact with the head or the helmet. If not, improper wearing is taken as the result of wearing the helmet. If in contact, the judgment time is extended to continuously monitor and determine whether the hand is in contact with the head or the helmet. If the contact time between the hand and the head or the helmet exceeds the preset prescribed time, evasion of wearing the helmet is taken as the wearing result.

7. A rapid helmet detection method according to claim 6, characterized in that: When the wearing result is not a standard wearing of a safety helmet, if the pixel positions and quantity change ratios occupied by the facial features in the head features of the image information collected by the multimodal sensor do not exceed the preset facial features change ratio, the shooting frequency of the multimodal sensor is increased, and the time span of image acquisition is increased; the pixel positions and quantity change ratios occupied by the facial features in the image information are obtained again as the correction ratio. If the correction ratio and the change ratio obtained for the first time do not exceed the preset facial features change ratio, and the similarity between the correction ratio and the change ratio obtained for the first time is higher than the preset similarity, the object that does not wear it in a standard manner is judged as non-human, and the wearing result is ignored.

8. A rapid helmet detection method according to claim 7, characterized in that: When annotating data, if no helmet feature is identified above the head feature identified in the image information, the edge contour of the head feature is identified to determine whether the edge contour of the head feature is circular. If it is circular, not wearing a helmet is taken as the wearing result; if it is not circular, the image information is used as reference data, the shooting frequency of the multimodal sensor is increased, and the time span of image acquisition is increased. The acquired image information is used as corrected image information, and the pixel position and number of the head feature and torso feature in the corrected image information are compared with the reference data to determine whether the pixel position and number of the head feature and torso feature in the corrected image have changed. The edge contour of the changed head feature is determined to be circular. If it is circular, it is determined whether there is a helmet feature above the head feature. If not, not wearing a helmet is taken as the wearing result; if there is a helmet feature, the head feature, the helmet feature and the torso feature are combined for processing, and the helmet wearing result is output.

Images