A method and device for identifying safety helmets at construction sites based on an independent training framework
By using complementary feature learning and a hard sample loss function within an independent training framework, the problem of low detection efficiency and low accuracy in safety helmet recognition at construction sites is solved, achieving efficient safety helmet recognition in complex scenarios and improving recognition accuracy and robustness.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA RAILWAY CONSTRUCTION ENGINEERING GROUP
- Filing Date
- 2022-12-09
- Publication Date
- 2026-06-30
AI Technical Summary
Existing technologies for identifying safety helmets at construction sites suffer from low detection efficiency, high cost, and low accuracy. In particular, deep learning models are prone to overfitting and have a high false positive rate in complex scenarios and when the sample distribution is uneven.
A complementary feature learning method based on an independent training framework is adopted, which combines Attention CNN and Attention LSTM models to extract the color and contour information of the safety helmet, and improves model training through a hard sample loss function mechanism to solve the sample imbalance problem.
It improves the accuracy and robustness of safety helmet recognition at construction sites, can automatically identify the wearing status of safety helmets in complex scenarios, saves manpower and resources, reduces misjudgments, and enhances the feature extraction capability of the model.
Smart Images
Figure CN116168430B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image detection and processing technology, specifically to a method and device for identifying safety helmets at construction sites based on an independent training framework. Background Technology
[0002] In the construction industry, wearing safety helmets protects the personal safety of workers on construction sites and effectively prevents accidents. Traditional safety helmet inspections at construction sites mainly rely on manual patrols and mutual supervision and reminders. This method not only requires a lot of manpower but also easily fosters a sense of complacency among workers. Furthermore, due to factors such as worker turnover and the complex environment of construction sites, manual monitoring is prone to oversights, resulting in low inspection efficiency and high costs.
[0003] With the rapid development of computer technology and in-depth research into deep learning, people have begun to use artificial intelligence image recognition technology to replace traditional manual inspection. This method reduces the input of human resources and greatly improves the efficiency of detecting whether construction workers are wearing safety helmets. Hu Tian et al. were the first to achieve intelligent safety helmet detection through algorithm design. Currently, safety helmet recognition technology based on deep learning methods mainly uses candidate region target detection methods. First, candidate regions are identified from the captured photos. Then, a deep learning neural network learns the abstract features within the candidate regions. Finally, an appropriate classifier is selected to determine whether a safety helmet is being worn based on the extracted abstract features. Mainstream neural networks include Convolutional Neural Networks (CNNs), Region Convolutional Neural Networks (R-CNNs), and Attention-based Convolutional Neural Networks (A-CNNs).
[0004] While the methods described above have verified the feasibility of using deep learning to improve the efficiency of safety helmet recognition and detection, safety helmet detection in some complex scenarios (such as complex worker postures, insufficient lighting, and long distances between objects) significantly impacts the system's recognition accuracy. Furthermore, safety helmet recognition in real-world scenarios is a typical imbalanced task (workers not wearing safety helmets as required are a very small minority). Training deep learning models in this environment is prone to overfitting and even produces many false positives, leading to system misjudgments.
[0005] Patent document CN114387495A discloses a method for detecting the wearing of safety helmets in workshop safety production. This method, by utilizing deep learning algorithms, can replace manual safety supervision, possessing all-weather, high-efficiency detection capabilities, and exhibiting excellent and stable detection results with greater adaptability to complex environments. However, this method still cannot solve the problem of uneven sample distribution in real-world safety helmet recognition tasks. Summary of the Invention
[0006] In response to the complex working environment of construction sites and to effectively detect whether workers are wearing safety helmets, this invention provides a method and device for identifying safety helmets on construction sites based on an independent training framework.
[0007] To achieve the above objectives, the technical solution of the present invention is as follows:
[0008] In a first aspect, the present invention provides a method for identifying safety helmets at construction sites based on an independent training framework, the method comprising:
[0009] The images of construction site workers captured are used as input to the complementary feature learning framework. Then, a convolution operation is performed on the input images to obtain the abstract representation information conv of the images. The complementary feature learning framework includes a parallel feature extraction channel Attention CNN model and an Attention LSTM model.
[0010] The convolutional vectors are fed into parallel feature extraction channels, namely the Attention CNN model and the Attention LSTM model, for feature learning.
[0011] The Attention CNN model outputs color information F1 about the helmet, while the Attention LSTM model outputs outline information F2 about the helmet.
[0012] The outputs of the Attention CNN model and the Attention LSTM model are combined into F, where F = [F1, F2], and F is input into the fully connected layer.
[0013] The output of the fully connected layer is input into the support vector machine to determine whether a helmet is being worn.
[0014] Furthermore, the Attention CNN model outputs F1 information about the color of the helmet, including:
[0015] In the Attention CNN model, the conv module first passes through four convolutional modules. After passing through the four convolutional modules, an attention mechanism is set up. Finally, the output of the attention mechanism is taken as the color information F1 of the safety helmet extracted by the Attention CNN model.
[0016] Furthermore, each convolutional module includes 3x3 convolution operations, average pooling, batch normalization, and non-linear function operations.
[0017] Furthermore, the Attention LSTM model's contour information F2 regarding the helmet includes:
[0018] In the Attention LSTM model, global average pooling is first used to downsample the abstract representation information conv. Secondly, two cascaded Attention LSTM modules are used to learn sequence-related information about the helmet's outline. An attention mechanism is then set after the Attention LSTM model to enhance the model's ability to recognize the correlation between sequences. Finally, the output of the attention mechanism is taken as the helmet outline information F2 extracted by the Attention LSTM model.
[0019] Furthermore, before performing the convolution operation on the input image, the image is preprocessed, including image grayscale conversion, data normalization, and image enhancement operations.
[0020] Furthermore, the complementary feature learning framework will be pre-trained using the cross-validation entropy loss function. Once the model begins to converge, it will start mining difficult samples and count the number of times each training sample is trained (c) and the number of times it is misclassified (e).
[0021] After the difficult sample mining is completed, the misclassification rate h of each training sample is calculated; at the same time, the distribution ratio α of the training samples is calculated.
[0022] Modify the loss function based on the calculated h and α for each training sample;
[0023] After the loss function is modified, the complementary feature learning framework will retrain the model on the new loss function.
[0024] Furthermore, the method for calculating the misclassification rate h of each training sample is as follows:
[0025]
[0026] Where e(i) represents the number of times the i-th training sample is misclassified during the pre-training process, and c(i) represents the total number of training iterations;
[0027] The larger h(i) is, the more times the sample is misclassified during the pre-training process, and the more difficult it is to identify the sample. Conversely, the smaller h(i) is, the easier it is to identify the sample.
[0028] Furthermore, the method for calculating the distribution ratio α of the training samples is not:
[0029]
[0030] The proportion of the class to which the i-th training sample belongs to the total number of training samples is such that a larger a indicates a larger number of samples, and vice versa.
[0031] Furthermore, the modified loss function consists of three parts: the sample distribution (1-α), the sample recognition difficulty (h), and the sample loss function entropy log(p(x)), as follows:
[0032] I(x)=-(1-α)*(1-h)*log(p(x))
[0033] For training sample x, 1-α is used to adjust the influence of sample distribution on model training, h is used to adjust the influence of sample recognition difficulty on model training, and log(p(x)) is the ordinary cross-validation entropy.
[0034] Secondly, the present invention provides a construction site safety helmet recognition device based on an independent training framework, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of any of the methods described above.
[0035] Compared with the prior art, the advantages of this invention are as follows:
[0036] 1. For the task of identifying whether construction site workers are wearing safety helmets, this invention proposes a complementary feature learning framework. This framework consists of parallel complementary feature extraction channels. Furthermore, based on the specific characteristics of the safety helmet features, attention mechanisms are introduced into different channels to further help the model focus on learning specific information, improving the model's feature learning ability and enhancing the system's recognition performance. This method effectively addresses the shortcomings of traditional deep learning-based single-feature safety helmet recognition methods.
[0037] 2. Since most workers on construction sites wear safety helmets as required, an imbalanced sample distribution can occur in safety helmet recognition tasks in real-world scenarios. To address this issue, this invention proposes a difficult sample loss function mechanism. This mechanism considers the difficulty of recognizing training samples and the sample distribution ratio, allowing the model to increase its learning on training samples that are difficult to recognize and have a small sample ratio. This alleviates the impact of imbalanced sample distribution on model training from an algorithmic perspective, further enhancing the model's feature extraction capabilities and improving the system's recognition performance. Attached Figure Description
[0038] Figure 1 A flowchart of the construction site safety helmet recognition method based on an independent training framework provided in Embodiment 1 of the present invention;
[0039] Figure 2 A schematic diagram of the components of a complementary feature learning framework;
[0040] Figure 3 Flowchart of the loss function mechanism for hard samples;
[0041] Figure 4 This is a schematic diagram of the components of a convolutional module;
[0042] Figure 5 This is a schematic diagram of the construction site safety helmet recognition device based on an independent training framework provided in Embodiment 1 of the present invention. Detailed Implementation
[0043] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0044] Example 1:
[0045] See figure Figure 1 As shown in the figure, the construction site safety helmet recognition method based on an independent training framework provided in this embodiment mainly includes the following steps:
[0046] The images of construction site workers are taken as input to the complementary feature learning framework. Then, the input images are convolved to obtain the abstract representation information of the images (conv). The complementary feature learning framework includes a parallel feature extraction channel Attention CNN model and an Attention LSTM model.
[0047] The convolutional vectors are fed into parallel feature extraction channels, namely the Attention CNN model and the Attention LSTM model, for feature learning.
[0048] The Attention CNN model outputs color information F1 about the helmet, while the Attention LSTM model outputs outline information F2 about the helmet.
[0049] The outputs of the Attention CNN model and the Attention LSTM model are combined into F, where F = [F1, F2], and F is input into the fully connected layer.
[0050] The output of the fully connected layer is input into the support vector machine to determine whether a helmet is being worn.
[0051] Therefore, this method designs a deep learning complementary feature learning framework to extract the color and outline information of the safety helmet in the photograph, enriching the feature vector from the perspective of the deep model and improving the robustness of the model features.
[0052] As a preferred embodiment of the above method, the image is further preprocessed before the convolution operation is performed on the input image. The preprocessing includes image grayscale conversion, data normalization, and image enhancement operations to improve the visual effect of the image.
[0053] Specifically, the deep model complementary feature learning framework involved in this method has the following structure: Figure 2 As shown, this learning framework consists of two parallel complementary feature extraction channels. Based on the needs of the actual scenario, an attention-based convolutional neural network (Attention CNN) and an attention-based long short-term memory network (Attention LSTM) were designed respectively.
[0054] To fully learn the color information of the safety helmet, this method designs an Attention CNN convolutional neural network. The introduction of this network helps to fully utilize the global feature learning capabilities of CNNs, while the attention mechanism also helps the model focus on learning the more prominent parts of the safety helmet within the region. In the Attention CNN, the convolutional network first passes through four convolutional modules, such as... Figure 4 As shown, each module includes 3*3 convolution operations, average pooling, batch normalization, and non-linear function operations. After four convolutional modules, an attention mechanism is set up, which helps the model focus on learning the darker areas of the helmet. Finally, the output of the attention mechanism is taken as the F1 score of the color information about the helmet extracted by the Attention CNN model.
[0055] Secondly, to fully capture the helmet's contour information, this method designs an Attention LSTM network based on an attention mechanism. The introduction of this network helps to fully utilize the model's sequence correlation characteristics and capture more information related to the helmet's contour. In the Attention LSTM, firstly, global average pooling is used to downsample the abstract representation information (conv), which helps to highlight the helmet's contour information and reduce model computational overhead. Secondly, two cascaded LSTM modules are used to learn sequence-related information about the helmet's contour. An attention mechanism is then set after the LSTM model to increase the model's recognition of the correlation between preceding and following sequences. Finally, the output of the attention mechanism is taken as the helmet contour information F2 extracted by the Attention LSTM model.
[0056] Finally, a fully connected neural network is introduced to linearly fit the features learned from the two parallel complementary feature channels. This involves merging the outputs of the Attention CNN and Attention LSTM into F (F = [F1, F2]), which is then input into the fully connected layer to enhance the non-linearity of the features. Finally, the output of the fully connected layer is fed into a Support Vector Machine (SVM) to determine whether the helmet is being worn. In this way, the designed deep model can obtain richer complementary features of the helmet, which helps to enrich the feature dimensions, improve the feature representation ability of the deep learning model, and thus improve the system's recognition accuracy.
[0057] On the other hand, to address the problem of imbalanced samples in real-world scenarios, this method, based on the designed complementary feature learning framework, incorporates a hard sample loss function mechanism to improve system performance at the algorithm level. The hard sample loss function mechanism is described in the following steps: Figure 3 As shown, firstly, the model is pre-trained based on the designed complementary feature learning framework, and difficult samples (samples that are misclassified many times during training) are mined during the pre-training process; secondly, the difficulty coefficient of the training samples and the sample distribution rate are calculated; finally, the loss function is modified, and the model is retrained using the modified loss function.
[0058] Specifically, firstly, the model is pre-trained and hard sample mining is performed. The model is pre-trained using the traditional CrossEntropy Loss function, as shown in the following formula:
[0059] I(x)=-log(P(x))
[0060] Once the model begins to converge (i.e., the model initially possesses discrimination ability), the difficult samples are mined, that is, the number of times each training sample is trained (c) and the number of times it is misclassified (e) during the pre-training process are counted.
[0061] Secondly, after the difficult sample mining is completed, the misclassification rate h of each training sample is calculated, as shown in the following formula:
[0062]
[0063] Where e(i) represents the number of times the i-th training sample is misclassified during the pre-training process, and c(i) represents the total number of training iterations. The calculated h(i) represents the difficulty of recognizing the training sample. The larger h(i) is, the more times the sample is misclassified during the pre-training process, and the more difficult it is to recognize. Conversely, the smaller h(i) is, the easier the sample is to recognize, requiring less model attention. Simultaneously, the sample distribution rate α is calculated (for example, if the i-th training sample belongs to someone wearing a helmet, and the entire training dataset has 1000 photos, of which 500 people are wearing helmets, then αi = 1 / 2), as shown in the formula below:
[0064]
[0065] That is, the proportion of the category to which the i-th training sample belongs to the total number of training samples. The larger the a, the more samples there are, and vice versa.
[0066] Based on the calculated h and α for each training sample, the loss function is modified, and the modified loss function is shown in the following formula:
[0067] I(x)=-(1-α)*(1-h)*log(p(x))
[0068] The modified loss function consists of three parts: the sample distribution (1-α), the sample recognition difficulty (h), and the sample loss function entropy log(p(x)). For training sample x, 1-α is used to adjust the influence of the sample distribution on model training, h is used to adjust the influence of the sample recognition difficulty on model training, and log(p(x)) is the ordinary cross-validation entropy.
[0069] Finally, the modified loss function is used to retrain the deep learning model until it converges. The modified loss function helps the deep model learn from training samples that are few in number and difficult to identify, improving the model's feature learning ability and effectively avoiding overfitting caused by imbalanced samples.
[0070] In summary, this application provides a method that can effectively improve the accuracy of safety helmet recognition at construction sites based on deep learning. This method can automatically identify the helmet wearing status of workers in complex scenarios, saving significant manpower and resources, and is highly feasible and portable. Specifically, compared to traditional deep learning schemes based on single features, the complementary feature learning framework proposed in this application can learn richer safety helmet-related information, exhibiting stronger robustness in helmet recognition tasks under complex scenarios. Secondly, addressing the common sample imbalance problem in helmet recognition tasks, the difficult sample loss function mechanism designed in this application can effectively mitigate the impact of sample distribution on model training at the algorithm level, maximizing the feature extraction capabilities of the deep model and further improving the helmet wearing status recognition performance at construction sites.
[0071] Example 2:
[0072] See Figure 5 As shown, the construction site safety helmet recognition device based on an independent training framework provided in this embodiment includes a processor 51, a memory 52, and a computer program 53 stored in the memory 52 and executable on the processor 51, such as a construction site safety helmet recognition program based on an independent training framework. When the processor 51 executes the computer program 53, it implements the steps of Embodiment 1 described above, for example... Figure 1 The steps are shown.
[0073] For example, the computer program 53 can be divided into one or more modules / units, which are stored in the memory 52 and executed by the processor 51 to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 53 in the construction site safety helmet recognition device based on an independent training framework.
[0074] The construction site safety helmet recognition device based on an independent training framework can be a computing device such as a desktop computer, laptop, handheld computer, or cloud server. The construction site safety helmet recognition device based on an independent training framework may include, but is not limited to, a processor 51 and a memory 52. Those skilled in the art will understand that... Figure 5 This is merely an example of a construction site safety helmet recognition device based on an independent training framework, and does not constitute a limitation on the construction site safety helmet recognition device based on an independent training framework. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the construction site safety helmet recognition device based on an independent training framework may also include input / output devices, network access devices, buses, etc.
[0075] The processor 51 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.
[0076] The memory 52 can be an internal storage unit of the construction site safety helmet recognition device based on the independent training framework, such as a hard drive or memory of the device. The memory 52 can also be an external storage device, such as a plug-in hard drive, SmartMedia Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device. Furthermore, the memory 52 can include both internal and external storage units. The memory 52 is used to store the computer program and other programs and data required by the construction site safety helmet recognition device. The memory 52 can also be used to temporarily store data that has been output or will be output.
[0077] The above embodiments are merely illustrative of the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent changes or modifications made based on the essence of the content of the present invention should be covered within the scope of protection of the present invention.
Claims
1. A method for identifying safety helmets at construction sites based on an independent training framework, characterized in that, The method includes: The images of construction site workers captured are used as input to the complementary feature learning framework. Then, a convolution operation is performed on the input images to obtain the abstract representation information conv of the images. The complementary feature learning framework includes a parallel feature extraction channel Attention CNN model and an Attention LSTM model. The convolutional vectors are fed into parallel feature extraction channels, namely the Attention CNN model and the Attention LSTM model, for feature learning. The Attention CNN model outputs color information F1 about the helmet, and the Attention LSTM model outputs outline information F2 about the helmet. The outputs of the Attention CNN model and the Attention LSTM model are combined into F, where F = [F1, F2], and F is input into the fully connected layer. The output of the fully connected layer is input into a support vector machine to determine whether a helmet is being worn. The Attention LSTM model outputs contour information F2 about the helmet, including: In the Attention LSTM model, global average pooling is first used to downsample the abstract representation information conv. Secondly, two cascaded Attention LSTM modules are used to learn sequence-related information about the helmet's outline. An attention mechanism is then set after the Attention LSTM model to enhance the model's ability to recognize the correlation between sequences. Finally, the output of the attention mechanism is taken as the helmet's outline information F2 extracted by the Attention LSTM model.
2. The construction site safety helmet recognition method based on an independent training framework as described in claim 1, characterized in that, The Attention CNN model outputs F1 information about the color of the helmet, including: In the Attention CNN model, the conv module first passes through four convolutional modules. After passing through the four convolutional modules, an attention mechanism is set up. Finally, the output of the attention mechanism is taken as the color information F1 of the safety helmet extracted by the Attention CNN model.
3. The construction site safety helmet recognition method based on an independent training framework as described in claim 2, characterized in that, Each convolutional module includes 3x3 convolution operations, average pooling, batch normalization, and non-linear function operations.
4. The construction site safety helmet recognition method based on an independent training framework as described in claim 1, characterized in that, Before performing convolution on the input image, the image is preprocessed, including image grayscale conversion, data normalization, and image enhancement operations.
5. The construction site safety helmet recognition method based on an independent training framework as described in claim 1, characterized in that, The complementary feature learning framework will use cross-validation entropy loss function for pre-training, and start to dig difficult samples when the model starts to converge, and count the number of training times c of each training sample i and the number of misclassification times e i ; After the difficult sample mining is completed, the misclassification rate h for each training sample is calculated. i Simultaneously, calculate the distribution ratio α of the training samples. i ; Based on the calculated h for each training sample i ɑ i Modify the loss function; After the loss function is modified, the complementary feature learning framework will retrain the model on the new loss function.
6. The construction site safety helmet recognition method based on an independent training framework as described in claim 5, characterized in that, The misclassification rate h for each training sample is calculated. i The method is as follows: h i The larger the value of h, the more times the sample is misclassified during pre-training, making the sample more difficult to identify; conversely, the smaller the value of h, the more likely the sample is to be misclassified. i The smaller the value, the easier the sample is to identify.
7. The construction site safety helmet recognition method based on an independent training framework as described in claim 6, characterized in that, The distribution ratio α of the calculated training samples i The method is as follows: The proportion of the category to which the i-th training sample belongs to the total number of training samples, α i The larger the value, the larger the sample size; conversely, the smaller the value, the smaller the sample size. i The smaller the value, the smaller the sample size.
8. The construction site safety helmet recognition method based on an independent training framework as described in claim 7, characterized in that, The modified loss function consists of three parts: the sample distribution 1-α, the sample recognition difficulty h, and the sample loss function entropy log(p(x)), as follows: For training sample x, 1-α is used to adjust the influence of sample distribution on model training, h is used to adjust the influence of sample recognition difficulty on model training, and log(p(x)) is the ordinary cross-validation entropy.
9. A construction site safety helmet recognition device based on an independent training framework, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.