A new method and system for identifying helmet wearing based on human posture estimation

Abstract

This invention discloses a novel method and system for identifying helmet wearing based on human posture estimation, relating to the field of safety management technology. The method includes: acquiring initial images of several individuals; using human posture estimation to obtain joint point information of the human body from the initial images; determining the center and diameter of a positioning circle based on the acquired joint point information, constructing the circumscribed square of the circle, and cropping a head and neck sub-image from the initial images; labeling the head and neck sub-image to obtain a dataset, and inputting the dataset into a constructed small classification network model for model training to obtain a trained classification network model; classifying the personnel to be tested according to the trained classification network model to identify whether the construction worker is wearing a helmet. This invention can identify construction workers in complex postures and against complex backgrounds, correctly distinguishing whether the personnel to be tested are wearing helmets, improving recognition accuracy and success rate.

Description

Technical Field

[0001] This invention relates to the field of safety management technology, and more specifically to a novel method and system for identifying helmet wearing based on human posture estimation. Background Technology

[0002] Safety in production is one of the fundamental guarantees for the development of productivity. With rapid economic development, the number of engineering projects is increasing, making the protection of production safety an increasingly important aspect, as it relates to the personal safety of workers and the vital interests of enterprises. Over the past two decades, the number of accidents in the construction industry has been three times that of all other industries, and the number of accidents and fatalities has generally been increasing year by year. The direct cause of most accidents is the failure to wear safety helmets or improper wearing of them. However, safety management personnel have limited time and resources and cannot ensure that every worker wears a safety helmet at all times.

[0003] Artificial intelligence is developing rapidly. Deep learning and computer vision methods can be used to automatically monitor whether construction workers are wearing safety helmets. This method has many advantages over manual helmet monitoring, including a wider recognition range, lower cost, and higher efficiency. However, current helmet recognition methods face challenges when identifying workers in complex postures and against complex backgrounds, including difficulty in recognition, insufficient accuracy, and the tendency to misclassify other hats as safety helmets.

[0004] Therefore, how to identify construction workers in different environments and conditions, correctly distinguish whether they are wearing safety helmets, and improve the accuracy and success rate of identification are technical problems that urgently need to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a novel helmet wearing identification method and system based on human posture estimation, which can have a good identification effect on the person being tested in different environments and states, and can correctly distinguish whether the person being tested is wearing a helmet.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A novel method for identifying helmet wearing based on human posture estimation includes the following steps:

[0008] Acquire initial images of several individuals, and use human pose estimation to obtain joint information of the individuals from the initial images;

[0009] Based on the obtained joint information of the human body, the center and diameter of the positioning circle are determined, the circumscribed square of the positioning circle is constructed, and the head and neck sub-image of the human body is extracted from the initial image.

[0010] The head and neck sub-image is labeled to obtain a dataset, and the dataset is input into a small classification network model to train the model and obtain the trained classification network model.

[0011] The trained classification network model is used to classify the test subjects and identify whether the construction workers are wearing safety helmets.

[0012] The technical effect of the above solution is that it can solve the problems of existing safety helmet wearing recognition methods, such as difficulty in recognizing construction workers in complex postures and backgrounds, insufficient accuracy, and easy misclassification of other hats as safety helmets. It can improve recognition accuracy and is easy to implement.

[0013] Optionally, obtaining the joint information of the human body specifically involves: collecting initial images of several people through public datasets or street photography, and using OpenPose as the human pose estimation method to estimate the pose of the several people. The specific process is as follows:

[0014] The first ten layers of VGG19 are used to extract features from the initial input image, and the extracted features are processed through a series of multi-stage convolutional neural networks to obtain optimized detection results.

[0015] PCM is used to represent the position information of joints;

[0016] PAF is used to represent the affinity information between joints; joints with high affinity will be connected.

[0017] Connect all the joints in the human body to obtain the necessary information about these joints.

[0018] Optionally, in the series of multi-stage convolutional neural networks, each multi-stage convolutional neural network includes two branches: a first branch for predicting the confidence of the joints and a second branch for predicting the affinity vector field of the joints; except for the first-stage convolutional neural network, the input of each stage convolutional neural network is the fusion of the outputs of the two branches of the previous stage convolutional neural network and the input of the initial image.

[0019] The technical effects achieved by the above technical solution are as follows: OpenPose is a bottom-up human pose recognition method based on supervised learning. It has good recognition ability in complex poses and scenes, and solves many problems of top-down human pose recognition estimation methods in recognizing multiple images.

[0020] Optionally, the step of extracting the head and neck sub-image from the initial image specifically involves: obtaining the coordinate information of the joints in the image based on the joint information of the human body, and selecting an appropriate cutting method based on different joint acquisition situations, which can be divided into the following two cases:

[0021] When the detected joints include the neck, right ear, and left ear, let the coordinates of the neck joint, right ear joint, and left ear joint be N(x0,y0), Re(x1,y1), and Le(x2,y2), respectively. Connect the left ear joint and the right ear joint to form a line segment ReLe. Take the symmetrical point N'(x'0,y'0) of the neck joint N(x0,y0) with line segment ReLe as the axis of symmetry. The right ear joint and the left ear joint can be replaced by the right eye joint and the left eye joint. Connect N(x0,y0) and N'(x'0,y'0) to form a line segment NN'. With the intersection of line segment ReLe and line segment NN' as the center, the circumscribed square of the circle with center c((x'0+x0) / 2,(y'0+y0) / 2) and diameter line segment NN' is used as the head and neck sub-image region for segmentation. The lengths of line segment ReLe and line segment NN' are calculated using the following formulas:

[0022]

[0023]

[0024] When the detected joints are only the neck and left ear, or the neck and right ear, let the coordinates of the neck joint and the ear joint be N(x0,y0) and E(x1,y1) respectively. Connect the neck joint and the ear joint to obtain the line segment NE. Then, the circumscribed square of the circle with the ear joint E(x1,y1) as the center and the line segment NE as the diameter is used as the head and neck sub-image region for segmentation. The formula for calculating the length of the line segment NE is as follows:

[0025]

[0026] The technical effect achieved by the above technical solution is as follows: the improved three-point positioning method is used to extract the head and neck sub-image, which solves the problems of existing technologies being unable to obtain ideal sub-images under complex pose conditions and having complex algorithms, and can cut the head and neck sub-image more accurately.

[0027] Optionally, the constructed small classification network model includes: 2 convolutional layers, 2 pooling layers, 2 dropout layers, and 3 fully connected layers; wherein the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the first fully connected layer, the first dropout layer, the second fully connected layer, the second dropout layer, and the third fully connected layer are connected in sequence.

[0028] The technical effect achieved by the above solution is that the network structures of ResNet and VGG, which are commonly used in target classification problems, are often deeper, wider, and more complex than other network structures. The small classification network constructed by this solution is suitable for the actual situation of this classification task and can improve the speed of the classification model.

[0029] Optionally, obtaining the trained classification network model specifically includes the following steps:

[0030] The head and neck sub-images labeled in the dataset are divided into three categories: those not wearing helmets, those wearing safety helmets, and those wearing other types of hats.

[0031] The dataset is processed using data augmentation methods such as rotation, inversion, and color transformation to obtain an expanded dataset;

[0032] The expanded dataset is input into the constructed small classification network model for model training. Precision, accuracy, recall, and loss are calculated until the model converges, at which point training stops and the trained classification network model is obtained.

[0033] The technical effects achieved by the above technical solution are as follows: adding the category of ordinary hats to the dataset can distinguish between ordinary hats and safety helmets, thereby improving the accuracy of identifying whether the person being tested is wearing a safety helmet; expanding the dataset can prevent the model from overfitting and improve the model's generalization ability.

[0034] The present invention also discloses a novel helmet wearing recognition system based on human posture estimation, comprising: an acquisition module, an interception module, an annotation module, a construction module, a training module, and a recognition module;

[0035] The acquisition module is used to acquire initial images of several people and to obtain joint information of the human body from the initial images using human pose estimation.

[0036] The cropping module determines the center and diameter of the positioning circle based on the acquired joint information of the human body, constructs the circumscribed square of the positioning circle, and crops the head and neck sub-image of the human body from the initial image.

[0037] The annotation module is used to annotate the head and neck sub-image to obtain a dataset;

[0038] The building module is used to build a small classification network model;

[0039] The training module is used to input the dataset into the constructed small classification network model for model training and obtain the trained classification network model.

[0040] The identification module classifies the test personnel according to the trained classification network model and identifies whether the construction workers are wearing safety helmets.

[0041] Optionally, the small classification network model includes: 2 convolutional layers, 2 pooling layers, 2 dropout layers and 3 fully connected layers; wherein the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the first fully connected layer, the first dropout layer, the second fully connected layer, the second dropout layer and the third fully connected layer are connected in sequence.

[0042] Optionally, the human pose estimation is OpenPose, whose network structure includes: a VGG19 network and a series of multi-stage convolutional neural networks; in the series of multi-stage convolutional neural networks, each multi-stage convolutional neural network includes two branches, the first branch is used to predict the confidence of the joints, and the second branch is used to predict the affinity vector field of the joints; except for the first stage convolutional neural network, the input of each stage convolutional neural network is the fusion of the output of the two branches of the previous stage convolutional neural network and the input of the initial image.

[0043] Optionally, the labeled head and neck sub-images in the dataset include three types: those without a helmet, those with a helmet, and those with other types of hats.

[0044] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a novel helmet wearing recognition method and system based on human posture estimation, which has the following beneficial effects:

[0045] (1) This invention obtains key node information of the human body from the image by human pose estimation, uses an improved three-point positioning method to extract the head and neck sub-image of the person to be tested, and classifies the extracted head and neck sub-image to identify whether the construction worker is wearing a safety helmet. This method is easy to implement and has stronger robustness in complex environments.

[0046] (2) This invention uses OpenPose as a method for human pose estimation. It is a bottom-up human pose recognition method for multiple people based on supervised learning. It has good recognition ability in complex poses and scenes and can solve many problems of top-down human pose recognition estimation methods when recognizing multiple people.

[0047] (3) The present invention designs a small classification network model, which is more suitable for the actual situation of this classification task. It can accurately locate the head of the person being tested, reduce the impact of complex background and complex posture of the person being tested on the model training, and make the model more likely to converge.

[0048] (4) The present invention adds a category of ordinary hats to the dataset, which can distinguish between ordinary hats and safety helmets and improve the accuracy of recognition; expanding the dataset can prevent the model from overfitting and improve the model's generalization ability. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0050] Figure 1 A flowchart for a novel helmet wearing recognition method based on human posture estimation;

[0051] Figure 2 This is a diagram of the OpenPose network architecture.

[0052] Figure 3 A schematic diagram of a human joint model;

[0053] Figure 4 A schematic diagram of the cutting result using three-point positioning;

[0054] Figure 5 A schematic diagram of the cutting result for two-point positioning;

[0055] Figure 6 This is a structural diagram of a small classification network model;

[0056] Figure 7 A diagram illustrating the classification of the dataset;

[0057] Figure 8 For training curves;

[0058] Figures 9(a)-9(c) These are the confusion matrices for ResNet18, VGG11, and the network of this scheme, respectively.

[0059] Figure 10 A comparison chart showing the recognition performance of ResNet18, VGG11, and the proposed solution network;

[0060] Figures 11(a)-11(c) These are schematic diagrams illustrating the recognition results for strong light, weak light, and blurred images, respectively.

[0061] Figures 12(a)-12(c) These are schematic diagrams showing the recognition results for a handheld safety helmet, a partially obscured target, and a target in a complex posture, respectively.

[0062] Figure 13This is a structural diagram of a novel helmet-wearing recognition system based on human posture estimation. Detailed Implementation

[0063] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0064] Example 1

[0065] To address the difficulties in identifying safety helmets under complex postures and backgrounds, and the potential for misidentifying other hats as safety helmets, this invention discloses a novel safety helmet wearing identification method based on human posture estimation. Figure 1 As shown, it includes the following steps:

[0066] Initial images of several individuals are acquired, and human pose estimation is used to extract joint information from these images. Based on this joint information, the center and diameter of a positioning circle are determined, and the circumscribed square of this circle is constructed. A head and neck sub-image is extracted from the initial images. The head and neck sub-image is labeled to obtain a dataset, which is then input into a small classification network model for training. The trained classification network model is then obtained. Based on the trained classification network model, the individuals to be tested are classified to identify whether construction workers are wearing safety helmets.

[0067] Human pose estimation determines the location of body parts from a given image or video. Currently, human pose estimation can be categorized into single-person pose estimation and multi-person pose estimation based on the target data, and into top-down and bottom-up human pose estimation based on the detection method. DeepPose directly regresses the coordinates of joints using multiple deep neural networks, but its performance is not ideal due to the complexity of human poses. Flow ConvNet uses convolutional neural networks for human pose estimation, improving the robustness of joint localization. Newell et al. introduced a graph stacked hourglass network to repeatedly acquire joint information from images at different scales, significantly improving accuracy. These are all single-person pose estimation methods, performing well when the image contains only a single person, but are difficult to apply to multiple people. AlphaPose is a top-down multi-person pose estimation method that first extracts high-quality single-person regions and then uses single-person pose estimation to estimate the human pose. DeepCut is a bottom-up multi-person pose estimation method that first uses a convolutional neural network to find all candidate joints and then determines which joint belongs to which person.

[0068] In this embodiment, initial images of several individuals are collected using public datasets or street photography. OpenPose is used as the human pose estimation method to estimate the poses of these individuals. It is a bottom-up human pose recognition method based on supervised learning for multiple people, and it has good recognition capabilities in complex poses and scenes. Top-down human pose estimation methods have many problems when recognizing multiple images. For example, too many people in the image can lead to a sharp increase in time complexity, or recognition failures are likely when people are occluded. As non-rigid objects, the human body is often in complex poses or occluded in construction scenes. In such cases, using top-down human pose estimation methods to recognize people often encounters recognition failures and inaccuracies.

[0069] When the human detector fails, top-down single-person pose estimation methods cannot proceed to the next step of identifying human keypoints. However, the OpenPose method used in this embodiment, as a bottom-up detection method, can effectively solve these problems. (Reference) Figure 2 The OpenPose architecture diagram shown first extracts features from the initial input image using the first ten layers of VGG19, and then processes these features through a series of multi-stage convolutional neural networks. Each multi-stage convolutional neural network includes two branches: the first branch predicts the PCM (part confidence map), representing the confidence of the joints; the second branch predicts the PAM (part affinity map), representing the affinity vector field of the joints. Except for the first-stage convolutional neural network, the input to each stage convolutional neural network is the fusion of the outputs of the two branches of the previous stage convolutional neural network and the input of the initial image. Subsequent stages further optimize the detection results using the information extracted from the previous stages. PCM represents the positional information of the joints, and PAF (part affiliation fifields) represents the affinity information between joints; joints with high affinity are connected. Finally, the joints of the entire human body are connected to obtain the human joint information required by this scheme.

[0070] In recent years, many scholars have studied methods that first locate the head and neck of the human body and then identify whether a helmet is being worn. For example, Geng Zhang et al. located the head and neck by detecting skin color and eyes and mouth, and then identified whether a helmet was being worn; Bo Zhang et al. used the head point set detected by OpenPose to obtain a head and neck sub-image, and then used Faster R-CNN to identify whether the person being tested was wearing a helmet. This method cannot obtain an ideal sub-image when the pose is complex; Wang et al. proposed a three-point localization method to extract the head and neck sub-image, and then used RetinaNet to identify whether the person being tested was wearing a helmet. This method produces a sub-image that is too large, increasing the complexity of the algorithm.

[0071] In this embodiment, OpenPose is used to detect and identify the joints of the person being tested, and the coordinates of the joints in the image can be obtained, such as... Figure 3 As shown, there are a total of 18 joints: nose, neck, right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, left ankle, right eye, left eye, right ear, and left ear. This solution proposes an improved three-point localization method to segment the sub-image based on existing head and neck sub-image localization methods. Using the coordinate information of the joints in the image provided by OpenPose, the required neck, left eye, right eye, left ear, and right ear joints are obtained. In practice, these joints are not always visible; different segmentation methods are used depending on the situation to achieve more accurate segmentation of the head and neck sub-image. Specifically, the following two cases are considered:

[0072] Case 1: The detected joints are the neck, right ear (if there is no right ear, then use the right eye), and left ear (if there is no left ear, then use the left eye). Let the coordinates of the neck joint, right ear joint, and left ear joint be N(x0,y0), Re(x1,y1), and Le(x2,y2), respectively. Connect the left ear joint and the right ear joint to form a line segment ReLe. Take the symmetric point N'(x'0,y'0) of the neck joint N(x0,y0) with line segment ReLe as the axis of symmetry. Connect N(x0,y0) and N'(x'0,y'0) to form a line segment NN'. With the intersection of line segment ReLe and line segment NN' as the center, the circumscribed square of the circle with center c((x'0+x0) / 2,(y'0+y0) / 2) and diameter line segment NN' is used as the head and neck sub-image region for segmentation. The segmentation result is as follows. Figure 4 As shown, the formulas for calculating the lengths of line segments ReLe and NN' are as follows:

[0073]

[0074]

[0075] The second scenario: Only the neck and left ear, or the neck and right ear, are detected as joint points. This typically occurs from a side-facing or side-rear-facing perspective, where only the left or right side of the person being tested is visible. Let the coordinates of the neck joint point and the ear joint point be N(x0,y0) and E(x1,y1), respectively. Connecting the neck joint point and the ear joint point yields line segment NE. The circumscribed square of the circle with the ear joint point E(x1,y1) as its center and line segment NE as its diameter is used as the sub-region of the head and neck for segmentation. The segmentation result is as follows: Figure 5 As shown, the formula for calculating the length of line segment NE is as follows:

[0076]

[0077] After segmenting the head and neck sub-image, target classification is required to determine the wear status of the test subject. ResNet and VGG both show significant performance in target classification, but these network structures often have higher depth, width, and complexity than other network structures. Therefore, considering the actual situation of this classification task, to improve the efficiency of this method, this embodiment designs a small classification network model, which is faster and smaller in size. The specific network structure is as follows: Figure 6 As shown, the model includes: 2 convolutional layers, 2 pooling layers, 2 dropout layers, and 3 fully connected layers; wherein the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the first fully connected layer, the first dropout layer, the second fully connected layer, the second dropout layer, and the third fully connected layer are sequentially connected. In this embodiment, the input size of the image is 224*224. The number of channels, the number of channels, and the size of the two convolutional layers are 3, 16, and 3*3 and 16, 16, and 3*3, respectively. After each convolution, a 2*2 max pooling layer is applied, followed by 3 fully connected layers with inputs and outputs of 50176 and 4096; 4096 and 512; and 512 and 3, respectively. A dropout layer with a coefficient of 0.5 is added between every two fully connected layers to prevent the model from overfitting.

[0078] Furthermore, obtaining the trained classification network model specifically includes the following steps:

[0079] The labeled head and neck sub-images in the dataset are divided into three categories: those not wearing hats, those wearing safety helmets, and those wearing other types of hats. Adding a category for ordinary hats to the dataset can further distinguish between ordinary hats and safety helmets, thereby improving the recognition accuracy.

[0080] The dataset is processed using data augmentation methods such as rotation, inversion, and color transformation to obtain an expanded dataset, which can prevent the model from overfitting and improve the model's generalization ability.

[0081] The expanded dataset is input into the constructed small classification network model for model training. Precision, accuracy, recall, and loss are calculated until the model converges, at which point training stops.

[0082] Each sample in the class of interest is called a positive sample, and the other samples are called negative samples. The formulas for calculating precision, accuracy, and recall are as follows:

[0083]

[0084]

[0085]

[0086] In the formula: A is accuracy, P is precision, R is recall; N TP N represents the number of samples that are predicted to be positive and have positive labels; TN N represents the number of samples that are predicted to be negative and have negative labels; FP N represents the number of samples predicted to be positive but labeled as negative; FN This indicates the number of samples that are predicted to be negative but have positive labels.

[0087] Example 2

[0088] In this embodiment, the process and results of this technical solution are further understood through specific experiments.

[0089] The experimental platform was a Windows 10 64-bit operating system with the following specifications: AMD Ryzen 5 3600 6-Core CPU @ 3.59GHz, 15.9GB of memory, RTX 2060, and Python 3.9.

[0090] A. Datasets and Data Processing

[0091] Datasets are crucial for deep learning, and their quality directly impacts training effectiveness. The dataset used in this embodiment contains 13,680 images. Initial images were first acquired using a publicly available dataset and camera captures. An improved triangulation method was then used to obtain head and neck sub-images, and each image was labeled. For example... Figure 7 As shown, the images are classified into three categories: those without hats, those with helmets, and those with other types of hats. As shown in Table 1, the dataset is randomly divided into training, test, and validation sets in a 7:2:1 ratio. The training set is used to train the parameters of the neural network, the test set is used to test the performance of the currently trained model, and the validation set is not used in training but can be used to verify the final model's performance. To prevent overfitting and improve the model's generalization ability, data augmentation methods such as rotation, inversion, and color transformation are applied to the dataset, expanding the training set to 38,328 images.

[0092] Table 1 Dataset Label Distribution

[0093] Target Category training set test set Validation set Not wearing 2870 815 410 Wear other hats 2422 690 346 Wear a hard hat 4290 1225 613

[0094] B. Experimental Results and Analysis

[0095] In this embodiment, three metrics—precision, accuracy, and recall—are used to measure the model's performance.

[0096] 1) Training parameters and results: In this embodiment, a total of 300 training rounds were performed, and the hyperparameters are shown in Table 2. Figure 8 It can be seen that when the training reaches 170 rounds, the accuracy value basically stabilizes, the training basically converges, the final loss value is about 0.05, and the highest accuracy is about 99.4%.

[0097] Table 2 Hyperparameters

[0098] parameter Value or method Learning rate 0.04 momentum 0.035 Batch size 128 loss function Cross-entropy Training times 300

[0099] 2) Comparison of different helmet classification models: To verify the effectiveness of the proposed method, ResNet18 and VGG11 were trained and validated using the un-segmented dataset, respectively. The results were then compared with the network in this embodiment trained using the segmented dataset. The confusion matrix is ​​an important metric for evaluating the performance of a classification network model in cases of imbalanced datasets and multi-class classification. Figures 9(a)-9(c) These are the confusion matrices for ResNet18, VGG11, and the network of our proposed solution, respectively, where the horizontal axis represents the true label and the vertical axis represents the predicted label.

[0100] The accuracy, precision, and recall of these classification networks on the validation set were calculated using the formula, and the results are shown in Table 3. It can be seen that the accuracy, precision, and recall of the classification network using this scheme all exceed those of ResNet18 and VGG11; in fact, ResNet18 and VGG11 cannot even correctly identify cases where the device is not wearing a mask. This technique can accurately locate the head of the person being tested, reducing the impact of complex backgrounds and complex poses on model training, making the model converge more easily. Figure 10 The image shows a comparison of the recognition performance of ResNet18, VGG11, and the network of this technology. It can be seen that the classification network using this technology is superior to ResNet18 and VGG11 in recognizing helmet wearing.

[0101] Table 3 Comparison of Classification Results

[0102]

[0103]

[0104] 2) Recognition performance in complex scenes: To evaluate the robustness of the model recognition method in this technology, detection experiments were conducted on images with strong light, low light, and blurry images. The recognition results are as follows: Figures 11(a)-11(c) As shown, the results demonstrate that the technology still achieves good recognition results even in complex image conditions. Furthermore, to verify that this technology remains effective in different scenarios, tests were conducted on various individuals. Figure 12(a) shows that the handheld helmet does not affect the recognition results. Figures 12(b)-12(c) This demonstrates that even if the target is partially occluded or in a complex pose, this technology can still provide relatively accurate results.

[0105] Example 3

[0106] This embodiment discloses a novel helmet wearing recognition system based on human posture estimation, such as... Figure 13 As shown, it includes: acquisition module, interception module, annotation module, construction module, training module, and recognition module;

[0107] The acquisition module is used to acquire initial images of several people and to obtain joint information of the human body from the initial images using human pose estimation.

[0108] The cropping module determines the center and diameter of the positioning circle based on the acquired joint information of the human body, constructs the circumscribed square of the positioning circle, and crops the head and neck sub-image of the human body from the initial image.

[0109] The annotation module is used to annotate the head and neck sub-images to obtain the dataset;

[0110] Builder modules are used to build small classification network models;

[0111] The training module is used to input the dataset into the constructed small classification network model for model training and obtain the trained classification network model.

[0112] The identification module classifies the test personnel based on the trained classification network model and identifies whether the construction workers are wearing safety helmets.

[0113] Furthermore, the small classification network model includes: 2 convolutional layers, 2 pooling layers, 2 dropout layers, and 3 fully connected layers; wherein the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the first fully connected layer, the first dropout layer, the second fully connected layer, the second dropout layer, and the third fully connected layer are connected in sequence.

[0114] Furthermore, the human pose estimation is OpenPose, whose network structure includes a VGG19 network and a series of multi-stage convolutional neural networks; in the series of multi-stage convolutional neural networks, each multi-stage convolutional neural network includes two branches, the first branch is used to predict the confidence of the joints, and the second branch is used to predict the affinity vector field of the joints; except for the first stage convolutional neural network, the input of each stage convolutional neural network is the fusion of the output of the two branches of the previous stage convolutional neural network and the input of the initial image.

[0115] Furthermore, the labeled head and neck sub-images in the dataset include three types: those without a helmet, those with a helmet, and those with other types of hats.

[0116] This technology utilizes deep learning and computer vision techniques to study a method for extracting head and neck sub-images using human pose estimation. An improved three-point localization method is proposed to determine the head and neck sub-image region. Three-class classification datasets (without wearing a helmet, wearing a regular hat, and wearing a safety helmet) are created for training. A network classification model is designed, reducing the space and time overhead, and successfully training the model to 99.0% accuracy. Finally, the extracted head and neck sub-images are fed into this classification model for training to determine whether the person being tested is wearing a safety helmet. Experimental results show that the proposed method has good recognition performance for people under different environments and conditions, and can correctly distinguish whether the person is wearing a safety helmet.

[0117] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0118] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A novel method for identifying helmet wearing based on human posture estimation, characterized in that, Includes the following steps: Acquire initial images of several individuals, and use human pose estimation to obtain joint information of the individuals from the initial images; Based on the obtained joint information of the human body, the center and diameter of the positioning circle are determined, the circumscribed square of the positioning circle is constructed, and the head and neck sub-image of the human body is extracted from the initial image. The head and neck sub-image is labeled to obtain a dataset, and the dataset is input into a small classification network model to train the model and obtain the trained classification network model. The trained classification network model is used to classify the test subjects and identify whether the construction workers are wearing safety helmets. Specifically, the step of extracting the head and neck sub-image from the initial image involves: obtaining the coordinate information of the joints in the image based on the joint information of the human body, and selecting an appropriate cutting method based on different joint acquisition situations, which can be divided into the following two cases: When the detected joints include the neck, right ear, and left ear, let the coordinates of the neck joint, right ear joint, and left ear joint be N(x0,y0), Re(x1,y1), and Le(x2,y2), respectively. Connect the left ear joint and the right ear joint to form a line segment ReLe. Take the symmetrical point N'(x'0,y'0) of the neck joint N(x0,y0) with line segment ReLe as the axis of symmetry. The right ear joint and the left ear joint can be replaced by the right eye joint and the left eye joint. Connect N(x0,y0) and N'(x'0,y'0) to form a line segment NN'. With the intersection of line segment ReLe and line segment NN' as the center, the circumscribed square of the circle with center c((x'0+x0) / 2,(y'0+y0) / 2) and diameter line segment NN' is used as the head and neck sub-image region for segmentation. The lengths of line segment ReLe and line segment NN' are calculated using the following formulas: When the detected joints are only the neck and left ear, or the neck and right ear, let the coordinates of the neck joint and the ear joint be N(x0,y0) and E(x1,y1) respectively. Connect the neck joint and the ear joint to obtain the line segment NE. Then, the circumscribed square of the circle with the ear joint E(x1,y1) as the center and the line segment NE as the diameter is used as the head and neck sub-image region for segmentation. The formula for calculating the length of the line segment NE is as follows: The constructed small classification network model includes: 2 convolutional layers, 2 pooling layers, 2 dropout layers, and 3 fully connected layers; wherein the first convolutional layer, the first pooling layer, the second convolutional layer, the second pooling layer, the first fully connected layer, the first dropout layer, the second fully connected layer, the second dropout layer, and the third fully connected layer are connected in sequence.

2. The novel helmet wearing recognition method based on human posture estimation according to claim 1, characterized in that, The acquisition of human joint information specifically involves: collecting initial images of several individuals using public datasets or street photography; and then using OpenPose as the human pose estimation method to estimate the poses of these individuals. The specific process is as follows: The first ten layers of VGG19 are used to extract features from the initial input image, and the extracted features are processed through a series of multi-stage convolutional neural networks to obtain optimized detection results. PCM is used to represent the position information of joints; PAF is used to represent the affinity information between joints; joints with high affinity will be connected. Connect all the joints in the human body to obtain the necessary information about these joints.

3. The novel helmet wearing recognition method based on human posture estimation according to claim 2, characterized in that, In the series of multi-stage convolutional neural networks, each multi-stage convolutional neural network includes two branches: the first branch is used to predict the confidence of the joints, and the second branch is used to predict the affinity vector field of the joints. Except for the first-stage convolutional neural network, the input of each stage convolutional neural network is the fusion of the outputs of the two branches of the previous stage convolutional neural network and the input of the initial image.

4. The novel helmet wearing recognition method based on human posture estimation according to claim 1, characterized in that, The process of obtaining the trained classification network model specifically includes the following steps: The head and neck sub-images labeled in the dataset are divided into three categories: those not wearing helmets, those wearing safety helmets, and those wearing other types of hats. The dataset is processed using data augmentation methods such as rotation, inversion, and color transformation to obtain an expanded dataset; The expanded dataset is input into the constructed small classification network model for model training. Precision, accuracy, recall, and loss are calculated until the model converges, at which point training stops and the trained classification network model is obtained.

5. A novel helmet-wearing recognition system based on human posture estimation, characterized in that, A novel helmet wearing recognition method based on human posture estimation as described in any one of claims 1-4 includes: an acquisition module, an extraction module, an annotation module, a construction module, a training module, and a recognition module; The acquisition module is used to acquire initial images of several people and to obtain joint information of the human body from the initial images using human pose estimation. The cropping module determines the center and diameter of the positioning circle based on the acquired joint information of the human body, constructs the circumscribed square of the positioning circle, and crops the head and neck sub-image of the human body from the initial image. The annotation module is used to annotate the head and neck sub-image to obtain a dataset; The building module is used to build a small classification network model; The training module is used to input the dataset into the constructed small classification network model for model training and obtain the trained classification network model. The identification module classifies the test personnel according to the trained classification network model and identifies whether the construction workers are wearing safety helmets.

6. A novel helmet-wearing recognition system based on human posture estimation according to claim 5, characterized in that, The human pose estimation method is OpenPose, whose network structure includes a VGG19 network and a series of multi-stage convolutional neural networks. In the series of multi-stage convolutional neural networks, each multi-stage convolutional neural network includes two branches: the first branch is used to predict the confidence of the joints, and the second branch is used to predict the affinity vector field of the joints. Except for the first-stage convolutional neural network, the input of each stage convolutional neural network is the fusion of the outputs of the two branches of the previous stage convolutional neural network and the input of the initial image.

7. A novel helmet-wearing recognition system based on human posture estimation according to claim 5, characterized in that, The labeled head and neck sub-images in the dataset include three types: those without a helmet, those with a safety helmet, and those with other types of hats.

Images