Helmet wearing recognition model training method, recognition method and storage medium

Abstract

The present invention discloses a helmet wearing recognition model training method, recognition method and storage medium, wherein the training method includes obtaining an industrial site picture set; labeling the industrial site picture set, wherein the label categories include helmets and workers; dividing the label-labeled industrial site picture set into a training set and a test set; constructing a helmet wearing recognition model framework, including a CNN-based feature extraction network and a GCN-based label classifier to establish a network, performing a dot product operation between the feature vector obtained by the feature extraction network and the label classifier to output a prediction score matrix; training the helmet wearing recognition model based on the training set and the test set to obtain the final helmet wearing recognition model. This solution can correctly and efficiently identify the situation where industrial site workers wear helmets, and provides a reliable and intelligent recognition method for reducing the probability of risk accidents caused by industrial site workers not wearing helmets.

Description

Technical Field

[0001] The present invention relates to the field of safe operation technology, and in particular to a helmet wearing recognition model training method, a recognition method and a storage medium based on a graph convolutional network. Background Art

[0002] Industrial sites are highly dangerous areas due to the complex environment, various construction vehicles, equipment and building materials, and the high frequency of falling objects. Safety helmets are a kind of safety product that can protect the human head, and are particularly important for protecting the safety of personnel in industrial sites. When people are impacted or squeezed by falling objects or hard objects, safety helmets can reduce the impact force, thereby offsetting or reducing the impact on the human head, and play an important role in protecting the lives of people in industrial sites.

[0003] In actual work, even though workers know that wearing a helmet is a good protective effect for their own safety, some workers still do not wear helmets on site, which leads to constant safety accidents. Traditional helmet wearing recognition relies on manual inspection and monitoring detection, which is prone to missed detection and false detection, and is difficult to adapt to helmet wearing recognition work in complex environments.

[0004] In recent years, various deep learning algorithms have been widely used in helmet wearing monitoring and detection. In the document "Research on Helmet Wearing Status Detection Based on ResNet50-SSD", Yue Shiqin et al. proposed a research method for helmet wearing status detection based on ResNet50-SSD. The model is based on the traditional SSD network, uses ResNet-50 instead of VGG-16 as the backbone network of SSD to extract features, and introduces the BN layer in the additional layer. The average precision (mAP) in the helmet wearing status detection task is 80.4%; the Chinese patent application with publication number CN112307984A discloses a helmet detection method and device based on a neural network. It is based on the YOLOv3 algorithm as the basic network, and improves the YOLOv3 structure, achieving a significant improvement in the detection accuracy of helmets and people. Although the above method realizes the intelligent recognition of helmet wearing, the average precision value of the method needs to be improved, which will undoubtedly increase the possibility of risk. Summary of the invention

[0005] The present invention provides a helmet wearing recognition model training method, a recognition method and a storage medium, so as to solve the problem that the recognition accuracy of the existing helmet wearing recognition scheme is not high enough.

[0006] In a first aspect, a method for training a helmet wearing recognition model is provided, comprising:

[0007] Obtain industrial site pictures containing safety helmets and workers under different working scenes and environmental conditions, and perform preprocessing to obtain an industrial site picture set;

[0008] Label a collection of industrial site images, where the label categories include hard hats and workers;

[0009] Divide the labeled industrial site image set into a training set and a test set;

[0010] The framework of the helmet wearing recognition model is constructed, including a CNN-based feature extraction network and a GCN-based label classifier network. The feature vector obtained by the feature extraction network is dot-producted with the label classifier to output a prediction score matrix, and the label classification result is obtained according to the prediction score matrix. Among them, the input of the feature extraction network is the industrial site picture, and the output is the feature vector x∈R D , D is the dimension of the feature vector; the input of the label classifier network is the word embedding vector constructed based on the labels in the industrial site pictures, and the output is the label classifier W∈R C×D , C is the number of label categories;

[0011] The helmet wearing recognition model is trained based on the training set and the test set to obtain the final helmet wearing recognition model, including the feature extraction network and the label classifier therein.

[0012] Furthermore, the feature extraction network is constructed based on a residual convolutional neural network, and the feature image output by the last convolutional pooling layer is converted into a one-dimensional feature vector x∈R through a global maximum pooling layer. D .

[0013] Furthermore, the label classifier establishes a network composed of multiple layers of GCN networks, and the processing formula of each layer of GCN network is expressed as follows:

[0014]

[0015] H 0 =Z∈R C×d

[0016] Among them, H l is the input of the lth layer, that is, the label feature output of the l-1th layer; H l+1 is the label feature output of the lth layer; the input H of the first layer 0 is the word embedding vector constructed based on the labels in the industrial site pictures, d is the number of features of the label; A is the cross-correlation matrix; is a nonlinear activation function; the label feature output by the last layer of the GCN network is the label classifier W∈R C×D .

[0017] Furthermore, the cross-correlation matrix is ​​obtained by the following method:

[0018] Count the number of times the training set label pairs co-occur and obtain the label co-occurrence matrix M∈R C×C , and then use the label co-occurrence matrix to get the conditional probability matrix. The processing formula is as follows:

[0019] P i =M i / N i

[0020] Among them, M represents the label co-occurrence matrix, M i represents all M corresponding to label i in M ij , N i represents the probability of label i appearing in the training set; P i The element P in ij =P(L j |L i ) represents the probability of label j appearing when label i appears;

[0021] The conditional probability matrix is ​​taken as the cross-correlation matrix A.

[0022] Furthermore, in the process of training with the training set, the cross-correlation matrix is ​​processed to prevent overfitting, as follows:

[0023] Randomly select q non-zero elements in the cross-correlation matrix A and set them to 0 to obtain A′;

[0024] Normalize A′ and get use Alternative A in

[0025] During the testing process using the test set, the cross-correlation matrix is ​​not processed to prevent overfitting.

[0026] Furthermore, the prediction score matrix It can be expressed as:

[0027]

[0028] During training, the loss function BCEloss is:

[0029]

[0030] Among them, y c ={0,1}, indicating whether there is a label c in the image; σ represents the sigmoid activation function; represents the prediction score of label c, where label c represents a hard hat or a worker.

[0031] Furthermore, the acquisition of industrial site pictures including safety helmets and workers under different working scenes and environmental conditions includes two acquisition methods:

[0032] The first is to obtain and filter the pictures of industrial sites collected by on-site surveillance cameras of industrial sites. The pictures must include pictures of industrial sites with safety helmets and workers in different working scenes and environmental conditions.

[0033] The second method is to use Internet crawlers to crawl pictures of industrial sites that include safety helmets and workers on the Internet.

[0034] Furthermore, the pretreatment process includes:

[0035] The obtained industrial site pictures are subjected to data enhancement processing by adopting at least one of the following methods: random cropping, rotation, and contrast change, so as to expand the industrial site pictures.

[0036] In a second aspect, a method for identifying wearing a helmet is provided, comprising:

[0037] Obtain pictures of the industrial sites to be inspected;

[0038] Input the image of the industrial site to be detected into the feature extraction network obtained by the helmet wearing recognition model training method as described above to obtain a feature vector;

[0039] Perform a dot product operation on the feature vector and the label classifier obtained by the helmet wearing recognition model training method as described above to obtain a prediction score matrix;

[0040] Based on the prediction score matrix, the label classification results of the industrial site images to be detected are obtained, and labels are annotated in the industrial site images to realize the recognition of helmet wearing.

[0041] In a third aspect, a computer storage medium is provided, which includes a computer program, and when the computer program is executed by a processor, the method for training a helmet wearing recognition model or the method for recognizing helmet wearing as described above is implemented.

[0042] Beneficial Effects

[0043] The present invention proposes a helmet wearing recognition model training method, a recognition method and a storage medium, which utilize a large number of industrial site image data sets, combine the advantages of CNN in image feature extraction and GCN in image label classification, and are superior to the traditional algorithm that only uses a single target detection algorithm to complete image feature extraction and classifier recognition tasks in terms of recognition accuracy and stability of target detection. It can correctly and efficiently recognize the situation of industrial site personnel wearing helmets, and provide a highly reliable intelligent recognition solution to reduce the probability of risk accidents caused by workers not wearing helmets. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0045] Figure 1 It is a framework diagram of a helmet wearing recognition model provided by an embodiment of the present invention;

[0046] Figure 2 is a schematic diagram of conditional probability between two labels provided by an embodiment of the present invention;

[0047] Figure 3 These are some examples of pictures of industrial sites provided by the embodiments of the present invention;

[0048] Figure 4 is a flowchart of a helmet wearing recognition model training provided by an embodiment of the present invention;

[0049] Figure 5 Schematic diagram of the relationship between the training set loss value and the number of iterations provided by an embodiment of the present invention;

[0050] Figure 6 Schematic diagram of the relationship between the test set loss value and the number of iterations provided by an embodiment of the present invention;

[0051] Figure 7 is a schematic diagram of a confusion matrix for image classification provided by an embodiment of the present invention;

[0052] Figure 8 is a relationship diagram between the model precision and recall provided by an embodiment of the present invention;

[0053] Fig. 9 This is an image annotation result diagram provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described in detail below. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other implementation methods obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0055] In view of the problem that the traditional algorithm only uses a single target detection algorithm to complete the image feature extraction and classifier recognition tasks, resulting in insufficient recognition accuracy, the patent of this invention proposes a new intelligent recognition solution for helmet wearing, which is based on the joint implementation of CNN (convolutional neural network) network and GCN (graph convolutional neural network) network. This solution uses the reasonable mapping of correlation and dependency between labels for modeling, and treats each label separately, converting the label classification problem into a binary classification problem, thereby establishing a mutually dependent classifier for each image label. The classifier absorbs information from the label map and applies it to the global image representation to perform the final classification result recognition. The technical solution of the present invention is further explained below in conjunction with specific embodiments.

[0056] Example 1

[0057] This embodiment provides a helmet wearing recognition model training method, such as Figure 4 As shown, including:

[0058] S1: Obtain industrial site images containing safety helmets and workers under different working scenes and environmental conditions, and perform preprocessing to obtain an industrial site image set.

[0059] Specifically, there are two ways to obtain industrial site images: First, obtain multiple industrial surveillance video files collected by on-site surveillance cameras in industrial sites, and use OpenCV tools to decompose them into several single-frame industrial site images in jpg format. Since a video contains a large number of frames, it is inevitable that the scenes and environments of adjacent frames will be similar. Therefore, we set the image to be rotated every 20-50 frames; then these industrial site images are screened, and the screening requires industrial site images containing safety helmets and workers under different working scenes and environmental conditions. Second, considering that the on-site surveillance cameras or surveillance video data of specific workplaces have certain regularities, in order to ensure the richness of the data set, Internet crawlers are used to crawl industrial site images containing safety helmets and workers on the Internet.

[0060] After obtaining the above-mentioned industrial site pictures, data enhancement preprocessing is performed to expand the industrial site pictures. The data enhancement preprocessing method includes at least one of random cropping, rotation, and changing contrast. Figure 3 Shown are pictures of some industrial sites.

[0061] S2: Label a collection of industrial site images, where the label categories include safety helmets and workers.

[0062] Specifically, the Labelimg image labeling tool is used to label the images in the industrial site image collection. The images are divided into two labels: safety helmet (hat) and staff (person). The VOC format is selected as the dataset generation format. The generated label file is in XML format and is named the same as the image.

[0063] In this embodiment, the industrial site image set includes 12182 samples, of which 5360 samples are safety helmets and 6822 samples are workers.

[0064] S3: Divide the labeled industrial site image set into a training set and a test set. When dividing, the number of images is divided according to the ratio of training set: test set = 8:2. Table 1 shows the statistics of the annotation information of the training set and test set in the industrial site image set.

[0065] Table 1 Statistics of annotation information of training set and test set

[0066]

[0067] S4: Build a helmet wearing recognition model framework, such as Figure 1 As shown in the figure, a network is established based on a feature extraction network based on CNN and a label classifier based on GCN. The feature vector obtained by the feature extraction network is dot-producted with the label classifier to output a prediction score matrix, and the label classification result is obtained according to the prediction score matrix. Among them, the input of the feature extraction network is an industrial site picture, and the output is a feature vector x∈R D , D is the dimension of the feature vector; the input of the label classifier network is the word embedding vector constructed based on the labels in the industrial site pictures, and the output is the label classifier W∈R C×D , C is the number of label categories.

[0068] In this embodiment, the Pytorch deep learning framework is used to train the entire model. In order to ensure the accuracy of the prediction results, reduce the amount of computer calculations, and reduce the use of storage space, the image size to be input into the data set is first cropped to 448×448 pixels during the training phase.

[0069] During implementation, the feature extraction network is constructed based on a residual convolutional neural network, and can be specifically constructed based on ResNet50 or ResNet101. In this embodiment, the feature extraction network constructed by ResNet101 is used as an example for explanation.

[0070] The feature extraction network is based on the ResNet101 network. ResNet101 is based on the VGG network structure, and a residual learning module is added to it based on the short-circuit mechanism. The purpose is to reduce the overfitting phenomenon that may occur during the network performance optimization process. The network has up to 101 layers. The residual block is composed of three convolutional layers of sizes 1×1, 3×3 and 1×1, which are connected in sequence and merged with the input. The Rule activation function is added after each layer. The residual module is used for training to establish an effective connection between the input and output, which solves the problem of gradient disappearance or gradient explosion caused by the deepening of the network layers, so that the neural network can maintain its feature expression ability while expanding in depth. Table 2 shows the parameter settings of the ResNet101 network. The input industrial site picture is feature extracted, and the feature image is output. Then, the global maximum pooling inter-layer feature image is converted into a 2048-dimensional feature vector x. The formula for obtaining x is expressed as follows:

[0071] x=f GMP (f ResNet101 (I,θ ResNet101 ))∈R D

[0072] In the formula, f GMP represents the global maximum pooling function, f ResNet101 Represents the function simulated by the residual network, θ ResNet101 Represents the network parameters, and D represents the dimension of 2048, that is, the input image is output as a 2048-dimensional feature vector after convolution and pooling.

[0073] Table 2 ResNet101 network parameter settings

[0074]

[0075] The label classifier network is based on the GCN network, and conditional probability is used to model the correlation and dependency between labels. By learning the graph structure, the label classifier W that is interdependent is learned from the label features based on the mapping function. The label classifier is applied to the image features (i.e., the feature vector x), image recognition is performed, the classification results are obtained, and the classification loss value is finally calculated.

[0076] The unique topological structure of the graph can model any pairwise label relationship. The present invention uses the correlation and dependency between labels in the image and uses a graph convolutional network to build a label classifier. The mapping function is used to learn the label classifier. In this embodiment, a two-layer graph convolutional network is stacked. The input of each graph convolutional network layer l takes the label features of the previous layer as input, and then outputs new label features. The processing formula of a single graph convolutional network layer is as follows:

[0077]

[0078] H 0 =Z∈R C×d

[0079] Among them, H l is the input of the lth layer, that is, the label feature output of the l-1th layer; H l+1 is the label feature output of the lth layer; the input H of the first layer 0 is the word embedding vector constructed based on the labels in the industrial site pictures, d is the number of features of the label; A is the cross-correlation matrix; is a nonlinear activation function; the label feature output by the last layer of the GCN network is the label classifier W∈R C×D .

[0080] Through the graph convolutional network stack, if the input of the first layer is the word embedding vector Z∈R C×d , the last layer outputs a W∈R C×D Image label classifier. In this case, the graph convolutional network learns the graph structure and embeds the label into a word vector Z∈R C×d Represents, C represents the number of label categories; construct a directed graph between these labels, learn a stacked GCN, and map these label representations to a set of interdependent label classifiers W∈R C×D .

[0081] The cross-correlation matrix is ​​obtained by the following method:

[0082] Count the number of times the training set label pairs co-occur and obtain the label co-occurrence matrix M∈R C×C , and then use the label co-occurrence matrix to get the conditional probability matrix. The processing formula is as follows:

[0083] P i =M i / N i

[0084] Among them, M represents the label co-occurrence matrix, M i represents all M corresponding to label i in M ij , N i represents the probability of label i appearing in the training set; P i The element P in ij =P(L j |L i ) represents the probability of label j appearing when label i appears;

[0085] The conditional probability matrix is ​​taken as the cross-correlation matrix A.

[0086] It should be noted that P(L j |L i) represents the probability of label j appearing when label i appears, P(L i |L j ) represents the probability of label i appearing when label j appears, so P(L j |L i ) and P(L i |L j ) are not necessarily equal, that is, the cross-correlation matrix is ​​not necessarily symmetric. Figure 2 Shown is a schematic diagram of the conditional probability between two labels.

[0087] In addition, the absolute number of co-occurrences in the training data and the test data may not be the same, so the cross-correlation matrix obtained from the training set may be overfitted. It is necessary to process the cross-correlation matrix to prevent overfitting, as follows: randomly select q non-zero elements in the cross-correlation matrix A, set them to 0, and obtain A′; normalize A′ to obtain use Alternative A in . However, when using the test set for testing, the cross-correlation matrix is ​​not processed to prevent overfitting.

[0088] S5: The helmet wearing recognition model is trained based on the training set and the test set to obtain the final helmet wearing recognition model, including the feature extraction network and the label classifier therein.

[0089] Prediction score matrix of helmet wearing recognition model It can be expressed as:

[0090]

[0091] The prediction score matrix corresponds to the final label classification result. According to the extraction results of the image features, the feature coordinates of a certain label of the image can be obtained, and the correct label and the label category can be marked using boxes of different colors.

[0092] Specifically, Represents the predicted score of label c, compares the predicted score with the preset score of the real label, and outputs all labels that exceed the preset score as the final label classification result. After obtaining the label classification result, according to the image feature extraction process, the feature coordinates of a certain label in the image can be obtained, and the label name and the label category are marked with different colored boxes. Count the number of hat and person labels in the image. If the number of person labels is greater than the number of hat labels, it means that there is no helmet worn in the image.

[0093] The predicted scores are compared with the original labels of the dataset, and the traditional multi-label classification loss function is used for training. The classification loss value is obtained by relying on the sigmoid activation function. The loss function BCEloss is:

[0094]

[0095] Among them, y c ={0,1}, indicating whether there is a label c in the image; σ represents the sigmoid activation function.

[0096] In this embodiment, the stochastic gradient descent method SGD (Stochastic Gradient Descent) is used to optimize the network model, the momentum rate is set to 0.9, the weight decay is set to 0.0005, the initial learning rate (lr) is set to 0.01, and it decays 10 times every 10 epochs, and a total of 100 epochs are trained.

[0097] Result analysis:

[0098] The training set and test set are respectively input into the above network structure, and the Loss curves obtained after 100 iterations are as follows: Figure 5 and Figure 6 As shown in the figure, the analysis of the results shows that when the iteration reaches 90 times, the Loss value of the training set reaches 2.0%, and when the iteration reaches 70 times, the Loss value of the test set reaches 4.0%. After that, the loss function tends to be stable and the model has fully converged, which proves that the helmet wearing recognition model proposed in this paper is effective and feasible.

[0099] Figure 7 This is the image classification confusion matrix. The confusion matrix is ​​used to further analyze the feature extraction and classifier classification capabilities of the model. The values ​​in the dark area on the diagonal represent the accuracy of each category of classification, and the values ​​in the light area represent the classification error rate.

[0100] Performance Verification:

[0101] Precision is used to determine the accuracy of the prediction results, that is, the total number of correct predictions / total number of samples; Recall indicates how many of all positive samples are identified. The algorithm of the present invention also uses the same method to calculate precision and recall. Precision is P, and recall is R. The calculation formula is as follows:

[0102]

[0103]

[0104] Among them, TP is the number of samples of a certain category (assuming the sample label is 1) that are correctly identified; FP is the number of other samples identified as category 1; FN is the number of samples of category 1 that are incorrectly identified; TN is the number of samples of other categories that are correctly identified. Figure 8 This is the relationship curve between model precision and recall.

[0105] The total number of samples is N, and the average precision AP is defined as follows:

[0106]

[0107] For the label category, first calculate the AP value of each category, and then calculate the average of all label categories according to the number of label types in the data set to get the overall mAP value:

[0108]

[0109] Where N class Indicates how many categories the dataset contains. mAP is the average AP of all label categories. This value is used as the evaluation standard for the entire algorithm. The larger the mAP, the better the algorithm performance. Figure 8 It can be read from the relationship curve between the model precision and the recall rate that the mAP value of the algorithm of the present invention is as high as 85.4%, which is sufficient to illustrate that the method of the present invention can correctly and efficiently identify the situation of personnel wearing safety helmets in industrial sites.

[0110] The results of labeling some images using the above helmet wearing recognition model are as follows: Fig. 9 As shown in the figure, it can be seen that the objects in the picture will be marked with rectangles of different colors according to the label category, and the name of the label category will be displayed above the rectangle. It can be more intuitive to see that the final result of the algorithm is effective in identifying safety helmets.

[0111] Model advancement verification:

[0112] In order to demonstrate the advancement of the method of the present invention, comparative experiments were conducted with the traditional target detection method ResNet50-SSD used for helmet wearing recognition and the basic network ResNet101 of the present invention.

[0113] (1) Evaluation indicators

[0114] In order to evaluate the performance of the above classification model, three indicators are used for evaluation: precision (P), recall (R), and mean average precision (mAP). The calculation formulas of each indicator are consistent with those presented in the algorithm performance verification.

[0115] (2) Comparative analysis

[0116] In order to reduce accidental errors, the three models were verified in the Pycharm environment with the image data set provided by the present invention under the same operating system and hardware configuration. The three methods were used for 4 experiments respectively, and the average values ​​of the indicators of the four experimental results were used as model evaluation indicators. The comparison results are shown in Table 3. The method used in the present invention obtained a score of 83.4% on the P index, which was 2.3% higher than the ResNet50-SSD method and 1.8% higher than the baseline ResNet101; a score of 95.0% was obtained on the R index, which was 2.1% higher than the ResNet50-SSD method and 0.3% higher than the baseline ResNet101; a score of 85.4% was obtained on the mAP index, which was 4.0% higher than the ResNet50-SSD method and 3.1% higher than the baseline ResNet101; therefore, the method of the present invention ensures high detection accuracy in the intelligent recognition of helmet wearing video images while meeting the real-time detection requirements.

[0117] Table 3 Comparison results with other methods

[0118]

[0119] Example 2

[0120] This embodiment provides a method for identifying helmet wearing, which is based on the helmet wearing identification model trained by the helmet wearing identification model training method provided in Example 1 and specifically includes:

[0121] Obtain pictures of the industrial sites to be inspected;

[0122] Input the image of the industrial site to be detected into the feature extraction network obtained by the helmet wearing recognition model training method described in Example 1 to obtain a feature vector;

[0123] Performing a dot product operation on the feature vector and the label classifier obtained by the helmet wearing recognition model training method described in Example 1 to obtain a prediction score matrix;

[0124] Based on the prediction score matrix, the label classification results of the industrial site images to be detected are obtained, and the labels are annotated in the industrial site images to be detected. After obtaining the label classification results, according to the image feature extraction process, the feature coordinates of a certain label in the image can be obtained, and the label name and the label category are marked with different color boxes. The number of hat and person labels in the image is counted. If the number of person labels is greater than the number of hat labels, it means that there is no helmet in the image, and the industrial site image to be detected can be transmitted to the control center for further manual review.

[0125] Example 3

[0126] This embodiment provides a computer storage medium, which includes a computer program. When the computer program is executed by a processor, it implements the helmet wearing recognition model training method as described in Example 1.

[0127] Example 4

[0128] This embodiment provides a computer storage medium, which includes a computer program. When the computer program is executed by a processor, it implements the helmet wearing identification method as described in Example 2.

[0129] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1The steps for the functions specified in one or more boxes.

[0133] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0134] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0135] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A helmet wearing recognition model training method, characterized in that: include: Obtain industrial site pictures containing safety helmets and workers under different working scenes and environmental conditions, and perform preprocessing to obtain an industrial site picture set; Label a collection of industrial site images, where the label categories include hard hats and workers; Divide the labeled industrial site image set into a training set and a test set; The framework of the helmet wearing recognition model is constructed, including a CNN-based feature extraction network and a GCN-based label classifier network. The feature vector obtained by the feature extraction network is dot-producted with the label classifier to output a prediction score matrix, and the label classification result is obtained according to the prediction score matrix. Among them, the input of the feature extraction network is the industrial site picture, and the output is the feature vector x∈R D , D is the dimension of the feature vector; the input of the label classifier network is the word embedding vector constructed based on the labels in the industrial site pictures, and the output is the label classifier W∈R C×D , C is the number of label categories; The helmet wearing recognition model is trained based on the training set and the test set to obtain a final helmet wearing recognition model, including a feature extraction network and a label classifier therein; The label classifier network is built by stacking multiple layers of GCN networks. The processing formula of each layer of GCN network is expressed as follows: H 0 =Z∈R C×d Among them, H l is the input of the lth layer, that is, the label feature output of the l-1th layer; H l+1 is the label feature output of the lth layer; the input H of the first layer 0 is the word embedding vector constructed based on the labels in the industrial site pictures, d is the number of features of the label; A is the cross-correlation matrix; is a nonlinear activation function; the label feature output by the last layer of the GCN network is the label classifier W∈R C ×D ; The cross-correlation matrix is ​​obtained as follows: Count the number of times the label pairs co-occur in the training set to obtain the label co-occurrence matrix M∈R C×C , and then use the label co-occurrence matrix to get the conditional probability matrix. The processing formula is as follows: P i =M i / N i Among them, M represents the label co-occurrence matrix, M i represents all M corresponding to label i in M ij , N i represents the probability of label i appearing in the training set; P i The element P in ij =P(L j |L i ) represents the probability of label j appearing when label i appears; The conditional probability matrix is ​​used as the cross-correlation matrix A; In the process of training with the training set, the cross-correlation matrix is ​​processed to prevent overfitting, as follows: Randomly select q non-zero elements in the cross-correlation matrix A and set them to 0 to obtain A ′ ; A ′ After normalization, we get use Alternative A in During the testing process using the test set, the cross-correlation matrix is ​​not processed to prevent overfitting.

2. The helmet wearing recognition model training method according to claim 1, characterized in that: The feature extraction network is constructed based on the residual convolutional neural network, and the feature image output by the last convolutional pooling layer is converted into a one-dimensional feature vector x∈R through a global maximum pooling layer. D .

3. The helmet wearing recognition model training method according to any one of claims 1 to 2, characterized in that: Prediction score matrix It can be expressed as: During training, the loss function BCEloss is: Among them, y c ={0,1}, indicating whether there is a label c in the image; σ represents the sigmoid activation function; represents the prediction score of label c, where label c represents a hard hat or a worker.

4. The helmet wearing recognition model training method according to claim 1, characterized in that: The acquisition of industrial site images including helmets and workers in different working scenes and environmental conditions includes two acquisition methods: The first is to obtain and filter the pictures of industrial sites collected by on-site surveillance cameras of industrial sites. The pictures must include pictures of industrial sites with safety helmets and workers in different working scenes and environmental conditions. The second method is to use Internet crawlers to crawl pictures of industrial sites that include safety helmets and workers on the Internet.

5. The helmet wearing recognition model training method according to claim 4, characterized in that: The pretreatment process includes: The obtained industrial site pictures are subjected to data enhancement processing by adopting at least one of the following methods: random cropping, rotation, and contrast change, so as to expand the industrial site pictures.

6. A method for identifying helmet wearing, characterized in that: include: Obtain pictures of the industrial sites to be inspected; Inputting the image of the industrial site to be detected into the feature extraction network obtained by the helmet wearing recognition model training method according to any one of claims 1 to 4 to obtain a feature vector; Performing a dot product operation on the feature vector and the label classifier obtained by the helmet wearing recognition model training method according to any one of claims 1 to 4 to obtain a prediction score matrix; Based on the prediction score matrix, the label classification results of the industrial site images to be detected are obtained, and labels are annotated in the industrial site images to be detected, thereby realizing the recognition of safety helmet wearing.

7. A computer storage medium comprising a computer program, characterized in that: The computer program implements the method according to any one of claims 1 to 6 when executed by a processor.

Images