Production safety detection method and system

Abstract

The invention discloses a production safety detection method and system, and the method specifically comprises the following steps: (1), collecting a head image of an operator before entering a production field, inputting the head image into a first safety detection model, and detecting whether the corresponding operator regularly wears a helmet or not through the first safety detection model; and (2) after entering the production field, acquiring an operation image of the operator, inputting the operation image into a second safety detection model, and detecting whether the operator has non-standard operation or not by the second safety detection model. The images collected by the camera in real time are input into the safety detection model, and the safety detection model can timely and accurately detect the operation eyes which do not normatively wear the helmet and the non-normative operation of the operator.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision technology, and more specifically, the present invention relates to a production safety detection method and system. Background Art

[0002] When performing strip shearing processing, large-scale machinery is mostly used for strip shearing. The operation of the machinery may cause cuts, entanglement, flying materials and other situations to the operators. The operators are exposed to personal safety issues such as mechanical injuries, object strikes, and lifting injuries.

[0003] In order to improve the personal safety of on-site construction workers, it is necessary to standardize the production behavior of operators, including requiring operators to wear helmets as required before entering the work site and perform related operations in the safe working area of ​​the operation site.

[0004] Existing production standardization mostly relies on the self-discipline of operators or manual supervision. Manual supervision is not in place, such as being unable to accurately identify whether the helmet is worn properly, or the occurrence of not wearing a helmet or working outside the safe area, which still poses a major safety hazard. Summary of the invention

[0005] The present invention provides a production safety detection method, aiming to improve the above-mentioned problem.

[0006] The present invention is implemented as follows: a production safety detection method, the method is specifically as follows:

[0007] (1) Before entering the production site, an operator's head image is collected and input into a first safety detection model, and the first safety detection model detects whether the corresponding operator is wearing a helmet in a standardized manner;

[0008] (2) After entering the production site, the operation images of the operators are collected and input into the second safety detection model. The second safety detection model detects whether the operators have any irregular operations.

[0009] Furthermore, the first security detection model is formed by adopting a multi-branch attention convolutional neural network model, and the first security detection model is formed after the multi-branch attention convolutional neural network model is trained by adopting the first type of samples;

[0010] Collect the head image of the operator before entering the production site, mark the rectangular frame containing the operator's head in the image, and mark the label information of the detection target, including whether the helmet is worn properly or not, to complete the construction of the first category of samples.

[0011] Furthermore, the second security detection model is formed by adopting a multi-branch attention convolutional neural network model, and the multi-branch attention convolutional neural network model is trained by the second type of samples to form the second security detection model;

[0012] Collect the operation images of operators at the production site, mark the rectangular frame containing the operator's operation part in the image, and mark the label information of the detection target, including standard operation and non-standard operation, to complete the construction of the second sample.

[0013] Furthermore, the multi-branch attention convolutional neural network model includes:

[0014] The first convolution layer, the first activation function ReLU, the first maximum pooling layer, the first maximum pooling layer is connected to the first branch structure and the second branch structure respectively, the outputs of the first branch structure and the second branch structure are connected to the Concat layer, and the Concat layer is sequentially connected to the second convolution layer, the third convolution layer, the second activation function ReLU, the second maximum pooling layer, the first Linear layer, the second Linear layer and the softmax layer; wherein:

[0015] The first branch structure consists of the fourth convolution layer, the third activation function ReLU, the fifth convolution layer, the fourth activation function ReLU, the third maximum pooling layer and the spatial attention module connected in sequence;

[0016] The second branch structure consists of the sixth convolutional layer, the fifth activation function ReLU, the seventh convolutional layer, the sixth activation function ReLU, the fourth maximum pooling layer and the attention channel module connected in sequence.

[0017] Furthermore, the spatial attention module includes:

[0018] A localized network and a spatial transformer connected to the input end, the localized network is connected to the spatial transformer, and the spatial transformer is connected to a bilinear interpolation function, wherein the localized network is composed of a convolutional layer, an activation function ReLU, a convolutional layer, an activation function ReLU, a Flatten layer and a fully connected layer FC connected in sequence.

[0019] Furthermore, the channel attention module includes: a global average pooling layer, connected to the input end, a fully connected network, connected to the global average pooling layer, and a weighted module, connected to the fully connected network and the input end; the fully connected network is composed of a fully connected layer FC1, a ReLU activation function, a fully connected layer FC2 and a sigmoid function connected in sequence.

[0020] Furthermore, the total loss of the multi-branch attention convolutional neural network model is The calculation formula is as follows:

[0021]

[0022] in, represents the cross entropy loss function, represents the loss function of the rectangular box position regression, and λ represents the weight parameter between the loss functions.

[0023] The present invention is implemented in this way: a production safety detection system, the system comprising:

[0024] A first camera, a second camera, and a processor;

[0025] The first camera and the second camera are connected to a processor, and the processor is integrated with a first safety detection model and a second safety detection model;

[0026] The first camera collects the head image of the operator and inputs it into the first safety detection model. The first safety detection model detects whether the corresponding operator wears the helmet in a standardized manner. The second camera collects the operation image of the operator and inputs it into the second safety detection model. The second safety detection model detects whether the operator has any irregular operation.

[0027] Further, the system includes: a prompt unit connected to the processor;

[0028] When the first safety detection model detects that the operator is not wearing a helmet properly or the second safety detection model detects that the operator is performing irregular operations, the prompt unit issues a relevant prompt.

[0029] The multi-branch attention convolutional neural network model designed in the present invention combines spatial and channel attention mechanisms, effectively focuses on key areas in the image, and effectively captures corresponding spatiotemporal information in different branches, so that the safety detection model can more effectively identify and distinguish different types of detection targets, thereby playing a more important role in production safety management; in addition, by inputting the images collected by the camera in real time into the safety detection model, the safety detection model can timely and accurately detect the eyes of operators who do not wear helmets properly and the irregular operations of operators. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A flowchart of a production safety detection method provided by an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of the structure of a multi-branch attention convolutional neural network model provided by an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of the structure of a production safety detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The specific implementation modes of the present invention are further explained in detail below by describing the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0034] Figure 1 The present invention provides a flowchart of a production safety detection method, which specifically includes the following steps:

[0035] (1) Before entering the production site, an operator's head image is collected and input into a first safety detection model, and the first safety detection model detects whether the corresponding operator is wearing a helmet in a standardized manner;

[0036] (2) After entering the production site, the operation images of the operators are collected and input into the second safety detection model. The second safety detection model detects whether the operators have any irregular operations.

[0037] In an embodiment of the present invention, the first security detection model and the second security detection model are constructed by using a multi-branch attention convolutional neural network model, the first type of samples are used to train the first security detection model, and the second type of samples are used to train the second security detection model;

[0038] Collect the head image of the operator before entering the production site, mark the rectangular frame containing the operator's head in the image, mark the label information of the detection target, including the standard wearing of the helmet and the non-standard wearing of the helmet. The non-standard wearing of the helmet includes the situation of not wearing a helmet and the situation of non-standard wearing of the helmet, and complete the construction of the first type of sample. Collect the operation image of the operator at the production site, mark the rectangular frame containing the operator's operating part in the image, and mark the label information of the detection target, including standard operation and non-standard operation, and complete the construction of the second sample.

[0039] In an embodiment of the present invention, in order to improve the recognition accuracy of the first safety detection model and the second safety detection model, the first type of sample and the second sample constructed include images obtained at different shooting angles and images shot at different brightness. The collected images are scaled to a standard size, and the standard size of the image is set to 640*640 pixels. After that, the Labelimg plug-in is used to frame each image, obtain the center point position coordinates (x, y), height and width (h, w) of the rectangular frame, mark the behavior type c in the rectangular frame, and obtain the label information corresponding to the detected target in each image as (x, y, h, w, c), forming a sample, and training the corresponding safety detection model based on the sample to ensure that the safety inspection model can accurately identify the corresponding irregular behavior in the image. In order to fully train the first safety detection model and the second safety detection, sample enhancement technology is used, including flipping (horizontal and vertical directions), cropping (0% to 20% scaling), and rotation (-10° to +10°). The safety detection model is trained by the enhanced sample, which can enhance the recognition accuracy of the first safety model and the second safety model.

[0040] Figure 2 A schematic diagram of the structure of a multi-branch attention convolutional neural network provided in an embodiment of the present invention. For ease of explanation, only the part related to the embodiment of the present invention is shown. The multi-branch attention convolutional neural network model includes

[0041] The first convolutional layer, the first activation function ReLU, the first maximum pooling layer, the first maximum pooling layer is connected to the first branch structure and the second branch structure respectively, the outputs of the first branch structure and the second branch structure are connected to the Concat layer, and the Concat layer is sequentially connected to the second convolutional layer, the third convolutional layer, the second activation function ReLU, the second maximum pooling layer, the first Linear layer, the second Linear layer and the softmax layer; among them, the first branch structure is composed of the fourth convolutional layer, the third activation function ReLU, the fifth convolutional layer, the fourth activation function ReLU, the third maximum pooling layer and the spatial attention module connected in sequence, and the second branch structure is composed of the sixth convolutional layer, the fifth activation function ReLU, the seventh convolutional layer, the sixth activation function ReLU, the fourth maximum pooling layer and the attention channel module connected in sequence.

[0042] First maximum pooling: perform a maximum pooling operation on the input image to reduce the image size and extract features; extract image features through the first branch structure, and dynamically assign weights to the importance of the input features through the spatial attention module to better focus on important areas. The second branch structure extracts image features, and the channel attention mechanism module dynamically assigns weights according to the importance of the features of different color channels (R, G, B, etc.) to improve the expression ability of the model; the Concat layer splices the outputs of the first branch structure and the second branch structure, and then performs feature fusion through the second convolution layer. The fused features are linearly transformed through the first Linear layer (also known as the fully connected layer), and then further feature transformation is performed through the second Linear layer. Finally, the output features are converted into probability distributions through the Softmax layer to represent the prediction probability corresponding to each action type. In the collected images of the workers, there are fewer pixel feature data of the helmet. The present invention designs a channel attention module and a spatial attention module based on the attention mechanism to improve the detection accuracy of the first safety detection model and the second safety detection model. These attention mechanisms can effectively focus on the key areas in the image, thereby enhancing the recognition ability of the first safety detection model and the second safety detection model for the helmet.

[0043] In addition, all convolutional layers in the first branch structure use 3×3 convolutional filters with a stride of 1 and ReLU as the activation function; all convolutional layers in the second branch structure use 5×5 convolutional filters with a stride of 1 and ReLU as the activation function; the remaining convolutional layers use 3×3 convolutional filters with a stride of 1 and ReLU as the activation function. The maximum pooling layer uses a 3×3 window and a stride of 2. In the fully connected layer (the first Linear layer), the first layer contains 4096 nodes and uses the ReLU activation function, and the second layer contains 128 nodes. Finally, the Softmax activation function is used to solve the problem of multi-target detection. This architecture design fully considers the multi-branch attention mechanism, aiming to improve the performance and generalization ability of the model, so as to effectively meet the challenges of safety detection in production sites.

[0044] In an embodiment of the present invention, spatial attention generates masks of corresponding weights at different spatial positions of the input feature map, so that the model has the ability to distinguish spatial regions. The spatial attention module includes: a localized network and a spatial transformer connected to the input end, the localized network is connected to the spatial transformer, and the spatial transformer is connected to the bilinear interpolation function, wherein the localized network is composed of a convolutional layer, an activation function ReLU, a convolutional layer, an activation function ReLU, a Flatten layer and a fully connected layer FC connected in sequence, and the affine transformation parameter θ is obtained from the input feature X through the localized network, and then the parameter θ is used to perform spatial transformation on the input feature image X, and finally the transformed output feature image X′ is obtained through bilinear interpolation.

[0045] The input feature image X is processed by the localized network to obtain the affine transformation parameter θ. In the present invention, the localized network is composed of a convolutional layer and a fully connected layer, and its output can be expressed as:

[0046] θ=FC(Flatten(ConvReLU(ConvReLU(X))));

[0047] Among them, ConvReLU means the convolutional layer is followed by the ReLU activation function, Flatten is the operation of flattening the output of the convolutional layer, and FC is the fully connected layer.

[0048] Then, by receiving the input feature image X and the empty affine transformation parameter θ as input, the transformation process can be achieved by matrix multiplication, as follows:

[0049]

[0050] Among them, (x, y) is the coordinate of the input feature image X, (x′, y′) is the coordinate of the image after the spatial transformer, and the spatial transformation parameter

[0051] Next, bilinear interpolation is performed on the input image X according to the transformed coordinates (x′, y′) to obtain the transformed output image X′, as follows:

[0052]

[0053] Where H and W are the height and width of the input image, respectively. i ,y i ) represents the coordinates in the input image, and the interpolate function interpolate is a bilinear interpolation function used to convert the transformed coordinates (x′, y′) and the coordinates in the input image (x i ,y i ) to calculate the interpolation weights.

[0054] In an embodiment of the present invention, channel attention generates corresponding weight masks for different channels of the input feature map, thereby giving the detection model channel-level discrimination ability, enhancing the convolutional neural network's weighting of the importance of input features to enhance the model's attention to useful information, thereby improving model performance. The channel attention module includes: a global average pooling layer connected to the input end, a fully connected network connected to the global average pooling layer, a weighting module connected to the fully connected network and the input end; the fully connected network is composed of a fully connected layer FC1, a ReLU activation function, a fully connected layer FC2, and a sigmoid function connected in sequence.

[0055] The input feature map Y is compressed into a feature vector z through a global average pooling operation, aiming to capture the global information of the input feature map, which is expressed as follows:

[0056] z = GlobalAvgPool(Y);

[0057] Among them, z is the feature vector formed after the feature map Y is compressed, and its dimension is the number of channels of the input feature map Y.

[0058] Next, a fully connected network is used to model the compressed feature vector to learn the correlation between features, aiming to dynamically adjust the weight of each channel in the input feature map z according to the global information, as shown below:

[0059] s = σ(FC2(δ(FC1(z))));

[0060] Among them, FC1 and FC2 represent two fully connected layers, δ represents the ReLU activation function, σ represents the sigmoid function, and s is the feature vector after excitation.

[0061] Finally, the input feature map Y is weighted using the stimulated feature vector s to enhance the model's attention to useful features. The weighting process can be expressed as:

[0062] Y′=Y·s T ;

[0063] Among them, Y′ is the weighted feature map of feature map Y, Y·s T It represents the weighting of each channel of the input feature map Y using the feature vector s. The channel attention mechanism improves the performance of the model by learning global information and dynamically adjusting the weight of each channel, so that the model can focus on important features more effectively.

[0064] In the embodiment of the present invention, the multi-branch attention convolutional neural network is trained using the cross entropy loss function and the mean square error loss function. The cross entropy loss function for the classification task The definition is as follows:

[0065]

[0066] Where log represents the natural logarithm, is the predicted category of the i-th sample output by the multi-branch attention convolutional neural network model, c i is the category label of the i-th sample, and N represents the number of samples involved in training.

[0067] In addition to the cross entropy loss function for classification tasks, it is also necessary to introduce a loss function for rectangular box position regression to further improve the accuracy of the model. This loss function can be defined as the mean square error (MSE), which represents the difference between the rectangular box position output by the model and the true label. Specifically, the loss function for rectangular box position regression is It is expressed as:

[0068]

[0069] Among them, p i represents the position of the rectangular box output by the multi-branch attention convolutional neural network model for the i-th sample, p i ′ represents the position of the rectangular box in the i-th sample label.

[0070] Combined cross entropy loss function and rectangular box position regression loss function, the total loss during training It is expressed as:

[0071]

[0072] in, represents the cross entropy loss function, represents the loss function of the rectangular box position regression, and λ represents the weight parameter between the loss functions, which is used to balance the importance of the two loss functions. By minimizing the cross entropy loss and the loss of the rectangular box position regression at the same time, a more accurate and sophisticated security detection model can be trained. Based on the above loss function, the branch attention convolutional neural network model is trained with relevant samples, and the stochastic gradient descent (SGD) optimizer is used to iterate the network parameter optimization to obtain the final first security detection model and the second security detection model.

[0073] Figure 3 The structural diagram of the production safety detection system provided by the embodiment of the present invention only shows the parts related to the embodiment of the present invention for the convenience of description. The system includes:

[0074] First camera, second camera, processor;

[0075] The first camera and the second camera are connected to a processor, and the processor is integrated with a first safety detection model and a second safety detection model;

[0076] a prompt unit connected to the processor;

[0077] Among them, the first camera collects the head image of the operator and inputs it into the first safety detection model. The first safety detection model detects whether the corresponding operator wears the helmet properly; the second camera collects the operation image of the operator and inputs it into the second safety detection model. The second safety detection model detects whether the operator has any irregular operation; when the operator does not wear the helmet properly or the operator has irregular operation, the prompt unit issues a relevant prompt.

[0078] The present invention has been described exemplarily. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A production safety detection method, characterized in that: The method is specifically as follows: (1) Before entering the production site, an operator's head image is collected and input into a first safety detection model, and the first safety detection model detects whether the corresponding operator is wearing a helmet in a standardized manner; (2) After entering the production site, the operator's operation image is collected and input into the second safety detection model. The second safety detection model detects whether the operator has any irregular operations.

2. The production safety detection method according to claim 1, characterized in that: The first security detection model is formed by adopting a multi-branch attention convolutional neural network model, and the multi-branch attention convolutional neural network model is trained by adopting the first type of samples to form the first security detection model; Collect the head image of the operator before entering the production site, mark the rectangular frame containing the operator's head in the image, and mark the label information of the detection target, including whether the helmet is worn properly or not, to complete the construction of the first category of samples.

3. The production safety detection method according to claim 1, characterized in that: The second security detection model is composed of a multi-branch attention convolutional neural network model, and the second security detection model is formed after the multi-branch attention convolutional neural network model is trained on the second type of samples; Collect the operation images of operators at the production site, mark the rectangular frame containing the operator's operation part in the image, and mark the label information of the detection target, including standard operation and non-standard operation, to complete the construction of the second sample.

4. The production safety detection method according to claim 2 or 3, characterized in that: The multi-branch attention convolutional neural network model includes: The first convolution layer, the first activation function ReLU, the first maximum pooling layer, the first maximum pooling layer is connected to the first branch structure and the second branch structure respectively, the outputs of the first branch structure and the second branch structure are connected to the Concat layer, and the Concat layer is sequentially connected to the second convolution layer, the third convolution layer, the second activation function ReLU, the second maximum pooling layer, the first Linear layer, the second Linear layer and the softmax layer; wherein: The first branch structure consists of the fourth convolution layer, the third activation function ReLU, the fifth convolution layer, the fourth activation function ReLU, the third maximum pooling layer and the spatial attention module connected in sequence; The second branch structure consists of the sixth convolutional layer, the fifth activation function ReLU, the seventh convolutional layer, the sixth activation function ReLU, the fourth maximum pooling layer and the attention channel module connected in sequence.

5. The production safety detection method according to claim 4, characterized in that: The spatial attention module includes: A localized network and a spatial transformer connected to the input end, the localized network is connected to the spatial transformer, and the spatial transformer is connected to a bilinear interpolation function, wherein the localized network is composed of a convolutional layer, an activation function ReLU, a convolutional layer, an activation function ReLU, a Flatten layer and a fully connected layer FC connected in sequence.

6. The production safety detection method according to claim 4, characterized in that: The channel attention module includes: a global average pooling layer, connected to the input end, a fully connected network, connected to the global average pooling layer, and a weighted module, connected to the fully connected network and the input end; the fully connected network is composed of a fully connected layer FC1, a ReLU activation function, a fully connected layer FC2 and a sigmoid function connected in sequence.

7. The production safety detection method according to claim 4, characterized in that: Total loss of multi-branch attention convolutional neural network model The calculation formula is as follows: in, represents the cross entropy loss function, represents the loss function of the rectangular box position regression, and λ represents the weight parameter between the loss functions.

8. A production safety detection system, characterized in that: The system comprises: A first camera, a second camera, and a processor; The first camera and the second camera are connected to a processor, and the processor is integrated with a first safety detection model and a second safety detection model; Among them, the first camera captures the operator's head image and inputs it into the first safety detection model. The first safety detection model detects whether the corresponding operator wears the helmet properly; the second camera captures the operator's operation image and inputs it into the second safety detection model. The second safety detection model detects whether the operator has any irregular operations.

9. The production safety detection system according to claim 8, characterized in that: The system includes: a prompt unit connected to the processor; When the first safety detection model detects that the operator is not wearing a helmet properly or the second safety detection model detects that the operator is performing irregular operations, the prompt unit issues a relevant prompt.

Images