Safety production target detection method

Abstract

The invention relates to a safety production target detection method, belongs to the technical field of computer vision, and solves the problems that an existing target detection method is low in target detection precision and low in calculation efficiency when facing a safety production scene. The method comprises the following steps: acquiring a safety production scene image including a to-be-detected target; inputting the safety production scene image and a preset to-be-detected target query vector into a trained safety production target detection model to obtain the classification and position of a to-be-detected target; wherein the safety production target detection model is an improved Transform model in which a convolutional neural network is used for carrying out feature extraction and a path aggregation feature pyramid is used for replacing a feedforward neural network in a coding and decoding process, and is used for enhancing multi-scale feature fusion and accurately detecting targets at different positions and scales.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision, and in particular to a method for detecting a safe production target. Background Art

[0002] Target detection in production safety is a technology that identifies and locates targets related to production safety. Through image target detection technology, it can automatically identify whether employees are wearing personal protective equipment such as helmets and safety belts correctly, thereby ensuring the safety of employees at work, and checking whether all personnel entering a specific area are wearing necessary protective equipment, or whether they are working in accordance with the prescribed operating procedures. It can also monitor potential risks in the workplace in real time and provide timely warning information for safety management.

[0003] In the production safety monitoring of large-scale industrial scenarios, there are a large number of situations where people and machines work together. The operational relationship is complex and the accuracy and real-time requirements for safety detection are extremely high. For example, in scenes such as chemical production workshops, construction sites, and tank warehouses, it is necessary to accurately identify whether workers are operating equipment in violation of regulations, whether they are in dangerous areas, whether they are placed in violation of regulations, or violate operating specifications. These scenes often involve complex backgrounds, multi-scale targets (such as individual workers in the near distance and large mechanical equipment in the distance, or workers in the distance cleaning dust, etc.) and detection challenges caused by changes in viewing angles and distances. Traditional target detection methods are difficult to effectively deal with these complex situations with specific safety requirements. Summary of the invention

[0004] In view of the above analysis, an embodiment of the present invention aims to provide a safe production target detection method to solve the problems of low target detection accuracy and low computational efficiency of existing target detection methods when facing safe production scenarios.

[0005] The purpose of the present invention is mainly achieved through the following technical solutions:

[0006] The present invention provides a method for detecting a production safety target, comprising the following steps:

[0007] Acquire a safe production scene image including a target to be detected;

[0008] The safety production scene image and the preset query vector of the target to be detected are input into the trained safety production target detection model to obtain the classification and position of the target to be detected; wherein, the safety production target detection model uses a convolutional neural network for feature extraction, and in the encoding and decoding process, uses a path aggregation feature pyramid to replace the improved Transformer model of the feedforward neural network to enhance multi-scale feature fusion and accurately detect targets of different positions and scales.

[0009] Furthermore, the step of inputting the safety production scene image and the preset query vector of the target to be detected into a trained safety production target detection model to obtain the classification and position of the target to be detected includes:

[0010] The safety production scene image is divided into a number of image blocks through the input layer, and the position code corresponding to each image block is obtained;

[0011] Each image block passes through the feature extraction layer to extract the preliminary features of each image block and obtain the preliminary feature vector;

[0012] The position encoding of the preliminary feature vector and each image block corresponding to each other is passed through the encoder of the production safety target detection model to obtain the encoding feature;

[0013] The coding feature is merged with the position coding to obtain a coding position feature;

[0014] The encoding feature, the encoding position feature and the query vector of the target to be detected are passed through a decoder of the safe production target detection model to obtain a decoding feature;

[0015] The decoded features are respectively passed through a softmax layer and a linear regression layer to obtain the classification and position of the target to be detected in the safety production scene image.

[0016] Furthermore, in the encoder of the safety production target detection model,

[0017] After fusing the preliminary feature vector with the position code, an encoder query vector and an encoder key vector are generated through linear transformation respectively;

[0018] The preliminary feature vector is transformed into an encoder value vector by linear transformation;

[0019] The encoder query vector, the encoder key vector, and the encoder value vector perform self-attention calculation in the encoder's multi-head attention module to capture internal dependencies and obtain output features of the encoder's multi-head attention module;

[0020] The output features of the encoder multi-head attention module and the preliminary features of the image are fused in the residual link and layer normalization module of the encoder to obtain encoder enhanced features;

[0021] The encoder enhanced features are subjected to multi-scale feature fusion through the path aggregation feature pyramid module of the encoder to obtain the encoding features.

[0022] Furthermore, in the decoder of the safety production target detection model,

[0023] The target query vector to be detected is self-attention-calculated by the first self-attention module of the decoder to obtain target features;

[0024] After the target feature is fused with the query vector of the target to be detected, the target feature is fused with the encoding position feature and the encoding feature, and a self-attention calculation is performed through the second self-attention module of the decoder to obtain a decoder enhanced feature;

[0025] The decoder enhancement feature is subjected to multi-scale feature fusion through the path aggregation feature pyramid module of the decoder to obtain the decoding feature.

[0026] Furthermore, in the first self-attention module,

[0027] The target query vector to be detected is fused with itself to generate a decoder first query vector and a decoder first key vector through two linear transformations respectively;

[0028] The target query vector to be detected generates a decoder first value vector through linear transformation;

[0029] The decoder first query vector, the decoder first key vector, and the decoder first value vector perform self-attention calculation in a multi-head attention module of the first self-attention module to capture internal dependencies and obtain output features of the first self-attention module;

[0030] The output features of the first self-attention module and the target query vector to be detected are fused in the residual link and layer normalization module of the first self-attention module to obtain the target features.

[0031] Furthermore, in the second self-attention module,

[0032] The target feature is fused with the query vector of the target to be detected to generate a second query vector of the decoder through linear transformation;

[0033] The encoding position feature generates a decoder second key vector through linear transformation;

[0034] The encoding feature generates a decoder second value vector through linear transformation;

[0035] The decoder second query vector, the decoder second key vector, and the decoder second value vector perform self-attention calculation in a multi-head attention module of the second self-attention module to capture internal dependencies and obtain output features of the second self-attention module;

[0036] The output features of the second self-attention module and the target features are fused in the residual link and layer normalization module of the second self-attention module to obtain decoder enhanced features.

[0037] Furthermore, when performing self-attention calculation, the attention matrix obtained based on the query vector, key vector and value vector is subjected to singular value decomposition to obtain an approximate low-rank matrix as the optimized attention matrix to reduce the computational complexity.

[0038] Furthermore, the path aggregation feature pyramid module of the encoder uses a path aggregation feature pyramid structure to perform refined feature extraction on objects of different sizes.

[0039] Furthermore, the path aggregation feature pyramid module of the decoder uses a simplified path aggregation feature pyramid structure for multi-scale feature fusion;

[0040] The simplified path aggregation feature pyramid structure retains the convolution layer C2, convolution layer C3 of the bottom-up path and the feature layer P4, feature layer P5 of the top-down path in the feature pyramid network structure, as well as the aggregation feature layer P2, aggregation feature layer P3 of the path aggregation network structure.

[0041] Furthermore, the safety production target detection model is trained using the following method:

[0042] Constructing a training data set for the safety production target detection model; wherein the training data set includes a safety production scene image set, a category label and a bounding box corresponding to the target in each image, and a target query vector for each image;

[0043] The training data set is loaded, and the weighted combination of the cross entropy loss function and the mean square error (MSE) loss function is used to train the safe production target detection model. The model parameters are updated using gradient back propagation. The training is terminated when the loss function value converges to obtain a trained safe production target detection model.

[0044] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0045] 1. The present invention inherits the long-distance dependency of Transformer in feature processing through the application of feature extraction based on convolutional neural network and improved Transformer, and can capture broader contextual information. In the encoding and decoding process, the path aggregation feature pyramid is combined to replace the traditional feedforward neural network for feature transformation and extraction, and the features output by the decoder are transformed and regularized. On the one hand, the ability of multi-scale feature fusion is enhanced to better adapt to the transformation of various complex scenes and targets. On the other hand, the feature representation ability in the underlying features is better utilized to more comprehensively detect targets of different positions and scales, reduce the missed detection rate, and thus improve the recall rate of target detection.

[0046] 2. During the decoding process, the present invention uses a simplified path aggregation feature pyramid, which reduces the computational complexity of the original path aggregation feature pyramid and reduces the processing of invalid features by subsequent decoders. Through the multi-path feature fusion method, the information bottleneck is effectively avoided, so that the features can retain more effective information during the transmission process, which is conducive to the subsequent decoders to better utilize these features for target reconstruction and prediction, while improving the computational efficiency.

[0047] 3. The solution of the present invention inherits the ability of the Transformer self-attention mechanism: it adopts a multi-head attention mechanism, can focus on multiple spatial positions at the same time, and effectively handle the relationship between different parts. It improves the detection accuracy, which plays an important role in the dynamic target detection in the field of safe production, improves the safety monitoring ability of automated equipment on workers, and reduces the risk of false alarms and missed alarms.

[0048] 4. In order to solve the problem of high bandwidth consumption of Transformer, the present invention uses the singular value decomposition (SVD) method to perform low-rank decomposition of the attention matrix when performing attention calculation. In this way, the data dimension and interaction amount can be reduced, thereby reducing the amount of calculation and bandwidth consumption to a certain extent.

[0049] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0051] Figure 1A schematic diagram of a process flow of a safe production target detection method in an embodiment of the present invention;

[0052] Figure 2 A schematic diagram of the structure of a safe production target detection model in an embodiment of the present invention;

[0053] Figure 3 Schematic diagram of the structure of the path aggregation feature pyramid module of the encoder in an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of the structure of the path aggregation feature pyramid module of the decoder in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0056] A specific embodiment of the present invention discloses a method for detecting a safe production target. Figure 1 As shown, the following steps are included:

[0057] Step S1, obtaining a safe production scene image including a target to be detected.

[0058] Specifically, images of safe production scenes captured by monitoring cameras installed at the safe production project site are obtained.

[0059] Step S2, input the safety production scene image and the preset query vector of the target to be detected into the trained safety production target detection model to obtain the classification and position of the target to be detected; wherein the safety production target detection model uses a convolutional neural network for feature extraction, and in the encoding and decoding process, uses a path aggregation feature pyramid to replace the improved Transformer model of the feedforward neural network to enhance multi-scale feature fusion and accurately detect targets of different positions and scales.

[0060] Furthermore, the structure of the safety production target detection model is as follows: Figure 2 As shown,

[0061] The safety production scene image is divided into a number of image blocks through the input layer, and the position code corresponding to each image block is obtained.

[0062] Specifically, the Transformer model used in the safety production target detection model is designed for processing sequence data, which relies on the position information of each element in the sequence. In the safety production target detection scenario, due to the image size captured by the camera and the requirements of the convolution input size in the model, it is necessary to divide the image into multiple small blocks and simulate the sequence input so that the Transformer model can effectively process the image data. Position encoding can provide the model with the position information of each small block in the image and understand the spatial layout of the target. For example, the position encoding can distinguish between the worker in the upper left corner of the image and the worker in the lower right corner.

[0063] It should be noted that the position encoding is calculated by using the sine and cosine functions based on the position and feature dimension of the input sequence, and the formula is as follows:

[0064]

[0065] Among them, PE represents position encoding; pos represents the position in the sequence; d represents the dimension of the feature, that is, the length of the feature vector; i represents the index of the dimension, i∈[0,d / 2].

[0066] Through this calculation method, each element of the position encoding vector contains position information, and due to the periodicity of the sine and cosine functions, these encoding vectors can remain unique even in very long sequences, allowing the model to process long sequence data and understand the relative position relationship of elements in the sequence.

[0067] Furthermore, each image block passes through a feature extraction layer to extract preliminary features of each image block and obtain a preliminary feature vector.

[0068] Specifically, the feature extraction layer uses a convolutional neural network for feature extraction. Exemplarily, the feature extraction network of the YOLO model can be used. The YOLO model itself can effectively extract target-related features in the image and provide high-quality initial features for subsequent processing. These initial features contain information such as the position, shape, and partial semantics of the target. In this embodiment, for images containing workers and machinery and equipment, the feature extraction network of the YOLO model can preliminarily identify basic features such as the outline of the worker and the approximate shape of the machinery and equipment. The resolution and semantic richness of these feature maps will vary depending on the different layers of its network. These features will pass through the Transformer's multi-head attention module to simultaneously focus on and process relationships in multiple positions. By weighted combination of features at different positions, the model can better understand the semantics of the entire scene, such as understanding the operational relationship between workers and surrounding equipment, rather than just looking at each target in isolation.

[0069] Furthermore, the position encoding corresponding to the preliminary feature vector and each image block is passed through the encoder of the safe production target detection model to obtain the encoded feature.

[0070] Specifically, the encoder of the safe production target detection model receives the preliminary feature vector and the position code corresponding to each image block, performs simultaneous attention and relationship processing on multi-position information, and enables the model to better understand the semantics of the entire scene through weighted combination of features at different positions, and performs multi-scale fusion of features at different levels, so that the model can better utilize the feature representation capabilities of the underlying features, more comprehensively detect targets of different positions and scales, and detect the status of small components on large equipment in the distance and the details of tools in the hands of nearby workers.

[0071] Furthermore, in the encoder of the safety production target detection model,

[0072] After fusing the preliminary feature vector with the position code, an encoder query vector and an encoder key vector are generated through linear transformation respectively;

[0073] The preliminary feature vector is transformed into an encoder value vector through linear transformation.

[0074] Specifically, the fused vector is linearly transformed to generate a query (Q) vector and a key (K) vector. These two vectors are the core of the self-attention mechanism and are used to calculate the attention score, that is, where the model should pay attention to more information.

[0075] The value (V) vector contains the actual feature information.

[0076] Furthermore, the encoder query vector, the encoder key vector and the encoder value vector perform self-attention calculation in the encoder's multi-head attention module to capture internal dependencies and obtain output features of the encoder's multi-head attention module.

[0077] Specifically, the multi-head attention module allows the model to capture information in different representation subspaces simultaneously. In the multi-head attention module, each head performs self-attention calculations on the input query (Q), key (K), and value (V) vectors to determine the influence of each image block in the input sequence on other elements.

[0078] The self-attention calculation is achieved by calculating the dot product between each query vector and all key vectors, and then the result is passed through a softmax function to obtain the attention weight matrix.

[0079] The output features of the encoder multi-head attention module are the concatenation of the output features of all heads. The output features of each head are generated by applying the weights calculated by self-attention to the value vector.

[0080] It should be noted that, since the attention mechanism requires interaction and integration of information at different locations, such a large amount of data interaction will lead to high bandwidth consumption. Therefore, this embodiment uses the singular value decomposition (SVD) method to perform low-rank decomposition of the attention matrix on the attention weight matrix, decomposing it into the product of two low-rank matrices, and using the low-rank matrix to approximate the original high-rank attention matrix. The low-rank matrix is ​​processed during the query process to reduce the data dimension and the amount of interaction, thereby reducing the bandwidth consumption during the query process.

[0081] Specifically, the singular value decomposition formula is as follows:

[0082] A=U∑V T

[0083] Among them, A represents the m×n attention matrix; U represents the m×m orthogonal matrix; V represents the n×n orthogonal matrix; ∑ represents the m×n diagonal matrix; V T Represents the transposed matrix of matrix V.

[0084] After the above singular value decomposition, the first k larger singular values ​​are selected to approximate the original matrix to obtain an approximate low-rank matrix, which is calculated as follows:

[0085] A k =U k ∑ k V k T

[0086] Among them, A k represents the result of approximating the original high-rank attention matrix A with a low-rank matrix; U k represents the m×k matrix consisting of the first k columns of matrix U; ∑ k V represents the k×k diagonal matrix consisting of the first k diagonal elements of the matrix Σ; k represents an n×k matrix consisting of the first k columns of the matrix V; k represents a hyperparameter, which can be a value between 50 and 150 by way of example.

[0087] Through the above method, the data dimension and interaction amount can be reduced, thereby reducing the amount of calculation and bandwidth consumption.

[0088] Furthermore, the output features of the encoder's multi-head attention module and the preliminary features of the image are fused in the residual link and layer normalization module of the encoder to obtain encoder enhanced features.

[0089] Specifically, the residual link of the encoder can alleviate the problem of gradient vanishing in deep neural networks, and the residual link directly learns the difference between input and output without complex transformation, thereby simplifying the network design.

[0090] The layer normalization helps to improve the generalization ability of the model by stabilizing the activation distribution of the hidden layer. At the same time, it improves the robustness of the model to input disturbances by reducing the scale variation of the input data.

[0091] Furthermore, the encoder enhancement feature is subjected to multi-scale feature fusion through a path aggregation feature pyramid module of the encoder to obtain the encoding feature.

[0092] The path aggregation feature pyramid module of the encoder uses a path aggregation feature pyramid structure to perform refined feature extraction on objects of different sizes.

[0093] Specifically, Figure 3 As shown in the figure, the path aggregation pyramid structure (PAFPN) adds a bottom-up path on the basis of FPN, which is mainly composed of bottom-up convolutional layers, top-down upsampling layers and lateral connections. It can better integrate feature information at different levels, so that the network can learn richer semantic information and detail information.

[0094] The extracted features are obtained in the path aggregation pyramid structure (PAFPN) in the bottom-up path to obtain lower-level features, and downsampled through convolution operations to gradually transfer feature information to higher levels. In this process, the size of the feature map gradually decreases, but the number of channels gradually increases, and the semantic information gradually becomes richer. Then the top-down path is connected horizontally. The top-down path is to upsample the high-level semantic feature map to gradually increase its size and fuse it with the feature map of the corresponding level in the bottom-up path. It should be noted that the upsampling in the structure adopts the bilinear interpolation method, and the horizontal connection is to better fuse the features of different levels, and the feature maps of the same scale in the bottom-up path and the top-down path are added.

[0095] Furthermore, the coding feature is fused with the position coding to obtain a coding position feature.

[0096] Specifically, during the processing of the encoder, the feature map will undergo multiple transformations and its position information may be lost. Here, the encoding features output by the encoder are fused with the position encoding again to strengthen the position information in the encoding features and ensure that the position sensitivity can continue to be maintained in subsequent processing.

[0097] Furthermore, the encoding feature, the encoding position feature and the query vector of the target to be detected are passed through a decoder of the safe production target detection model to obtain a decoding feature.

[0098] Specifically, the query vector of the target to be detected is a query vector containing information of the target to be detected in safe production, so as to determine whether the target to be detected exists in the image.

[0099] Specifically, the targets to be detected may be whether an employee is wearing or not wearing a safety helmet, whether an employee is in or not in a dangerous position, whether an employee violates operating specifications, whether equipment is operating normally, and the placement of dangerous items.

[0100] The query vector containing the information of the target to be detected is a vector composed of the feature encodings of each candidate target category.

[0101] Furthermore, in the decoder of the safety production target detection model,

[0102] The target query vector to be detected is subjected to self-attention calculation by the first self-attention module of the decoder to obtain target features.

[0103] Specifically, the query vector of the target to be detected is fused through the self-attention mechanism in the first self-attention module of the decoder to capture the dependencies within and between the targets to be detected, enhance feature representation, and improve the model's ability to discriminate targets. At the same time, the model's ability to handle multiple targets, small targets, and complex scenes is enhanced, and the robustness and generalization ability of the model are enhanced.

[0104] Furthermore, in the first self-attention module,

[0105] After the target query vector to be detected is fused with itself, two linear transformations are respectively used to generate a decoder first query vector and a decoder first key vector.

[0106] The target query vector to be detected generates a decoder first value vector through linear transformation.

[0107] The decoder first query vector, the decoder first key vector and the decoder first value vector are self-attention calculated in the multi-head attention module of the first self-attention module to capture internal dependencies and obtain output features of the first self-attention module.

[0108] The output features of the first self-attention module and the target query vector to be detected are fused in the residual link and layer normalization module of the first self-attention module to obtain the target features.

[0109] Specifically, the multi-head attention module and the residual link and layer normalization module in the first self-attention module of the decoder have the same processing process as the multi-head attention module and the residual link and layer normalization module of the encoder, and will not be repeated here.

[0110] It should be noted that when calculating the self-attention of the first self-attention module of the decoder, it is also necessary to use the singular value decomposition (SVD) method to perform low-rank decomposition of the attention matrix.

[0111] Furthermore, after the target feature is fused with the query vector of the target to be detected, self-attention calculation is performed with the encoding position feature and the encoding feature through the second self-attention module of the decoder to obtain a decoder enhanced feature.

[0112] Specifically, the second attention module of the decoder fuses the target features with the query vector of the target to be detected, and performs self-attention calculation in combination with the encoding position features and the encoding features, thereby integrating global and local information, enhancing the contextual perception ability of the target, and improving the model's positioning and recognition accuracy of the target. At the same time, it strengthens the expression ability of the features, providing a richer and more accurate feature basis for subsequent classification and positioning tasks.

[0113] Furthermore, in the second self-attention module,

[0114] The target feature is fused with the query vector of the target to be detected to generate a second query vector of the decoder through linear transformation;

[0115] The encoding position feature generates a decoder second key vector through linear transformation;

[0116] The encoding feature generates a decoder second value vector through linear transformation;

[0117] The decoder second query vector, the decoder second key vector, and the decoder second value vector perform self-attention calculation in a multi-head attention module of the second self-attention module to capture internal dependencies and obtain output features of the second self-attention module;

[0118] The output features of the second self-attention module and the target features are fused in the residual link and layer normalization module of the second self-attention module to obtain decoder enhanced features.

[0119] Specifically, the processing process of the second self-attention module of the decoder is the same as that of the first self-attention module of the decoder. It should be noted that when calculating the self-attention of the second self-attention module of the decoder, it is also necessary to use the singular value decomposition (SVD) method to perform low-rank decomposition of the attention matrix.

[0120] Furthermore, the decoder enhancement feature is subjected to multi-scale feature fusion through the path aggregation feature pyramid module of the decoder to obtain the decoding feature.

[0121] Further, such as Figure 4As shown, the path aggregation feature pyramid module of the decoder uses a simplified path aggregation feature pyramid structure for multi-scale feature fusion.

[0122] The simplified path aggregation feature pyramid structure retains the convolution layer C2, convolution layer C3 of the bottom-up path and the feature layer P4, feature layer P5 of the top-down path in the feature pyramid network structure, as well as the aggregation feature layer P2, aggregation feature layer P3 of the path aggregation network structure.

[0123] Specifically, in the path aggregation feature pyramid module of the encoder, a 4-layer path aggregation feature pyramid (PAFPN) is used to process the image input in order to more finely extract large and small targets. In the decoder, because there is no need to add too many pyramid structures on the basis of the processing of each branch, what is needed is the joint feature extraction of multiple features. Therefore, in the path aggregation feature pyramid module of the decoder, a simplified version of the path aggregation feature pyramid (PAFPN) is used, which reduces the computational complexity of the original PAFPN and reduces the subsequent processing of invalid features.

[0124] Furthermore, the decoded features are respectively passed through a softmax layer and a linear regression layer to obtain the classification and position of the target to be detected in the safe production scene image.

[0125] Specifically, the softmax layer is used to classify the decoded features and convert the decoded features into probability distribution, where each category corresponds to a probability value, indicating the possibility that the feature belongs to the category.

[0126] The linear regression layer is used to predict the specific position of the target, specifically to output the bounding box coordinates of the target.

[0127] Furthermore, the safety production target detection model is trained using the following method:

[0128] Construct a training data set for the safety production target detection model; wherein the training data set includes a set of safety production scene images, category labels and bounding boxes corresponding to targets in each image, and target query vectors for each image.

[0129] Specifically, for each image in the safety production scene image set, the target position and target category are firstly marked through an automated labeling tool to obtain the detection box and category of each target; then, the targets with marking errors or position offsets, as well as combined marks that require manual marking, are manually corrected. For example, if a person is not wearing a safety helmet, the head mark and the safety helmet mark are combined as an example of a person wearing a safety helmet. The situations where there is no safety helmet worn on the person's head or no safety helmet appears in the picture are all examples of a person not wearing a safety helmet.

[0130] The training data set is loaded, and the weighted combination of the cross entropy loss function and the mean square error (MSE) loss function is used to train the safe production target detection model. The model parameters are updated using gradient back propagation. The training is terminated when the loss function value converges to obtain a trained safe production target detection model.

[0131] Specifically, the cross entropy loss function is used to measure the model target classification task, the mean square error MSE loss function is used to measure the target position prediction, and the comprehensive loss is obtained by the weighted combination of classification loss and regression loss.

[0132] It should be noted that when the loss value is less than 0.001 in 10 consecutive training rounds, the training is terminated.

[0133] In summary, a safe production target detection method according to an embodiment of the present invention has the following beneficial effects:

[0134] 1. The present invention inherits the long-distance dependency of Transformer in feature processing through the application of feature extraction based on convolutional neural network and improved Transformer, and can capture broader contextual information. In the encoding and decoding process, the path aggregation feature pyramid is combined to replace the traditional feedforward neural network for feature transformation and extraction, and the features output by the decoder are transformed and regularized. On the one hand, the ability of multi-scale feature fusion is enhanced to better adapt to the transformation of various complex scenes and targets. On the other hand, the feature representation ability in the underlying features is better utilized to more comprehensively detect targets of different positions and scales, reduce the missed detection rate, and thus improve the recall rate of target detection.

[0135] 2. During the decoding process, the present invention uses a simplified path aggregation feature pyramid, which reduces the computational complexity of the original path aggregation feature pyramid and reduces the processing of invalid features by subsequent decoders. Through the multi-path feature fusion method, the information bottleneck is effectively avoided, so that the features can retain more effective information during the transmission process, which is conducive to the subsequent decoders to better utilize these features for target reconstruction and prediction, while improving the computational efficiency.

[0136] 3. The solution of the present invention inherits the ability of the Transformer self-attention mechanism: it adopts a multi-head attention mechanism, can focus on multiple spatial positions at the same time, and effectively handle the relationship between different parts. It improves the detection accuracy, which plays an important role in the dynamic target detection in the field of safe production, improves the safety monitoring ability of automated equipment on workers, and reduces the risk of false alarms and missed alarms.

[0137] 4. In order to solve the problem of high bandwidth consumption of Transformer, the present invention uses the singular value decomposition (SVD) method to perform low-rank decomposition of the attention matrix when performing attention calculation. In this way, the data dimension and interaction amount can be reduced, thereby reducing the amount of calculation and bandwidth consumption to a certain extent.

[0138] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for detecting a production safety target, characterized in that: The steps include: Acquire a safe production scene image including a target to be detected; The safety production scene image and the preset query vector of the target to be detected are input into the trained safety production target detection model to obtain the classification and position of the target to be detected; wherein, the safety production target detection model uses a convolutional neural network for feature extraction, and in the encoding and decoding process, uses a path aggregation feature pyramid to replace the improved Transformer model of the feedforward neural network to enhance multi-scale feature fusion and accurately detect targets of different positions and scales.

2. The method according to claim 1, characterized in that: The step of inputting the safety production scene image and the preset query vector of the target to be detected into the trained safety production target detection model to obtain the classification and position of the target to be detected includes: The safety production scene image is divided into a number of image blocks through the input layer, and the position code corresponding to each image block is obtained; Each image block passes through the feature extraction layer to extract the preliminary features of each image block and obtain the preliminary feature vector; The position encoding of the preliminary feature vector and each image block corresponding to each other is passed through the encoder of the production safety target detection model to obtain the encoding feature; The coding feature is merged with the position coding to obtain a coding position feature; The encoding feature, the encoding position feature and the query vector of the target to be detected are passed through a decoder of the safe production target detection model to obtain a decoding feature; The decoded features are respectively passed through a softmax layer and a linear regression layer to obtain the classification and position of the target to be detected in the safety production scene image.

3. The method according to claim 2, characterized in that: In the encoder of the production safety target detection model, After fusing the preliminary feature vector with the position code, an encoder query vector and an encoder key vector are generated through linear transformation respectively; The preliminary feature vector is transformed into an encoder value vector by linear transformation; The encoder query vector, the encoder key vector, and the encoder value vector perform self-attention calculation in the encoder's multi-head attention module to capture internal dependencies and obtain output features of the encoder's multi-head attention module; The output features of the encoder multi-head attention module and the preliminary features of the image are fused in the residual link and layer normalization module of the encoder to obtain encoder enhanced features; The encoder enhanced features are subjected to multi-scale feature fusion through the path aggregation feature pyramid module of the encoder to obtain the encoding features.

4. The method according to claim 3, characterized in that: In the decoder of the production safety target detection model, The target query vector to be detected is self-attention-calculated by the first self-attention module of the decoder to obtain target features; After the target feature is fused with the query vector of the target to be detected, the target feature is fused with the encoding position feature and the encoding feature, and a self-attention calculation is performed through the second self-attention module of the decoder to obtain a decoder enhanced feature; The decoder enhancement feature is subjected to multi-scale feature fusion through the path aggregation feature pyramid module of the decoder to obtain the decoding feature.

5. The method according to claim 4, characterized in that: In the first self-attention module, The target query vector to be detected is fused with itself to generate a decoder first query vector and a decoder first key vector through two linear transformations respectively; The target query vector to be detected generates a decoder first value vector through linear transformation; The decoder first query vector, the decoder first key vector, and the decoder first value vector perform self-attention calculation in a multi-head attention module of the first self-attention module to capture internal dependencies and obtain output features of the first self-attention module; The output features of the first self-attention module and the target query vector to be detected are fused in the residual link and layer normalization module of the first self-attention module to obtain the target features.

6. The method according to claim 5, characterized in that: In the second self-attention module, The target feature is fused with the query vector of the target to be detected to generate a second query vector of the decoder through linear transformation; The encoding position feature generates a decoder second key vector through linear transformation; The encoding feature generates a decoder second value vector through linear transformation; The decoder second query vector, the decoder second key vector, and the decoder second value vector perform self-attention calculation in a multi-head attention module of the second self-attention module to capture internal dependencies and obtain output features of the second self-attention module; The output features of the second self-attention module and the target features are fused in the residual link and layer normalization module of the second self-attention module to obtain decoder enhanced features.

7. The method according to claim 6, characterized in that: When performing self-attention calculation, the attention matrix obtained based on the query vector, key vector and value vector is subjected to singular value decomposition to obtain an approximate low-rank matrix as the optimized attention matrix to reduce the computational complexity.

8. The method according to any one of claims 4 to 7, characterized in that: The path aggregation feature pyramid module of the encoder uses a path aggregation feature pyramid structure to perform refined feature extraction on objects of different sizes.

9. The method according to claim 8, characterized in that: The path aggregation feature pyramid module of the decoder uses a simplified path aggregation feature pyramid structure for multi-scale feature fusion; The simplified path aggregation feature pyramid structure retains the convolution layer C2, convolution layer C3 of the bottom-up path and the feature layer P4, feature layer P5 of the top-down path in the feature pyramid network structure, as well as the aggregation feature layer P2, aggregation feature layer P3 of the path aggregation network structure.

10. The method according to claim 1, characterized in that: The safety production target detection model is trained using the following method: Constructing a training data set for the safety production target detection model; wherein the training data set includes a safety production scene image set, a category label and a bounding box corresponding to the target in each image, and a target query vector for each image; The training data set is loaded, and the weighted combination of the cross entropy loss function and the mean square error (MSE) loss function is used to train the production safety target detection model. The model parameters are updated using gradient back propagation. The training is terminated when the loss function value converges to obtain a trained production safety target detection model.

Images