Characteristic detection method in industrial environment

Abstract

The invention relates to a feature detection method in an industrial environment, and belongs to the field of industrial Internet of Things. The method comprises the following steps: firstly, adding an attention mechanism of concerning position information into a Bcheck structure of MobileNetV3, constructing a CA-Bcheck structure and a lightweight YOLOv5s backbone network, and reducing model parameter quantity and calculation complexity so as to effectively improve real-time detection speed; for the problems of feature shielding and multi-feature coexistence, a space factor is designed to measure the relative space position of a detection frame and a real frame, so that the effectiveness of the detection frame is evaluated more accurately under the condition of feature overlapping. And finally, a self-adaptive channel pruning method is put forward to compress the model, the optimized model is issued to an industrial edge intelligent terminal, and features are detected on edge equipment, so that the model recognition precision and the reasoning speed can be effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of industrial Internet of Things and relates to a feature detection method in an industrial environment. Background Art

[0002] In recent years, with the continuous development of social and economic development, the construction of industrial Internet of Things has achieved very remarkable results. With the continuous expansion of the scale of industrial Internet of Things and the improvement of ecological environment, the safety and reliability in industrial environment have become increasingly important. Industrial environment is usually faced with various potential risks, including characteristics such as balloons and hanging garbage, which may cause equipment running in industrial Internet of Things to fail, which has a great impact on people's production and life. Therefore, regular inspection of industrial equipment and timely investigation of risks are of great significance to the safe operation of equipment in industrial environment.

[0003] Traditional feature detection methods mainly rely on manual visual inspection and simple screening techniques. These methods are not only inefficient, but also easily affected by human factors, making it difficult to ensure the accuracy and consistency of detection. With the development of technologies such as computer vision, sensor technology, and machine learning, more and more automated detection systems are being introduced into industrial production. These systems can monitor products on the production line in real time, quickly identify and locate features through image processing and data analysis, thereby improving detection efficiency and accuracy.

[0004] Even though deep learning and edge computing technologies have made significant progress in the deployment and application of industrial environments, there are still some problems and challenges that need to be solved in practice. For example, most edge terminal devices cannot directly run feature detection models that require a lot of computing resources due to computing resource constraints. At the same time, the communication cost of transmitting feature images from the edge side to the cloud server over long distances is high. Existing intelligent perception terminals can only complete simple image acquisition tasks and still need to send feature images to the cloud data center for processing at regular intervals. A large number of feature images will cause a heavy load on the cloud data center and even cause data explosion. Moreover, a large upload time interval will cause the underreporting of features, causing serious accidents, and the emergency handling capacity is seriously insufficient, making it difficult to meet the requirements of feature detection in industrial environments for the effectiveness of cloud-edge data transmission. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a feature detection method in an industrial environment. Taking advantage of edge computing, feature processing and analysis are pushed to the edge device closest to the feature source to achieve real-time feature detection and reduce detection delay.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A feature detection method in an industrial environment comprises the following steps:

[0008] S1. Obtain an industrial environment feature image dataset and preprocess the dataset;

[0009] S2, using the Bneck structure in the MobileNetV3 network, combined with the CA attention mechanism, constructing the CA-Bneck structure, and using CA-Bneck to reconstruct the Yolov5s backbone network, and proposing a fast sigmoid function to construct the MobileNetV3 activation function;

[0010] S3. Based on the traditional loss function, we design adjustable parameters and spatial factors to construct an adaptive loss function Adaptive-IoU (A-IoU), which can provide more reasonable scores when the feature distance is close or the occlusion is severe.

[0011] S4. An adaptive channel pruning method is proposed to perform dynamic sparse training on the feature detection model to obtain the optimal feature detection model and improve the inference speed of the feature detection model.

[0012] In S1, the visual image annotation tool LabelImg is used to annotate the feature images collected in the industrial environment, and Make Sense is used for semi-automatic annotation to generate a txt format annotation file.

[0013] In S2, the Bneck structure in the MobileNetV3 network is adopted and combined with the CA attention mechanism to construct the CA-Bneck structure.

[0014] Furthermore, the CA attention mechanism described in S2 includes the aggregation of feature information in two directions: aggregation in the horizontal direction and the vertical direction; applying pooling kernels of size (H, 1) and (1, W); encoding each channel in the horizontal and vertical coordinate directions respectively to capture long-distance relationships and position information. The CA attention mechanism can adaptively select and adjust the feature weights of different channels to better express the input data. It helps the model discover and utilize important channel information in the input data, and improves the discriminative ability and distinctiveness of the features. By suppressing unimportant channels, the redundant information in the input data can be reduced, and the model's attention to key features can be increased. Using CA-Bneck to reconstruct the Yolov5s backbone network can greatly reduce the number of model parameters and computational complexity, and help the model better capture the correlation and dependency between channels, thereby improving the model's ability to understand the input data.

[0015] Furthermore, in S2, MobileNetV3 consists of two parts: depthwise separable convolution and inverse residual structure. Depthwise separable convolution, which is composed of two independent layers, depthwise convolution and pointwise convolution, separates spatial filtering and feature generation, reducing the number of model parameters and computational cost compared to traditional convolution. In order to further improve the detection speed and performance, a more lightweight activation function Parametric Fast-Sigmoid(x) is proposed. In depthwise separable convolution, this activation function can effectively reduce the computational overhead of each layer output, while maintaining numerical stability and computational efficiency in the layer-by-layer transmission of the inverse residual structure. The activation function Parametric Fast-Sigmoid(x) formula is as follows:

[0016]

[0017] Among them, x is the input variable, Parametric Fast-Sigmoid(x) is the output result, α controls the output gain, β affects the compression degree of the input, and γ is the offset. Through the dynamic adjustment of α, β, and γ, the activation function can better adapt to different feature distributions, enhance the expression ability of the model, reduce complex nonlinear calculations, and greatly improve the calculation speed of edge devices.

[0018] Furthermore, in S3, traditional loss functions include classification loss, confidence loss, and bounding box loss, which have problems in dealing with feature occlusion and multi-feature overlap. Therefore, designing spatial factors and confidence factors to construct an adaptive loss function A-IoU can help make more reasonable scores when the feature distance is close or the occlusion is severe. The A-IoU loss function calculation formula is as follows:

[0019] A-IoU=(IoU α Confidence β SpatialFactor γ )

[0020] Among them, IoU is a traditional loss function, which represents the overlap between the detection box and the true box, Confidence represents the confidence of the detection box, SpatialFactor is used to quantify the relative position of the detection box and the true box in space, α, β and γ are adjustable parameters used to balance the impact of IoU, confidence and spatial factor on the final score. The spatial factor is the distance factor between the center point of the detection box and the center point of the true box, and the formula is as follows:

[0021] SpatialFactor = exp(-d / σ)

[0022] Where d represents the Euclidean distance between the center of the detection box and the true box, and σ is the smoothing parameter.

[0023] Furthermore, in S4, the model compression is achieved by pruning the model. Before pruning the model, the constructed MobileNetV3-CA-Bneck model needs to be dynamically sparsely trained to distinguish the importance of channels. The batch normalization (BN) layer can accelerate the convergence of neural network training and improve the generalization performance of the network. Its principle is to keep the activation input of each layer in the same distribution during training, so that the activation input value is always in the activation sensitive area, away from the derivative saturation area, and alleviate the gradient vanishing problem in back propagation.

[0024] In the formula, γ i is the scale factor of channel i in the BN layer, which is an indicator for screening the importance of channels. The L1 norm is calculated for the scale factors of these channels, and the scale factor L1 is regularized to make it approach 0. The convolution kernel corresponding to the scale factor close to 0 contributes little to the network, and pruning it will not have much impact on the accuracy. The calculation formula after L1 regularization is:

[0025]

[0026] Add a scale scaling factor γ to the network loss function i The L1 norm term of , the reconstructed loss function is:

[0027]

[0028] In the formula, L original is the original loss function, λ is the sparse regularization factor, which is used to control the weight of the sparse regularization term in the loss function, and γ i is the scale scaling factor of channel i in the BN layer, and p(t) is the dynamic sparsity, which is dynamically adjusted with the training process t. Dynamic sparsity adjustment can help the network gradually learn the importance of channels, make the pruning process smoother, and reduce the impact on accuracy.

[0029] The beneficial effects of the present invention are:

[0030] 1. This paper adopts the Bneck structure in the MobileNetV3 network and combines it with the CA attention mechanism to construct the CA-Bneck structure. It also uses CA-Bneck to reconstruct the YOLOv5s backbone network, so that the model pays more attention to the main information, thereby enhancing the feature extraction capability in the industrial environment, improving the network detection accuracy, and effectively reducing the number of network parameters and complexity.

[0031] 2. The present invention proposes a fast sigmoid function to construct the MobileNetV3 activation function, which reduces complex nonlinear calculations, so that the optimized model can be locally deployed on edge devices.

[0032] 3. Based on the traditional loss function, the present invention designs adjustable parameters and spatial factors to construct an adaptive loss function Adaptive-IoU (A-IoU), which can help to make more reasonable scoring when the feature distance is close or the occlusion is severe.

[0033] 4. The present invention proposes an adaptive channel pruning method to perform dynamic sparse training on the feature detection model. After the model is pruned, the number of parameters is reduced, but the detection performance will also be slightly reduced. In order to maximize the performance of the detection network, a dynamic sparse strategy is adopted in the sparse training process. By gradually adjusting the scale scaling factor of the channel, unimportant channels are sparsed to the target range, and continuous iterations are performed to obtain the best pruned model.

[0034] 5. The feature detection method in this industrial environment adopts edge computing based on NPU neural network processor devices to deploy the optimized feature detection model directly on the edge device, greatly reducing the training and inference time of the model.

[0035] Other advantages, objectives and features of the present invention will be described in the following description to some extent, and to some extent, will be obvious to those skilled in the art based on the following examination and study, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention will be described in detail below in conjunction with the accompanying drawings, wherein:

[0037] Figure 1 It is a flow chart of the industrial environment feature detection method in the present invention;

[0038] Figure 2 The CA-Bneck structure diagram constructed in the present invention;

[0039] Figure 3 This is the structural diagram of the CA attention mechanism in the present invention;

[0040] Figure 4 A structural diagram of the network model constructed in the present invention;

[0041] Figure 5 This is a flowchart of the adaptive channel pruning proposed in the present invention. DETAILED DESCRIPTION

[0042] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner, and the following embodiments and features in the embodiments can be combined with each other without conflict.

[0043] Among them, the drawings are only used for illustrative explanations, and they only represent schematic diagrams rather than actual pictures, and should not be understood as limitations on the present invention. In order to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of actual products. For those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0044] The same or similar numbers in the drawings of the embodiments of the present invention correspond to the same or similar parts; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "front", "rear", etc. indicate the orientation or position relationship, they are based on the orientation or position relationship shown in the drawings, which is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the terms describing the position relationship in the drawings are only used for illustrative purposes and cannot be understood as limiting the present invention. For ordinary technicians in this field, the specific meanings of the above terms can be understood according to specific circumstances.

[0045] See also Figure 1 to Figure 5 , the present invention provides a technical solution: a feature detection method in an industrial environment, comprising the following steps:

[0046] S1: Use a public industrial environment feature image dataset as a test, training, and validation dataset; generate more diverse samples by using a variety of data enhancement preprocessing methods such as random cropping, random scaling, and arbitrary combination and splicing to expand the dataset, and finally obtain 4,000 images for model training and testing. Use the visual image annotation tool LabelImg to annotate the collected feature data, and use MakeSense for semi-automatic annotation to generate a txt format annotation file. The processed dataset is randomly divided into training set, validation set, and test set in a ratio of 8:1:1.

[0047] S2: Use the improved CA-Bneck as the basic feature extraction module, which includes the backbone feature extraction structure, attention structure, and inverse residual structure. CA-BNeck uses an inverse residual structure. First, the input feature map is dimensionalized by a 1×1 convolution, so that the network can extract more features. Then, a 3×3 convolution is performed for depth-separable convolution. The output feature map is passed through the attention module to obtain the weights of different channels. Then, a 1×1 convolution is used to reduce the dimension to obtain the output Output. When the number of input and output feature layers is the same, the residual is directly connected to the Output and then output. For details, Figure 2 shown.

[0048] Among them, in order to enhance the model's attention to features in industrial environments, the CA attention mechanism is integrated into the Bneck structure. The CA attention mechanism is as follows Figure 3 As shown in the figure, the performance of mobile networks is enhanced by embedding position information into channel attention. Unlike channel attention, which converts feature tensors into a single feature vector through 2D global pooling, coordinate attention decomposes channel attention into two 1D feature encoding processes, aggregating features along two spatial directions respectively. In this way, long-range dependencies that are critical to visual tasks can be captured; in addition, position information is embedded into channel attention, enabling mobile networks to focus on larger areas without introducing significant computational overhead. In order to alleviate the loss of position information caused by 2D global pooling, channel attention is decomposed into two parallel 1D feature encoding processes to effectively integrate spatial coordinate information into the generated attention map. Specifically, two 1D global pooling operations are used to aggregate input features along the vertical and horizontal directions, respectively, to generate two feature maps containing direction-specific information. These two feature maps are then encoded into two attention maps, each of which is able to capture the long-range dependencies of the input feature map along one spatial direction. This improves the model's ability to accurately locate the target and enhances the feature extraction effect. By integrating the CA attention mechanism into the CA-Bneck structure, the backbone network of YOLOv5s is reconstructed to obtain a lightweight backbone network. The specific structure is as follows Figure 4 shown.

[0049] Since the activation function sigmoid of MobileNetV3 itself needs to use Taylor series expansion iteration to solve the weights corresponding to each channel of the feature detection model, the calculation on the edge device is slow. In order to further improve the detection speed and performance, a more lightweight activation function Parametric Fast-Sigmoid(x) is proposed. In the depth-separable convolution, this activation function can effectively reduce the computational overhead of each layer output, while maintaining numerical stability and computational efficiency in the layer-by-layer transmission of the inverse residual structure. The activation function Parametric Fast-Sigmoid(x) formula is as follows:

[0050]

[0051] Among them, x is the input variable, Parametric Fast-Sigmoid(x) is the output result, α controls the output gain, β affects the compression degree of the input, and γ is the offset. Through the dynamic adjustment of α, β, and γ, the activation function can better adapt to different feature distributions, enhance the expression ability of the model, reduce complex nonlinear calculations, and greatly improve the calculation speed of edge devices.

[0052] S3: Traditional loss functions include classification loss, confidence loss, and bounding box loss. They have problems in dealing with feature occlusion and multi-feature overlap. Therefore, designing spatial factors and confidence factors to construct an adaptive loss function A-IoU can help make more reasonable scores when the feature distance is close or the occlusion is severe. The A-IoU loss function calculation formula is as follows:

[0053] A-IoU=(IoU α Confidence β SpatialFactor γ )

[0054] Among them, IoU is a traditional loss function, which represents the overlap between the detection box and the true box, Confidence represents the confidence of the detection box, SpatialFactor is used to quantify the relative position of the detection box and the true box in space, α, β and γ are adjustable parameters used to balance the impact of IoU, confidence and spatial factor on the final score. The spatial factor is the distance factor between the center point of the detection box and the center point of the true box, and the formula is as follows:

[0055] SpatialFactor = exp(-d / σ)

[0056] Where d represents the Euclidean distance between the center of the detection box and the true box, and σ is a smoothing parameter. The A-IoU loss function proposed in the present invention can fully capture the image feature information and significantly improve the detection model's recognition and positioning accuracy of small features. Combined with the role of the CA attention mechanism, this method effectively reduces the interference of complex background on feature recognition while enhancing the model's feature recognition ability, allowing the model to focus more on key information. Especially when dealing with overlapping and close features in complex scenes, the synergy of the A-IoU loss function and the CA attention mechanism improves the saliency and recognition accuracy of the features, enabling the model to accurately detect and locate key information even in complex backgrounds.

[0057] S4: From the perspective of cost-effectiveness of equipment, it is relatively difficult to deploy large-scale feature detection network models on edge devices with limited computing power. In order to pursue a balance between detection speed and accuracy, model compression methods are usually used to improve feature detection network models. The present invention proposes a pruning strategy for dynamic sparse training to achieve model compression. This strategy gradually sparsifies the model parameters during the training process, and gradually reduces the convolution kernels or channels that contribute less to the model by dynamically adjusting the importance of weights, thereby reducing computational redundancy. Unlike traditional static pruning, dynamic sparse training can continuously update the sparsity of weights during training, and flexibly select structures to retain or remove based on the loss feedback of the model, thereby avoiding unnecessary computational overhead and improving the performance of the model after compression. In this way, the model can significantly reduce the number of parameters and computational costs while maintaining recognition accuracy, making it more suitable for deployment in resource-constrained edge terminal environments. The adaptive channel pruning process is as follows: Figure 5 shown.

[0058] Before pruning the model, the constructed MobileNetV3-CA-Bneck model needs to be dynamically sparsely trained to distinguish the importance of channels. The batch normalization (BN) layer can accelerate the convergence of neural network training and improve the generalization performance of the network. Its principle is to keep the activation input of each layer in the same distribution during training, so that the activation input value is always in the activation sensitive area and away from the derivative saturation area, alleviating the gradient vanishing problem in back propagation.

[0059] In the formula, γ i is the scale factor of channel i in the BN layer, which is an indicator for screening the importance of channels. The L1 norm is calculated for the scale factors of these channels, and the scale factor L1 is regularized to make it approach 0. The convolution kernel corresponding to the scale factor close to 0 contributes little to the network, and pruning it will not have much impact on the accuracy. The calculation formula after L1 regularization is:

[0060]

[0061] Add a scale scaling factor γ to the network loss function i The L1 norm term of , the reconstructed loss function is:

[0062]

[0063] In the formula, L original is the original loss function, λ is the sparse regularization factor, which is used to control the weight of the sparse regularization term in the loss function, and γ iis the scale scaling factor of channel i in the BN layer, and p(t) is the dynamic sparsity, which is dynamically adjusted with the training process t. Dynamic sparsity adjustment can help the network gradually learn the importance of channels, make the pruning process smoother, and reduce the impact on accuracy.

[0064] Finally, after the model completes channel pruning, the detection performance of the model will decrease slightly due to the reduction in parameters and the change in network structure. Therefore, in order to maximize the performance of the detection network, fine-tuning training is required to compensate for the impact of pruning on the accuracy of the detection model. The above steps are iterated continuously to obtain the network model with the best effect after pruning.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution, which should be included in the scope of the claims of the present invention.

Claims

1. A feature detection method in an industrial environment, characterized in that: The method comprises the following steps: S1. Obtain an industrial environment feature image dataset and preprocess the dataset; S2, using the Bneck structure in the MobileNetV3 network, combined with the CA attention mechanism, constructing the CA-Bneck structure, and using CA-Bneck to reconstruct the Yolov5s backbone network, and proposing a fast sigmoid function to construct the MobileNetV3 activation function; S3. Based on the traditional loss function, we design adjustable parameters and spatial factors to construct an adaptive loss function Adaptive-IoU, called A-IoU, which can provide more reasonable scores when the feature distance is close or the occlusion is severe. S4. An adaptive channel pruning method is proposed to perform dynamic sparse training on the feature detection model to obtain the optimal feature detection model and improve the inference speed of the feature detection model.

2. The feature detection method in an industrial environment according to claim 1, characterized in that: In S1, the feature images collected in the industrial environment are labeled using the visual image labeling tool LabelImg, and semi-automatic labeling is performed using MakeSense to generate a txt format labeling file.

3. The feature detection method in an industrial environment according to claim 1, characterized in that: In S2, the Bneck structure in the MobileNetV3 network is adopted and combined with the CA attention mechanism to construct a CA-Bneck structure. The CA attention mechanism can effectively capture the channel relationship in the image and improve the feature representation capability. CA-Bneck is used to reconstruct the Yolov5s backbone network to reduce the number of parameters and computational complexity of the model.

4. The feature detection method in an industrial environment according to claim 1, characterized in that: In S2, MobileNetV3 is composed of a depthwise separable convolution and an inverse residual structure; the depthwise separable convolution is composed of two independent layers, a depthwise convolution and a pointwise convolution, which separates spatial filtering and feature generation, reducing the number of parameters and computational cost of the model; in order to improve the detection speed and performance, an activation function Parametric Fast-Sigmoid (x) is proposed. In the depthwise separable convolution, the activation function reduces the computational overhead of each layer output, while maintaining numerical stability and computational efficiency in the layer-by-layer transmission of the inverse residual structure; the activation function Parametric Fast-Sigmoid (x) formula is as follows: Among them, x is the input variable, Parametric Fast-Sigmoid(x) is the output result, α controls the output gain, β affects the compression degree of the input, and γ is the offset. Through the dynamic adjustment of α, β, and γ, the activation function can better adapt to different feature distributions, enhance the expressiveness of the model, reduce complex nonlinear calculations, and greatly improve the computing speed of edge devices.

5. The feature detection method in an industrial environment according to claim 1, characterized in that: In S3, an adaptive loss function A-IoU of a spatial factor and a confidence factor is designed, and the calculation formula of the A-IoU loss function is as follows: A-IoU=(IoU α ·Confidence β ·SpatialFactor γ ) Among them, IoU is a traditional loss function, which indicates the overlap between the detection box and the true box. Confidence indicates the confidence of the detection box. SpatialFactor is used to quantify the relative position of the detection box and the true box in space. α, β and γ are adjustable parameters used to balance the impact of IoU, confidence and spatial factor on the final score. The spatial factor is the distance factor between the center point of the detection box and the center point of the true box. The formula is as follows: SpatialFactor = exp(-d / σ) Where d represents the Euclidean distance between the center of the detection box and the true box, and σ is the smoothing parameter.

6. The feature detection method in an industrial environment according to claim 1, characterized in that: In the S4, the method of pruning the model is used to achieve model compression. Before pruning the model, the constructed MobileNetV3-CA-Bneck model is dynamically sparsely trained to distinguish the importance of channels; the batch normalization BN layer accelerates the convergence of neural network training and improves the generalization performance of the network, so that the activation input of each layer is kept identically distributed during training, so that the activation input value is always in the activation sensitive area, away from the derivative saturation area, and alleviates the gradient vanishing problem in back propagation; γ i is the scale factor of channel i in the BN layer, which is an indicator of the importance of the screening channel; The L1 norm is calculated for the scale factors of these channels, and the scale factor L1 is regularized to make it approach 0. The convolution kernel corresponding to the scale factor close to 0 contributes little to the network, and pruning it will not have much impact on the accuracy. The calculation formula after L1 regularization is: Add a scale scaling factor γ to the network loss function i The L1 norm term of , the reconstructed loss function is: In the formula, L original is the original loss function, λ is the sparse regularization factor, which is used to control the weight of the sparse regularization term in the loss function, and γ i is the scale scaling factor of channel i in the BN layer, and p(t) is the dynamic sparsity, which is dynamically adjusted with the training process t. The dynamic sparsity adjustment helps the network gradually learn the importance of the channel and makes the pruning process smooth.

Images