A feature detection method in an industrial environment

Abstract

This invention relates to a feature detection method in an industrial environment, belonging to the field of Industrial Internet of Things (IIoT). The method includes the following steps: First, an attention mechanism focusing on location information is added to the Bneck structure of MobileNetV3 to construct a CA-Bneck structure, lightweighting the YOLOv5s backbone network to reduce the number of model parameters and computational complexity, thereby effectively improving real-time detection speed. To address the problems of feature occlusion and multiple feature coexistence, a spatial factor is designed to measure the relative spatial position of the detection box and the ground truth box, thus more accurately evaluating the effectiveness of the detection box even when features overlap. Finally, an adaptive channel pruning method is proposed to compress the model, and the optimized model is distributed to an industrial edge intelligent terminal. By detecting features on the edge device, the model's recognition accuracy and inference speed can be effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of industrial Internet of Things and relates to a feature detection method in an industrial environment. Background Technology

[0002] In recent years, with the continuous development of the social economy, the construction of the Industrial Internet of Things (IIoT) has achieved remarkable results. As the scale of the IIoT continues to expand and the ecological environment improves, the safety and reliability of industrial environments have become increasingly important. Industrial environments typically face various potential risks, including features such as balloons and hanging debris, which can cause equipment malfunctions within the IIoT, significantly impacting people's production and lives. Therefore, regular inspections of industrial equipment and timely risk assessment are of great significance for the safe operation of equipment in industrial environments.

[0003] Traditional feature detection methods mainly rely on manual visual inspection and simple screening techniques. These methods are not only inefficient but also susceptible to human factors, making it difficult to guarantee the accuracy and consistency of detection. With the development of technologies such as computer vision, sensor technology, and machine learning, more and more automated inspection systems are being introduced into industrial production. These systems can monitor products on the production line in real time, and through image processing and data analysis, quickly identify and locate features, thereby improving inspection efficiency and accuracy.

[0004] Even though deep learning and edge computing technologies have made significant progress in industrial deployments, several problems and challenges remain to be addressed in practice. For example, most edge devices, due to limited computing resources, cannot directly run feature detection models that require substantial computational resources. Furthermore, the long-distance transmission of feature images from the edge to the cloud server incurs high communication costs. Existing intelligent sensing terminals can only perform simple image acquisition tasks and still need to periodically send feature images to the cloud data center for processing. A large number of feature images will overload the cloud data center, potentially causing a data explosion. Moreover, excessively long upload intervals can lead to missed feature detections, triggering serious incidents and severely compromising emergency response capabilities, making it difficult to meet the timeliness requirements of cloud-edge data transmission for feature detection in industrial environments. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a feature detection method in an industrial environment. Leveraging the advantages of edge computing, feature processing and analysis are pushed to the edge device closest to the feature source, achieving real-time feature detection and reducing detection latency.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A feature detection method in an industrial environment includes the following steps:

[0008] S1. Obtain the industrial environment feature image dataset and preprocess the dataset;

[0009] S2. Adopting the Bneck structure in the MobileNetV3 network and combining it with the CA attention mechanism, construct the CA-Bneck structure, and use the CA-Bneck to reconstruct the Yolov5s backbone network. Propose a fast sigmoid function to construct the MobileNetV3 activation function.

[0010] S3. Based on the traditional loss function, we design adjustable parameters and spatial factors to construct an adaptive loss function Adaptive-IoU (A-IoU), which provides a more reasonable score when the feature distance is close or the occlusion is severe.

[0011] S4. An adaptive channel pruning method is proposed to perform dynamic sparse training on the feature detection model, obtain the optimal feature detection model, and improve the inference speed of the feature detection model.

[0012] In S1, the visual image annotation tool LabelImg is used to annotate the feature images collected in the industrial environment, and Make Sense is used for semi-automatic annotation to generate txt format annotation files.

[0013] In S2, the Bneck structure from the MobileNetV3 network is adopted, and a CA attention mechanism is combined to construct a CA-Bneck structure.

[0014] Furthermore, the CA attention mechanism described in S2 includes feature information aggregation in two directions: horizontal and vertical. It applies pooling kernels of sizes (H,1) and (1,W), encoding each channel in both the horizontal and vertical coordinate directions to capture long-distance relationships and positional information. The CA attention mechanism adaptively selects and adjusts the feature weights of different channels to better represent the input data. This helps the model discover and utilize important channel information in the input data, improving the discriminative power and distinguishability of features. By suppressing unimportant channels, redundant information in the input data can be reduced, increasing the model's focus on key features. Reconstructing the Yolov5s backbone network using CA-Bneck significantly reduces the number of model parameters and computational complexity, helping the model better capture the correlations and dependencies between channels, thereby improving the model's understanding of the input data.

[0015] Furthermore, in S2, MobileNetV3 consists of two parts: depthwise separable convolutions and inverse residual structures. The depthwise separable convolutions, composed of two independent layers—depthwise convolutions and pointwise convolutions—separate spatial filtering and feature generation, reducing the number of parameters and computational cost compared to traditional convolutions. To further improve detection speed and performance, a more lightweight activation function, Parametric Fast-Sigmoid(x), is proposed. In depthwise separable convolutions, this activation function effectively reduces the computational cost of each layer's output, while maintaining numerical stability and computational efficiency during the layer-by-layer propagation of the inverse residual structure. The formula for the Parametric Fast-Sigmoid(x) activation function is as follows:

[0016]

[0017] Here, x is the input variable, Parametric Fast-Sigmoid(x) is the output, α controls the output gain, β affects the degree of input compression, and γ is the offset. By dynamically adjusting α, β, and γ, the activation function can better adapt to different feature distributions, enhance the model's expressive power, reduce complex nonlinear calculations, and significantly improve the computing speed of edge devices.

[0018] Furthermore, in S3, traditional loss functions, including classification loss, confidence loss, and bounding box loss, have problems handling feature occlusion and multiple feature overlap. Therefore, an adaptive loss function, A-IoU, is constructed by designing spatial and confidence factors. This can help achieve more reasonable scoring when features are close together or severely occluded. The formula for calculating the A-IoU loss function is as follows:

[0019] A-IoU=(IoU α Confidence β SpatialFactor γ )

[0020] Where IoU is the traditional loss function, representing the overlap between the detection box and the ground truth box; Confidence represents the confidence level of the detection box; SpatialFactor quantifies the spatial relative position of the detection box and the ground truth box; and α, β, and γ are adjustable parameters used to balance the influence of IoU, confidence level, and spatial factor on the final score. The spatial factor is the distance between the center point of the detection box and the center point of the ground truth box, and the formula is as follows:

[0021] SpatialFactor = exp(-d / σ)

[0022] Where d represents the Euclidean distance between the center of the detection box and the center of the ground truth box, and σ is the smoothing parameter.

[0023] Furthermore, in S4, model compression is achieved through model pruning. Before pruning, the constructed MobileNetV3-CA-Bneck model needs to undergo dynamic sparse training to differentiate the importance of channels. Batch normalization (BN) layers can accelerate neural network training convergence and improve network generalization performance. Its principle is to ensure that the activation inputs of each layer remain identically distributed during training, keeping the activation input values ​​always within the activation-sensitive region and away from the derivative saturation region, thus mitigating the gradient vanishing problem in backpropagation.

[0024] In the formula γ i Let be the scaling factor for channel i in the Batch Normalization (BN) layer, serving as an indicator for selecting channel importance. The L1 norm of these scaling factors is calculated, and L1 regularization is applied to bring the scaling factors closer to 0. Convolutional kernels corresponding to scaling factors close to 0 contribute little to the network, and pruning them will not significantly impact accuracy. The L1 regularized calculation formula is as follows:

[0025]

[0026] Adding a scaling factor γ to the network loss function i The L1 norm term yields the reconstructed loss function as follows:

[0027]

[0028] In the formula, L original Here, λ is the original loss function, λ is the sparsity regularization factor used to control the weight of the sparsity regularization term in the loss function, and γ is the weight of the sparse regularization term. i denoted as the scaling factor for channel i in the BN layer, and p(t) as the dynamic sparsity, which is dynamically adjusted as training progresses. Adjusting the dynamic sparsity helps the network gradually learn the importance of channels, making the pruning process smoother and reducing the impact on accuracy.

[0029] The beneficial effects of this invention are as follows:

[0030] 1. This invention adopts the Bneck structure in the MobileNetV3 network and combines it with the CA attention mechanism to construct a CA-Bneck structure. The CA-Bneck is then used to reconstruct the YOLOv5s backbone network, making the model focus more on the main information, thereby enhancing the feature extraction capability in industrial environments, improving the network detection accuracy, and effectively reducing the number of network parameters and complexity.

[0031] 2. This invention proposes a fast sigmoid function to construct the MobileNetV3 activation function, which reduces complex nonlinear calculations and enables the optimized model to be deployed locally on edge devices.

[0032] 3. Based on the traditional loss function, this invention designs an adjustable parameter and spatial factor to construct an adaptive loss function, Adaptive-IoU (A-IoU), which can help to make more reasonable scores when the feature distance is short or the occlusion is severe.

[0033] 4. This invention proposes an adaptive channel pruning method for dynamic sparse training of the feature detection model. After pruning, the number of parameters is reduced, but the detection performance will also decrease slightly. In order to maximize the performance of the detection network, a dynamic sparsity strategy is adopted in the sparse training process. By gradually adjusting the scaling factor of the channels, the unimportant channels are sparsified to the target range. Through continuous iteration, the optimal pruning model is obtained.

[0034] 5. The feature detection method in this industrial environment adopts edge computing based on NPU neural network processor devices, and deploys the optimized feature detection model directly on the edge device, which greatly reduces the training and inference time of the model.

[0035] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0037] Figure 1 This is a flowchart of the industrial environment feature detection method in this invention;

[0038] Figure 2 This is a diagram of the CA-Bneck structure constructed in this invention;

[0039] Figure 3 This is a structural diagram of the CA attention mechanism in this invention;

[0040] Figure 4 This is a diagram of the network model structure constructed in this invention;

[0041] Figure 5 This is a flowchart of the adaptive channel pruning process proposed in this invention. Detailed Implementation

[0042] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0043] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0044] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0045] Please see Figures 1-5 The present invention provides a technical solution: a feature detection method in an industrial environment, comprising the following steps:

[0046] S1: A publicly available industrial environment feature image dataset was used as the test, training, and validation dataset. Various data augmentation preprocessing methods were employed to generate more diverse samples, including random cropping, random scaling, and arbitrary combination and stitching. Ultimately, 4000 images were obtained for model training and testing. The collected feature data was labeled using the visual image annotation tool LabelImg, and semi-automated annotation was performed using MakeSense to generate TXT format annotation files. The processed dataset was then randomly divided into training, validation, and test sets in an 8:1:1 ratio.

[0047] S2: The improved CA-Bneck is used as the basic feature extraction module, including a backbone feature extraction structure, an attention structure, and an inverse residual structure. CA-BNeck employs an inverse residual structure, first performing dimensionality upscaling on the input feature map through a 1×1 convolution, allowing the network to extract more features. Then, a depthwise separable convolution is performed using a 3×3 convolution. The output feature map passes through an attention module to obtain weights for different channels. Finally, a 1×1 convolution is used for dimensionality reduction to obtain the output. When the number of input and output feature layers is the same, the residual is directly concatenated with the output before output. Specifically... Figure 2 As shown.

[0048] To enhance the model's attention to features in industrial environments, a CA attention mechanism is integrated into the Bneck structure. The CA attention mechanism is as follows: Figure 3 As shown, the performance of mobile networks is enhanced by embedding location information into channel attention. Unlike channel attention, which transforms feature tensors into a single feature vector through 2D global pooling, coordinate attention decomposes channel attention into two 1D feature encoding processes, aggregating features along two spatial directions respectively. This approach captures long-range dependencies crucial for visual tasks; furthermore, embedding location information into channel attention allows mobile networks to focus on larger regions without introducing significant computational overhead. To mitigate the loss of location information caused by 2D global pooling, channel attention is decomposed into two parallel 1D feature encoding processes to effectively integrate spatial coordinate information into the generated attention map. Specifically, two 1D global pooling operations are used to aggregate input features along the vertical and horizontal directions respectively, generating two feature maps containing orientation-specific information. These two feature maps are then encoded into two attention maps, each capable of capturing long-range dependencies of the input feature map along a spatial direction. This improves the model's ability to accurately locate targets and enhances feature extraction performance. By incorporating the CA attention mechanism into the CA-Bneck structure, a lightweight backbone network is obtained by reconstructing the YOLOv5s backbone network, as shown in the specific structure below. Figure 4 As shown.

[0049] Because the sigmoid activation function in MobileNetV3 requires iterative Taylor series expansion to calculate the weights for each channel in the feature detection model, it is computationally slow on edge devices. To further improve detection speed and performance, a more lightweight activation function, Parametric Fast-Sigmoid(x), is proposed. In depthwise separable convolutions, this activation function can effectively reduce the computational overhead of each layer's output, while maintaining numerical stability and computational efficiency in the layer-by-layer propagation of the inverse residual structure. The formula for the Parametric Fast-Sigmoid(x) activation function is as follows:

[0050]

[0051] Here, x is the input variable, Parametric Fast-Sigmoid(x) is the output, α controls the output gain, β affects the degree of input compression, and γ is the offset. By dynamically adjusting α, β, and γ, the activation function can better adapt to different feature distributions, enhance the model's expressive power, reduce complex nonlinear calculations, and significantly improve the computing speed of edge devices.

[0052] S3: Traditional loss functions, including classification loss, confidence loss, and bounding box loss, have problems handling feature occlusion and multiple feature overlap. Therefore, designing spatial factors and confidence factors to construct an adaptive loss function, A-IoU, can help achieve more reasonable scoring when features are close together or severely occluded. The formula for calculating the A-IoU loss function is as follows:

[0053] A-IoU=(IoU α Confidence β SpatialFactor γ )

[0054] Where IoU is the traditional loss function, representing the overlap between the detection box and the ground truth box; Confidence represents the confidence level of the detection box; SpatialFactor quantifies the spatial relative position of the detection box and the ground truth box; and α, β, and γ are adjustable parameters used to balance the influence of IoU, confidence level, and spatial factor on the final score. The spatial factor is the distance between the center point of the detection box and the center point of the ground truth box, and the formula is as follows:

[0055] SpatialFactor = exp(-d / σ)

[0056] Where d represents the Euclidean distance between the detection box and the center of the ground truth box, and σ is a smoothing parameter. The A-IoU loss function proposed in this invention can fully capture image feature information, significantly improving the accuracy of the detection model in recognizing and locating small features. Combined with the CA attention mechanism, this method enhances the model's feature recognition ability while effectively reducing the interference of complex backgrounds on feature recognition, allowing the model to focus more on key information. Especially when dealing with overlapping and close features in complex scenes, the synergistic effect of the A-IoU loss function and the CA attention mechanism improves the saliency and recognition accuracy of features, enabling the model to accurately detect and locate key information even in complex backgrounds.

[0057] S4: From a cost-effectiveness perspective, deploying large feature detection network models on edge devices with limited computing power is quite difficult. To achieve a balance between detection speed and accuracy, model compression is often used to improve feature detection network models. This invention proposes a dynamic sparse training pruning strategy to achieve model compression. This strategy gradually sparsifies model parameters during training, dynamically adjusting the importance of weights and gradually reducing convolutional kernels or channels that contribute less to the model, thereby reducing computational redundancy. Unlike traditional static pruning, dynamic sparse training continuously updates the sparsity of weights during training, flexibly selecting which structures to retain or remove based on the model's loss feedback, thus avoiding unnecessary computational overhead and improving the model's performance after compression. Through this method, the model can significantly reduce the number of parameters and computational cost while maintaining recognition accuracy, making it more suitable for deployment in resource-constrained edge terminal environments. The adaptive channel pruning process is as follows: Figure 5 As shown.

[0058] Before pruning the model, the constructed MobileNetV3-CA-Bneck model needs to be dynamically sparsely trained to differentiate the importance of channels. Batch normalization (BN) layers can accelerate neural network training convergence and improve network generalization performance. Its principle is to keep the activation inputs of each layer identically distributed during training, so that the activation input values ​​are always in the activation-sensitive region and far from the derivative saturation region, thus alleviating the gradient vanishing problem in backpropagation.

[0059] In the formula γ i Let be the scaling factor for channel i in the Batch Normalization (BN) layer, serving as an indicator for selecting channel importance. The L1 norm of these scaling factors is calculated, and L1 regularization is applied to bring the scaling factors closer to 0. Convolutional kernels corresponding to scaling factors close to 0 contribute little to the network, and pruning them will not significantly impact accuracy. The L1 regularized calculation formula is as follows:

[0060]

[0061] Adding a scaling factor γ to the network loss function i The L1 norm term yields the reconstructed loss function as follows:

[0062]

[0063] In the formula, L original Here, λ is the original loss function, λ is the sparsity regularization factor used to control the weight of the sparsity regularization term in the loss function, and γ is the weight of the sparse regularization term. idenoted as the scaling factor for channel i in the BN layer, and p(t) as the dynamic sparsity, which is dynamically adjusted as training progresses. Adjusting the dynamic sparsity helps the network gradually learn the importance of channels, making the pruning process smoother and reducing the impact on accuracy.

[0064] Finally, after channel pruning, the model's detection performance will slightly decrease due to the reduction in parameters and the change in network structure. Therefore, to maximize the performance of the detection network, fine-tuning is needed to compensate for the impact of pruning on the accuracy of the detection model. By iterating through these steps, the optimal network model after pruning can be obtained.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A feature detection method in an industrial environment, characterized in that: The method includes the following steps: S1. Obtain the industrial environment feature image dataset and preprocess the dataset; S2. The Bneck structure from the MobileNetV3 network is adopted, combined with the CA attention mechanism to construct a CA-Bneck structure. The Yolov5s backbone network is then reconstructed using the CA-Bneck, and a fast sigmoid function is proposed to construct the MobileNetV3 activation function. In S2, MobileNetV3 consists of depthwise separable convolutions and inverse residual structures. The depthwise separable convolutions are composed of two independent layers: depthwise convolution and pointwise convolution, separating spatial filtering and feature generation to reduce the number of model parameters and computational cost. To improve detection speed and performance, the activation function Parametric Fast-Sigmoid(x) is proposed. In the depthwise separable convolutions, the activation function reduces the computational cost of each layer's output, while maintaining numerical stability and computational efficiency in the layer-by-layer propagation of the inverse residual structure. The formula for the activation function Parametric Fast-Sigmoid(x) is as follows: Where x is the input variable, Parametric Fast-Sigmoid(x) is the output result, α controls the output gain, β affects the degree of input compression, and γ is the offset. By dynamically adjusting α, β, and γ, the activation function can better adapt to different feature distributions, enhance the expressive power of the model, reduce complex nonlinear calculations, and significantly improve the computing speed of edge devices. S3. Based on the traditional loss function, an adaptive loss function, A-IoU, is constructed by designing adjustable parameters and a spatial factor. This function provides a more reasonable score when feature distances are short or occlusion is severe. In S3, the adaptive loss function A-IoU, which incorporates spatial and confidence factors, is calculated using the following formula: A-IoU=(IoU α ·Confidence β ·SpatialFactor γ ) Where IoU is the traditional loss function, representing the overlap between the detection box and the ground truth box; Confidence represents the confidence level of the detection box; SpatialFactor quantifies the spatial relative position of the detection box and the ground truth box; α, β, and γ are adjustable parameters used to balance the influence of IoU, confidence level, and spatial factor on the final score; the spatial factor is the distance factor between the center point of the detection box and the center point of the ground truth box, and the formula is as follows: SpatialFactor = exp(-d / σ) Where d represents the Euclidean distance between the center of the detection box and the center of the ground truth box, and σ is the smoothing parameter; S4. An adaptive channel pruning method is proposed to perform dynamic sparse training on the feature detection model, obtain the optimal feature detection model, and improve the inference speed of the feature detection model. In S4, the model pruning method is used to compress the model. Before pruning the model, the constructed MobileNetV3-CA-Bneck model is dynamically sparsely trained to distinguish the importance of channels. Batch normalized (BN) layers accelerate the convergence of neural network training and improve the network generalization performance, so that the activation input of each layer is kept in the same distribution during training, so that the activation input value is always in the activation sensitive region and far away from the derivative saturation region, thus alleviating the gradient vanishing problem in backpropagation. γ i Let be the scaling factor of channel i in the BN layer, which is an indicator for selecting channel importance. The L1 norm of these scaling factors is calculated, and L1 regularization is applied to make the scaling factors approach 0. Convolutional kernels corresponding to scaling factors close to 0 contribute little to the network, and removing them will not significantly affect accuracy. The calculation formula after L1 regularization is: Adding a scaling factor γ to the network loss function i The L1 norm term yields the reconstructed loss function as follows: In the formula, L original Here, λ is the original loss function, λ is the sparsity regularization factor used to control the weight of the sparsity regularization term in the loss function, and γ is the sparse regularization factor. i is the scaling factor for channel i in the BN layer, and p(t) is the dynamic sparsity, which is dynamically adjusted as training progresses. The adjustment of the dynamic sparsity helps the network gradually learn the importance of channels, making the pruning process smooth.

2. The feature detection method in an industrial environment according to claim 1, characterized in that: In step S1, the visual image annotation tool LabelImg is used to annotate the feature images collected in the industrial environment, and MakeSense is used for semi-automatic annotation to generate a txt format annotation file.

3. The feature detection method in an industrial environment according to claim 1, characterized in that: In S2, the Bneck structure in the MobileNetV3 network is adopted, and the CA attention mechanism is combined to construct the CA-Bneck structure. The CA attention mechanism can effectively capture the channel relationship in the image and improve the feature representation capability. The Yolov5s backbone network is reconstructed using CA-Bneck to reduce the number of model parameters and computational complexity.

Images