Industrial scene lightweight target detection method based on NFL-DETR network

Abstract

The invention discloses an industrial scene lightweight target detection method based on an NFL-DETR network, and the method comprises the steps: obtaining an industrial scene source image, and adjusting the size of the image to 1280 * 800; the adjusted image is input into an NDF-DETR network, and the NDF-DETR network is composed of a TriCBAM-NET feature extraction network, an NBF-NET normalized deviation feature fusion network and a DETR-Transform coding and decoding network; in the TriCBAM-NET feature extraction network, an image is processed in four stages in sequence, and each stage comprises an embedded layer, a CBAM attention layer and a plurality of Tri-Blocks, so that image features are extracted step by step; inputting the image features extracted in the four stages into an NBF-NET normalized deviation feature fusion network, and carrying out feature integration and normalization processing; and the processed features are inputted into a DETR-Transform coding and decoding network, and a prediction bounding box is generated.

Description

Technical Field

[0001] This invention relates to a lightweight target detection method for industrial scenarios based on NFL-DETR networks. Background Technology

[0002] With the iterative development of artificial intelligence technology, especially the breakthroughs in deep learning methods, the field of object detection has shown a significant shift in methodology. In industrial scenarios, object detection faces continuous technical challenges, specifically as follows:

[0003] Limitations of traditional detection methods:

[0004] Early two-stage detection frameworks primarily relied on a combination of Region Proposal Networks (RPNs) and Convolutional Neural Networks (CNNs). While these methods achieved breakthroughs in detection accuracy, the generated region proposals exhibited significant spatial redundancy, necessitating post-processing using heuristic algorithms such as Non-Maximum Suppression (NMS).

[0005] While anchor-based single-stage detectors offer greater computational efficiency, they struggle to balance accuracy and speed, making it difficult to simultaneously meet the high demands of industrial applications for both precision and speed.

[0006] The rise of the Transformer architecture:

[0007] The Transformer architecture initially demonstrated superior modeling capabilities compared to recurrent neural networks in the field of natural language processing, with its self-attention mechanism effectively capturing long-range dependencies. Based on this, the Detection Transformer (DETR) achieved, for the first time, a detection process without manually designing anchor boxes or NMS through an end-to-end bipartite graph matching strategy, providing a new technological paradigm for object detection in industrial scenarios. However, the DETR detection method has the following shortcomings:

[0008] (1) The training convergence speed is significantly slower than that of traditional CNN detectors, and its complexity is higher than that of other networks, resulting in a longer training time required to achieve a specific detection result.

[0009] (2) Directly encoding the high-level features extracted by CNN and passing them to Transformer will lose information from small local regions, especially for small objects in the image, and the accuracy is generally poor in small target detection.

[0010] (2) It cannot adaptively adjust the weights and scales of the feature map, lacks strong generalization ability, and has average accuracy when dealing with targets of different shapes and poses. Summary of the Invention

[0011] This invention provides a lightweight target detection method for industrial scenarios based on NFL-DETR networks to address the problems existing in the prior art.

[0012] The technical solutions adopted in this invention are as follows:

[0013] A lightweight target detection method for industrial scenarios based on NFL-DETR networks includes the following steps:

[0014] S1: Acquire source images of industrial scenes and adjust the image size to 1280*800;

[0015] S2: Input the adjusted image into the NFL-DETR network, which consists of a TriCBAM-NET feature extraction network, an NBF-NET normalized bias feature fusion network, and a DETR-Transformer encoder-decoder network;

[0016] In the TriCBAM-NET feature extraction network, the image is processed through four stages in sequence. Each stage includes an embedding layer, a CBAM attention layer, and multiple Tri-Block blocks to extract image features step by step.

[0017] The image features extracted in the four stages are input into the NBF-NET normalized bias feature fusion network for feature integration and normalization.

[0018] The processed features are input into the DETR-Transformer encoder-decoder network to generate predicted bounding boxes, including: location information, category, and category confidence.

[0019] The TriCBAM-NET feature extraction network combines depthwise separable convolutions and 1*1 convolutions to achieve network lightweighting while maintaining network accuracy and improving network generalization ability; the NBF-NET normalized bias feature fusion network integrates and normalizes features from different stages to maintain feature integrity, especially in small target detection.

[0020] Furthermore, in each stage of the TriCBAM-NET feature extraction network,

[0021] The embedding layer uses convolutional layers with different kernel sizes to extract image features, change image size, and number of channels;

[0022] The CBAM attention layer is used to enhance the feature extraction capability of convolutional neural networks. It achieves adaptive feature enhancement by first selecting the feature type and then determining the attention position through refined processing.

[0023] The Tri-Block consists of two parts: depthwise separable convolution and 1*1 convolution. It uses three separately distributed 1*1 convolution kernels to map the input to a non-linear feature space, and sets residual connections in each Tri-Block to compensate for the loss of inter-channel correlation information caused by depthwise separable convolution.

[0024] Furthermore, the NBF-NET normalized bias feature fusion network includes: deconvolution, convolution, normalization, and LeakyReLU activation functions.

[0025] The image features from the four stages of the TriCBAM-NET feature extraction network are integrated and normalized to highlight the spatial reduction effect of small target features, so that the features of small objects can remain intact during further processing.

[0026] Furthermore, the DETR-Transformer encoder-decoder network employs an end-to-end bipartite graph matching strategy and uses the Hungarian algorithm to uniquely match the prediction results with the ground truth labels, thereby achieving end-to-end training.

[0027] The present invention has the following beneficial effects:

[0028] (1) This invention achieves network lightweighting while maintaining network accuracy by using a combination of depthwise separable convolution and 1*1 convolution, which significantly reduces the computational complexity and number of parameters of the model. Compared with traditional detection methods, this lightweight design can significantly improve the speed of target detection, making it more suitable for industrial scenarios with high real-time requirements, and effectively solving the problem of slow detection speed in existing technologies.

[0029] (2) The TriCBAM-NET feature extraction network can improve the network's generalization ability while maintaining its accuracy. This network extracts image features step by step through its four stages, each stage containing an embedding layer, a CBAM attention layer, and several Tri-Block blocks. This step-by-step feature extraction method can capture key information in the image more efficiently, reduce redundant computation, and further improve detection efficiency.

[0030] (3) The NBF-NET normalized bias feature fusion network can integrate image features from different stages of the TriCBAM-NET feature extraction network and perform normalization processing. This process helps to highlight the spatial reduction effect of small target features, so that the features of small objects can remain intact during further processing, thereby effectively improving the detection accuracy of small targets.

[0031] (4) Embedding the CBAM attention layer during the feature extraction process can perform adaptive feature enhancement on the feature map.

[0032] (5) The training convergence speed of the present invention is faster. By optimizing the network structure and feature processing flow, better detection results can be achieved in a shorter time, reducing training costs, improving the efficiency of model development, and enabling the model to be applied to real industrial scenarios more quickly. Attached Figure Description

[0033] Figure 1 This is a diagram of the NFL-DETR network model.

[0034] Figure 2 This is a diagram of the TriCBAM-NET feature extraction network model.

[0035] Figure 3 The diagram shows the NBF-NET normalized bias feature fusion network model. Detailed Implementation

[0036] The invention will now be further described with reference to the accompanying drawings.

[0037] To address the issues of slow target detection speed, mediocre accuracy in detecting small objects, and weak generalization ability in industrial scenarios, this invention discloses a lightweight target detection method for industrial scenarios based on the NFL-DETR network, which utilizes depthwise convolution and pointwise convolution. This invention achieves model lightweighting while integrating features from the feature extraction network and normalizing them to ensure feature integrity during further processing. The method includes the following steps:

[0038] 1. Acquire source images of industrial scenes and adjust the image size to 1280*800.

[0039] 2. Input the image from step 1 into the NFL-DETR network. The NFL-DETR network model is as follows: Figure 1 As shown, the network consists of three parts: the TriCBAM-NET feature extraction network, the NBF-NET normalized bias feature fusion network, and the DETR-Transformer encoding and decoding network.

[0040] After the image enters the NFL-DETR network, it is first fed into the TriCBAM-NET feature extraction network. The TriCBAM-NET feature extraction network is as follows: Figure 2 As shown, the TriCBAM-NET feature extraction network extracts image features step by step through its four stages, each stage containing an embedding layer, a CBAM attention layer, and several Tri-Block blocks.

[0041] (1) The embedding layer is a convolutional layer with different kernel sizes, used to extract image features, change image size and number of channels.

[0042] (2) The CBAM attention layer is a lightweight neural network module used to enhance the feature extraction capability of convolutional neural networks (CNNs). It achieves adaptive feature enhancement by first selecting the feature type and then determining the attention position through refined processing.

[0043] (3) The Tri-Block consists of two parts: depthwise separable convolution and 1*1 convolution. It uses three separately distributed 1*1 convolution kernels to map the input to a very high nonlinear feature space.

[0044] Unlike traditional methods that improve performance by increasing network width, separable distributed 1x1 convolutional kernels achieve a nonlinear high-dimensional mapping similar to a multinomial kernel function through cross-channel feature multiplication. As Tri-Block blocks are stacked in the network model, each layer increases the complexity of the hidden dimensions. This feature fusion method enables the TriCBAM-NET feature extraction network to achieve near-infinite dimensions within a compact feature space, greatly enhancing the model's representational power and overall performance. Depthwise separable convolution is an efficient alternative to standard convolution; its core function is to significantly reduce computational cost and parameter count while maintaining model performance by decoupling spatial filtering and channel feature fusion.

[0045] After the image enters the Tri-Block, it first undergoes depthwise separable convolution to generate a set of feature maps. These feature maps are then fed into two separate 256*1*1 convolutions. The two resulting feature maps are pixel-wise summed to synthesize a single feature map, which is then fed into another 256*1*1 convolution to obtain a separable 1*1 convolution kernel feature map. This feature map is then fed into a depthwise separable convolution to obtain the final feature map. To compensate for the loss of inter-channel correlation information caused by depthwise separable convolution, residual connections are included in each Tri-Block.

[0046] 4. After the image enters the TriCBAM-NET feature extraction network, it first enters Stage 1. Stage 1 uses 256*4*4 convolutional kernels to obtain the corresponding feature maps. The obtained feature maps are then input into the CBAM network module to achieve adaptive feature enhancement. Finally, the enhanced feature maps are input into two Tri-Block blocks to extract the image features of Stage 1. .

[0047] 5. The image features obtained in the four steps are... Input the TriCBAM-NET feature extraction network Stage 2. Stage 2 uses 512*2*2 convolutional kernels to obtain the corresponding feature maps. The obtained feature maps are then input into the CBAM network module to achieve adaptive feature enhancement. Finally, the enhanced feature maps are input into four Tri-Block blocks to extract the Stage 2 image features. .

[0048] 6. The image features obtained in the five steps are: Input the TriCBAM-NET feature extraction network Stage 3. Stage 3 uses 1024*2*2 convolutional kernels to obtain the corresponding feature maps. The obtained feature maps are then input into the CBAM network module to achieve adaptive feature enhancement. Finally, the enhanced feature maps are input into 8 Tri-Block blocks to extract the Stage 3 image features. .

[0049] 7. The image features obtained in the six steps are: Input the TriCBAM-NET feature extraction network Stage 4. Stage 4 uses 2048*2*2 convolutional kernels to obtain the corresponding feature maps. The obtained feature maps are then input into the CBAM network module to achieve adaptive feature enhancement. Finally, the enhanced feature maps are input into four Tri-Block blocks to extract the Stage 4 image features. .

[0050] 8. Generate image features for the four stages. , , , Simultaneously inputting the NBF-NET normalized bias feature fusion network, the NBF-NET normalized bias feature fusion network is as follows: Figure 3 As shown.

[0051] The NBF-NET Normalized Bias Feature Fusion Network consists of four parts: deconvolution, convolution, normalization, and LeakyReLU activation function. It integrates features from the TriCBAM-NET feature extraction network, normalizes them, and then passes them to the DETR-Transformer encoding and decoding network. The integrated features provide more accurate spatial localization in subsequent processing. The normalization operation of the NBF-NET Normalized Bias Feature Fusion Network helps to highlight the spatial reduction effect of small target features. Furthermore, it integrates image features from different stages of the TriCBAM-NET feature extraction network. , , , By integrating them, the features of small objects can be preserved during further processing.

[0052] 9. Image Features , , , Simultaneously inputting the NBF-NET normalized bias feature fusion network, first using a 1024*2*2 deconvolution to... The feature dimension is reduced by half, while its length and width are doubled. Then, a 1024*3*3 convolution is used to extract features, and finally, a normalization operation is performed to generate a feature map.

[0053] 10. Input a 1024*3*3 convolution to extract features, then perform a normalization operation to generate a feature map. Add this feature map to the feature map generated in step nine at the pixel level, and then input the LeakyReLU activation function to activate it and obtain the feature map.

[0054] 11. Apply a 512*2*2 deconvolution to the feature map obtained in step 10 to halve its dimensions, while doubling its length and width. Then, combine it with... Perform pixel-level summation. First, use a 256*2*2 deconvolution to halve the feature dimension of the obtained feature map, while doubling its length and width. Then, input a 256*3*3 convolution to extract features. Finally, perform normalization to generate the feature map.

[0055] 12. The input is a 256*3*3 convolution to extract features, then a normalization operation is performed to generate a feature map. This feature map is then added pixel-wise to the feature map generated in step eleven, and finally activated using the LeakyReLU activation function to obtain the final feature map.

[0056] 13. Input the feature map obtained in step 12 into the DETR-Transformer encoding and decoding network to finally generate the predicted bounding box.

[0057] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be considered within the scope of protection of the present invention.

Claims

1. A lightweight target detection method for industrial scenarios based on NFL-DETR networks, characterized in that: Includes the following steps: S1: Acquire source images of industrial scenes and adjust the image size to 1280*800; S2: Input the adjusted image into the NFL-DETR network, which consists of a TriCBAM-NET feature extraction network, an NBF-NET normalized bias feature fusion network, and a DETR-Transformer encoder-decoder network; In the TriCBAM-NET feature extraction network, the image is processed through four stages in sequence. Each stage includes an embedding layer, a CBAM attention layer, and multiple Tri-Block blocks to extract image features step by step. The image features extracted in the four stages are input into the NBF-NET normalized bias feature fusion network for feature integration and normalization. The processed features are input into the DETR-Transformer encoder-decoder network to generate predicted bounding boxes.

2. The lightweight target detection method for industrial scenarios based on NFL-DETR network as described in claim 1, characterized in that: In each stage of the TriCBAM-NET feature extraction network The embedding layer uses convolutional layers with different kernel sizes to extract image features, change image size, and number of channels; The CBAM attention layer is used to enhance the feature extraction capability of convolutional neural networks. It achieves adaptive feature enhancement by first selecting the feature type and then determining the attention position through refined processing. The Tri-Block consists of two parts: depthwise separable convolution and 1*1 convolution. It uses three separately distributed 1*1 convolution kernels to map the input to a nonlinear feature space, and sets residual connections in each Tri-Block to compensate for the loss of inter-channel correlation information caused by depthwise separable convolution.

3. The lightweight target detection method for industrial scenarios based on NFL-DETR network as described in claim 1, characterized in that: The NBF-NET normalized bias feature fusion network includes: deconvolution, convolution, normalization, and LeakyReLU activation function.

4. The lightweight target detection method for industrial scenarios based on NFL-DETR network as described in claim 1, characterized in that: The DETR-Transformer encoder-decoder network achieves end-to-end training by employing an end-to-end bipartite graph matching strategy and using the Hungarian algorithm to uniquely match the predicted results with the ground truth labels.

Images