A target detection method, system, computer device, and computer-readable storage medium based on lightweight multi-scale feature fusion.

Abstract

This invention discloses a target detection method, system, computer device, and computer-readable storage medium based on lightweight multi-scale feature fusion, relating to the fields of computer vision and intelligent detection technology. The method comprises: image acquisition and preprocessing, backbone network feature extraction, neck network bidirectional weighted feature fusion, decoupled detection head prediction, post-processing, and result output. The system includes an image acquisition module, a preprocessing module, a feature extraction module, a feature fusion module, and a detection output module. By constructing a collaborative architecture of the RepViT lightweight backbone network and the RepFPN weighted bidirectional feature pyramid, this invention significantly improves the detection accuracy of small targets, occluded targets, and complex background scenes while maintaining real-time inference capabilities. It is particularly suitable for edge deployment scenarios such as UAV inspection, industrial quality inspection, and remote sensing monitoring.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and intelligent detection technology, and in particular to a target detection method, system, computer device, and computer-readable storage medium based on lightweight multi-scale feature fusion. Background Technology

[0002] Object detection is one of the core foundational tasks in computer vision, aiming to classify and spatially locate objects of interest in images or videos. With the rapid development of deep learning technology, object detection methods have mainly evolved into two major technical routes: two-stage detection methods and single-stage detection methods. The former, represented by region proposal networks, typically has high detection accuracy, but its model structure is relatively complex, computationally expensive, and inference speed is slow. The latter, represented by the YOLO series, significantly improves detection speed while maintaining considerable accuracy thanks to its end-to-end network design and efficient inference capabilities, making it more suitable for applications with high real-time requirements, such as drone inspection, industrial site quality inspection, and mobile edge devices.

[0003] In the field of power facility inspection, taking the defect detection of insulators in transmission lines as an example, insulators, as key components, are exposed to complex outdoor environments for extended periods, making them prone to various defect forms such as dirt, cracking, breakage, and spontaneous explosion. Traditional manual inspection methods suffer from inherent drawbacks such as low efficiency, significant safety hazards, and strong subjectivity, making it difficult to meet the needs of large-scale inspections. While the introduction of drone inspection technology has effectively improved the efficiency of image data acquisition, defect targets in inspection images often exhibit characteristics such as small scale, varied shapes, and blurred edges. Furthermore, the background often contains complex interference factors such as the sky, conductors, towers, and vegetation, placing extremely high demands on the performance of subsequent intelligent detection algorithms. Similar technical challenges exist in practical application scenarios such as surface defect detection of industrial products, condition assessment of road traffic facilities, and small target recognition in remote sensing images. These scenarios generally face problems such as small target scale, strong background interference, and weak defect features.

[0004] However, existing lightweight object detection models still face significant technical bottlenecks in small object detection tasks under the aforementioned complex backgrounds. On the one hand, to strictly control the number of model parameters and computational complexity to adapt to edge deployment requirements, lightweight backbone networks often sacrifice the ability to extract shallow, fine-grained texture features to some extent. This results in the model struggling to effectively capture the discriminative features of targets when they are small, have blurred edges, or are disturbed by complex backgrounds, leading to missed or false detections. On the other hand, most current mainstream multi-scale feature fusion networks adopt equal-weight connections or fixed topologies, lacking the ability to adaptively adjust the contribution of features at different levels to the final detection task. This makes it difficult for shallow features rich in detail and high-level features rich in semantic information to form an effective collaborative expression, limiting the model's ability to perceive multi-scale targets, especially small targets. Therefore, this invention proposes an object detection method, system, computer device, and computer-readable storage medium based on lightweight multi-scale feature fusion to address the problems existing in the prior art. Summary of the Invention

[0005] To address the aforementioned problems, the present invention aims to propose a target detection method, system, computer device, and computer-readable storage medium based on lightweight multi-scale feature fusion, thereby solving the problems of insufficient feature extraction, unbalanced multi-scale fusion, and insufficient edge deployment accuracy of existing lightweight target detection networks in complex backgrounds, small targets, and occluded scenarios.

[0006] To achieve the objectives of this invention, the present invention is implemented through the following technical solution: a target detection method based on lightweight multi-scale feature fusion, comprising the following steps:

[0007] S1. Acquire the image to be detected and preprocess the acquired image to be detected;

[0008] S2. Input the preprocessed image into an improved target detection network that includes a backbone network, a neck network, and a detection head. Use the backbone network to extract feature maps of different scales from the input image.

[0009] S3. Input the extracted feature maps into the neck network using the RepFPN random bidirectional feature pyramid network. Use learnable weights to perform bidirectional weighted fusion of feature maps at different scales from top to bottom and from bottom to top, and output the enhanced multi-scale fused features.

[0010] S4. Input the output multi-scale fusion features into the detection head to perform target category prediction and bounding box regression respectively;

[0011] S5. Post-process the output of the detection head to output the final target detection result.

[0012] A further improvement is made in the following steps: In step S1, the specific steps of preprocessing are as follows: scaling the original image of the image to be detected to a preset input resolution, standardizing the pixel values ​​using mean-variance normalization, and performing at least one data augmentation operation, wherein the data augmentation operation is selected from one or more of random flipping, random cropping, color perturbation, Mosaic enhancement, or MixUp enhancement.

[0013] A further improvement is that in step S2, the backbone network is a lightweight visual transformer RepViT, and the extracted and output feature maps of different scales include at least low-level feature maps, mid-level feature maps, and high-level feature maps. The low-level feature maps are used to represent edge texture information, the mid-level feature maps are used to represent local structural information, and the high-level feature maps are used to represent high-level semantic information.

[0014] A further improvement is made in step S2, where the improved target detection network is built on the YOLO framework, replacing the original backbone network with the RepViT backbone network and the original neck network with the RepFPN network. The RepFPN network introduces a learnable weight and random bidirectional feature fusion mechanism on the basis of the bidirectional feature pyramid structure to improve the detection capability of small targets and targets with complex backgrounds.

[0015] A further improvement is made in step S3, where the random bidirectional feature pyramid network RepFPN normalizes and fuses the input features using learnable weights, and its fusion expression is:

[0016]

[0017] Where O represents the output feature map, Iᵢ represents the i-th input feature map, wᵢ represents the non-negative learnable weight parameter corresponding to the i-th input feature map, ε is a stability constant, and w j Indicates the relationship with the first Non-negative learnable weight parameters corresponding to each input feature map.

[0018] A further improvement lies in the following: the random bidirectional feature pyramid network RepFPN transforms the fusion result after the feature fusion node using depthwise separable convolutions, batch normalization layers, and nonlinear activation functions. For the 4th layer features, its bidirectional fusion expression is:

[0019]

[0020]

[0021] in, This represents the output after fusing features from layer 5 to layer 4. This represents the feature input of the 4th layer. This represents the feature input of the 5th layer. This represents the output after feature fusion from layer 3 to layer 4. This represents the output after feature fusion from layer 2 to layer 3. Up(·) represents upsampling operation, Down(·) represents downsampling operation, and Conv(·) represents a combination of depthwise separable convolution, batch normalization, and nonlinear activation function.

[0022] A further improvement is made in step S4, where the detection head adopts a decoupled detection head structure, the classification branch outputs the target category probability, the regression branch outputs the bounding box center coordinates, width and height parameters, and positioning quality parameters, and the total loss function used when constructing the improved target detection network is a weighted sum of WIoU bounding box regression loss, classification loss, and distribution focus loss.

[0023] A target detection system based on lightweight multi-scale feature fusion includes:

[0024] The image acquisition module is used to acquire the image to be detected;

[0025] The preprocessing module is used to resize, normalize, and enhance the image to be detected;

[0026] The feature extraction module is used to extract multi-scale feature maps using the RepViT backbone network;

[0027] The feature fusion module is used to perform bidirectional weighted fusion of multi-scale feature maps using the RepFPN network, and output the enhanced multi-scale fused features.

[0028] The detection output module is used to perform target classification and bounding box regression by decoupling the detection head, and to perform post-processing to output the detection results.

[0029] A computer device includes a processor and a memory, the memory storing a computer program that runs on the processor, the processor executing the computer program to implement the target detection method described in any of the preceding claims.

[0030] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements the target detection method described in any of the preceding claims.

[0031] The beneficial effects of this invention are as follows: By constructing a collaborative architecture of the RepViT lightweight backbone network and the RepFPN weighted bidirectional feature pyramid, this invention significantly improves the target detection performance in complex scenes. First, the RepViT backbone, while maintaining low computational complexity, can effectively extract multi-scale features, taking into account both shallow texture details and deep semantic information, laying the foundation for small target detection. Second, the RepFPN neck network introduces a learnable weight mechanism to adaptively weight and fuse features at different levels, overcoming the limitations of traditional static fusion methods and significantly enhancing the representation ability of targets in complex backgrounds. Furthermore, by combining depthwise separable convolution and a decoupled detection head design, classification and localization accuracy are improved while maintaining model lightweightness. Finally, by introducing the WIoU loss function for joint optimization, the stability and convergence speed of bounding box regression are further improved. In summary, this invention significantly improves the detection accuracy of small targets, occluded targets, and complex background scenes while maintaining real-time inference capabilities, making it particularly suitable for edge deployment scenarios such as UAV inspection, industrial quality inspection, and remote sensing monitoring. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the target detection method based on lightweight multi-scale feature fusion of the present invention.

[0033] Figure 2 This is a schematic diagram of the improved target detection network structure of the present invention;

[0034] Figure 3 This is a schematic diagram of the RepFPN weighted bidirectional feature fusion node of the present invention;

[0035] Figure 4 This is a schematic diagram of the integrated training and deployment process of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] It should be noted that the technical means not described in detail in the following embodiments are all conventional means in the art, are not the key points of the invention, and will not be elaborated upon.

[0038] Example 1

[0039] See Figure 1 , Figure 2 , Figure 3This embodiment provides a target detection method based on lightweight multi-scale feature fusion, and the specific implementation process is as follows:

[0040] S1. Image Acquisition and Preprocessing

[0041] The image to be inspected is acquired using an image acquisition device (such as a visible light camera mounted on a drone, an industrial line scan camera, etc.). After acquiring the image, the following preprocessing operations are performed:

[0042] Size adjustment: The original image is uniformly scaled to a preset input resolution, which is set to 640×640 pixels in this embodiment;

[0043] Pixel normalization: The pixel values ​​are normalized to the [0,1] interval or a standard normal distribution using mean-variance normalization.

[0044] Data augmentation: During the training phase, one or more of the following operations are performed depending on the application scenario: random flipping, random cropping, color perturbation (brightness, contrast, saturation adjustment), Mosaic augmentation, or MixUp augmentation, to expand the sample distribution and improve the robustness of the model.

[0045] S2, Backbone Network Feature Extraction

[0046] The preprocessed image is input into the improved object detection network. In this embodiment, as... Figure 2 As shown, the improved object detection network is built based on the YOLO framework, and its specific structure is as follows:

[0047] The backbone network employs the lightweight visual transformer RepViT. RepViT is constructed by combining lightweight convolutional units and visual transformer units, outputting feature maps at three scales at different stages:

[0048] Low-level feature map P3 (larger size, smaller receptive field): retains rich edge texture information, suitable for small target detection;

[0049] Mid-level feature map P4 (medium size): takes into account both local structure and semantic information;

[0050] High-level feature map P5 (smaller size, larger receptive field): has stronger global semantic expression capabilities.

[0051] Neck network: A random bidirectional feature pyramid network RepFPN is used;

[0052] Detection head: A decoupled detection head structure is adopted.

[0053] S3, neck network bidirectional weighted feature fusion

[0054] The feature maps P3, P4, and P5 are input into the RepFPN neck network. RepFPN includes top-down and bottom-up paths, and normalized weighted fusion is performed at each fusion node using learnable non-negative weights. The specific fusion process is as follows:

[0055] For any fusion node, its output feature map O is obtained by weighted fusion of multiple input feature maps Iᵢ:

[0056]

[0057] Where Iᵢ represents the i-th input feature map, wᵢ is the non-negative learnable weight parameter corresponding to the i-th input feature map, which is updated through backpropagation during training, and ε is a stability constant, usually taken as 10. -5 w is used to avoid the denominator being zero. j Indicates the relationship with the first Non-negative learnable weight parameters corresponding to each input feature map.

[0058] Taking the bidirectional fusion of features at the 4th layer as an example, the specific calculation process is as follows:

[0059] Top-down fusion:

[0060]

[0061] Bottom-up integration:

[0062]

[0063] in, This represents the output after fusing features from layer 5 to layer 4. This represents the feature input of the 4th layer. This represents the feature input of the 5th layer. This represents the output after feature fusion from layer 3 to layer 4. This represents the output after feature fusion from layer 2 to layer 3. Up(·) represents an upsampling operation (usually using bilinear interpolation or transposed convolution), Down(·) represents a downsampling operation (usually using convolution with a stride of 2 or pooling), and Conv(·) represents a combination of depthwise separable convolution, batch normalization, and the SiLU activation function. The use of depthwise separable convolution significantly reduces the number of parameters and computational cost. Figure 3 This is a schematic diagram of the RepFPN weighted bidirectional feature fusion node in this embodiment.

[0064] S4, Decoupling Detection Head Prediction

[0065] The fused multi-scale feature maps are then input into the corresponding decoupled detection heads. Each detection head contains two independent branches:

[0066] Classification branch: Outputs the probability that each candidate region belongs to a different target category;

[0067] Regression branch: Outputs the center coordinates (x, y), width and height (w, h) of the bounding box, as well as localization quality parameters (such as IoU score).

[0068] During the training phase, a joint loss function is used to optimize the network:

[0069] L total =λ1·L WIoU +λ2·L cls +λ3·L DFL

[0070] Among them, L total Let L be the total loss function. WIoU L is the WIoU bounding box regression loss, used to improve localization accuracy. cls For classification loss (usually binary cross-entropy loss), L DFL λ1, λ2, and λ3 are preset weight coefficients, which are set to 7.5, 0.5, and 1.5 respectively in this embodiment.

[0071] S5. Post-processing and Result Output

[0072] During the inference phase, the following post-processing operations are performed on the candidate boxes output by the detection head:

[0073] Confidence threshold filtering: Remove candidate boxes with a confidence level lower than a preset threshold (e.g., 0.5);

[0074] Non-maximum suppression: Suppresses multiple overlapping candidate boxes of the same target while preserving the best detection result;

[0075] Output: The final output includes the target's category label, confidence score, and bounding box coordinates (x, y, w, h), which are used for defect location, quality assessment, or alarm triggering.

[0076] Example 2

[0077] This embodiment applies the method of the present invention to the scenario of defect detection in transmission line insulators. The specific process is as follows:

[0078] Dataset Construction: Images from UAV inspections were collected to construct training, validation, and test sets. The images include categories such as normal insulators, pollution flashover insulators, damaged insulators, and insulators with foreign objects attached. Defect areas are typically small (possibly only a few dozen pixels in the image) and have complex backgrounds containing interference factors such as the sky, conductors, towers, and trees. This embodiment utilizes RepFPN to jointly enhance shallow fine-grained information and deep semantic information, effectively reducing the false negative and false positive rates.

[0079] Training configuration:

[0080] Input resolution: 640×640 pixels;

[0081] Data augmentation: random rotation (±10°), random cropping, horizontal flipping, brightness perturbation, and Mosaic enhancement;

[0082] Batch size: 16;

[0083] Optimizer: AdamW, initial learning rate 1×10 -3 ;

[0084] Learning rate strategy: cosine annealing;

[0085] Training epochs: 300 epochs;

[0086] Loss function weights: λ1=7.5, λ2=0.5, λ3=1.5.

[0087] Results analysis: After training, the model performance was evaluated on the test set.

[0088] Deployment and Application: The trained model can be deployed on GPU servers, edge industrial computers, or onboard processing units of inspection drones. On NVIDIA Jetson Xavier NX edge devices, the model inference speed reaches 35 FPS, meeting real-time detection requirements.

[0089] Figure 4 This is a schematic diagram of the integrated training and deployment process of the present invention.

[0090] Example 3

[0091] This embodiment applies the method of the present invention to the detection of surface defects in metal sheets. The specific process is as follows:

[0092] Application characteristics: Scratches, cracks, pits and other defects on the surface of metal sheets are usually characterized by small size, low contrast and irregular shape, and the background may have texture interference.

[0093] Implementation: Similar to Example 2, images from industrial sites are collected to construct a dataset. Due to the high real-time requirements of industrial scenarios, the model can be lightweighted and trimmed.

[0094] Reduce the number of channels in the RepViT backbone network (e.g., from 96 to 64).

[0095] Reduce the number of repeated fusions in RepFPN (e.g., from 2 to 1).

[0096] The input resolution has been adjusted to 416×416 pixels.

[0097] Results: While maintaining the detection accuracy at a basically unchanged level, the number of model parameters was reduced by about 30%, and the inference speed was increased to more than 50 FPS, meeting the detection needs of high-speed production lines.

[0098] Example 4

[0099] This embodiment provides a target detection system based on lightweight multi-scale feature fusion. The system consists of an image acquisition module, a preprocessing module, a feature extraction module, a feature fusion module, and a detection output module, wherein:

[0100] The image acquisition module is used to acquire the image to be detected;

[0101] The preprocessing module is used to resize, normalize, and enhance the image to be detected;

[0102] The feature extraction module is used to extract multi-scale feature maps using the RepViT backbone network;

[0103] The feature fusion module is used to perform bidirectional weighted fusion of multi-scale feature maps using the RepFPN network, and output the enhanced multi-scale fused features.

[0104] The detection output module is used to perform target classification and bounding box regression by decoupling the detection head, and to perform post-processing to output the detection results.

[0105] Example 5

[0106] This embodiment provides a computer device, including a processor, a memory, an image input interface, and an output interface. The memory stores a computer program configured to execute the target detection method provided in Embodiment 1.

[0107] Workflow:

[0108] The image input interface receives the image to be detected (such as real-time acquisition via a camera or reading a local image).

[0109] The processor executes the program in memory, sequentially completing image preprocessing, feature extraction, feature fusion, classification and regression, and post-processing;

[0110] The output interface sends the detection results (category, confidence level, bounding box) to a display device, storage device, or control actuator.

[0111] The computer device can be a general-purpose server, an industrial control computer, an embedded edge computing device (such as the NVIDIA Jetson series or the Huawei Atlas series), or an unmanned aerial vehicle (UAV) onboard computing unit.

[0112] Example 6

[0113] This embodiment provides a computer-readable storage medium on which a computer program is stored. The storage medium can be a USB flash drive, external hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or any other medium capable of storing program code. When the computer program is executed by a processor, it implements the target detection method provided in Embodiment 1 of this invention.

[0114] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A target detection method based on lightweight multi-scale feature fusion, characterized in that, Includes the following steps: S1. Acquire the image to be detected and preprocess the acquired image to be detected; S2. Input the preprocessed image into an improved target detection network that includes a backbone network, a neck network, and a detection head. Use the backbone network to extract feature maps of different scales from the input image. S3. Input the extracted feature maps into the neck network using the RepFPN random bidirectional feature pyramid network. Use learnable weights to perform bidirectional weighted fusion of feature maps at different scales from top to bottom and from bottom to top, and output the enhanced multi-scale fused features. S4. Input the output multi-scale fusion features into the detection head to perform target category prediction and bounding box regression respectively; S5. Post-process the output of the detection head to output the final target detection result.

2. The target detection method based on lightweight multi-scale feature fusion according to claim 1, characterized in that: In step S1, the specific steps of preprocessing are as follows: scaling the original image of the image to be detected to a preset input resolution, standardizing the pixel values ​​using mean-variance normalization, and performing at least one data augmentation operation, wherein the data augmentation operation is selected from one or more of random flipping, random cropping, color perturbation, Mosaic enhancement, or MixUp enhancement.

3. The target detection method based on lightweight multi-scale feature fusion according to claim 1, characterized in that: In step S2, the backbone network is a lightweight visual transformer RepViT. The extracted and output feature maps of different scales include at least low-level feature maps, mid-level feature maps, and high-level feature maps. The low-level feature maps are used to represent edge texture information, the mid-level feature maps are used to represent local structural information, and the high-level feature maps are used to represent high-level semantic information.

4. The target detection method based on lightweight multi-scale feature fusion according to claim 1, characterized in that: In step S2, the improved target detection network is built based on the YOLO framework, replacing the original backbone network with the RepViT backbone network and the original neck network with the RepFPN network. The RepFPN network introduces a learnable weight and random bidirectional feature fusion mechanism on the basis of the bidirectional feature pyramid structure to improve the detection capability of small targets and complex background targets.

5. The target detection method based on lightweight multi-scale feature fusion according to claim 1, characterized in that: In step S3, the random bidirectional feature pyramid network RepFPN normalizes and fuses the input features using learnable weights, and its fusion expression is as follows: Where O represents the output feature map, Iᵢ represents the i-th input feature map, wᵢ represents the non-negative learnable weight parameter corresponding to the i-th input feature map, ε is a stability constant, and w j Indicates the relationship with the first Non-negative learnable weight parameters corresponding to each input feature map.

6. The target detection method based on lightweight multi-scale feature fusion according to claim 5, characterized in that: The RepFPN random bidirectional feature pyramid network transforms the fusion result after the feature fusion node using depthwise separable convolutions, batch normalization layers, and nonlinear activation functions. For the 4th layer features, its bidirectional fusion expression is: in, This represents the output after fusing features from layer 5 to layer 4. This represents the feature input of the 4th layer. This represents the feature input of the 5th layer. This represents the output after feature fusion from layer 3 to layer 4. This represents the output after feature fusion from layer 2 to layer 3. Up(·) represents upsampling operation, Down(·) represents downsampling operation, and Conv(·) represents a combination of depthwise separable convolution, batch normalization, and nonlinear activation function.

7. The target detection method based on lightweight multi-scale feature fusion according to claim 1, characterized in that: In step S4, the detection head adopts a decoupled detection head structure. The classification branch outputs the target category probability, and the regression branch outputs the bounding box center coordinates, width and height parameters, and positioning quality parameters. The total loss function used when constructing the improved target detection network is a weighted sum of WIoU bounding box regression loss, classification loss, and distribution focus loss.

8. A target detection system based on lightweight multi-scale feature fusion, characterized in that, include: The image acquisition module is used to acquire the image to be detected; The preprocessing module is used to resize, normalize, and enhance the image to be detected; The feature extraction module is used to extract multi-scale feature maps using the RepViT backbone network; The feature fusion module is used to perform bidirectional weighted fusion of multi-scale feature maps using the RepFPN network, and output the enhanced multi-scale fused features. The detection output module is used to perform target classification and bounding box regression by decoupling the detection head, and to perform post-processing to output the detection results.

9. A computer device, characterized in that: It includes a processor and a memory, the memory storing a computer program that runs on the processor, and the processor executing the computer program to implement the target detection method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The device contains a computer program that, when executed by a processor, implements the target detection method according to any one of claims 1-7.

Images