Multi-scale target detection method
Through the multi-scale object detection method, combined with the clustering algorithm and the dual attention mechanism, the target box and feature pyramid are optimized, and the problem of unbalanced detection accuracy of multi-objective types in the existing technology is solved, and the target detection effect with high accuracy and low leakage detection rate is achieved.
Patent Information
- Application Number
- CN202411494219.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-03-04
AI Technical Summary
The existing target detection methods have universality problems in multi-objective type detection, especially when the types of land and water targets are diverse, the scale distribution is uneven, and the presence of occlusion, resulting in a decrease in detection accuracy.
Multi-scale object detection method is adopted to detect target objects of different sizes through multi-scale target boxes, combine the clustering algorithm to optimize the target boxes, introduce a dual attention mechanism and a feature pyramid of multiple expansion rates, and extract and fuse multi-scale feature information.
It improves the accuracy and overall performance of target detection, reduces the missed detection rate, and can effectively detect target objects of multiple scales and types.
Smart Images
Figure CN119559371A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of target detection, and in particular to a multi-scale target detection method. Background Art
[0002] Target detection and recognition are the main tasks that need to be studied in the vision system of unmanned platforms. Correctly identifying and classifying multiple targets on land and water, and improving the accuracy of target detection in the vision system are also research focuses, which are of great significance for ensuring the safety of land, water and air environments.
[0003] Among the existing target detection methods, whether based on template matching, target structure or deep learning methods, there are certain universality problems. These methods have good detection effects on single targets, but poor detection capabilities for multiple target types. In actual scenarios, there are various types of land and water targets. Taking the water surface as an example, there are cargo ships, cruise ships, sailboats, fleets, birds, etc. For the same type of target, the intra-class differences will also be large. The target scale distribution varies greatly, and there are many occlusions. The existence of these factors will lead to a decrease in the accuracy of target detection. Therefore, the uneven distribution of target size and the uncertainty of the target's posture, scale, and perspective greatly increase the difficulty of target detection. It is a challenging task to solve the uneven target detection accuracy in multi-scale target detection and improve the overall performance of current target detectors. Summary of the invention
[0004] In view of the above problems, the present invention proposes a multi-scale target detection method, which detects target objects of different sizes through a multi-scale target frame, has high detection accuracy and low missed detection rate.
[0005] The technical solution of the present invention is: a multi-scale target detection method, comprising the following steps:
[0006] 1) Establish and train a multi-scale target detection neural network model;
[0007] 1.1) Image preprocessing;
[0008] Collect four consecutive frames of images, randomly scale and randomly crop the four frames of images to obtain multiple images, randomly arrange these images to form an image data set, and obtain the prior frames of the target objects in these images;
[0009] 1.2) Use clustering algorithm to optimize the target box;
[0010] 1.3) Multi-scale feature extraction;
[0011] The images in the image dataset are input into the neural network model respectively, and after 3×3 convolution, the low-level global feature map F of each image is obtained. CONV1After normalizing and activating each low-level global feature map, we can obtain the second feature map F CONV2 , each second feature map F CONV2 Multiply the channel attention map and the spatial attention map to obtain the third feature map F CONV3 , each third feature map F CONV3 The input is a 3×3 maximum pooling feature pyramid, and a variety of expansion rates are introduced into the feature pyramid to obtain receptive fields of different scales, and extract multi-scale feature information of each image;
[0012] 1.4) Multi-scale feature fusion;
[0013] The neural network model conveys strong semantic features from the top to the bottom of the Neck layer based on the multi-scale feature information extracted from each image, and the feature pyramid conveys strong positioning features from the bottom to the top, and performs multi-scale feature fusion on different detection layers from different backbone layers;
[0014] 1.5) Multi-scale image detection;
[0015] After multi-scale feature fusion, a multi-scale final feature map is obtained, and a multi-scale target box is used to detect the target object in each final feature map;
[0016] 1.6) Calculation of loss;
[0017] The loss function is used to calculate the difference between the position of the target object in the target frame and the position of the target object in the prior frame. If the difference is greater than the set threshold, the model is retrained. If the difference is less than the set threshold, the model training is completed.
[0018] 2) Collect images and input them into a multi-scale object detection neural network model;
[0019] 3) The multi-scale target detection neural network model uses multi-scale target boxes to frame the target objects in the image.
[0020] Preferably, the multi-sized target frames include target frames of three sizes: large, medium and small.
[0021] Preferably, in step 1.2), the formula of the clustering algorithm used is:
[0022]
[0023] R(x) is the calculation of the initial cluster center point to each data sample x i The distance between the two clusters is , and P(x) is the probability of each sample becoming the next cluster center.
[0024] Preferably, in step 1.3), each second feature map F CONV2Multiply the channel attention map and the spatial attention map to obtain the third feature map F CONV3 The formula is:
[0025]
[0026] Where A represents the second feature map F CONV2 , N c is the channel attention map, A' represents the channel output feature map, N s is the spatial attention map, A' is the third feature map F CONV3 , Represents element-wise multiplication.
[0027] Preferably, in step 1.6), the formula of the loss function is:
[0028] Central loss =-φs(1-t) η -(1-φ)(1-s)t η log(1-t)
[0029] In the formula, Central loss is the loss function, s represents the label of the real sample, t represents the predicted output value after the softmax function, φ is the balance factor, and η is the factor added on the basis of the binary cross entropy.
[0030] The advantages of the present invention are:
[0031] 1. The present invention uses a clustering algorithm to optimize the target frame, so that the scale distribution of the target frame is wider, ensuring that the target frame can detect target objects of various scales.
[0032] 2. The present invention embeds a dual attention mechanism (i.e., channel attention and spatial attention) in the backbone network of the multi-scale target detection neural network model, which improves the problem of excessive difference in detection accuracy between large-scale and small-scale target objects.
[0033] 3. The present invention improves the feature pyramid by introducing a variety of different expansion rates, thereby enhancing the perception ability of the multi-scale target detection neural network model to segment targets of different scales, improving the problems of resolution degradation and local information loss when the model detects target objects, and making the detection accuracy of the model high and the missed detection rate low. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 A flowchart of establishing and training a multi-scale target detection neural network model of the present invention;
[0035] Figure 2 This is a diagram of the above-water detection results of the present invention;
[0036] Figure 3 This is a diagram of land detection results of the present invention. DETAILED DESCRIPTION
[0037] See also Figures 1 to 3 , a multi-scale target detection method, comprising the following steps:
[0038] 1) Establish and train a multi-scale target detection neural network model;
[0039] 1.1) Image preprocessing;
[0040] Collect four consecutive frames of images, randomly scale and randomly crop the four frames of images to obtain multiple images, randomly arrange these images to form an image data set, and obtain the prior frames of the target objects in these images;
[0041] 1.2) Use clustering algorithm to optimize the target box;
[0042] A point is randomly selected from the image in the image data set as the first cluster center point, and the distance from any other data sample point on the image to the selected cluster center point is calculated, and then a new point is reselected as the cluster center point according to the roulette method. Repeat the process repeatedly until the categories of all data sample points have not changed.
[0043] The formula of the clustering algorithm used is:
[0044]
[0045] R(x) is the calculation of the initial cluster center point to each data sample point x i The distance between the two clusters is , and P(x) is the probability of each sample becoming the next cluster center.
[0046] 1.3) Multi-scale feature extraction;
[0047] The images in the image dataset are input into the neural network model respectively, and after 3×3 convolution, the low-level global feature map F of each image is obtained. CONV1 After normalizing and activating each low-level global feature map, we can obtain the second feature map F CONV2 , each second feature map F CONV2 Multiply the channel attention map and the spatial attention map to obtain the third feature map F CONV3 , each third feature map F CONV3 The input is a 3×3 maximum pooling feature pyramid, and a variety of different expansion rates are introduced into the feature pyramid to obtain receptive fields of different scales, and the multi-scale feature information of each image is extracted; each second feature map F CONV2Multiply the channel attention map and the spatial attention map to obtain the third feature map F CONV3 The formula is:
[0048]
[0049] Where A represents the second feature map F CONV2 , N c is the channel attention map, A' represents the channel output feature map, N s is the spatial attention map, A' is the third feature map F CONV3 , Represents element-wise multiplication.
[0050] 1.4) Multi-scale feature fusion;
[0051] The neural network model conveys strong semantic features from the top to the bottom of the Neck layer based on the multi-scale feature information extracted from each image, and the feature pyramid conveys strong positioning features from the bottom to the top, and performs multi-scale feature fusion on different detection layers from different backbone layers;
[0052] 1.5) Multi-scale image detection;
[0053] After multi-scale feature fusion, a multi-scale final feature map is obtained, and a multi-size target frame is used to detect the target object in each final feature map; the multi-size target frame includes three sizes of target frames: large, medium and small. The large target object is detected by the large-size target frame, the medium target object is detected by the medium-size target frame, and the small target object is detected by the small-size target frame. Among them, large targets, medium targets and small targets are statistically analyzed for the target scale distribution using the definition of absolute targets. Targets larger than 0.38% of the pixel area of the entire image are regarded as large targets, such as fleets, cruise ships, cargo ships, etc. Targets larger than 0.12% of the pixel area of the entire image and smaller than 0.38% are regarded as medium targets, such as sailboats, yachts, etc. Targets smaller than 0.12% of the pixel area of the entire image are regarded as small targets, such as people, birds, fish, etc.
[0054] 1.6) Calculation of loss;
[0055] The loss function is used to calculate the difference between the position of the target object in the target frame and the position of the target object in the prior frame. If the difference is greater than the set threshold, the model is retrained. If the difference is less than the set threshold, the model training is completed. The threshold is set according to specific needs. The more accurate the detection result, the smaller the threshold setting. The formula of the loss function is:
[0056] Central loss =-φs(1-t) η -(1-φ)(1-s)t η log(1-t)
[0057] In the formula, Central loss is the loss function, s represents the label of the real sample, t represents the predicted output value after the softmax function, φ is the balance factor, and η is the factor added on the basis of the binary cross entropy.
[0058] 2) Collect images and input them into a multi-scale object detection neural network model;
[0059] 3) The multi-scale target detection neural network model uses multi-scale target boxes to identify the target objects in the image.
Claims
1. A multi-scale target detection method, characterized in that: The following steps are involved: 1) Establish and train a multi-scale target detection neural network model; 1.1) Image preprocessing; Collect four consecutive frames of images, randomly scale and randomly crop the four frames of images to obtain multiple images, randomly arrange these images to form an image data set, and obtain the prior frames of the target objects in these images; 1.2) Use clustering algorithm to optimize the target box; 1.3) Multi-scale feature extraction; The images in the image dataset are input into the neural network model respectively, and after 3×3 convolution, the low-level global feature map F of each image is obtained. CONV1 After normalizing and activating each low-level global feature map, we can obtain the second feature map F CONV2 , each second feature map F CONV2 Multiply the channel attention map and the spatial attention map to obtain the third feature map F CONV3 , each third feature map F CONV3 The input is a 3×3 maximum pooling feature pyramid, and a variety of expansion rates are introduced into the feature pyramid to obtain receptive fields of different scales, and extract multi-scale feature information of each image; 1.4) Multi-scale feature fusion; The neural network model conveys strong semantic features from the top to the bottom of the Neck layer based on the multi-scale feature information extracted from each image, and the feature pyramid conveys strong positioning features from the bottom to the top, and performs multi-scale feature fusion on different detection layers from different backbone layers; 1.5) Multi-scale image detection; After multi-scale feature fusion, a multi-scale final feature map is obtained, and a multi-scale target box is used to detect the target object in each final feature map; 1.6) Calculation of loss; The loss function is used to calculate the difference between the position of the target object in the target frame and the position of the target object in the prior frame. If the difference is greater than the set threshold, the model is retrained. If the difference is less than the set threshold, the model training is completed. 2) Collect images and input them into a multi-scale object detection neural network model; 3) The multi-scale target detection neural network model uses multi-scale target boxes to frame the target objects in the image.
2. A multi-scale target detection method according to claim 1, characterized in that: The multi-sized target frames include target frames of three sizes: large, medium and small.
3. The multi-scale target detection method according to claim 1, characterized in that: In step 1.2), the formula of the clustering algorithm used is: R(x) is the calculation of the initial cluster center point to each data sample x i The distance between the two clusters is , and P(x) is the probability of each sample becoming the next cluster center.
4. The multi-scale target detection method according to claim 1, characterized in that: In the step 1.3), each second feature map F CONV2 Multiply the channel attention map and the spatial attention map to obtain the third feature map F CONV3 The formula is: Where A represents the second feature map F CONV2 , N c is the channel attention map, A' represents the channel output feature map, N s is the spatial attention map, A' is the third feature map F CONV3 , Represents element-wise multiplication.
5. The multi-scale target detection method according to claim 1, characterized in that: In step 1.6), the formula of the loss function is: Central loss =-φs(1-t) η -(1-φ)(1-s)t η log(1-t) In the formula, Central loss is the loss function, s represents the label of the real sample, t represents the predicted output value after the softmax function, φ is the balance factor, and η is the factor added on the basis of the binary cross entropy.