Lightweight small target detection method based on efficient vit improved YOLO model

Abstract

The application provides a kind of light weight small target detection method based on EfficientVIT improved YOLO model, method includes: the YOLO model based on EfficientVIT fusion multiscale attention improvement is established, including: the main network based on EfficientViT improves Yolov8 network, the depth convolution is promoted as group convolution, the activation value after normalization is " proxy correction", and the light weight ReLU multiscale linear attention mechanism is introduced and the limitation of ReLU linear attention is weakened;Collect small target image data in multiple scenes, and pretreat, construct data set;The data set is divided into training set and test set;The improved YOLO model is trained using the training set, and a light weight small target detection model is obtained;Collect the image of the region to be detected, input into the light weight small target detection model, and output the small target detection result.The application provides an innovative solution that takes into account both theoretical and practical value for the perception and tracking technology of small targets such as unmanned aerial vehicles in complex scenarios.

Description

Technical Field

[0001] This invention belongs to the field of low-altitude small target visual detection technology, and in particular, it is a lightweight small target detection method based on the EfficientVIT improved YOLO model. Background Technology

[0002] With the widespread application of drones, small aircraft and drones in low-altitude areas have also brought serious challenges to public safety and airspace management. In particular, the monitoring and countermeasures against illegal or unauthorized drone activities have become important technical problems that urgently need to be solved. Low-altitude small target detection has the following typical characteristics: small target size, strong concealment, high flight speed, and often in complex and changing environmental backgrounds. Currently, the mainstream detection methods are mostly based on visual sensors (such as visible light cameras) or multimodal fusion technology.

[0003] In recent years, with the rapid development of deep learning technology, target detection algorithms based on convolutional neural networks (CNNs) and Transformer architectures have achieved significant breakthroughs in performance. Among them, the YOLO series of algorithms has become a popular research direction for low-altitude target detection due to its end-to-end detection capabilities and real-time advantages. However, existing YOLO models still face the problem of insufficient accuracy when detecting small targets, especially at long distances and in complex backgrounds, making it difficult to effectively distinguish targets from the environment. At the same time, traditional YOLO networks typically have high computational complexity, making it difficult to adapt to the deployment requirements of lightweight and low-power devices. Summary of the Invention

[0004] The purpose of this invention is to address the problems existing in the prior art by providing a lightweight small target detection method based on EfficientVIT fusion with a multi-scale attention-based improved YOLO model. This method focuses on the field of UAV visual target detection, providing technical support for low-altitude security management scenarios such as UAV countermeasures. By enhancing the detection capability of small targets, reducing model complexity, and improving real-time performance, this method provides an efficient and reliable solution for low-altitude area security monitoring and UAV countermeasures.

[0005] The technical solution to achieve the purpose of this invention is: a lightweight small target detection method based on the EfficientVIT improved YOLO model, the method comprising the following steps:

[0006] Step 1: Establish a YOLO model based on EfficientViT and multi-scale attention improvement, including: improving the YOLOv8 network based on the backbone network of EfficientViT, generalizing depthwise convolution to grouped convolution, performing "surrogate correction" on the normalized activation values, and introducing a lightweight ReLU multi-scale linear attention mechanism to weaken the limitations of ReLU linear attention.

[0007] Step 2: Collect small target image data in various scenarios, perform preprocessing, and construct a dataset; the definition of small targets is customized according to requirements.

[0008] Step 3: Divide the dataset into a training set and a test set;

[0009] Step 4: Use the training set to train the YOLO model based on EfficientVIT fusion multi-scale attention improvement to obtain a lightweight small target detection model.

[0010] Step 5: Acquire an image of the area to be detected, input it into the lightweight small target detection model, and output the small target detection result.

[0011] Furthermore, the improvement of the Yolov8 network based on the EfficientViT backbone network described in step 1 specifically includes:

[0012] Step 1-1: Replace the Yolov8 backbone network with the EfficientViT backbone network;

[0013] Steps 1-2 establish an improved Sandwich layout that incorporates dynamically adjusted weights, specifically represented as follows:

[0014]

[0015] Among them, X i X i+1 These represent the features of the i-th and (i+1)-th layers in the Sandwich layout, respectively. This represents the i-th layer of the FNN. This represents the self-attention mechanism of the i-th layer; α and β are dynamically adjusted parameters that can be adaptively optimized according to the input features, and σ is an activation function used to introduce nonlinear mapping;

[0016] Steps 1-3 introduce a cascaded group attention module to solve the redundancy problem in multi-head self-attention;

[0017] The attention calculation for each head in the cascaded attention module is as follows:

[0018]

[0019] Wherein, the j-th head calculates the input feature X i Self-attention corresponding to the segmentation above, X ij The input feature X i The j-th segment, X i It consists of all the segments, i.e., X i =[X i1 ,Xi2 ,…,X ih And 1≤j≤h, where h represents the total number of heads, and each segment passes through a different projection layer. and Input features X i The projection layers, mapped to different subspaces, are ultimately passed through a linear layer. Project the output features of the connection back to the same dimension as the input; X represents ij Self-attention This indicates that the Concat function is used to apply the changes to X. ij With the input features X of the (i+1)th layer i+1 The connection;

[0020] At the same time, the output of each head is added to the subsequent head:

[0021]

[0022] Among them, X i ' j The input feature X i The j-th segment X ij With the (j-1)th head output The sum of these features is used as the new input feature for calculating the j-th head.

[0023] Steps 1-4 involve parameter reallocation by increasing the channel width of critical modules while decreasing the channel width of unimportant modules to redistribute parameters in the network; the critical and unimportant modules are defined in a custom way.

[0024] Furthermore, in steps 1-2, the parameters α and β are dynamically adjusted by adjusting the input features X. i The statistics are obtained through calculation;

[0025] α = mean(X) i )

[0026] β=std(X i )

[0027] Where, mean(X) i ) and std(X i ) represent X respectively i The mean and standard deviation.

[0028] Furthermore, steps 1-4 specifically include:

[0029] For Q and K projections, the channel size in each attention mechanism across all learning stages is reduced; for V projection, it is allowed to have the same dimension as the input embedding, i.e., the number of channels; while the expansion ratio of FFN is reduced to 2.

[0030] Furthermore, in step 1, depthwise convolution is extended to grouped convolution, which involves dividing the input channels into multiple groups and performing standard convolution on each group separately.

[0031] Furthermore, in step 1, "surrogate correction" is performed on the normalized activation values. Specifically, based on hierarchical normalization and statistical constraint optimization using surrogate variables, "surrogate correction" is performed on the normalized activation values. The process includes:

[0032] Steps 1-5, GroupNorm normalization:

[0033] Divide the input activation X into G groups, and compute the normalized output for each group. Where μ and σ are the within-group mean and standard deviation, respectively;

[0034] Steps 1-6: Construct proxy variables:

[0035] Generate an ideal standard Gaussian distribution variable Z ~ N(0,1) as a proxy variable;

[0036] Steps 1-7: Share affine transformation and activation function:

[0037] right Applying the same affine transformation parameters (γ, β) and activation function φ as Z, we obtain... Y proxy :

[0038]

[0039] Steps 1-8, Statistical offset correction:

[0040] Use Y proxy Statistical property correction At the same time, a hierarchical normalization method is introduced to perform the normalization of proxy variables in multiple levels:

[0041]

[0042] Where ε is the adjustment factor, γ l and β l These are the normalized weight parameters of the l-th layer, learned independently in each layer, μ proxy and σ proxy These are the mean and standard deviation of the hierarchical proxy variables, respectively, calculated using the following formula:

[0043]

[0044] In the formula, Let L be the proxy variable for the l-th layer, and L be the total number of layers.

[0045] Zproxy Statistical constraints introduced for the distribution of proxy variables:

[0046] Z proxy =Proj N(0,1 (Z) init )

[0047] Among them, Proj N(0,1) It is a projection operator used to project the initial agent Z. init Mapped to a standard Gaussian distribution:

[0048]

[0049] μ init =mean(Z) init )

[0050] σ init =std(Z init )

[0051] In the formula, mean(Z) init ) and std(Z init ) represent Z respectively init The mean and standard deviation.

[0052] Furthermore, the introduction of a lightweight ReLU multi-scale linear attention mechanism and the mitigation of the limitations of ReLU linear attention in step 1 specifically includes:

[0053] A lightweight ReLU multi-scale linear attention mechanism is introduced to enable the global receptive field;

[0054] Insert a depthwise convolution into each FFN layer to enhance ReLU linear attention and mitigate its limitations.

[0055] Furthermore, step 1 also includes improving the loss function of the Yolov8 network, including improving the MPDIoU loss function based on bounding box regression and improving the Focal Loss loss function based on classification.

[0056] Furthermore, step 2 involves collecting small target image data from various scenarios, performing preprocessing, and constructing a dataset, specifically including:

[0057] Step 2-1: For a specific detection scenario, collect small target image data with various resolutions and backgrounds to construct an initial dataset;

[0058] Step 2-2: Perform noise filtering and enhancement processing on the images in the initial dataset;

[0059] Steps 2-3 involve annotating the images with information, including the coordinates of the target bounding boxes and category information, thereby obtaining the final dataset.

[0060] Further, in step 2-2, noise filtering is performed using an improved bilateral filtering noise removal method. The filtering formula for the improved bilateral filtering noise removal method is as follows:

[0061]

[0062] Among them, I in (x) is the gray value of pixel x in the input image, I in (y) is the gray value of pixel y in the input image, I out (x) is the gray value of pixel x in the input image, Ω represents the neighborhood of pixel x, i.e., the window size, and f s (‖xy||) is defined as the space kernel function:

[0063]

[0064] In the formula, σ s It is the smoothing range of the control domain;

[0065] Among them, f r (|I in (x)-I in (y)|) is the intensity kernel function, defined as:

[0066]

[0067] In the formula, σ r Weights are used to control the similarity of pixel values;

[0068] In particular, gradient domain constraints are introduced into the weight calculation of bilateral filtering:

[0069]

[0070] In the formula, |I in (x)-I in (y)| represents the difference between the pixel values ​​of pixel x and pixel y. It is the gradient difference between pixel x and pixel y, σ g Used to control gradient sensitivity

[0071] Compared with the prior art, the significant advantages of this invention are:

[0072] (1) Improving the YOLO model by combining EfficientVIT can not only significantly improve the detection accuracy of small targets at low altitudes, but also reduce the computational complexity of the model and achieve lightweight deployment.

[0073] (2) A Sandwich layout with dynamically adjusted weights has been innovatively established to optimize the YOLOv8 backbone network for efficient visual tasks. This layout can dynamically adjust the feature ratio and make the network more capable of nonlinear mapping.

[0074] (3) While making the model convolution lightweight, it innovatively proposes hierarchical normalization and statistical constraint optimization based on surrogate variables. By performing "surrogate correction" on the normalized activation values, the non-normalization problem is solved. At the same time, the feature distribution can be adjusted in a finer granular manner to ensure that the surrogate variables always maintain the target distribution.

[0075] (4) A lightweight ReLU multi-scale linear attention mechanism is introduced. At the same time, the limitations of ReLU linear attention are weakened according to the characteristics of this model, which can further optimize the feature expression ability and computational efficiency of the model, thereby meeting the needs of real-time detection in complex environments.

[0076] (5) An innovative improved bilateral filtering noise removal method is proposed, which incorporates a gradient domain constraint to enhance edge preservation. This can more effectively maintain image edge details and avoid edge blurring. For image structures with small gradient changes or complex structures, the gradient domain constraint can also guide the filter to better distinguish noise and important edges, thereby enhancing image quality.

[0077] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0078] Figure 1 This is a flowchart of a lightweight small target detection method based on the EfficientVIT improved YOLO model.

[0079] Figure 2 This is a schematic diagram of the network structure of the YOLOv8 model based on EfficientVIT in one embodiment.

[0080] Figure 3 This is a schematic diagram of the ReLU multiscale linear attention mechanism in one embodiment. Detailed Implementation

[0081] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0082] It should be noted that if the embodiments of the present invention involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of the components in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0083] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.

[0084] In one embodiment, combined Figure 1 This paper presents a lightweight small target detection method based on the EfficientVIT improved YOLO model, the method comprising the following steps:

[0085] Step 1: Establish a YOLO model based on EfficientVIT fusion with multi-scale attention improvements, such as... Figure 2 As shown, it includes: improving the Yolov8 network based on the EfficientViT backbone network, generalizing depthwise convolution to grouped convolution, performing "surrogate correction" on the normalized activation values, and introducing a lightweight ReLU multi-scale linear attention mechanism to weaken the limitations of ReLU linear attention.

[0086] Step 2: Collect small target image data in various scenarios, perform preprocessing, and construct a dataset; the definition of small targets is customized according to requirements; (the small targets in this invention are for, but not limited to, drones).

[0087] Step 3: Divide the dataset into a training set and a test set;

[0088] Step 4: Use the training set to train the YOLO model based on EfficientVIT fusion multi-scale attention improvement to obtain a lightweight small target detection model.

[0089] Step 5: Acquire an image of the area to be detected, input it into the lightweight small target detection model, and output the small target detection result.

[0090] Furthermore, in one embodiment, the improvement of the Yolov8 network based on the EfficientViT backbone network described in step 1 specifically includes:

[0091] Step 1-1: Replace the Yolov8 backbone network with the EfficientViT backbone network;

[0092] Steps 1-2 establish an improved Sandwich layout that incorporates dynamically adjusted weights, specifically represented as follows:

[0093]

[0094] Among them, X i X i+1 These represent the features of the i-th and (i+1)-th layers in the Sandwich layout, respectively. This represents the i-th layer of the FNN. This represents the self-attention mechanism of the i-th layer; α and β are dynamically adjusted parameters that can be adaptively optimized according to the input features, and σ is an activation function used to introduce nonlinear mapping;

[0095] Here, the specific analysis is as follows:

[0096] The original Sandwich layout is:

[0097]

[0098] It can be seen that the inner layer first passes through multiple The mixture is then fed into a single self-attention layer. Finally, the outer layer passes through Further processing is needed.

[0099] The improved Sandwich layout of this invention is in and The network incorporates dynamically adjustable parameters α and β, allowing for dynamic adjustment of feature proportions. Furthermore, an activation function σ is added to enhance the network's nonlinear mapping capabilities.

[0100] Steps 1-3 introduce a cascaded group attention module to solve the redundancy problem in multi-head self-attention;

[0101] The attention calculation for each head in the cascaded attention module is as follows:

[0102]

[0103] Wherein, the j-th head calculates the input feature X i Self-attention corresponding to the segmentation above, X ij The input feature X i The j-th segment, Xi It consists of all the segments, i.e., X i =[X i1 ,X i2 ,...,X ih And 1≤j≤h, where h represents the total number of heads, and each segment passes through a different projection layer. and Input features X i The projection layers, mapped to different subspaces, are ultimately passed through a linear layer. Project the output features of the connection back to the same dimension as the input; X represents ij Self-attention This indicates that the Concat function is used to apply the changes to X. ij With the input features X of the (i+1)th layer i+1 The connection;

[0104] Here, feature segmentation is used for each head instead of full features, which is more efficient and saves computational overhead. Attention maps for each head are computed in a cascaded manner, and the output of each head is added to subsequent heads.

[0105]

[0106] Where, X′ ij The input feature X i The j-th segment X ij With the (j-1)th head output The sum of these features is used as the new input feature for calculating the j-th head.

[0107] Steps 1-4 involve parameter reallocation by increasing the channel width of critical modules while decreasing the channel width of unimportant modules to redistribute parameters in the network; the critical and unimportant modules are defined in a custom way.

[0108] Preferably, in some embodiments, the dynamic adjustment of parameters α and β in steps 1-2 is achieved by adjusting the input feature X. i The statistics are obtained through calculation;

[0109] α = mean(X) i )

[0110] β=std(X i )

[0111] Where, mean(X) i ) and std(X i ) represent X respectively i The mean and standard deviation.

[0112] Preferably, in some embodiments, steps 1-4 specifically include:

[0113] For Q and K projections, the channel size in each attention mechanism across all learning stages is reduced; for V projection, it is allowed to have the same dimension as the input embedding, i.e., the number of channels, to ensure that it can fully convey the information of the input features; at the same time, the scaling ratio of FFN is reduced to 2 (by default, the hidden layer dimension of FFN is 4 times the input dimension).

[0114] The solution in this embodiment can effectively reduce the number of parameters and computational load.

[0115] Furthermore, in one embodiment, step 1 extends depthwise convolution to grouped convolution, whereby the input channels are divided into multiple groups, and each group undergoes standard convolution independently. A detailed analysis follows:

[0116] Assuming we use a 3×3 convolution kernel for convolution, and only perform convolution on each channel separately instead of convolving them all at once, that is, each input channel is convolved independently with a convolution kernel, the computational complexity is:

[0117] FLOPs = H out ·W out ·K 2 ·C in C out

[0118] Where K is the kernel size, H out and W out It is the output feature map size, C in C out This represents the input and output quantities.

[0119] Grouped convolution divides the input channels into multiple groups, and each group undergoes a standard convolution independently. The choice of the number of groups G affects the computational cost, i.e., the computational complexity becomes:

[0120]

[0121] That is, the number of parameters generated by grouped convolution is the same as the number of parameters of standard convolution. The multiple reduction in parameters (by a factor of 1) indicates that grouped convolutions can significantly reduce the number of parameters, thereby lowering computational complexity and memory usage. Furthermore, it offers flexibility; by adjusting G, a suitable balance between computational efficiency and model performance can be found for specific tasks.

[0122] Furthermore, in one embodiment, step 1 involves "surrogate correction" of the normalized activation values. Specifically, this involves "surrogate correction" of the normalized activation values ​​based on hierarchical normalization and statistical constraint optimization of the surrogate variable. The process includes:

[0123] Steps 1-5, Group Norm normalization:

[0124] Divide the input activation X into G groups, and compute the normalized output for each group. Where μ and σ are the within-group mean and standard deviation, respectively;

[0125] Steps 1-6: Construct proxy variables:

[0126] Generate an ideal standard Gaussian distribution variable Z ~ N(0,1) as a proxy variable;

[0127] Steps 1-7: Share affine transformation and activation function:

[0128] right Applying the same affine transformation parameters (γ, β) and activation function φ as Z, we obtain... Y proxy :

[0129]

[0130] Y proxy =φ(γZ+β)

[0131] Steps 1-8, Statistical offset correction:

[0132] Use Y proxy Statistical property correction At the same time, a hierarchical normalization method is introduced to perform the normalization of proxy variables in multiple levels:

[0133]

[0134] Where Y represents the correction value, ε is the adjustment factor, and γ l and β l These are the normalized weight parameters of the l-th layer, learned independently in each layer, μ proxy and σ proxy These are the mean and standard deviation of the hierarchical proxy variables, respectively, calculated using the following formula:

[0135]

[0136] In the formula, Let L be the proxy variable for the l-th layer, and L be the total number of layers.

[0137] This design, through a hierarchical proxy normalization method, enables finer-grained adjustment of feature distribution;

[0138] Z proxy Statistical constraints introduced to the distribution of proxy variables (to make the generation of proxy variables more stable):

[0139] Zproxy =Proj N(0,1) (Z init )

[0140] Among them, Proj N(0,1) It is a projection operator used to project the initial agent Z. init Mapping to a standard Gaussian distribution (ensuring that the proxy variable always maintains the target distribution and avoids the influence of distribution shift during training):

[0141]

[0142] μ init =mean(Z) init )

[0143] σ init =std(Z init )

[0144] In the formula, mean(Z) init ) and std(Z init ) represent Z respectively init The mean and standard deviation.

[0145] Here, we innovatively propose a hierarchical normalization and statistical constraint optimization based on surrogate variables. By performing "surrogate correction" on the normalized activation values, we can solve the non-normalization problem and adjust the feature distribution with finer granularity, ensuring that the surrogate variables always maintain the target distribution.

[0146] Furthermore, in one embodiment, the introduction of a lightweight ReLU multi-scale linear attention mechanism (such as...) in step 1... Figure 3 (As shown) and weakens the limitations of ReLU linear attention, specifically including:

[0147] Steps 1-9 introduce a lightweight ReLU multi-scale linear attention mechanism to enable the global receptive field;

[0148] Given input The generalized form of softmax attention can be written as:

[0149]

[0150] Where Q = xW Q K = xW K V = xW V And W Q / W K / It is a learnable linear projection matrix. O iLet O represent the i-th row of matrix O, and Sim(·,·) be the similarity function. Utilizing the associative properties of matrix multiplication, the computational complexity and memory usage are reduced from quadratic to linear without altering its functionality:

[0151]

[0152] Steps 1-10 involve inserting a depthwise convolution into each FFN layer to enhance ReLU linear attention and mitigate its limitations. Multi-scale tokens are obtained by aggregating information from nearby Q / K / V tokens to enhance the multi-scale learning capability of ReLU linear attention. This information aggregation process is independent for each Q, K, and V in each head. After obtaining the multi-scale tokens, ReLU linear attention is applied to them to extract multi-scale global features. Finally, the features are concatenated along the head dimension and fed into the final linear projection layer to fuse the features.

[0153] Here, a lightweight ReLU multi-scale linear attention mechanism is introduced. At the same time, the limitations of ReLU linear attention are weakened according to the characteristics of this model, which can further optimize the feature representation ability and computational efficiency of the model, thereby meeting the needs of real-time detection in complex environments.

[0154] Furthermore, in one embodiment, step 1 further includes improving the loss function of the Yolov8 network, including:

[0155] Steps 1-11: Improvement of MPDIoU loss function based on bounding box regression;

[0156] Given any two convex shapes: Input image width and height: w, h, for A and B, Indicate the coordinates of the top left and bottom right corners of A. Let d1 and d2 represent the coordinates of the top-left and bottom-right corners of B. Now, redefine the distances d1 and d2:

[0157]

[0158] Therefore, we get:

[0159]

[0160] MPDIoU is chosen as the loss for bounding box regression to ensure that each bounding box B predicted by the model... prd =[x prd ,y prd ,w prd ,h prd ] T Approximately its true bounding box B gt =[x gt,y gt ,w gt ,h gt ] T During training, the following loss function is minimized:

[0161]

[0162] Where Θ represents the parameters of the deep regression model, based on the definition of MPDIoU. Therefore, the loss function based on MPDIoU is defined as follows:

[0163] L MPDIoU =1-MPDIoU.

[0164] Steps 1-12: Improvement of the classification-based Focal Loss function;

[0165] Focal Loss is an improved loss function based on cross-entropy (CE). The formula for cross-entropy (CE) is as follows:

[0166]

[0167] Here, y takes values ​​{1, -1} to represent positive and negative samples, and p is the label probability predicted by the model. Typically, p > 0.5 is considered a positive sample, otherwise it's considered a negative sample. For ease of representation, p is redefined. t :

[0168]

[0169] Thus, the CE function can be expressed as:

[0170] CE(p,y)=CE(p t ) = -log(p t )

[0171] A common method to resolve class imbalance is to introduce a weighting factor α∈[0,1] for class 1 and a weighting factor 1-α for class -1. α can be set using the inverse class frequency. For notation convenience, α is defined as... t Methods and definitions of p t The approach is similar. The α-balanced CE loss can be written as:

[0172] CE(p t )=-α t log(p t )

[0173] While α balances the importance of positive / negative examples, it doesn't distinguish between easy / hard examples. Therefore, the loss function is redefined to reduce the weight of easy examples, thus focusing training on hard negative examples. A modulation factor (1-p) is added to the cross-entropy loss. t ) γ Where the adjustable focusing parameter γ ≥ 0. This yields the final Focal Loss function, which is defined as:

[0174] FL(p t )=-(1-p t ) γ log(p t ).

[0175] Step 2 involves collecting small target image data from various scenarios, preprocessing it, and constructing a dataset. Specifically, this includes:

[0176] Step 2-1: For a specific detection scenario, collect small target image data with various resolutions and backgrounds to construct an initial dataset;

[0177] Step 2-2: Perform noise filtering and enhancement processing on the images in the initial dataset;

[0178] Steps 2-3 involve annotating the images with labels (using annotation tools to generate YOLO format label files), including the coordinates of the target bounding boxes and category information, thereby obtaining the final dataset.

[0179] Preferably, in some embodiments, in step 2-2, noise filtering is performed using an improved bilateral filtering noise removal method, wherein the filtering formula of the improved bilateral filtering noise removal method is:

[0180]

[0181] Among them, I in (x) is the gray value of pixel x in the input image, I in (y) is the gray value of pixel y in the input image, I out (x) is the gray value of pixel x in the input image, Ω represents the neighborhood of pixel x, i.e., the window size, and f s (||xy||) is defined as the space kernel function:

[0182]

[0183] In the formula, σ s It is the smoothing range of the control domain;

[0184] Among them, f r (|I in (x)-Iin (y)|) is the intensity kernel function, defined as:

[0185]

[0186] In the formula, σ r Weights are used to control the similarity of pixel values;

[0187] In particular, gradient domain constraints are introduced into the weight calculation of bilateral filtering:

[0188]

[0189] In the formula, |I in (x)-I in (y)| represents the difference between the pixel values ​​of pixel x and pixel y. It is the gradient difference between pixel x and pixel y, σ g Used to control gradient sensitivity.

[0190] Here, the traditional improved bilateral filtering noise removal method can smooth noise while preserving edge details. Although bilateral filtering can preserve edge information well, it is prone to edge blurring and insufficient sensitivity to gradient details in some high-noise or complex structure scenarios. This embodiment adds a gradient domain constraint on the basis of spatial and intensity domains to enhance edge preservation, thus more effectively distinguishing edge details from noise.

[0191] Preferably, in some embodiments, in step 2-2, an image enhancement method based on contrast-limited adaptive histogram equalization is used for image enhancement processing. This image enhancement method based on contrast-limited adaptive histogram equalization improves upon the problem of excessive noise amplification inherent in adaptive histogram equalization, aiming to limit contrast amplification and reduce the impact of noise. Based on adaptive histogram equalization, the concept of contrast limitation is introduced:

[0192] When performing histogram equalization, a threshold T is calculated to control the equalization effect on sub-blocks:

[0193]

[0194] Where, N x N represents the number of pixels in the horizontal direction of the sub-block. y C represents the number of pixels in the vertical direction of the sub-block. clip As a parameter limiting contrast, the cropping factor is used. For the cropped portion, pixels exceeding T are uniformly redistributed across all gray levels, with M gray levels.

[0195] Further, in one embodiment, step 4, training the YOLO model based on EfficientVIT fusion multi-scale attention improvement using the training set, includes:

[0196] The preprocessed data is input into an improved YOLO model based on EfficientVIT and multi-scale attention, and forward and backward propagation are performed according to the model structure. An early stopping strategy is employed during training to avoid overfitting, and the training process is continuously adjusted using real-time monitoring metrics (mAP, Precision, Recall).

[0197] Here, step 4 also includes model validation: evaluating model performance using a validation set to ensure that the recall rate for small targets meets the needs of practical applications. The effectiveness of the model improvements is verified through comparative experiments (compared to benchmark models such as YOLOv8).

[0198] In summary, this method provides an innovative solution with both theoretical and practical value for the perception and tracking of small targets, such as small drones, in complex scenarios.

[0199] In one embodiment, a lightweight small target detection system based on the EfficientVIT improved YOLO model is provided, the system comprising:

[0200] The first module is used to build a YOLO model based on EfficientVIT and multi-scale attention improvement, including: improving the YOLOv8 network based on the backbone network of EfficientViT, generalizing depthwise convolution to grouped convolution, performing "surrogate correction" on the normalized activation values, and introducing a lightweight ReLU multi-scale linear attention mechanism to weaken the limitations of ReLU linear attention.

[0201] The second module is used to collect small target image data in various scenarios, perform preprocessing, and construct a dataset; the definition of small targets is customized according to requirements.

[0202] The third module is used to divide the dataset into a training set and a test set;

[0203] The fourth module is used to train the YOLO model based on EfficientVIT fusion multi-scale attention improvement using the training set to obtain a lightweight small target detection model.

[0204] The fifth module is used to acquire images of the area to be detected, input them into the lightweight small target detection model, and output the small target detection results.

[0205] Specific limitations regarding the lightweight small target detection system based on the EfficientVIT-improved YOLO model can be found in the limitations of the lightweight small target detection method based on the EfficientVIT-improved YOLO model mentioned above, and will not be repeated here. Each module in the aforementioned lightweight small target detection system based on the EfficientVIT-improved YOLO model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0206] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements:

[0207] Step 1: Establish a YOLO model based on EfficientViT and multi-scale attention improvement, including: improving the YOLOv8 network based on the backbone network of EfficientViT, generalizing depthwise convolution to grouped convolution, performing "surrogate correction" on the normalized activation values, and introducing a lightweight ReLU multi-scale linear attention mechanism to weaken the limitations of ReLU linear attention.

[0208] Step 2: Collect small target image data in various scenarios, perform preprocessing, and construct a dataset; the definition of small targets is customized according to requirements.

[0209] Step 3: Divide the dataset into a training set and a test set;

[0210] Step 4: Use the training set to train the YOLO model based on EfficientVIT fusion multi-scale attention improvement to obtain a lightweight small target detection model.

[0211] Step 5: Acquire an image of the area to be detected, input it into the lightweight small target detection model, and output the small target detection result.

[0212] For specific limitations on each step, please refer to the limitations on the lightweight small object detection method based on the EfficientVIT improved YOLO model mentioned above, which will not be repeated here.

[0213] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being implemented when executed by a processor:

[0214] Step 1: Establish a YOLO model based on EfficientViT and multi-scale attention improvement, including: improving the YOLOv8 network based on the backbone network of EfficientViT, generalizing depthwise convolution to grouped convolution, performing "surrogate correction" on the normalized activation values, and introducing a lightweight ReLU multi-scale linear attention mechanism to weaken the limitations of ReLU linear attention.

[0215] Step 2: Collect small target image data in various scenarios, perform preprocessing, and construct a dataset; the definition of small targets is customized according to requirements.

[0216] Step 3: Divide the dataset into a training set and a test set;

[0217] Step 4: Use the training set to train the YOLO model based on EfficientVIT fusion multi-scale attention improvement to obtain a lightweight small target detection model.

[0218] Step 5: Acquire an image of the area to be detected, input it into the lightweight small target detection model, and output the small target detection result.

[0219] For specific limitations on each step, please refer to the limitations on the lightweight small object detection method based on the EfficientVIT improved YOLO model mentioned above, which will not be repeated here.

[0220] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention without departing from its spirit and scope should be included within the protection scope of the present invention.

Claims

1. A lightweight small target detection method based on the EfficientVIT improved YOLO model, characterized in that, The method includes the following steps: Step 1: Establish a YOLO model based on EfficientViT and multi-scale attention improvement, including: improving the YOLOv8 network based on the backbone network of EfficientViT, generalizing depthwise convolution to grouped convolution, performing surrogate correction on the normalized activation values, and introducing a lightweight ReLU multi-scale linear attention mechanism to weaken the limitations of ReLU linear attention. Step 2: Collect small target image data in various scenarios, perform preprocessing, and construct a dataset; the definition of small targets is customized according to requirements. Step 3: Divide the dataset into a training set and a test set; Step 4: Use the training set to train the YOLO model based on EfficientVIT fusion multi-scale attention improvement to obtain a lightweight small target detection model. Step 5: Acquire an image of the area to be detected, input it into the lightweight small target detection model, and output the small target detection result; The improvement of the Yolov8 network based on the EfficientViT backbone network described in step 1 specifically includes: Step 1-1: Replace the Yolov8 backbone network with the EfficientViT backbone network; Steps 1-2 establish an improved Sandwich layout that incorporates dynamically adjusted weights, specifically represented as follows: Among them, X i X i+1 These represent the features of the i-th and (i+1)-th layers in the Sandwich layout, respectively. This represents the i-th layer of the FNN. This represents the self-attention mechanism of the i-th layer; α and β are dynamically adjusted parameters that can be adaptively optimized according to the input features, and σ is an activation function used to introduce nonlinear mapping; Steps 1-3 introduce a cascaded group attention module to solve the redundancy problem in multi-head self-attention; The attention calculation for each head in the cascaded attention module is as follows: Wherein, the j-th head calculates the input feature X i Self-attention corresponding to the segmentation above, X ij The input feature X i The j-th segment, X i It consists of all the segments, i.e., X i =[X i1 ,X i2 ,…,X ih And 1≤j≤h, where h represents the total number of heads, and each segment passes through a different projection layer. and Input features X i The projection layers, mapped to different subspaces, are ultimately passed through a linear layer. Project the output features of the connection back to the same dimension as the input; X represents ij Self-attention This indicates that the Concat function is used to apply the changes to X. ij With the input features X of the (i+1)th layer i+1 The connection; At the same time, the output of each head is added to the subsequent head: Among them, X i ' j The input feature X i The j-th segment X ij With the (j-1)th head output The sum of these features is used as the new input feature for calculating the j-th head. Steps 1-4 involve parameter reallocation by increasing the channel width of critical modules while decreasing the channel width of unimportant modules to redistribute parameters in the network; the critical and unimportant modules are defined in a custom way. Step 1 involves surrogate correction of the normalized activation values. Specifically, based on hierarchical normalization and statistical constraint optimization using surrogate variables, surrogate correction is performed on the normalized activation values. The process includes: Steps 1-5, Group Norm normalization: Divide the input activation X into G groups, and compute the normalized output for each group. Where μ and σ are the within-group mean and standard deviation, respectively; Steps 1-6: Construct proxy variables: Generate an ideal standard Gaussian distribution variable Z ~ N(0,1) as a proxy variable; Steps 1-7: Share affine transformation and activation function: right Applying the same affine transformation parameters (γ, β) and activation function φ as Z, we obtain... Y proxy : Steps 1-8, Statistical offset correction: Use Y proxy Statistical property correction At the same time, a hierarchical normalization method is introduced to perform the normalization of proxy variables in multiple levels: Where ε is the adjustment factor, γ l and β l These are the normalized weight parameters of the l-th layer, learned independently in each layer, μ proxy and σ proxy These are the mean and standard deviation of the hierarchical proxy variables, respectively, calculated using the following formula: In the formula, Let L be the proxy variable for the l-th layer, and L be the total number of layers. Z proxy Statistical constraints introduced for the distribution of proxy variables: WITH proxy =Project N(0,1) (WITH init ) Among them, Proj N(0,1) It is a projection operator used to project the initial agent Z. init Mapped to a standard Gaussian distribution: σ init =std(Z init ) In the formula, mean(Z) init ) and std(Z init ) represent Z respectively init The mean and standard deviation; The introduction of a lightweight ReLU multi-scale linear attention mechanism and the mitigation of the limitations of ReLU linear attention in step 1 specifically includes: A lightweight ReLU multi-scale linear attention mechanism is introduced to enable the global receptive field; Insert a depthwise convolution into each FFN layer to enhance ReLU linear attention and mitigate its limitations.

2. The lightweight small target detection method based on the EfficientVIT improved YOLO model according to claim 1, characterized in that, In steps 1-2, the parameters α and β are dynamically adjusted by adjusting the input features X. i The statistics are obtained through calculation; α=mean(X i ) β=std(X i ) Where, mean(X) i ) and std(X i ) represent X respectively i The mean and standard deviation.

3. The lightweight small target detection method based on the EfficientVIT improved YOLO model according to claim 1, characterized in that, Steps 1-4 specifically include: For Q and K projections, the channel size in each attention mechanism across all learning stages is reduced; for V projection, it is allowed to have the same dimension as the input embedding, i.e., the number of channels; while the expansion ratio of FFN is reduced to 2.

4. The lightweight small target detection method based on the EfficientVIT improved YOLO model according to claim 1, characterized in that, In step 1, depthwise convolution is extended to grouped convolution, which involves dividing the input channels into multiple groups and performing standard convolution on each group separately.

5. The lightweight small target detection method based on the EfficientVIT improved YOLO model according to claim 1, characterized in that, Step 1 also includes improving the loss function of the Yolov8 network, including improvements to the MPDIoU loss function based on bounding box regression and the Focal Loss loss function based on classification.

6. The lightweight small target detection method based on the EfficientVIT improved YOLO model according to claim 1, characterized in that, Step 2 involves collecting small target image data from various scenarios, preprocessing it, and constructing a dataset. Specifically, this includes: Step 2-1: For a specific detection scenario, collect small target image data with various resolutions and backgrounds to construct an initial dataset; Step 2-2: Perform noise filtering and enhancement processing on the images in the initial dataset; Steps 2-3 involve annotating the images with information, including the coordinates of the target bounding boxes and category information, thereby obtaining the final dataset.

7. The lightweight small target detection method based on the EfficientVIT improved YOLO model according to claim 1, characterized in that, In step 2-2, noise filtering is performed using an improved bilateral filtering noise removal method. The filtering formula for the improved bilateral filtering noise removal method is as follows: Among them, I in (x) is the gray value of pixel x in the input image, I in (y) is the gray value of pixel y in the input image, I out (x) is the gray value of pixel x in the input image, Ω represents the neighborhood of pixel x, i.e., the window size, and f s (||xy||) is defined as the space kernel function: In the formula, σ s It is the smoothing range of the control domain; Among them, f r (|I in (x)-I in (y)|) is the intensity kernel function, defined as: In the formula, σ r Weights are used to control the similarity of pixel values; In particular, gradient domain constraints are introduced into the weight calculation of bilateral filtering: In the formula, |I in (x)-I in (y)| represents the difference between the pixel values ​​of pixel x and pixel y. It is the gradient difference between pixel x and pixel y, σ g Used to control gradient sensitivity.

Images