Image small target detection method based on fusion feature visual model

Abstract

The invention discloses an image small target detection method based on a fusion feature visual model, and relates to the technical field of computer visual detection. And the fusion feature visual detection model performs the following processing on an input image and outputs a detection result: adopting adaptive Daubechies wavelet transform to enhance high-frequency details, combining multi-scale feature fusion with a double attention mechanism to enhance small target feature expression, utilizing regression calculation to predict bounding box coordinates, and realizing category probability distribution prediction based on a Softmax function. A multi-task loss function and an Adam optimizer are adopted for end-to-end training, and the model performance is improved by balancing classification and regression loss. The method effectively solves the problems that small target features are deficient and are susceptible to background interference, significantly improves the detection precision while maintaining the calculation efficiency, and is especially suitable for unmanned aerial vehicle patrol and other practical application scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision detection technology, specifically to a method for detecting small objects in images based on a fusion feature visual model. Background Technology

[0002] Image target detection is a core task in computer vision, with wide applications in security monitoring, autonomous driving, remote sensing analysis, and drone patrols. However, while existing detection technologies have achieved significant results in recognizing large and medium-sized targets, their performance in detecting small targets in images still falls far short of practical needs. Small targets are typically defined as those occupying a small pixel area in an image (e.g., defined in the COCO dataset as targets with a pixel area less than 32×32), and their feature representation and accurate localization face fundamental challenges.

[0003] Specifically, small target detection mainly faces the following major technical bottlenecks: Limited inherent features and weak representational ability: Small targets contain only extremely limited pixel information in images, resulting in their visual features (such as texture and shape) being indistinct and having low resolution. When the input image is downsampled multiple times by a deep convolutional neural network (stride convolution or pooling operations), these already weak features almost disappear completely in the high-level feature maps, causing severe information loss and making it difficult for the detector to perform effective class differentiation and location regression.

[0004] Limitations of existing network architectures: Traditional Convolutional Neural Networks (CNNs) rely primarily on local receptive fields and inductive biases (such as translation invariance) for feature extraction. This locality makes it difficult to capture the correlation between small objects and the long-range context of the image, while global contextual information is crucial for identifying small objects in complex backgrounds. Although Transformer-based detectors (such as DETR) possess powerful global modeling capabilities through self-attention mechanisms, mitigating the locality problem of CNNs, their computational complexity is proportional to the square of the image size. When processing high-resolution input images containing numerous small objects, this results in unbearable computational overhead and memory consumption, severely limiting their application in real-time detection scenarios (such as video stream analysis).

[0005] Background noise interference and sample imbalance: Due to the tiny size of small targets, their effective features are easily overwhelmed by complex background noise. At the same time, in typical datasets, the number of small target instances is usually far greater than that of large and medium targets. However, during model training, the gradient contribution for small targets is easily diluted by the numerically dominant background regions and large targets, resulting in insufficient attention from the model to them, thereby further reducing detection accuracy.

[0006] Therefore, developing a small target detection method capable of balancing feature richness, global context awareness and computational efficiency has become a key technical problem to be solved in the field. SUMMARY

[0007] In view of the deficiencies in the prior art, the present application provides an image small target detection method based on a fusion feature visual model to solve the technical problems mentioned in the background art.

[0008] To achieve the above object, the present application is implemented by the following technical solutions: An image small target detection method based on a fusion feature visual model, comprising the following steps: inputting a to-be-detected image into a fusion feature visual detection model, performing the following processing on the input image by the fusion feature visual detection model and outputting a detection result: S1, feature extraction and enhancement: performing convolution processing on the input image to generate an initial feature map, performing wavelet transform on the initial feature map, and adaptively selecting the base function order of the wavelet transform based on the gradient amplitude variance to enhance the high-frequency details, and obtaining an enhanced feature map through inverse wavelet transform; S2, fusion of multi-scale features and highlighting of small target features: performing multi-scale fusion on the enhanced feature maps from different layers, and applying channel and spatial attention mechanisms for weighting to obtain a weighted feature map for prediction; S3, predicting small target position and category: based on the weighted feature map and a preset anchor box, regression calculation of the predicted bounding box coordinates of the small target; based on the weighted feature map, calculation of the category score and conversion thereof into a probability distribution to determine the predicted category of the small target; output of the final detection result containing the target position and category probability.

[0009] As a further scheme of the present application, the specific steps of the feature extraction and enhancement are as follows: S11, after preprocessing of normalization and image enhancement, inputting the to-be-detected image into a multi-layer convolution structure to generate an initial feature map; S12, transforming the generated initial feature map using Daubechies biorthogonal wavelet to decompose it into low-frequency components and horizontal, vertical and diagonal high-frequency components; S13, adaptively selecting the base function order of the Daubechies wavelet according to the feature complexity of the small target: calculating the gradient amplitude variance of the initial feature map; if the gradient amplitude variance is greater than a preset threshold, selecting a high-order Daubechies wavelet, i.e. a Daubechies wavelet with an order greater than or equal to 4; If the gradient amplitude variance is less than or equal to a preset threshold, a low-order Daubechies wavelet is selected, i.e., a Daubechies wavelet with an order less than or equal to 2; S14, the low-frequency component and the horizontal direction high-frequency component, the vertical direction high-frequency component and the diagonal direction high-frequency component obtained by decomposition are respectively subjected to convolution processing, and the detail features are further strengthened; S15, the low-frequency component and the horizontal direction high-frequency component, the vertical direction high-frequency component and the diagonal direction high-frequency component after convolution processing are subjected to inverse wavelet transform, recombined into a complete feature map, and the original size and information of the image are restored, and finally an enhanced feature map is obtained.

[0010] As a further scheme of the application, the specific steps of fusing multi-scale features and highlighting small target features are as follows: S21, the enhanced feature maps from different convolution layer outputs are respectively subjected to upsampling or downsampling operation, so that the sizes of the feature maps of each layer are consistent: The upsampling uses bilinear interpolation, and the calculation formula is: ; Where X i,j is the value of the feature map after upsampling at position (i,j), w m,n is the interpolation weight, which is used to enlarge the size of the feature map, and is calculated according to the principle of bilinear interpolation ; The downsampling adopts maximum pooling, and the calculation formula is: ; Where Y i,j is the value of the feature map after downsampling at position (i,j), s is the pooling step, is the value of the corresponding region in the original feature map; S22, the feature maps of each layer after size unification are spliced according to the channel dimension to obtain a fused feature map F fusion , and the splicing relationship is: ; S23, an attention mechanism is introduced, and the channel attention weight A c and the spatial attention weight A s are calculated respectively; The calculation method of the channel attention weight A c is: performing global average pooling compression on the spatial dimension of the fused feature map F fusion to obtain a channel-level global feature GAP(F fusion ); then inputting the global feature into a first fully connected layer FC1, a ReLU activation function δ(x)=max(0,x), a second fully connected layer FC2 and a Sigmoid function for processing, and finally obtaining Ac , the calculation formula is: ; Wherein FC1 is used for dimension reduction of global features, and FC2 is used for restoring the dimension-reduced features to the original channel number; The spatial attention weight A s is calculated as follows: the fusion feature map F fusion is respectively subjected to maximum pooling and average pooling, and then is spliced by channel, and finally is subjected to convolution layer Conv1, ReLU activation function, convolution layer Conv2 and Sigmoid function processing, and finally the A s is obtained, and the calculation formula is: ; Wherein max-pool(F fusion ) is global maximum pooling, and the maximum value in the spatial dimension is taken for each channel of F fusion ; avg-pool(F fusion ) is global average pooling, and the average value in the spatial dimension is taken for each channel of F fusion ; S24, the channel attention weight A c , the spatial attention weight A s and the fusion feature map F fusion are multiplied to obtain the final weighted feature map F weighted , and the calculation formula is: .

[0011] As a further scheme of the present application, the specific steps of predicting the small target position and category are as follows: S31, the weighted feature map is subjected to convolution processing, and based on the convolution-processed features, the bounding box prediction offset of the small target is output by the bounding box prediction fully connected layer , and the K category prediction scores are output by the category prediction fully connected layer , wherein K is an integer greater than 1; S32, the regression function is combined with the center coordinates (x p , y p ) of the preset anchor frame, the width w p and the height h p , and the bounding box prediction offset (t x , t y , t w , t h ) to calculate the final bounding box coordinates of the small target , specifically including: The center coordinates of the final bounding box are: ; ​The width of the final bounding box and height for: ; Where σ is the Sigmoid function, (t w ,t h ) represents the predicted width and height offset of the bounding box, used to adjust the width and height of the anchor box; S33. Input the K category prediction scores output by the fully connected layer into the Softmax function to calculate the probability of each category. The calculation formula is: ; Where z c Let K be the predicted score for category c, and K be the total number of categories. To predict the probability that a sample belongs to class c; S34. Coordinates of the bounding box of the small target in the final output image. and corresponding category probability As a result of the test.

[0012] As a further aspect of the present invention, the fused feature visual detection model is obtained through the following training method: Step 1, Forward Propagation: Input the training image into the initial model, and sequentially perform the feature extraction and enhancement, multi-scale feature fusion and highlighting of small target features, and prediction of the location and category of small targets to obtain the model prediction results; Step 2, Loss Calculation and Parameter Update: Construct a multi-task loss function based on the difference between the model's prediction results and the true labels. Minimize the multi-task loss function using an optimization algorithm that combines momentum and adaptive learning rate mechanisms. Dynamically adjust the learning rate to update the model parameters using a learning rate decay strategy. At the same time, prevent overfitting by continuously monitoring the loss and performance metrics on the training and validation sets to ensure that the model effectively converges to the optimal state. Step 3: Iterative training until convergence: Repeat steps 1 to 2 until the model converges, and obtain the final fused feature visual detection model.

[0013] As a further aspect of the present invention, the specific method for constructing the multi-task loss function in step two is as follows: P1. Define the overall structure of the total loss function, expressed as follows: ; Where L cls For classification loss, L reg The regression loss is α, where α is the balance coefficient and α ∈ [0, 1]. P2. The classification loss uses cross-entropy loss, which measures the difference between the predicted class probability and the true label. For a training batch containing N samples, each sample belongs to one of K classes. The classification loss is calculated using the following formula: ; where L cls is the classification loss, N is the number of training batch samples, K is the total number of classes, y i,c is the true label of sample i, is the probability of the model predicting that sample i belongs to class c; P3, the regression loss adopts a smooth L1 loss, which measures the difference between the predicted bounding box and the true bounding box, and the regression loss calculation formula is: ; where is the offset or scaling factor of the model predicting sample i in dimension m, t i,m is the corresponding true offset or scaling factor; The SmoothL1 function is defined as follows: ; When |x|<1, a quadratic function is adopted to avoid gradient explosion; When |x|≥1, a linear function is adopted to enhance the punishment of large errors.

[0014] As a further scheme of the application, in step two, the specific way of minimizing the multi-task loss function by combining the momentum and adaptive learning rate mechanism optimization algorithm and dynamically adjusting the learning rate to update the model parameters is as follows: T1, calculate the gradient of the multi-task loss function to the current model parameters: For the multi-task loss function L total , calculate the gradient of the multi-task loss function L total to the model parameters θ t at the t-th step, expressed as: ; where θ t is the model parameter at the t-th step, including the weights and biases of the neural network; L total is the multi-task loss function of the model, composed of the classification loss and the regression loss; T2, update the first moment and the second moment estimates: Update the first moment based on the gradient , expressed as: ; where m t is the first moment estimate at the t-th step, β1 is the momentum decay coefficient, m t-1 is the first moment estimate at the t-1-th step; Update the second moment based on the gradient , expressed as: ; where v t ​For the second moment estimation of the t-th step, beta2 is an adaptive attenuation coefficient, and v t-1 For the first moment estimation of the t-1-th step; T3, correct the initial bias of the first moment and the second moment: The first moment m t is corrected for bias to obtain the corrected first moment , and the expression is: ; The second moment v t is corrected for bias to obtain the corrected second moment , and the expression is: ; Where t is the current training step, counted from 1; T4, update the model parameters: based on the corrected first moment and the second moment , update the model parameters theta t for the t+1-th step theta t+1 , and the expression is: ; Where eta is the learning rate, and o is a small constant to prevent the denominator from being zero, with a value range of 10 -9 -10 -7 ; T5, dynamically adjust the learning rate: use the exponential decay strategy to dynamically adjust the learning rate eta, and the learning rate eta t of the t-th training round, and the expression is: ; Where eta0 is the initial learning rate, gamma is the attenuation factor, with a value range of (0, 1), and t is the training round.

[0015] Compared with the prior art, the following beneficial effects are achieved: 1. By introducing wavelet transform, especially Daubechies wavelet with adaptive selection order, and combining with convolution features, the application can actively and specifically enhance high-frequency details in the image, which are the key features of small targets. This directly compensates for the inherent defects of small targets, such as few pixels and weak features, effectively preventing the features from being submerged or filtered out during the convolution downsampling process.

[0016] 2. After fusing multi-scale features, the application introduces a channel and spatial dual attention mechanism. This allows the model to not only integrate features from different depths and consider semantics and details, but also automatically learn and focus on channels and spatial positions that are crucial for small target detection. This is equivalent to allowing the model to learn to accurately locate and focus on small targets that are easily overlooked, significantly suppressing background noise interference and improving the quality and discriminability of feature representation.

[0017] 3. This invention uses a CNN as the main architecture, supplemented by wavelet transform and attention mechanisms for enhancement. This design typically has a much lower computational complexity than a pure Transformer model, making it more suitable for processing high-resolution images and meeting the real-time application requirements such as UAV video stream analysis. Simultaneously, through multi-scale fusion and spatial attention, the model also possesses a certain degree of context awareness, overcoming the shortcomings of traditional CNNs being too localized, and achieving a good balance between efficiency and performance.

[0018] 4. This invention incorporates multiple adaptive strategies. The adaptive wavelet basis selection dynamically chooses the wavelet order based on the gradient variance of the feature map, enabling the feature enhancement strategy to adapt to image content of varying complexity, thus improving the model's generalization ability and robustness. The adaptive learning rate and learning rate decay strategies make the model training process more stable, converge faster, and better approximate the optimal solution. The multi-task loss function coordinates classification and regression tasks through balancing coefficients, guiding the model to learn more evenly and further improving the final detection accuracy. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the overall method steps of the present invention; Figure 2 This is a flowchart of step S2 of the present invention; Figure 3 This is a flowchart of step S3 of the present invention; Figure 4 This is a flowchart of step S4 of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 like Figure 1 As shown, this embodiment of the invention provides a method for detecting small targets in images based on a fusion feature visual model, including the following steps: inputting the image to be detected into the fusion feature visual detection model, and having the fusion feature visual detection model perform the following processing on the input image and output the detection result: S1. Feature extraction and enhancement: The input image is convolved to generate an initial feature map. Then, the initial feature map is subjected to wavelet transform. The order of the basis function of the wavelet transform is adaptively selected based on the gradient magnitude variance to enhance high-frequency details. The enhanced feature map is obtained through inverse wavelet transform. S2, fuse multi-scale features and highlight small target features: fuse the enhanced feature maps from different layers in a multi-scale manner, and apply channel and spatial attention mechanisms for weighting to obtain weighted feature maps for prediction; S3, predict small target position and category: based on the weighted feature maps and preset anchor boxes, the predicted bounding box coordinates of the small target are calculated by regression; based on the weighted feature maps, the category score is calculated and converted into a probability distribution to determine the predicted category of the small target; and the final detection result containing the target position and category probability is output.

[0022] As shown in Figure 2 , the specific steps of feature extraction and enhancement are as follows: S11, after the input image to be detected is preprocessed by normalization and image enhancement, an initial feature map is generated by inputting a multi-layer convolution structure; Normalization processing (standardization processing): by subtracting the mean value of the image data set from the pixel value and then dividing by the standard deviation of the data set, the pixel value is mapped to the [0, 1] interval, and the formula is ; wherein μ and σ are the mean and standard deviation of the image data set, respectively; Data enhancement processing: one or more of random cropping, flipping and rotation are used to expand the data diversity to obtain preprocessed data; After preprocessing, the input image is represented as , wherein H, W and C represent the height, width and number of channels of the image (common color image C=3). In the feature extraction process, a multi-layer convolution structure is used; each layer of convolution operation uses a convolution kernel , and the convolution operation formula is as follows: ; , wherein represents the value of the output channel of the output feature map Y at position (i, j), which is obtained by weighted summation of the input feature map X in the region centered at (i, j) and adding a bias term ; is the weight value of the convolution kernel K at position (m, n), input channel c to output channel , the size and value of which determine the ability of the convolution operation to capture information at different positions and channels; k h and k w are the height and width of the convolution kernel, respectively, which control the size of the local receptive field of the convolution operation; after the above series of operations, an initial feature map is generated.

[0023] S12, in order to further enhance the ability to capture high-frequency details of small targets, the generated initial feature map The Daubechies biorthogonal wavelet is used for transformation, where h and w are the height and width of the feature map after convolution, and d is the number of channels thereof. After wavelet transformation, the feature map is decomposed into a low-frequency component F LL and three high-frequency components, i.e., a horizontal high-frequency component F LH , a vertical high-frequency component F HL , and a diagonal high-frequency component F HH .

[0024] S13, according to the feature complexity of the small target, the base function order N of the Daubechies wavelet is adaptively selected, and the feature complexity of the small target is measured by calculating the gradient amplitude variance V of the feature map. The gradient amplitude variance V of the feature map is calculated, and the formula is as follows: ; wherein is the gradient amplitude of the feature map at the (i, j) position, which reflects the local feature change degree of the position, and the square sum of the gradient amplitudes in the horizontal and vertical directions is calculated and then the square root is obtained, is the average gradient amplitude of the feature map, which is used to measure the overall feature change trend; If V > a preset threshold T (determined according to a large amount of experimental data and experience), it indicates that the feature complexity of the small target is high, and a high-order Daubechies wavelet (order ≥ 4) is selected, because the high-order wavelet has finer frequency resolution and can better capture complex high-frequency details; If V ≤ the preset threshold T, a low-order Daubechies wavelet (order ≤ 2) is selected to reduce the calculation amount and improve the processing efficiency.

[0025] S14, the three high-frequency components obtained by decomposition are respectively subjected to convolution processing to further strengthen the detail features; Suppose that the convolution operation is performed on the horizontal high-frequency component F LH , a convolution kernel K is used, and the convolution formula is as follows: ; wherein is the high-frequency component F LH after convolution, is the weight of the convolution kernel K LH , is a bias term, k h1 and k w1 are the sizes of the convolution kernel, the same convolution operation is performed on the vertical high-frequency component F HL and the diagonal high-frequency component F HH as on the horizontal high-frequency component F LH .

[0026] S15, the horizontal direction high frequency component after convolution processing , the vertical direction high frequency component , the diagonal direction high frequency component and the low frequency component F LL , inverse wavelet transform, recombine into a complete feature map , restore the original size and information of the image, and finally obtain the enhanced feature map; here the inverse wavelet transform process is the inverse operation of wavelet transform.

[0027] It is worth mentioning that: Daubechies wavelet (Daubechies Wavelets, abbreviated as DB wavelet) is a class of compactly supported orthogonal wavelets proposed by Belgian mathematician Ingrid Daubechies in 1988, and is one of the most influential and most widely used wavelets in the field of wavelet analysis. Its core value lies in solving the problem of "insufficient smoothness and limited localization ability" of early wavelets (such as Haar wavelet), and laying a key foundation for the transition of wavelet analysis from theory to engineering application.

[0028] Daubechies wavelet is named as "DB+order", different orders correspond to different characteristics, and is suitable for different scenes: DB1: that is, the basic Haar wavelet, the support length is 1, and the vanishing moment is 1; its function has a discontinuity (not smooth), but the calculation efficiency is very high, and it is suitable for fast rough division and preliminary feature extraction of signals; DB2: the support length is 3, the vanishing moment is 2, and the function is slightly smooth; because of its small amount of calculation and fast local response, it is often used in scenes that require real-time performance, such as signal denoising and edge detection; DB4: the support length is 7, the vanishing moment is 4, and the smoothness is better. It is the core wavelet of JPEG2000 image compression standard (default option), which balances compression efficiency and image quality; DB6 / DB8: higher order, longer support length (DB6 is 11, and DB8 is 15), stronger smoothness, suitable for scenes with high precision requirements (such as medical image processing and high-resolution image analysis).

[0029] In the Daubechies wavelet system, DB4 and above are usually considered as high order, especially in engineering scenes with high precision requirements; in the embodiment of the application, the low-order wavelet refers to the wavelet with order ≤2, and the high-order wavelet refers to the wavelet with order ≥4.

[0030] As shown in Figure 3 , the specific steps of fusing multi-scale features and highlighting small target features are as follows: S21, the enhanced feature map from the output of different convolution layers , respectively, to perform upsampling or downsampling operation, so that the size of each layer feature map remains consistent: The upsampling uses bilinear interpolation, the formula is: ; Where X i,j is the value of the feature map after upsampling at position (i, j), w m,n is the interpolation weight, used to enlarge the size of the feature map, calculated according to the principle of bilinear interpolation ; The downsampling uses max pooling, the formula is: ; Where Y i,j is the value of the feature map after downsampling at position (i, j), s is the pooling step, used to reduce the size of the feature map, which determines the ratio of downsampling, is the value of the corresponding region in the original feature map.

[0031] S22, the size of each layer feature map is unified and spliced according to the channel dimension to obtain the fusion feature map F fusion , the splicing relationship is: .

[0032] S23, in order to highlight the small target feature, the attention mechanism is introduced, and the channel attention weight A c and the spatial attention weight A s are calculated respectively; The calculation formula of the channel attention weight A c is: ; Where GAP(F fusion ) represents the global average pooling operation on the fusion feature map F fusion , which compresses the feature map in the spatial dimension to obtain the global feature representation of each channel; FC1 is the first fully connected layer to reduce the dimension of the global feature, and FC2 is the second fully connected layer to restore the reduced feature to the original channel number, FC1 and FC2 are used to transform and map the global feature; δ is the ReLU activation function, which increases the nonlinear expression ability of the model, and its formula is δ(x)=max(0, x); σ is the Sigmoid function, which maps the output value to the interval [0,1] to obtain the channel attention weight, and its formula is: ; The calculation formula of the spatial attention weight A s is: ; Where max-pool(F fusion ) is the global max pooling, and F fusionFor each channel, the maximum value is taken in the spatial dimension; avg-pool(F fusion For global average pooling, F fusion Each channel is averaged across the spatial dimension; [max-pool(F fusion ); avg-pool(F fusion )] represents the fused feature map F fusion Max pooling and average pooling operations are performed separately, and the results are concatenated along the channel dimension. Conv1 and Conv2 are convolutional layers used to extract spatial features. Spatial attention weights are then obtained after processing with ReLU activation and Sigmoid functions.

[0033] S24. Adjust the channel attention weights A c Spatial attention weight A s With fusion feature map F fusion Multiplying them together yields the final weighted feature map F. weighted The calculation formula is: .

[0034] like Figure 4 As shown, the location and category of small targets are predicted based on the weighted feature map. A combination of convolutional and fully connected layers is used. The convolutional layer further extracts features, and the fully connected layer outputs the prediction results, predicting the bounding box coordinates (x, y, w, h) and the category probability P(c) of the small target, where (x, y) is the center coordinate of the bounding box, and w and h are the width and height.

[0035] The specific steps for predicting the location and category of small targets are as follows: S31. The weighted feature map is convolved, and based on the result of the convolution, the bounding box prediction offset of the small target is output by the fully connected layer. The fully connected layer for category prediction outputs K category prediction scores. , where K is an integer greater than 1.

[0036] S32. By combining the center coordinates (x) of the preset anchor frame with a regression function. p , y p Width w p and height h p Combined with bounding box prediction offset (t) x , t y , t w , t h ), calculate the final bounding box coordinates of the small target. Specifically, it includes: The center coordinates of the final bounding box for: ; The width of the final bounding box and height for: ; Where σ is the Sigmoid function, used to map the predicted offset to a suitable range; (t w ,t h ) represents the width and height offset of the bounding box prediction, used to adjust the width and height of the anchor frame.

[0037] S33. Input the K category prediction scores output by the fully connected layer into the Softmax function to calculate the probability of each category. The calculation formula is: ; Where z c Let K be the predicted score for category c, and K be the total number of categories. To predict the probability that a sample belongs to class c, the sum of the probabilities of all classes is 1.

[0038] S34. Coordinates of the bounding box of the small target in the final output image. and corresponding category probability As the detection result, the bounding box coordinates can clearly express the position of small targets. This step outputs structured detection results, including the precise bounding box of each detected target (defined by center coordinates, width and height) and its corresponding class probability. This output format is a common standard for target detection tasks and can be directly applied to subsequent vision tasks, such as target tracking and behavior analysis, realizing a complete process from input image to usable detection results.

[0039] Example 2 This invention also provides a training method for a fusion feature visual detection model, used to train the fusion feature visual detection model in Embodiment 1. The fusion feature visual detection model is obtained through the following training method: Step 1, Forward Propagation: Input the training image into the initial model, perform feature extraction and enhancement as described in Example 1, fuse multi-scale features and highlight small target features, and predict the location and category of small targets to obtain the prediction results; Step 2, Loss Calculation and Parameter Update: Construct a multi-task loss function based on the difference between the model prediction results and the true labels. Minimize the multi-task loss function by combining momentum and adaptive learning rate mechanisms. Dynamically adjust the learning rate to update the model parameters through a learning rate decay strategy. At the same time, prevent overfitting by continuously monitoring the loss and performance indicators on the training and validation sets to ensure that the model effectively converges to the optimal state. Step 3: Iterative training until convergence: Repeat steps 1 to 2 until the model converges to obtain the fused feature visual detection model.

[0040] Specifically, the specific way of constructing the multi-task loss function is as follows: P1, define the overall structure of the total loss function, the expression is: ; Where L cls is the classification loss, L reg is the regression loss, and ɑ is the balance coefficient, ɑ ∈ [0,1], the optimal value needs to be determined through experiments combined with the characteristics of the data set and the task requirements: When the number of small targets is small and the class distinction is difficult, increase ɑ to strengthen the classification loss weight; When the positioning accuracy of small targets is high, reduce ɑ to strengthen the regression loss weight.

[0041] P2, the classification loss adopts cross-entropy loss, which measures the difference between the predicted class probability and the true label. For a training batch containing N samples, each sample belongs to one of the K classes, and the classification loss is calculated as: ; Where N is the number of training batch samples, K is the total number of classes, y i,c is the true label of sample i (binary value, 1 represents "yes" and 0 represents "no"), and represents whether sample i belongs to class c, is the probability that the model predicts that sample i belongs to class c, which is obtained by the Softmax function output, and the value range is between (0,1), and for each sample .

[0042] P3, the regression loss adopts smooth L1 loss, which measures the difference between the predicted bounding box and the true bounding box; for each sample i, four parameters of the bounding box need to be predicted: center coordinate offset and width and height scaling factor ; The regression loss is calculated as follows: ; Where is the offset or scaling factor of the model prediction of sample i in dimension m (4 dimensions: center x offset, center y offset, width scaling, height scaling), t i,m is the corresponding true offset, and the SmoothL1 function is defined as follows: ; When the error |x| between the predicted value and the true value is small (|x|<1), the SmoothL1 function adopts a quadratic function, and the gradient changes relatively gently, avoiding gradient explosion; When the error |x| between the predicted value and the true value is large (|x|≥1), SmoothL1 adopts a linear function form to ensure the penalty for large errors, prompting the model to adjust the predicted value more quickly.

[0043] The Adam optimizer is a widely used parameter optimization algorithm in the field of deep learning. Its core objective is to update model parameters (such as the weights and biases of a neural network) by minimizing the loss function. The algorithm's name comes from "Adaptive Moment Estimation," successfully integrating the ideas of two classic optimization algorithms. Momentum method: Similar to giving a ball inertia, allowing it to roll not only according to the current slope but also according to its previous velocity direction. This helps to accelerate convergence and escape local optima and saddle points; RMSProp: It has an adaptive learning rate mechanism that can "personalize" the learning rate for different parameters—using a smaller learning rate (small steps) for parameters that are updated frequently, and a larger learning rate (big steps) for parameters that are updated sparsely.

[0044] Furthermore, the optimization algorithm that combines momentum and adaptive learning rate mechanisms to minimize the multi-task loss function, and dynamically adjusts the learning rate to update model parameters through a learning rate decay strategy, is described in the following specific way: T1. Calculate the gradient of the multi-task loss function with respect to the current model parameters. For the multi-task loss function L total Calculate the multi-task loss function L at training step t. total For model parameters θ t gradient The calculation formula is: ; in It reflects the degree to which small changes in parameters affect the total loss, and serves as a "direction indicator" for parameter updates; θ t L represents the model parameters (including learnable parameters such as weights and biases of the neural network) at the t-th training step. total This is the multi-task loss function of the model, which is composed of a weighted average of classification loss and regression loss.

[0045] T2. Update the first moment (momentum term) and the second moment (adaptive learning rate term). First moment (momentum term): Tracks the mean of the gradient, simulating "physical momentum," making parameter updates more inertial (accelerating convergence and escaping local optima). The calculation formula is: ; Where m tFor the first moment estimate of the t-th step (momentum term), the "historical average trend" of the gradient is tracked, simulating physical inertia; β1 is the momentum decay coefficient (hyperparameter, usually 0.9), controlling the weight of historical gradients (the larger β1, the stronger the influence of historical gradients, the more stable the parameter update); m t-1 For the first moment estimate of the t-1-th step (momentum value of the previous step); Second moment (adaptive learning rate term): track the squared mean of the gradient, achieve "personalized learning rate" (reduce the step size for frequently updated parameters, increase the step size for sparse updated parameters), the calculation formula is: ; Where v t is the second moment estimate of the t-th step (adaptive term), used to track the "historical average trend" of the gradient, and achieve personalized adjustment of the learning rate; β2 is the adaptive decay coefficient (hyperparameter, usually 0.99), controlling the weight of historical gradients (the larger β2, the smoother the second moment estimate); t-1 is the first moment estimate of the t-1-th step (adaptive term value of the previous step).

[0046] T3, correct the initial bias of the first and second moments Since m0=0 and v0=0 at the beginning, directly using m t , v t will have a bias, which needs to be eliminated through "bias correction": Bias correction is performed on the first moment m t to obtain the corrected first moment , the expression is: ; Where t is the current training step (counting from 1), is the t-th power of β1 (tending to 0 as t increases, the correction intensity gradually weakens), is the corrected first moment estimate, eliminating the bias caused by the initial value of 0; Bias correction is performed on the second moment v t to obtain the corrected second moment , the expression is: ; Where t is the current training step (counting from 1), is the t-th power of β2 (tending to 0 as t increases, the correction intensity gradually weakens), is the corrected second moment estimate, eliminating the bias caused by the initial value of 0.

[0047] T4, update model parameters Based on the corrected first moment and the second moment , update the model parameters θ tThe parameter θ at step t+1 t+1 The expression is: ; Where θ t+1 The model parameters are updated at step t+1; θ t Here are the model parameters that haven't been updated at step t; η is the learning rate (a hyperparameter, usually initialized to 1e-4 or 3e-4, controlling the "step size" of parameter updates); it needs to be adjusted based on the characteristics of the model and dataset. A larger learning rate may cause the model to converge quickly in the early stages of training, but it may easily skip the optimal solution; a smaller learning rate will make the training process too slow, increasing training time. Typically, a large initial learning rate is set at the beginning of training, and then gradually decreased as training progresses. ω is the square root of the corrected second moment (used to "scale" the learning rate for adaptive adjustment); ω is a local constant (ranging from 10). -9 -10 -7 Usually 10 -8 This is to prevent the denominator from being 0 (to avoid numerical calculation errors).

[0048] T5, Dynamically Adjust Learning Rate The learning rate η is dynamically adjusted using an exponential decay strategy. The learning rate η in the t-th training round is... t The expression is: ; Where η t Let η be the learning rate for the t-th training round; η0 is the initial learning rate, i.e., the learning rate value set at the beginning of training; γ is the decay factor (a hyperparameter, usually ranging from (0,1), controlling the rate of learning rate decay), which increases with t. t The value of η gradually decreases, thus reducing the learning rate η. t Gradually decrease.

[0049] Before training begins, the model parameters need to be initialized. Common initialization methods include random initialization, initialization based on a normal distribution, or uniform distribution. In this embodiment of the invention, an improved initialization strategy is adopted based on the structure and function of each layer of the model. For the weights of the convolutional layers, initial values ​​are sampled from a truncated normal distribution based on the size of the convolutional kernel and the number of input channels, ensuring that the mean of the weights is 0. The standard deviation is calculated according to a preset rule, such as... , where k h and k w is the height and width of the convolution kernel, and C is the number of input channels; for the bias term, it is initialized to 0 or a small constant, and the weights and biases of the fully connected layer are initialized using similar logic.

[0050] After the training data is prepared, the model is inputted in batches to start training: in each training batch, the model first performs forward propagation, the input image is subjected to feature extraction and enhancement, multi-scale feature fusion and small target feature highlighting, small target position and category prediction, and the predicted bounding box coordinates and category probability are outputted; then, according to the preset multi-task loss function loss value of the current batch, the multi-task loss function gradient of the model parameters is obtained through the back propagation algorithm (based on the chain rule, the gradient of each parameter to the loss is calculated in turn from the multi-task loss function, and is realized by relying on an automatic derivative tool).

[0051] After the gradient is obtained, the aforementioned Adam optimizer is called to complete parameter updating. During the training process, the loss values and evaluation indexes of the training set and the validation set need to be monitored in real time: the model performance is evaluated on the validation set every certain number of training rounds, if the validation set loss value does not decrease continuously for multiple rounds or the evaluation index stagnates, it is determined that overfitting may occur, at this time, adjustment is made through measures such as reducing the learning rate, increasing the regularization term, and terminating the training in advance. Through continuous optimization of the training parameters and monitoring of the performance, the fused feature visual detection model gradually converges to a stable state, and finally a model that can effectively detect small targets in images is obtained.

[0052] The above embodiments are only used to illustrate the technical method of the present application and not to limit it, although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A method for detecting small targets in images based on a fusion feature visual model, characterized in that, The method comprises the following steps: The image to be detected is input into the fusion feature visual detection model, and the input image is processed by the fusion feature visual detection model and the detection result is output: S1, feature extraction and enhancement: performing convolution processing on the input image to generate an initial feature map, performing wavelet transform on the initial feature map, and adaptively selecting the basis function order of the wavelet transform based on the gradient amplitude variance to enhance the high-frequency details, and obtaining the enhanced feature map through inverse wavelet transform; S2, fusion of multi-scale features and highlighting of small target features: performing multi-scale fusion on the enhanced feature maps from different layers, and applying channel and spatial attention mechanisms for weighting to obtain a weighted feature map for prediction; S3, predicting the position and category of the small target: based on the weighted feature map and the preset anchor box, the predicted bounding box coordinates of the small target are calculated by regression; based on the weighted feature map, the category score is calculated and converted into a probability distribution to determine the predicted category of the small target; and outputting the final detection result containing the target position and category probability. 2.The image small target detection method based on the fusion feature visual model according to claim 1, characterized in that, The specific steps of the feature extraction and enhancement are as follows: S11, after the image to be detected is preprocessed by normalization and image enhancement, the image is input into a multi-layer convolution structure to generate an initial feature map; S12, the generated initial feature map is transformed by Daubechies biorthogonal wavelet to decompose it into low-frequency components and horizontal, vertical and diagonal high-frequency components; S13, according to the feature complexity of the small target, the basis function order of the Daubechies wavelet is adaptively selected: Calculate the gradient amplitude variance of the initial feature map; If the gradient amplitude variance is greater than a preset threshold, select a high-order Daubechies wavelet, i.e., a Daubechies wavelet with an order greater than or equal to 4; If the gradient amplitude variance is less than or equal to a preset threshold, select a low-order Daubechies wavelet, i.e., a Daubechies wavelet with an order less than or equal to 2; S14, the low-frequency components and the horizontal, vertical and diagonal high-frequency components obtained by decomposition are respectively subjected to convolution processing to further strengthen the detail features; S15, the low-frequency components and the horizontal, vertical and diagonal high-frequency components subjected to convolution processing are subjected to inverse wavelet transform to recombine into a complete feature map, and the original size and information of the image are restored, and finally an enhanced feature map is obtained. 3.The image small target detection method based on the fusion feature visual model of claim 1, wherein, The specific steps of the fusion of multi-scale features and the highlighting of small target features are as follows: S21, enhanced feature maps from different convolutional layer outputs respectively perform upsampling or downsampling operations to keep the sizes of the feature maps of each layer consistent: The up-sampling uses bilinear interpolation, and the calculation formula is: ; where X i,j is the value of the up-sampled feature map at position (i, j), w m,n is the interpolation weight used to upsize the feature map, which is calculated according to the principle of bilinear interpolation ; The down-sampling adopts the maximum pooling, and the calculation formula is: ; where Y i,j is the value of the down-sampled feature map at position (i, j), s is the pooling step size, is the value of the corresponding region in the original feature map; S22, the size of each layer feature map is unified and spliced in the channel dimension to obtain a fusion feature map F fusion , the splicing relationship is: ; S23, introduce attention mechanism, respectively calculate channel attention weight A c and spatial attention weight A s ; The channel attention weight A c The calculation method is: performing global average pooling compression space dimension on the fusion feature map F fusion fusion ); then inputting the global feature into a first fully connected layer FC1, a ReLU activation function δ(x) = max(0, x), a second fully connected layer FC2 and a Sigmoid function processing in sequence, and finally obtaining A c , and the calculation formula is: ;​ FC1 is used to reduce the dimensionality of the global feature, and FC2 is used to restore the dimensionality-reduced feature to the original channel number; The spatial attention weight A s The calculation method is: performing maximum pooling and average pooling on the fusion feature map F fusion respectively, splicing by channel, and processing through a convolution layer Conv1, a ReLU activation function, a convolution layer Conv2, and a Sigmoid function Finally, A s is obtained, and the calculation formula is: ； where max-pool(F fusion ) is a global max pooling that takes the maximum value over the spatial dimensions for each channel of F fusion ; avg-pool(F fusion ) is a global average pooling that takes the average value over the spatial dimensions for each channel of F fusion ; S24, the channel attention weight A c , the spatial attention weight A s is multiplied with the fusion feature map F fusion to obtain the final weighted feature map F weighted , and the calculation formula is as follows: .

4. The image small target detection method based on the fusion feature visual model according to claim 1, characterized in that, The specific steps of the prediction of the position and category of the small target are as follows: S31, performing convolution processing on the weighted feature map, and based on the feature after the convolution processing, outputting a bounding box prediction offset of the small target by a bounding box prediction fully connected layer , outputting K category prediction scores by a category prediction fully connected layer , wherein K is an integer greater than 1; S32, combine the center coordinates (x p , y p ) of the preset anchor frame, the width w p and the height h p by a regression function, combine the bounding box to predict the offset (t x , t y , t w , t h ), calculate the final bounding box coordinates of the small target , specifically including: center coordinates of the final bounding box are: ; width of the final bounding box and height are: ; where σ is a Sigmoid function, (t w ,t h ) is the width and height offset of the bounding box prediction, used to adjust the width and height of the anchor box. S33, input the K category prediction scores output by the category prediction full connection layer into a Softmax function to calculate the probability of each category , the calculation formula is: ; where z c is the predicted score for class c, K is the total number of classes, is the probability that the sample belongs to class c. S34, the bounding box coordinates of the small target in the final output image and the corresponding category probability are output as the detection result.

5. The image small target detection method based on the fusion feature visual model according to claim 1, characterized in that, The fusion feature visual detection model is obtained by the following training method: Step one, forward propagation: input the training image into the initial model, and sequentially perform the feature extraction and enhancement, the fusion of multi-scale features and the highlighting of small target features, and the prediction of the position and category of the small target as described in claim 1 to obtain the model prediction result; Step two, loss calculation and parameter update: according to the difference between the model prediction result and the real label, a multi-task loss function is constructed, the optimization algorithm combining momentum and adaptive learning rate mechanism is used to minimize the multi-task loss function, and the learning rate is dynamically adjusted by the learning rate decay strategy to update the model parameters; at the same time, the loss and performance indicators on the training and validation sets are continuously monitored to prevent overfitting and ensure that the model converges to the best state; Step three, iterative training to convergence: repeat the execution of steps one to two until the model converges, and obtain the final fusion feature visual detection model.

6. The image small target detection method based on the fusion feature visual model according to claim 5, characterized in that, In step two, the specific way of constructing the multi-task loss function is as follows: P1, defines the overall structure of the total loss function, expressed as: ; where L cls is a classification loss, L reg is a regression loss, a is a balancing coefficient and a e [0, 1]; P2, the classification loss adopts cross-entropy loss, measures the difference between the predicted category probability and the true label, for a training batch containing N samples, each sample belongs to one of K categories, and the classification loss calculation formula is: ; where L cls is the classification loss, N is the number of training batch samples, K is the total number of classes, y i,c is the true label of sample i, is the probability that the model predicts sample i belongs to class c; P3, the regression loss adopts a smooth L1 loss, measures the difference between the predicted bounding box and the real bounding box, and the regression loss calculation formula is: ; wherein is the offset or scaling factor for model prediction sample i in dimension m, t i,m is the corresponding true offset or scaling factor; the SmoothL1 function is defined as follows: ； When |x|<1, a quadratic function is used to avoid gradient explosion; When |x|≥1, a linear function is used to enhance the punishment for large errors.

7. The image small target detection method based on the fusion feature visual model according to claim 6, characterized in that, In step two, the specific way of using the optimization algorithm combining momentum and adaptive learning rate mechanism to minimize the multi-task loss function and dynamically adjusting the learning rate by the learning rate decay strategy to update the model parameters is as follows: T1, calculate the gradient of the multi-task loss function to the current model parameters: For the multi-task loss function L total , the gradient total of the multi-task loss function L t with respect to the model parameters θ at the t-th step of training is calculated, and the expression is as follows: ; where θ t is the model parameter of the t-th step, including the weights and biases of the neural network; L total is the multi-task loss function of the model, composed of the classification loss and the regression loss weighted. T2, update the first moment and second moment estimates: Based on the gradient The first moment is updated, expressed as: ; where m t is the first moment estimate of the tth step, β1is the momentum decay coefficient, m t-1 is the first moment estimate of the t-1th step; Based on the gradient The second moment is updated, with the expression: ; where v t is the second moment estimate for step t, β2is an adaptive decay coefficient, v t-1 is the first moment estimate for step t - 1. T3, correct the initial deviation of the first moment and second moment: The first moment m t The bias correction is performed to obtain the corrected first moment The expression is: ; The second-order moment v t The bias correction is performed to obtain the corrected second-order moment , and the expression is ; Where t is the current training step, starting from 1; T4, update model parameters: based on the corrected first moment and second moment , update model parameters θ t for the parameters θ t+1 of the t+1 step, the expression is: ; where η is the learning rate, and 0 is a small constant to prevent the denominator from being zero, with a range of 10 -9 -10 -7 ; T5, dynamically adjust the learning rate: adopt exponential decay strategy to dynamically adjust the learning rate η, the learning rate η of the t training t , the expression is: ; Where η0 is the initial learning rate, γ is the decay factor, the value range is (0, 1), and t is the training round.

Images