A method for infrared small target detection based on data expansion and weak feature enhancement

Through data expansion and weak feature enhancement technology, the problems of positive and negative samples imbalance and feature loss in infrared small object detection are solved, the generalization ability and efficiency of the detection model are improved, and more accurate small object detection is achieved.

CN118736241BActive Publication Date: 2024-11-01CHANGCHUN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411225051.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-01
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

The existing infrared small-object detection technology has extremely unbalanced positive and negative samples, resulting in poor overfitting and generalization capabilities, and traditional feature extraction methods lead to a large number of missing small-objective features, with high computational volume and complexity.

Method used

Using a method based on data augmentation and weak feature enhancement, the feature interactivity and generalization ability of the detection model are improved through target adaptive proliferation, sparse sampling-mixed filtering feature extraction and quad-in-one feature enhancement, combined with the joint loss function.

Benefits of technology

It effectively reduces the loss of target features, improves the generalization ability and feature enhancement effect of the detection model, reduces the computational complexity, and improves the accuracy and efficiency of infrared small object detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118736241B_ABST
    Figure CN118736241B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of target detection, and in particular, is a method for detecting small infrared targets based on data expansion and weak feature enhancement, including: S1, preparing data sets: preparing three infrared small target detection data sets, data set one and data set two are public data sets, used for network training and model fine-tuning; data set three is a self-made data set, used for model testing. The present invention is an infrared small target detection method based on data expansion and feature enhancement, constructs a new infrared small target detection model, designs a target adaptive proliferation method, a feature extraction method based on sparse sampling-hybrid filtering, and a four-in-one feature enhancement method. Through experimental data, it is proved that the present invention can solve the problem of imbalance between positive and negative samples, prevent overfitting of the model and poor generalization ability; at the same time, the present invention can reduce the loss of scarce features and gradually enhance weak features. The present invention shows good performance in the qualitative and quantitative evaluation of infrared small target detection tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of target detection and is an infrared small target detection method based on data expansion and weak feature enhancement. Background Art

[0002] Object detection is an important research direction in the field of computer vision, which aims to detect the position and outline of specific objects in images or videos. Object detection technology is often used in scenarios such as security monitoring, intelligent driving, drone inspection, industrial automation, etc., and is of great significance for improving the efficiency and accuracy of various fields. Among them, in the fields of modern warfare, maritime rescue and traffic management, the detection technology for small targets such as small drones and floating objects plays an important role. However, due to the characteristics of small target size, long imaging distance and strong concealment, when using visible light cameras to shoot and detect small targets, there are problems such as missed detection and false detection; while infrared imaging technology has the advantages of all-weather operation and strong robustness in the face of complex backgrounds. Using infrared imaging to assist in the detection of small targets can accurately capture long-distance and small-sized target images. Therefore, using infrared imaging to detect small-sized targets is the focus of current scholars.

[0003] The Chinese patent publication number is "CN118379601A", and its name is "A method for detecting small infrared targets based on a stepped interactive attention and pixel feature enhancement network". This method first inputs saliency maps of different scales in the backbone network into the stepped interactive attention module, and enhances the features with the help of pixel-by-pixel operations in the pixel feature enhancement module; then, the multi-dimensional feature fusion module fuses feature maps of different dimensions to obtain a saliency image, and performs deep supervision on it through the binary labels of small infrared targets.

[0004] However, this method does not solve the problem of extreme imbalance of positive and negative samples in the detection field, which easily leads to overfitting of the detection network and poor generalization ability in complex environments. At the same time, the pixel-by-pixel enhancement process greatly increases the computational complexity and complexity of the detection method. In addition, the traditional feature extraction backbone leads to a large loss of scarce features of small targets. Therefore, designing an infrared small target detection method that can reduce target feature loss, solve the problem of overfitting of the detection model, and achieve efficient feature enhancement is a problem that the present invention urgently needs to solve. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides an infrared small target detection method based on data expansion and weak feature enhancement, which solves the problems of overfitting and poor generalization ability caused by the extreme imbalance of positive and negative samples; then, it alleviates the phenomenon of feature loss of small targets in the process of continuous downsampling; finally, it improves the problem of poor feature interactivity between encoding and decoding.

[0007] (II) Technical solution

[0008] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical solutions:

[0009] An infrared small target detection method based on data expansion and weak feature enhancement specifically comprises the following steps:

[0010] S1, prepare data sets: prepare three infrared small target detection data sets. Data set 1 and data set 2 are public data sets used for network training and model fine-tuning; data set 3 is a self-made data set used for model testing;

[0011] S2, build an infrared small target detection model: the detection model includes four parts: target adaptive proliferation, sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding;

[0012] S3, training network model: training infrared small target detection network model, inputting data set 1 and data set 2 prepared in step S1 into the target adaptive proliferation method constructed in step S2, so as to obtain a high-quality expanded data set; inputting the obtained expanded data set into the sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding constructed in step S2 and training;

[0013] S4, select a suitable loss function and determine the optimal evaluation index of this method: select a suitable loss function to minimize the difference and loss between the output target prediction result and the input target image, set the training loss threshold, and continuously iterate and optimize the model until the number of training times reaches the set threshold or the value of the loss function reaches the set threshold range. The model parameters can be considered to have been pre-trained and the model parameters are saved; select the test image in the data set 3 and input it into the solidified model to obtain the target prediction result, and use the optimal evaluation index of the target detection result to measure the accuracy and performance of the model;

[0014] S5, fine-tuning the model, using the infrared small target detection dataset 2 to train and fine-tune the model, optimize the model parameters, further improve the performance of the detection network, and obtain target prediction results with clearer edges and more accurate positions;

[0015] S6, save the model, and after the fine-tuning training in S5 is completed, solidify the fine-tuned network parameters and determine the final small target detection model.

[0016] Furthermore, in S1, dataset one is the IRSTD-1k dataset; dataset two is the SIRST v1 dataset; and dataset three is a self-made infrared small target detection dataset.

[0017] Furthermore, in S2, the infrared small target detection model includes four parts: target adaptive proliferation, sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding; the target adaptive proliferation part can effectively increase the number and types of targets in each image in the data set; the sparse sampling-hybrid filtering feature extraction part can flexibly capture the complex shape and edge information of small targets, and reduce the loss and displacement of scarce features; the four-in-one feature enhancement part can utilize vertical and horizontal bidirectional attention and multi-scale feature encoding to establish a stable feature interaction channel between encoding and decoding, and further enhance the key features of small targets; the decoding part can gradually and effectively restore the edge features and spatial information of small targets.

[0018] Further, in said S2, the target adaptive proliferation method is composed of threshold segmentation, grayscale cumulative distribution function, histogram matching, collision detection, random scaling and rotation operations;

[0019] The threshold segmentation operation is used to segment targets with regular shapes and simple backgrounds into target and background areas with smoother transitions, thereby obtaining a pure target database;

[0020] The grayscale cumulative distribution function operation is used to describe the grayscale value distribution of the infrared background image in the self-made data set and the image in the public data set, so as to achieve histogram matching of the image, thereby effectively expanding the background material of the data set;

[0021] The collision detection, random scaling and rotation operations are used to further enhance the expression ability and presentation form of small objects in the expanded data set;

[0022] Furthermore, in S2, the sparse sampling-hybrid filtering feature extraction part is composed of feature extraction block 1, feature extraction block 2, feature extraction block 3 and feature extraction block 4; each feature extraction block has the same structure, consisting of 1 layer of hybrid filter, 1 layer of sparse sampling convolution, 1 layer of 1×1 convolution, 2 layers of 3×3 convolution and 3 layers of L-type activation layers, which are used to reduce the information loss between sub-regions during the downsampling process and retain the shallow key features of small targets;

[0023] Furthermore, in S2, the four-in-one feature enhancement part consists of four-in-one feature enhancement block 1, four-in-one feature enhancement block 2 and four-in-one feature enhancement block 3; each four-in-one feature enhancement block has the same structure, consisting of one multi-scale feature refinement module and two multi-scale linear feature modules, and can enhance the interaction between deep and shallow features through a bidirectional attention mechanism and multi-scale feature encoding;

[0024] Furthermore, in S2, the decoding part is composed of decoding block 1, decoding block 2, decoding block 3 and decoding block 4; each decoding block has the same structure, consisting of 1 layer of 1×1 convolution, 1 layer of Instance Norm and 1 layer of L-type activation layer;

[0025] Further, in S4, the loss function is a joint loss, including focal loss, binary cross entropy loss and L2 loss;

[0026] The focus loss is used to balance the probability distribution between the target and the background, so that the detection network pays more attention to the small target area with a smaller size ratio;

[0027] The binary cross entropy loss is used to provide a stable gradient for the training process, helping the model to converge smoothly and making the training process more reliable;

[0028] The L2 loss is used to reduce noise, process outliers, and effectively capture the segmentation information of small objects;

[0029] Furthermore, in S4, the process of training the network model also includes evaluating the performance of the algorithm prediction results through evaluation indicators.

[0030] The beneficial effects of the present invention are as follows:

[0031] (1) This paper designs a new end-to-end infrared small target detection network model. By designing a sparse sampling mechanism based on the number of adaptive convolution sampling points, the complex shape and edge information of small targets can be captured more flexibly. At the same time, a hybrid filtering method based on BlurPool is designed to solve the problem of strengthening high-frequency noise and suppressing low-frequency target areas, thereby reducing the loss of scarce features.

[0032] (2) The present invention designs a new end-to-end infrared small target detection network model, introduces a multi-scale feature enhancement module, uses vertical-horizontal bidirectional attention and multi-scale feature encoding to establish a stable feature interaction channel between the encoder and decoder, generates deep semantic linear features to guide shallow edge enhancement, integrates long-range and short-range features in the spatial domain, and enhances the features of weak small targets.

[0033] (3) The present invention designs a new end-to-end infrared small target detection network model and introduces a data adaptive proliferation strategy. It creates a target database through threshold segmentation, adjusts the homemade infrared image through the grayscale cumulative distribution function, and uses collision testing, rotation and scaling to effectively expand the infrared small target data, thereby solving the problem of imbalance between positive and negative samples and enhancing the generalization and expression capabilities of the detection data set.

[0034] (4) The present invention designs a new end-to-end infrared small target detection network model and introduces a joint loss that is an effective combination of focal loss, binary cross entropy loss and L2 loss, which helps the detection network pay more attention to target areas with smaller sizes and provides a stable gradient for the training process, making the training process more reliable.

[0035] (5) The detection model proposed in this paper has shown good results in the SIRST v1 dataset, IRSTD-1k dataset and self-made dataset, and the five quantitative evaluation indicators have all achieved significant improvements, indicating that the detection method proposed in this paper has a very strong generalization ability in single / multi-target and different complex backgrounds, and can adapt to most small target detection tasks and scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a flow chart of an infrared small target detection method based on data expansion and weak feature enhancement;

[0037] Figure 2 The schematic diagram of the infrared small target detection model constructed by the present invention;

[0038] Figure 3 This is a structural diagram of the sparse sampling-hybrid filtering feature extraction of the present invention;

[0039] Figure 4 This is a structural diagram of the hybrid filter of the present invention;

[0040] Figure 5 This is a schematic diagram of the sparse sampling convolution principle of the present invention;

[0041] Figure 6 This is a structural diagram of the four-in-one feature enhancement module of the present invention;

[0042] Figure 7 This is a structural diagram of the multi-scale linear feature module of the present invention;

[0043] Figure 8 This is a structural diagram of the multi-scale feature refinement module of the present invention;

[0044] Fig. 9 This is a structural diagram of the decoding block of the present invention;

[0045] Fig.10 This is a qualitative comparison effect diagram of the infrared small target detection method described in this invention and the existing methods;

[0046] Fig.11 This is a schematic diagram comparing the evaluation indicators of the infrared small target detection method described in this invention with the existing methods. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] Example 1

[0049] like Figure 1 As shown, the invention implementation example 1 provides a flowchart of an infrared small target detection method based on data expansion and weak feature enhancement, and the method specifically includes the following steps:

[0050] S1, prepare the data set: prepare the IRSTD-1k data set 1 for network training, train the entire detection network; prepare the SIRST v1 data set 2 for model fine-tuning; prepare the self-made infrared small target detection data set 3 for end-to-end model testing;

[0051] S2, build an infrared small target detection model: build a detection network model consisting of target adaptive proliferation, sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding;

[0052] Target adaptive proliferation consists of threshold segmentation, grayscale cumulative distribution function, histogram matching, collision detection, random scaling and rotation operations; the threshold segmentation operation is used to segment the target and background areas with smoother transitions for targets with regular shapes and simple backgrounds, thereby obtaining a pure target database; the grayscale cumulative distribution function operation is used to describe the grayscale value distribution of the infrared background image in the self-made data set and the image in the public data set, and to achieve image histogram matching, thereby effectively expanding the background material of the data set; collision detection, random scaling and rotation operations are used to further enhance the expression ability and presentation form of small targets in the expanded data set; the sparse sampling-hybrid filtering feature extraction part consists of feature extraction block 1, feature extraction block 2, feature extraction block 3 and feature extraction block 4; each feature extraction block The structure is the same, consisting of 1 layer of hybrid filter, 1 layer of sparse sampling convolution, 1 layer of 1×1 convolution, 2 layers of 3×3 convolution and 3 layers of L-type activation layers, which are used to reduce the information loss between sub-regions during the downsampling process and retain the shallow key features of small targets; the four-in-one feature enhancement part is composed of four-in-one feature enhancement block 1, four-in-one feature enhancement block 2 and four-in-one feature enhancement block 3; each four-in-one feature enhancement block has the same structure, consisting of 1 multi-scale feature refinement module and 2 multi-scale linear feature modules, which can strengthen the interaction between deep features and shallow features through bidirectional attention mechanism and multi-scale feature encoding; the decoding part is composed of decoding block 1, decoding block 2, decoding block 3 and decoding block 4; each decoding block has the same structure, consisting of 1 layer of 1×1 convolution, 1 layer of Instance Norm and 1 layer of L-type activation layer.

[0053] S3, training the detection network model: training the detection network model, inputting the data set prepared in step S1 into the detection network model constructed in step S2 for training.

[0054] S4, select a suitable loss function and determine the optimal evaluation index of this method: select a suitable loss function to minimize the difference and loss between the output target prediction result and the input target image, set the training loss threshold, and continuously iterate and optimize the model until the number of training times reaches the set threshold or the value of the loss function reaches the set threshold range. The model parameters can be considered to have been pre-trained and the model parameters can be saved; select the test image in the data set 3 to input the network model to obtain the target prediction result, and use the optimal evaluation index of the target detection result to measure the accuracy and performance of the model; in the training process, the loss function is selected to use a joint loss function, including focal loss, binary cross entropy loss and L2 loss; the joint loss can balance the probability distribution between the target and the background, so that the detection network pays more attention to the small target area with a smaller size ratio; then, provide a stable gradient for the training process, help the model converge smoothly, and make the training process more reliable; at the same time, it can reduce noise, handle outliers, and effectively capture the segmentation information of small targets; suitable evaluation indicators are selected from normalized intersection over union (nIoU), precision, recall, detection rate (Pd) and false alarm rate (Fa).

[0055] S5, fine-tuning the model: Use dataset 2 to train and fine-tune the model, optimize the model parameters, and further improve the performance of the detection network;

[0056] S6, save the model: after the fine-tuning training in step S5 is completed, solidify the fine-tuned network parameters and determine the final detection network model; if an infrared small target detection task is performed, the infrared small target image can be directly input into the trained end-to-end network model to obtain the final prediction result.

[0057] Embodiment 2:

[0058] As shown in FIG1 , a method for detecting small infrared targets based on data expansion and weak feature enhancement includes the following steps:

[0059] S1, prepare the data set, prepare the data set 1 for detection network training, train the entire detection network, data set 1 is the IRSTD-1k data set, the data set includes small drones, bright spots, airplanes and other small-sized targets in complex backgrounds such as ocean, city, grassland, sky and clouds, with a total of 1001 infrared target images, each image size is 512×512; prepare SIRST v1 data set 2 for model fine-tuning, the data set includes multi-rotor drones, fixed-wing drones, bright spots, ships and other small-sized targets in complex backgrounds such as city, building, sky, clouds, ocean and ground, with a total of 427 infrared target images; prepare a self-made infrared small target data set, the data set includes 100 complex infrared backgrounds such as urban rail transit, parks, commercial streets, and a total of 14534 infrared small target images of small fixed-wing drones in complex backgrounds.

[0060] S2, build an infrared small target detection model, such as Figure 2 The figure shows the principle diagram of the infrared small target detection model constructed by the present invention, which includes four parts: target adaptive proliferation, sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding;

[0061] Target adaptive proliferation includes threshold segmentation, grayscale cumulative distribution function, histogram matching, collision detection, random scaling and rotation operations;

[0062] Threshold segmentation is applied to targets with regular shapes and simple backgrounds to obtain pure target pixel areas. When processing targets with irregular shapes and complex backgrounds, the processing areas with an expansion distance of 10 are first cropped out in batches, and then the manual segmentation method is used to accurately segment the complex targets.

[0063] Grayscale cumulative distribution function matches the histogram. This paper assumes that the average temperature of the shooting scene is the same, and the background of the public data set is statistically analyzed. With homemade background Grayscale cumulative distribution function. Since the pixel value ranges of different images may be different, the present invention normalizes the grayscale cumulative distribution function to eliminate scale and brightness differences. Then, through histogram matching and table lookup, For each gray level, find the closest gray level and establish a corresponding mapping relationship. Finally, the present invention adjusts the mean and variance of the homemade infrared background image to match the mean and variance of the source image (public infrared dataset) to obtain the adjusted infrared background library. (including the adjusted background and the background of the public dataset); after obtaining the target library after threshold segmentation The present invention calculates the minimum, maximum and average grayscale values ​​of the target area, and then calculates the grayscale value of the target area under the infrared background. Randomly select 5 areas of random size (less than or equal to 30×30) from each background in the image. The present invention selects background areas in the upper and lower halves of the image in a 3:2 ratio. Then, the lowest, highest and average grayscale values ​​of each area are calculated respectively, and subtracted from the target area respectively. Finally, the background area whose difference is within the threshold range is selected as the new position of the target. According to the analysis of the target and background areas of different scales, the present invention sets the threshold range to 51-52.7. This range includes part of the background area. , thereby enhancing the abstract semantic expression ability of the proliferated target;

[0064] Collision detection, random scaling and rotation. In order to further enhance the expression ability and presentation form of the small target context area, the present invention performs collision detection on the background area (obtained from step 2), and determines whether there is overlap between the background areas through the IoU indicator to obtain the final background area. The present invention sets the IoU threshold to 0.7. The present invention assumes that the background area and .in, and Indicates the coordinates of the upper left corner of areas A and B; and Represents the coordinates of the lower right corner of areas A and B. Therefore, the collision test can be expressed as follows:

[0065]

[0066] The conditions for two background regions to collide are as follows: If the above conditions are met, the background area is retained. The present invention performs random scaling (0.5 to 2 times), rotation and other operations on the target area. Finally, the processed target area is combined with the background area.

[0067] The sparse sampling-hybrid filtering feature extraction part consists of feature extraction block 1, feature extraction block 2, feature extraction block 3 and feature extraction block 4; each feature extraction block has the same structure, consisting of 1 layer of hybrid filter, 1 layer of sparse sampling convolution, 1 layer of 1×1 convolution, 2 layers of 3×3 convolution and 3 layers of L-type activation layers, which are used to reduce the information loss between sub-regions during the downsampling process and retain the shallow key features of small targets;

[0068] The hybrid filter consists of a Gaussian high-pass filter, a bilateral filter, a maximum pooling filter, and a BlurPool filter. After passing through Gaussian high-pass filter and bilateral filter respectively, the high-frequency response area is obtained and low frequency response area .in, It includes most of the target area and a few high-frequency bright spots. It includes most of the background and a few high-frequency bright spots. and The common features of the two parts are Background and bright spots in , so that a purer high-frequency target feature can be obtained .at last, After Max and BlurPool, high-frequency target features with anti-aliasing function are obtained ;

[0069] Sparse sampling convolution can achieve adaptive learning of target context features through three steps: similarity measurement, sampling point selection and feature update. Features Decompose into vector form , the window features and center features are encoded and normalized through two feature encoders. The process can be expressed by the formula:

[0070]

[0071] in, and Represent the window feature and center feature respectively ( represents the number of sampling points), and Represent the encoding matrices of window features and center features respectively, represents the normalization operation. Then, the sparse sampling convolution is directly calculated and The absolute difference between them is used to measure the correlation between the window feature and the center feature, thus obtaining the correlation matrix . This operation is expressed as follows:

[0072]

[0073] The correlation calculation method proposed in the present invention is significantly different from the matrix multiplication of self-attention. In the self-attention mechanism, features are usually represented as a set of vectors and processed by matrix multiplication. In contrast, the present invention simplifies the features into a single eigenvalue and uses subtraction operations instead of multiplication. Therefore, when performing feature processing in a local window, the dimension and complexity of the infrared image features are much lower than searching in the entire global range. Therefore, this method not only ensures that the network performance is not affected, but also achieves the purpose of improving computing and storage efficiency.

[0074] Sparse sampling mechanism, after normalizing the correlation matrix, the present invention introduces a learnable relative position encoding. The above operation is expressed as follows:

[0075]

[0076] in, The present invention selects The sampling points with the highest similarity are selected. Different from the cosine similarity used by the self-attention mechanism, the present invention uses absolute difference to quantify the correlation. At the same time, if the highest similarity is defined only by the maximum value, the upper limit of the correlation metric calculation cannot be guaranteed, resulting in the potential risk of data overflow during training or inference. Therefore, the present invention defines the minimum value of similarity as 0. The above processing is expressed as follows:

[0077]

[0078] in, is the index of each sampling point in the window.

[0079] After determining the sampling point index, the sparse sampling convolution is performed from the expanded original features The present invention is ( ) A soft attention mechanism is introduced to perform feature weighting operations to obtain a feature set The above operation process is expressed as follows:

[0080]

[0081] Feature update, in order to achieve an operation equivalent to convolution, the present invention will Reshape , and defines a set of learnable weight parameters and Perform matrix multiplication operations. At the same time, by using skip connections or splicing operations, the sampled features and the input central feature vector are spliced ​​at the channel level, which effectively alleviates problems such as gradient disappearance or gradient explosion, thereby promoting stable gradient propagation. The above strategy is expressed as follows:

[0082]

[0083] in, Represents a concatenation operation.

[0084] The four-in-one feature enhancement consists of four-in-one feature enhancement block 1, four-in-one feature enhancement block 2 and four-in-one feature enhancement block 3; each four-in-one feature enhancement block has the same structure, consisting of a multi-scale feature refinement module and two multi-scale linear feature modules, which can enhance the interaction between deep features and shallow features through a bidirectional attention mechanism and multi-scale feature encoding; the four-in-one feature enhancement module is as follows Figure 6 As shown in the figure, there are 5 branches, which are divided into 2 parts: local information and global information. In the local information, in the first branch, the shallow horizontal features are processed through the bottleneck structure, k×1 sparse convolution and Sigmoid function. In the second branch, the shallow vertical features are processed through the bottleneck structure, 1×k sparse convolution and Sigmoid function. In the third branch, the deep features are processed through the multi-scale linear feature module to obtain the component Q. In the fourth branch, the deep features are processed through the multi-scale linear feature module to obtain the components Q, K and V. In the fifth branch, the shallow features are processed through the multi-scale feature refinement module. The results of the first and second branches are multiplied with the component Q of the third branch, respectively, and the components Q, K and V are obtained. and The weights are assigned, and then the two results are added pixel by pixel. In the global information, the component Q obtained by the fourth branch is matrix transposed, and then matrix multiplied with the component K. The result is matrix multiplied with the result in the local information, and then Softmax is performed, and then matrix multiplied with the component V in the fourth path. After the result is matrix transposed, it is pixel-multiplied with the result of the fifth branch to obtain the feature enhancement result.

[0085] Multi-scale linear feature modules such as Figure 7 As shown in the figure, the module has three branches; in the first branch, the input features are respectively subjected to 1×1 depthwise separable convolution and 1×1 group convolution to obtain the output result; in the second branch, the input features are respectively subjected to 3×3 depthwise separable convolution and 1×1 group convolution to obtain the output result; in the third branch, the input features are respectively subjected to 5×5 depthwise separable convolution and 1×1 group convolution to obtain the output result; the output results of the three branches are added in dimension to obtain the final output result;

[0086] Multi-scale feature refinement module such as Figure 8 As shown in the figure, the module consists of 2 hybrid filters and 3 deconvolutions, with a total of 3 branches; the first branch inputs the image features without any processing, and the second branch first passes the input features through a hybrid filter, and the obtained features are processed by deconvolution and hybrid filters respectively, and are added to the features of the same size in the three branches in terms of dimension; in the third branch, the result of the hybrid filter in the second branch is deconvolved and then added to the unprocessed and deconvolution results in the second branch in terms of dimension;

[0087] The decoding part is as follows Fig. 9 As shown in the figure, the decoding part consists of decoding block 1, decoding block 2, decoding block 3 and decoding block 4; each decoding block has the same structure, consisting of 1 layer of 1×1 convolution, 1 layer of Instance Norm and 1 layer of L-type activation layer.

[0088] S3, training the detection network model, inputting the data set prepared in step S1 into the detection network model constructed in step S2 for training.

[0089] S4, select a suitable loss function and determine the optimal evaluation index of this method: select a suitable loss function to minimize the difference and loss between the output prediction result and the input target image, set the training loss threshold, and continuously iterate and optimize the model until the number of training times reaches the set threshold or the value of the loss function reaches the set threshold range. The model parameters can be considered to have been pre-trained and the model parameters can be saved; select the test image in the data set 3 and input it into the network model to obtain the target prediction result, and use the optimal evaluation index of the target prediction result to measure the accuracy and performance of the model;

[0090] The output and label loss functions of the network in S4 are calculated using a joint loss function, including focal loss, binary cross entropy loss and L2 loss; focal loss By reducing the weight of easy-to-classify samples, the model can focus more on difficult-to-classify samples during training. It can be expressed using the formula as follows:

[0091]

[0092] in, Predict the probability for the target or background, and To balance parameters and focus parameters;

[0093] Binary Cross Entropy Loss , the cross entropy principle is used to further process unbalanced data and provide a stable gradient for the training process, which helps the smooth convergence of the model and makes the training process more reliable. It can be expressed using the formula as follows:

[0094]

[0095] in, Indicates the probability of the prediction result (between 0 and 1). represents the target label (0 or 1);

[0096] The loss, through the square operation, gives a higher penalty to larger errors, encouraging the model to pay more attention to larger deviations. It can be expressed as follows:

[0097]

[0098] in, is the total number of samples, is the target label data, is the model prediction result;

[0099] Appropriate evaluation indicators include normalized intersection over union (nIoU), precision, recall, detection rate (Pd) and false alarm rate (Fa).

[0100] The appropriate evaluation indicators in S5 are normalized intersection over union (nIoU), precision (Precision), recall (Recall), detection rate (Pd) and false alarm rate (Fa);

[0101] Normalized intersection-over-union ratio, by adding normalization operation, makes the detection results of objects of different scales more fair, reduces the impact of scale on the evaluation results, and improves the sensitivity to partial overlap. Normalized intersection-over-union ratio can be expressed by the following formula:

[0102]

[0103] in, Represents the total number of samples in the dataset; Indicates The IoU value of samples, Indicates The number of pixels of the real mask of the sample target; Indicates The number of pixels of the segmentation result predicted by each sample; Indicates The number of true mask pixels in the sample predicted segmentation result.

[0104] Precision, among all the pixels identified, The ratio of pixels to pixels.

[0105]

[0106] in, Indicates that the positive sample is correctly identified as a positive sample; It represents a false positive sample, that is, a negative sample is mistakenly identified as a positive sample.

[0107] Recall rate is the proportion of all positive samples in the test set that are correctly identified as positive samples.

[0108]

[0109] in, Indicates that the positive sample is correctly identified as a positive sample; Represents a false negative sample, that is, a positive sample is mistakenly identified as a negative sample.

[0110] The detection rate is the number of correctly predicted target pixels. All pixels of the infrared image The detection rate can be expressed using the formula as follows:

[0111]

[0112] The false alarm rate is the number of incorrectly predicted target pixels. All pixels of the infrared image The false alarm rate can be expressed using the formula as follows:

[0113]

[0114] During network training, the learning rate is set to 0.0001, the batch size is set to 8, and the total iteration is 100,000 rounds. The Adam optimizer is used to continuously update the network parameters, and its exponential decay rate and eps value are set to (0.9, 0.999) and 1e-08 respectively. The entire training process lasts about 20 hours. In order to ensure that the four loss values ​​in the total loss function are as close to the same order of magnitude as possible, the four loss weights are set to =0.1, =0.001, =1.

[0115] S5, fine-tuning model: The model is trained and fine-tuned again using the SIRST v1 dataset, with the learning rate set to 0.005, for a total of 500 rounds, and other parameters remain unchanged, to further improve the performance of the detection network;

[0116] S6, save the model: after the training in step S4 is completed, solidify the fine-tuned network parameters, wait for S5 to fine-tune the model, and determine the final detection network model; if the infrared small target detection task is performed, the infrared small target image can be directly input into the trained end-to-end network model to obtain the final prediction result;

[0117] Among them, the implementation of convolution, activation function, addition operation, etc. are algorithms well known to those skilled in the art, and the specific processes and methods can be found in corresponding textbooks or technical literature.

[0118] The present invention constructs an end-to-end infrared small target detection method based on data expansion and weak feature enhancement, which can expand the number of infrared small target data sets, improve the quality of target data, and directly obtain small target prediction results without going through other intermediate steps, avoiding the existing manual design of data expansion, feature extraction and enhancement strategies, and simplifying the detection steps and making them more efficient. The prediction results of the prior art and the method proposed in the present invention are qualitatively compared. Fig.10 As shown in the figure; under the same conditions, the relevant indicators of target prediction results and target labels are obtained by calculating the existing methods, which further verifies the feasibility and superiority of this method.

[0119] A schematic diagram comparing the evaluation indicators of the prior art and the method proposed in the present invention is shown in FIG. Fig.11 As shown in the figure, it can be seen that the method proposed in the present invention has higher normalized intersection-over-union ratio and detection rate than the existing method, achieves a better balance between precision and recall rate, and also has a lower false alarm rate; these indicators further illustrate that the method proposed in the present invention achieves better infrared small target detection performance and achieves the expected effect.

[0120] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for detecting small infrared targets based on data expansion and weak feature enhancement, characterized in that: The method specifically comprises the following steps: S1, prepare data sets: prepare three infrared small target detection data sets, data sets 1 and 2 are used for network training and model fine-tuning, and data set 3 is used for model testing; S2, build an infrared small target detection model: the detection model includes four parts: target adaptive proliferation, sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding; In S2, the sparse sampling-hybrid filtering feature extraction part is composed of feature extraction block 1, feature extraction block 2, feature extraction block 3 and feature extraction block 4; each feature extraction block has the same structure, consisting of 1 layer of hybrid filter, 1 layer of sparse sampling convolution, 1 layer of 1×1 convolution, 2 layers of 3×3 convolution and 3 layers of L-type activation layers, which are used to reduce the information loss between sub-regions during the downsampling process and retain the shallow key features of small targets; In S2, the four-in-one feature enhancement part consists of four-in-one feature enhancement block 1, four-in-one feature enhancement block 2 and four-in-one feature enhancement block 3; each four-in-one feature enhancement block has the same structure, consisting of one multi-scale feature refinement module and two multi-scale linear feature modules, and can enhance the interaction between deep features and shallow features through a bidirectional attention mechanism and multi-scale feature encoding; S3, training network model: training infrared small target detection network model, inputting data set 1 and data set 2 prepared in step S1 into the target adaptive proliferation method constructed in step S2, so as to obtain a high-quality expanded data set; inputting the obtained expanded data set into the sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding constructed in step S2 and training; S4, select a suitable loss function and determine the optimal evaluation index of this method: select a suitable loss function to minimize the difference and loss between the output target prediction result and the input target image, set the training loss threshold, and continuously iterate and optimize the model until the number of training times reaches the set threshold or the value of the loss function reaches the set threshold range. The model parameters can be considered to have been pre-trained and the model parameters are saved; select the test image in the data set 3 and input it into the solidified model to obtain the target prediction result, and use the optimal evaluation index of the target detection result to measure the accuracy and performance of the model; S5, fine-tuning the model, using the infrared small target detection dataset 2 to train and fine-tune the model, optimize the model parameters, further improve the performance of the detection network, and obtain target prediction results with clearer edges and more accurate positions; S6, save the model, and after the fine-tuning training in S5 is completed, solidify the fine-tuned network parameters and determine the final small target detection model.

2. The infrared small target detection method based on data expansion and weak feature enhancement according to claim 1 is characterized in that: In S1, dataset 1 is the IRSTD-1k dataset; dataset 2 is the SIRST v1 dataset; and dataset 3 is a self-made infrared small target detection dataset.

3. The infrared small target detection method based on data expansion and weak feature enhancement according to claim 1 is characterized in that: In S2, the infrared small target detection model includes four parts: target adaptive proliferation, sparse sampling-hybrid filtering feature extraction, four-in-one feature enhancement and decoding; The object adaptive multiplication part can effectively increase the number and types of objects in each image in the data set; The sparse sampling-hybrid filtering feature extraction part can flexibly capture the complex shape and edge information of small targets and reduce the loss and displacement of scarce features; The four-in-one feature enhancement part can use vertical and horizontal bidirectional attention and multi-scale feature encoding to establish a stable feature interaction channel between the encoding stage and the decoding stage, and further enhance the key features of small targets; The decoding part can gradually and effectively restore the edge features and spatial information of small objects.

4. The infrared small target detection method based on data expansion and weak feature enhancement according to claim 1 is characterized in that: In said S2, the target adaptive proliferation method is composed of threshold segmentation, grayscale cumulative distribution function, histogram matching, collision detection, random scaling and rotation operations; The threshold segmentation operation is used to segment targets with regular shapes and simple backgrounds into target and background areas with smoother transitions, thereby obtaining a pure target database; The grayscale cumulative distribution function operation is used to describe the grayscale value distribution of the infrared background image in the self-made data set and the image in the public data set, so as to achieve histogram matching of the image, thereby effectively expanding the background material of the data set; The collision detection, random scaling and rotation operations are used to further enhance the expression ability and presentation form of small objects in the expanded data set.

5. The infrared small target detection method based on data expansion and weak feature enhancement according to claim 1 is characterized in that: In S2, the decoding part consists of decoding block 1, decoding block 2, decoding block 3 and decoding block 4; each decoding block has the same structure, consisting of 1 layer of 1×1 convolution, 1 layer of Instance Norm and 1 layer of L-type activation layer.

6. The infrared small target detection method based on data expansion and weak feature enhancement according to claim 1 is characterized in that: In S4, the loss function is a joint loss, including focal loss, binary cross entropy loss and L2 loss; The focus loss is used to balance the probability distribution between the target and the background, so that the detection network pays more attention to the small target area with a smaller size ratio; The binary cross entropy loss is used to provide a stable gradient for the training process, helping the model to converge smoothly and making the training process more reliable; The L2 loss is used to reduce noise, handle outliers, and effectively capture the segmentation information of small targets.

7. The infrared small target detection method based on data expansion and weak feature enhancement according to claim 1 is characterized in that: In S4, the process of training the network model also includes evaluating the performance of the algorithm prediction results through evaluation indicators.