Infrared weak and small target detection method and system, electronic equipment and storage medium

Abstract

The invention provides an infrared weak and small target detection method and system, electronic equipment and a storage medium, and belongs to the technical field of target detection, and the method comprises the steps: improving a YOLOv8 model for the features of an infrared weak and small target image; and performing target detection on the infrared weak and small target image by using the improved YOLOv8 model. According to the invention, a stronger infrared weak and small target detection capability is provided for the YOLOv8 model, and the problem of target detection precision bottleneck of a traditional model in a complex scene is effectively solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and in particular relates to a method, system, electronic equipment and storage medium for detecting infrared weak and small targets. Background Art

[0002] With the rapid development of computer vision, target detection technology has made continuous breakthroughs. The YOLO series of models has become the mainstream choice in actual detection tasks due to its high efficiency and real-time performance. However, with the increasing diversification and complexity of application scenarios, the traditional convolutional neural network architecture is unable to cope with problems such as complex backgrounds, noise interference, and infrared weak target detection. Therefore, further improving the performance of YOLOv8 in infrared weak target detection has important application value and research significance.

[0003] Although the YOLO series of models have achieved remarkable results in the field of target detection, especially in terms of real-time performance and detection accuracy, they still have shortcomings when dealing with complex backgrounds and weak target detection. Due to the limitation of local receptive field, traditional convolutional neural networks are difficult to effectively capture long-distance or global feature dependencies, which makes it difficult for the model to fully utilize global information for accurate recognition in infrared weak target detection tasks. Weak targets in infrared images usually have weak signals and are easily masked by noise in complex backgrounds. The detection accuracy of traditional convolution operations in such scenarios is often not high, and missed detections and false detections are prone to occur. In addition, the basic feature extraction network of YOLOv8 has limitations in its expression ability when generating high-dimensional nonlinear features. Although increasing the feature width or depth can partially improve the feature expression of the model, when faced with complex backgrounds or high-noise environments, traditional convolutions are difficult to generate detailed target features, especially when dealing with infrared targets with rich details or weak signals, the model performance is insufficient. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides an infrared dim small target detection method, system, electronic device and storage medium. The present invention provides a stronger infrared dim small target detection capability for the YOLOv8 model, effectively solving the target detection accuracy bottleneck problem of the traditional model in complex scenes.

[0005] In order to achieve the above objectives, in a first aspect, the present invention provides an infrared small target detection method, comprising the following steps:

[0006] S1. Improve the YOLOv8 model based on the characteristics of infrared weak target images;

[0007] S2. Use the improved YOLOv8 model to detect infrared small target images.

[0008] Furthermore, the step S1 includes the following steps:

[0009] S101. In view of the characteristics of infrared weak target images, a non-local attention mechanism is introduced into the backbone network of the YOLOv8 model;

[0010] S102. In the feature extraction module of the YOLOv8 model, introduce star convolution and design a loss function to improve the YOLOv8 model.

[0011] Furthermore, the improved YOLOv8 model includes:

[0012] The non-local attention mechanism module is used to obtain the relationship between long-distance features by establishing global dependencies at all positions of the infrared dim target feature map and calculating the similarity weights between different positions of the infrared dim target feature map;

[0013] The star convolution module is used to map the infrared dim target feature map processed by the backbone feature extraction module into a high-dimensional nonlinear feature space;

[0014] The target detection module is used to obtain the detected infrared weak target according to the mapping result.

[0015] Furthermore, the relationship between long-distance features is obtained by establishing a global dependency relationship at all positions of the infrared dim small target feature map and calculating the similarity weights between different positions of the infrared dim small target feature map, which is specifically:

[0016] Input infrared weak target feature map;

[0017] The input infrared weak target feature map is linearly mapped to obtain the feature θ, feature Feature g;

[0018] On the feature θ branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a first infrared dim small target feature map after dimensionality reduction;

[0019] In Features On the branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a second infrared dim small target feature map after dimensionality reduction;

[0020] On the feature g branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a third infrared dim small target feature map after dimensionality reduction, wherein the sizes of the first infrared dim small target feature map, the second infrared dim small target feature map and the third infrared dim small target feature map after dimensionality reduction are all consistent;

[0021] The feature θ branch and the feature The output results of the branches are matrix multiplied to obtain the similarity relationship between all positions of the infrared weak target feature map;

[0022] Normalize the similarity relationship and generate the attention weight matrix;

[0023] Multiply the attention weight matrix with the output of the feature g branch to obtain a weighted global feature representation to construct global dependencies;

[0024] The weighted global feature representation is restored to the original number of channels through a convolution operation, and residual processing is performed with the input infrared dim target feature map to obtain the infrared dim target feature map after residual processing, wherein the infrared dim target feature map after residual processing includes the relationship between long-distance features.

[0025] Furthermore, the infrared dim small target feature map processed by the backbone feature extraction module is mapped into a high-dimensional nonlinear feature space, which is specifically:

[0026] Use a convolutional layer as the input layer and construct star blocks, where each star block contains two linear transformations and one star operation;

[0027] According to the infrared small target feature map processed by the backbone feature extraction module, linear processing is performed by the weight matrix W1 and the weight matrix W2 respectively, and the infrared small target feature map W1X and the infrared small target feature map W2X after linear processing are obtained one by one;

[0028] The star operation is realized by applying activation function to the infrared small target feature map W1X and performing element-wise multiplication with the infrared small target feature map W2X;

[0029] The result of the star operation is fused with the infrared dim small target feature map processed by the backbone feature extraction module to obtain a fused infrared dim small target feature map;

[0030] The fused infrared dim target feature map is subjected to feature extraction, wherein multiple star blocks are repeatedly used in each stage to extract high-level features, and at the end of each stage, depthwise separable convolution is used for downsampling.

[0031] In the last stage, global average pooling is used to convert the extracted feature map into a one-dimensional feature vector;

[0032] The fully connected layer is used to map the one-dimensional feature vector to the number of categories, completing the mapping process of the high-dimensional nonlinear feature space.

[0033] Furthermore, the loss function of the improved YOLOv8 model is expressed as follows:

[0034]

[0035] Among them, MPDIoU represents the loss function, IoU represents the ratio of the intersection and union of the predicted box and the real box, h represents the height of the box, w represents the width of the box, d1 represents the straight-line distance between the upper left corner coordinates of the predicted box and the real box, d2 represents the straight-line distance between the lower right corner coordinates of the predicted box and the real box, B represents the bounding box, B gt represents the ground-truth bounding box, Indicates the horizontal coordinate of the upper left corner of the prediction box, Indicates the horizontal coordinate of the upper left corner of the real box, Indicates the ordinate of the upper left corner of the prediction box, Indicates the ordinate of the upper left corner of the real box, Indicates the horizontal coordinate of the lower right corner of the prediction box, Indicates the horizontal coordinate of the lower right corner of the real box, Indicates the ordinate of the lower right corner of the prediction box, It represents the ordinate of the lower right corner of the real box, prd represents the predicted value, and gt represents the real value.

[0036] Furthermore, step S2 includes the following steps:

[0037] S201, obtaining an infrared data set, and dividing the infrared data set, wherein the training set is used to train the improved YOLOv8 model, the test set is used to evaluate the generalization ability of the trained YOLOv8 on unknown data, and the validation set is used to evaluate the YOLOv8 during the training process and adjust the hyperparameters;

[0038] S202, using the divided data set to train the YOLOv8 model, and selecting the optimal weight parameters in the training process;

[0039] S203, using the YOLOv8 model with optimal weight parameters to detect small target images in the infrared data set.

[0040] In a second aspect, the present invention provides an infrared small target detection system for executing an infrared small target detection method, comprising:

[0041] The first processing module is used to improve the YOLOv8 model based on the characteristics of infrared weak target images;

[0042] The second processing module is used to perform target detection on infrared weak target images using the improved YOLOv8 model.

[0043] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of any of the infrared weak small target detection methods.

[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the steps of any one of the infrared dim small target detection methods are implemented.

[0045] The beneficial effects of the present invention are:

[0046] The present invention proposes a target detection method combining a non-local attention mechanism and star convolution. The non-local attention mechanism breaks through the limitation of the local receptive field, effectively locates small targets by capturing global context information, and maintains high detection accuracy even in complex scenes. Star convolution enhances the feature expression ability at a low computational cost through the nonlinear combination of feature channels, and improves the YOLOv8 model's representation effect on details and complex backgrounds. This improvement provides the YOLOv8 model with a stronger infrared dim target detection capability, and effectively solves the accuracy bottleneck of traditional models in complex scenes. The present invention enhances the feature extraction capability and adaptability of the YOLOv8 model through a non-local attention mechanism and star convolution.

[0047] The present invention aims to overcome the limitation that traditional convolution operations can only capture local features by introducing a non-local attention mechanism. When extracting features layer by layer, convolutional neural networks often rely on local convolution kernels, so it is difficult to capture global features with long-distance dependencies. The non-local attention mechanism captures rich global context information by calculating the global relationship between any positions in the feature map, so that the YOLOv8 model can focus on the target area more accurately. Specifically, this mechanism has significant advantages in weak target detection. When the target volume is small, the signal is weak and easily disturbed by background noise, the non-local attention mechanism can enhance the use of contextual information, making the model more accurate in locating weak targets and reducing the probability of false detection and missed detection. This is particularly important in infrared image detection, because infrared targets usually appear as low-contrast, low-resolution weak signals that are easily masked by complex backgrounds.

[0048] The star convolution module in the present invention is a novel feature combination method. The module performs nonlinear mapping on the input data through star operation (element-wise multiplication) and converts it into a higher-dimensional feature space, thereby generating a richer feature expression. Unlike traditional convolution operations, star convolution does not simply increase the channel width or depth of the network, but achieves feature enhancement at a lower computational cost through nonlinear combination of feature channels. Especially when processing infrared images with rich details or complex scenes, star convolution can significantly improve the effect of feature extraction within limited computing resources. Compared with directly increasing the number of channels, star convolution can also effectively separate the target and background under complex backgrounds, further improving the detection accuracy of the model for weak targets.

[0049] By introducing non-local attention mechanism and star convolution into the backbone network of YOLOv8, the present invention effectively improves the feature expression ability of the model, making it more robust and generalizable in small target detection. These two improvements not only improve the detection accuracy, but also make the YOLOv8 model perform better when dealing with complex scenes and multi-scale targets. Overall, the introduction of non-local attention mechanism and star convolution module provides a new technical route for the improvement of YOLOv8, enabling the model to complete infrared small target detection tasks more efficiently and accurately. These improvements enhance the practicality and reliability of YOLOv8 in small target detection, and provide solid technical support for practical applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 The figure is a flow chart of the method of the present invention.

[0051] Figure 2 This is a diagram of the improved YOLOv8 network structure in the present invention.

[0052] Figure 3 This is a structural diagram of the non-local attention mechanism module in the present invention.

[0053] Figure 4 This is a structural diagram of the star convolution module in the present invention.

[0054] Figure 5 This is the result diagram of the original data set annotation in the present invention.

[0055] Figure 6 This is the original YOLOv8 detection result diagram in the present invention.

[0056] Figure 7 This is the improved YOLOv8 detection result diagram in the present invention.

[0057] Figure 8 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0058] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0059] Example 1

[0060] The purpose of the present invention is to improve the detection accuracy and robustness of the YOLOv8 model in complex scenes for the difficulties in infrared weak target detection. Due to the limitations of traditional convolutional neural networks in capturing long-distance or global feature dependencies, especially in infrared images, weak target signals are weak and easily disturbed by background noise, which often leads to missed detection or false detection. In order to solve these problems, the present invention introduces a non-local attention mechanism and a star convolution module to enhance the model's global information capture ability and complex feature representation ability. The non-local attention mechanism can break through the limitations of the local receptive field, capture the global context information of the input image, and enable the model to effectively identify small targets under a noisy background. The star convolution module maps the input data to a higher-dimensional nonlinear feature space by nonlinearly combining feature channels to generate a more expressive feature expression. Compared with the method of directly increasing the number of convolution channels, the star convolution enhances the model's ability to extract complex background and detail information while maintaining computational efficiency. By introducing these improved modules, the present invention aims to improve the performance of YOLOv8 in infrared weak target detection tasks, especially to improve the recognition accuracy and generalization ability of the model in different scales and complex background environments.

[0061] like Figure 1 As shown, the present invention provides an infrared small target detection method, and its implementation method is as follows:

[0062] S1. According to the characteristics of infrared weak target images, the YOLOv8 model is improved. The implementation method is as follows:

[0063] S101. In view of the characteristics of infrared weak target images, a non-local attention mechanism is introduced into the backbone network of the YOLOv8 model;

[0064] S102. In the feature extraction module of the YOLOv8 model, introduce star convolution and design a loss function to improve the YOLOv8 model.

[0065] The improved YOLOv8 model includes:

[0066] The non-local attention mechanism module is used to obtain the relationship between long-distance features by establishing global dependencies at all positions of the infrared dim target feature map and calculating the similarity weights between different positions of the infrared dim target feature map. Specifically, it is as follows:

[0067] Input infrared weak target feature map;

[0068] The input infrared weak target feature map is linearly mapped to obtain the feature θ, feature Feature g;

[0069] On the feature θ branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a first infrared dim small target feature map after dimensionality reduction;

[0070] In Features On the branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a second infrared dim small target feature map after dimensionality reduction;

[0071] On the feature g branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a third infrared dim small target feature map after dimensionality reduction, wherein the sizes of the first infrared dim small target feature map, the second infrared dim small target feature map and the third infrared dim small target feature map after dimensionality reduction are all consistent;

[0072] The feature θ branch and the feature The output results of the branches are matrix multiplied to obtain the similarity relationship between all positions of the infrared weak target feature map;

[0073] Normalize the similarity relationship and generate the attention weight matrix;

[0074] Multiply the attention weight matrix with the output of the feature g branch to obtain a weighted global feature representation to construct global dependencies;

[0075] The weighted global feature representation is restored to the original number of channels through a convolution operation, and residual processing is performed with the input infrared dim target feature map to obtain the infrared dim target feature map after residual processing, wherein the infrared dim target feature map after residual processing includes the relationship between long-distance features.

[0076] like Figure 3 As shown, the input feature map X of size T×H×W×1024 is linearly mapped to obtain θ, And g features, for the feature θ branch, apply 1x1 convolution to the input feature map X, reduce the number of channels to 512, and the output size is T×H×W×512, which reduces the dimension of the input features, keeps the spatial dimension unchanged, and reduces the amount of calculation. The feature g branch also applies a 1x1 convolution to the input feature map X. The output size is T×H×W×512, which is consistent with the feature θ branch and is used to calculate the similarity in the spatial dimension. For the feature g branch, a 1x1 convolution is also applied to the input feature map X. The output size is T×H×W×512, which is consistent with the first two branches. The input feature representation with reduced dimension is generated and used for feature aggregation after weighting in the attention mechanism. This branch carries the information of the input features and will be multiplied by the weight matrix to generate the output. The branches perform matrix multiplication to calculate the similarity relationship between all positions. Then it is normalized by softmax to generate an attention weight matrix, and the output is THW×THW, which represents the similarity between each position in the input feature map. The attention weight matrix (THW×THW) is then multiplied with the output feature of the feature g branch (T×H×W×512) to obtain a weighted global feature representation (T×H×W×512). Finally, the weighted features are restored to the original number of channels 1024 through 1x1 convolution, and the residual structure is performed with the original input feature map X, and the feature map of size T×H×W×1024 is output. The residual structure retains the original feature information and superimposes the globally weighted features, which can retain local details and combine global information. Among them, T represents the number of frames, H represents the height, and W represents the width.

[0077] In this embodiment, the present invention linearly transforms the input infrared dim target feature map through a weight matrix. This step provides a basic feature representation for capturing global dependencies. The output results of the two branches are matrix-multiplied, which is actually calculating the similarity or relationship between any two positions of the feature map. This step is the key to building global dependencies. The attention weight matrix generated by normalizing the similarity relationship reflects the importance of features at different positions to the current position. This attention weight matrix reflects the global dependencies. The attention weight matrix is ​​multiplied by the output results of the feature branch to obtain a weighted global feature representation. This step uses attention weights to strengthen or suppress features from different positions, thereby building global dependencies because it takes into account the contextual information of the entire feature map.

[0078] In this embodiment, the present invention uses a normalization method to process similarity relationships, and the generated attention weight matrix reflects the contribution of features at different positions to the current position. This step is particularly important because it captures the relationship between long-distance features by assigning appropriate weights to features at farther positions; the attention weight matrix is ​​multiplied by the output result of the feature branch to obtain a weighted global feature representation. This step strengthens the relationship between long-distance features by considering the information of all positions in the entire feature map. Enhanced feature representation in the present invention: The infrared weak target feature map after residual processing contains the relationship between long-distance features. Through the non-local attention mechanism, the model can capture the dependencies between these long-distance features, thereby enhancing the representation of these relationships in the feature map; information integration: residual processing allows the model to integrate the global information brought by non-local operations while maintaining the original feature information.

[0079] The star convolution module is used to map the infrared weak target feature map processed by the backbone feature extraction module to a high-dimensional nonlinear feature space. Specifically:

[0080] Use a convolutional layer as the input layer and construct star blocks, where each star block contains two linear transformations and one star operation;

[0081] According to the infrared small target feature map processed by the backbone feature extraction module, linear processing is performed by the weight matrix W1 and the weight matrix W2 respectively, and the infrared small target feature map W1X and the infrared small target feature map W2X after linear processing are obtained one by one;

[0082] The star operation is realized by applying activation function to the infrared small target feature map W1X and performing element-wise multiplication with the infrared small target feature map W2X;

[0083] The result of the star operation is fused with the infrared dim small target feature map processed by the backbone feature extraction module to obtain a fused infrared dim small target feature map;

[0084] The fused infrared dim target feature map is subjected to feature extraction, wherein multiple star blocks are repeatedly used in each stage to extract high-level features, and at the end of each stage, depthwise separable convolution is used for downsampling.

[0085] In the last stage, global average pooling is used to convert the extracted feature map into a one-dimensional feature vector;

[0086] The fully connected layer is used to map the one-dimensional feature vector to the number of categories, completing the mapping process of the high-dimensional nonlinear feature space.

[0087] In this embodiment, Figure 4As shown in the figure, the network is initialized first, and a convolution layer is used as the input layer to reduce the resolution of the input image and increase the number of channels. Then, a star block is constructed, each of which contains two linear transformations and a star operation. For the input feature X, two weight matrices W1 and W2 are used to perform linear changes, respectively, to obtain the infrared weak target feature map W1X and the infrared weak target feature map W2X after linear processing. Then, the activation function is applied to the weight matrix W1 to obtain act(W1X), and the activated feature act(W1X) is element-wise multiplied with W2X to obtain act(W1X)*W2X to complete the star operation, and the result of the star operation is added to the original input feature X to complete the feature fusion. Then, feature extraction is performed, and multiple star blocks are repeatedly used in each stage to extract higher-level features. At the end of each stage, a depth-separable convolution is used for downsampling to double the number of channels. In the last stage, global average pooling is applied to convert the feature map into a one-dimensional feature vector, and a fully connected layer is used to map the feature vector to the final number of categories to complete the classification task. Among them, W represents the weight matrix, and act represents the activation function.

[0088] The target detection module is used to obtain the detected infrared weak target according to the mapping result.

[0089] In this embodiment, the YOLOv8 basic training model is improved. Figure 2 As shown in the figure, the improved target detection network includes a backbone feature extraction module (Backbone), a feature fusion module (Neck), and a target prediction module (Head) connected from left to right.

[0090] In this embodiment, a non-local attention mechanism is introduced into the backbone network of the YOLOv8 model to enhance the global feature extraction capability of the YOLOv8 model. The idea is to improve the YOLOv8 model's understanding of global contextual information by capturing the global dependencies in the feature graph. Unlike traditional convolutional neural networks, the non-local attention mechanism can calculate the similarity between each position in the feature graph and all other positions, and is not limited to local features. It generates query, key, and value features, calculates the similarity matrix, and then weightedly combines the global features, and finally maintains the combination of local and global feature information through residual connections. When dealing with scenes with complex backgrounds, the efficiency of capturing weak targets is improved. Non-local attention mechanisms such as Figure 3 shown.

[0091] In this embodiment, in the feature extraction module of YOLOv8, star convolution StarNet is introduced to perform nonlinear combination between feature channels through star operation. Star operation is similar to kernel function, especially polynomial kernel function, which can perform pairwise multiplication between different feature channels to generate high-dimensional nonlinear features. Unlike traditional convolution operations, star operations can map input data to a higher-dimensional nonlinear feature space to generate richer feature expressions. Compared with the method of directly increasing the number of channels, star convolution improves the expressive ability of the model while maintaining a low computational overhead, especially when detecting complex scenes and small targets. Star convolution module such as Figure 4 shown.

[0092] In this embodiment, a loss function MPDIoU (minimum point distance intersection-over-union ratio) is added on the basis of the above improvements. It is a new bounding box similarity comparison metric based on the minimum point distance, which directly minimizes the upper left and lower right point distances between the predicted bounding box and the actual annotated bounding box, and includes all relevant factors considered in the existing loss function. Its design simplifies the similarity comparison between two bounding boxes and is suitable for overlapping or non-overlapping bounding box regression. Therefore, in computer vision tasks, MPDIoU can well replace the intersection-over-union ratio as a measure of all performance indicators, improve the training effect of bounding box regression, and improve the convergence speed and regression accuracy.

[0093] In this embodiment, the loss function based on MPDIoU is used to optimize the bounding box regression problem. Compared with some traditional IoU (intersection-over-union) loss functions, such as GIoU (generalized intersection-over-union) takes into account the inclusion relationship and spatial distribution between bounding boxes, DIoU (distance intersection-over-union) takes into account the distance between the center points of the bounding boxes to improve alignment and scale inconsistency, CIoU (complete intersection-over-union) combines the characteristics of DIoU and adds aspect ratio considerations to further improve the accuracy of the bounding box, and EIoU (efficient intersection-over-union) takes into account the expected similarity between the predicted bounding box and the true bounding box. MPDIoU is developed on the basis of these existing metrics and aims to provide an easy-to-implement solution by directly minimizing the key point distance between the predicted box and the true box. MPDIoU measures the similarity between the predicted box and the true box by introducing the minimum point distance (Minimum PointsDistance). It not only takes into account the overlapping area of ​​the bounding box, the center point distance, and the deviation of the width and height, but also simplifies the calculation process. MPDIoU can more accurately optimize bounding boxes with different aspect ratios, improve the convergence speed and enhance the positioning accuracy of the model, especially when dealing with detection tasks in complex scenes. Its expression is as follows:

[0094]

[0095] Among them, MPDIoU represents the loss function, IoU represents the ratio of the intersection and union of the predicted box and the real box, h represents the height of the box, w represents the width of the box, d1 represents the straight-line distance between the upper left corner coordinates of the predicted box and the real box, d2 represents the straight-line distance between the lower right corner coordinates of the predicted box and the real box, B represents the bounding box, B gt represents the ground-truth bounding box, Indicates the horizontal coordinate of the upper left corner of the prediction box, Indicates the horizontal coordinate of the upper left corner of the real box, Indicates the ordinate of the upper left corner of the prediction box, Indicates the ordinate of the upper left corner of the real box, Indicates the horizontal coordinate of the lower right corner of the prediction box, Indicates the horizontal coordinate of the lower right corner of the real box, Indicates the ordinate of the lower right corner of the prediction box, It represents the ordinate of the lower right corner of the real box, prd represents the predicted value, and gt represents the real value.

[0096] S2. Use the improved YOLOv8 model to detect infrared weak target images. The method is as follows:

[0097] S201, obtaining an infrared data set, and dividing the infrared data set, wherein the training set is used to train the improved YOLOv8 model, the test set is used to evaluate the generalization ability of the trained YOLOv8 on unknown data, and the validation set is used to evaluate the YOLOv8 during the training process and adjust the hyperparameters;

[0098] S202, using the divided data set to train the YOLOv8 model, and selecting the optimal weight parameters in the training process;

[0099] S203, using the YOLOv8 model with optimal weight parameters to detect small target images in the infrared data set.

[0100] In this embodiment, the detection of weak target images is further described below by taking an infrared ship as an example.

[0101] The original infrared ship dataset released by infiray is input into the most basic YOLOv8 model for training. The best weight file after training is used as the baseline network. The best result obtained by training the improved YOLOv8 model multiple times is used as the optimal YOLOv8 model, and the performance of the improved YOLOv8 model is evaluated using the relevant infrared ship test set. The evaluation indicators include accuracy (P), mean Average Precision (mAP), model volume, parameter quantity, etc. The specific steps are as follows:

[0102] In this embodiment, Figure 5 As shown, the data set used in the present invention is an infrared marine ship data set disclosed by infiray (Ruichuang Micro-Nano Infrared Open Source Platform), and 9400 infrared data were collected at sea, ports and seaside in different scenes, different time periods and different resolutions, including cruise ships, bulk carriers, warships, sailboats, kayaks, container ships and fishing boats. The data set is randomly divided into a training set, a validation set and a test set according to a ratio of 7:1:2. The training set is used for the training of the initial target network, and the validation set is used to evaluate the performance of the YOLOv8 model and adjust the hyperparameters during the training process. The test set is used to finally evaluate the generalization ability of the YOLOv8 model on unseen data, and the best weight file obtained after training multiple times is used as the baseline network, and then the test set is used to evaluate the basic YOLOv8 model and the improved YOLOv8 model. Finally, the experimental results of the basic YOLOv8 model and the improved YOLOv8 model are compared to complete the detection of infrared weak target images.

[0103] In this embodiment, the system environment used for training is Windows 10, the graphics card model is NVIDIA 4070SUPER (12GB), and the training and testing environment is built based on CUDA 12.6 and Pytorch 2.4.0 framework. Further, the required parameters are set, and the training parameters include epoch (number of iterations), batch size (number of images in each batch), lr0 (initial learning rate), lrf (final learning rate) momentum (momentum parameter), etc. Among them, epoch is 300, batch size is 64, lr0 is 0.01, lrf is 0.01, momentum is 0.937, iou threshold is 0.75, and conf threshold is 0.5. The network model parameters are iteratively updated using the SGD optimizer.

[0104] After the YOLOv8 model training is completed, the best weight parameter (best.pt) is selected during the training process, and the divided test set is used to identify and evaluate the model performance of seven types of targets in the infrared ship image data, including cruise ships, bulk carriers, warships, sailboats, kayaks, container ships, and fishing boats. The performance of the YOLOv8 model is evaluated mainly from two aspects: detection accuracy and model complexity.

[0105] The evaluation indicators of detection accuracy include precision, recall, mean average precision (mAP) and inference speed. The model complexity includes model size and number of parameters.

[0106] When defining indicators, TP represents the number of positive classes predicted as positive classes, FP represents the number of negative classes predicted as positive classes, and FN represents the number of positive classes predicted as negative classes. Based on this, the calculation formulas for precision, recall, and mean average precision (mAP) are as follows:

[0107]

[0108] Among them, AP stands for Average Precision, P(R) stands for Precision under different recall rates R, dR stands for a small change in recall rate R, n stands for the number of categories, and AP i represents the average precision of the i-th category.

[0109] like Figure 6 and Figure 7 As shown, based on the same infrared ship image data set, the original YOLOv8 model (scale selection YOLOv8n) and the method of the present invention (YOLOv8 improvement) are trained, and the target objects are set as liner, bulk carrier, warship, sailboat, canoe, container ship, fishing boat, and the trained target detection network is evaluated as above, so as to obtain the results shown in Table 1, which is a comparative experimental result table.

[0110] Table 1

[0111]

[0112] As shown in Table 1 above, the method of the present invention is superior to the original YOLOv8 model and the YOLOv8+Non_Local model with only the non-local attention mechanism added in all indicators. Specifically, the precision (Precision) of YOLOv8+Non_Local+StarNet is improved to 0.903, and the recall rate (Recall) reaches 0.821, which is significantly improved compared to the precision of 0.887 and the recall rate of 0.802 of the basic YOLOv8 model. In addition, mAP50 is improved from 0.861 of YOLOv8 to 0.892, and mAP50-95 is improved from 0.624 to 0.635. This shows that the non-local attention mechanism and star convolution module of the present invention can effectively enhance the detection accuracy and feature extraction ability of the model in infrared dim target detection, especially the target detection performance in complex scenes is relatively outstanding, and it has certain practical value.

[0113] Example 2

[0114] like Figure 8 As shown, the present invention provides an infrared small target detection system for executing an infrared small target detection method, comprising:

[0115] The first processing module is used to improve the YOLOv8 model based on the characteristics of infrared weak target images;

[0116] The second processing module is used to perform target detection on infrared weak target images using the improved YOLOv8 model.

[0117] like Figure 8 The infrared dim small target detection system provided in the illustrated embodiment can execute the technical solution shown in the infrared dim small target detection method in the above method embodiment, and its implementation principle and beneficial effects are similar and will not be repeated here.

[0118] In this embodiment, the present application can divide the functional units according to the infrared weak small target detection method. For example, each function can be divided into each functional unit, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the present invention is schematic and is only a logical division. There may be other division methods in actual implementation.

[0119] In this embodiment, in order to realize the principles and beneficial effects of the infrared dim small target detection method, the infrared dim small target detection system includes hardware structures and / or software modules corresponding to the execution of various functions. Those skilled in the art should easily realize that, in combination with the various schematic units and algorithm steps described in the embodiments disclosed in the present invention, the present invention can be implemented in the form of hardware and / or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven manner depends on the specific application and design constraints of the technical solution. Different methods can be used for each specific application to implement the described function, but such implementation should not be considered to exceed the scope of this application.

[0120] Example 3

[0121] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the infrared weak small target detection method described in any one of Example 1.

[0122] In this embodiment, the electronic device may include: a processor, a memory, a bus and a communication interface. The processor, the communication interface and the memory are connected via a bus. The memory stores a computer program that can be run on the processor. When the processor runs the computer program, it executes part or all of the steps of the infrared weak and small target detection method provided in the aforementioned embodiment 1 of the present application.

[0123] Example 4

[0124] In a fourth aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the steps of the infrared weak small target detection method described in Example 1 are implemented.

[0125] The computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk, and the readable storage medium can be any available medium that can be accessed by a general or special-purpose computer. The readable storage medium is coupled to the processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. The readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application-specific integrated circuit (ASIC). The processor and the readable storage medium can also exist as discrete components in the point de-registration system.

[0126] The embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. With reference to the process and / or block diagram of the method, device (device), and computer program product according to the embodiments of the present invention, it should be understood that each process and / or block diagram in the flow chart and / or block diagram, as well as the combination of the flow chart and / or block diagram, can be implemented by computer program instructions, and these computer program instructions can be provided to a computer-readable memory of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device that works in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the flow chart. Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing one or more processes and / or blocks in the flowchart. Figure 1 The steps for the functions specified in one or more boxes.

[0127] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the present invention.

Claims

1. A method for detecting infrared small targets, characterized in that: The following steps are involved: S1. Improve the YOLOv8 model based on the characteristics of infrared weak target images; S2. Use the improved YOLOv8 model to detect infrared small target images.

2. The infrared small target detection method according to claim 1, characterized in that: The step S1 comprises the following steps: S101. In view of the characteristics of infrared weak target images, a non-local attention mechanism is introduced into the backbone network of the YOLOv8 model; S102. In the feature extraction module of the YOLOv8 model, introduce star convolution and design a loss function to improve the YOLOv8 model.

3. The infrared small target detection method according to claim 2, characterized in that: The improved YOLOv8 model includes: The non-local attention mechanism module is used to obtain the relationship between long-distance features by establishing global dependencies at all positions of the infrared dim target feature map and calculating the similarity weights between different positions of the infrared dim target feature map; The star convolution module is used to map the infrared dim target feature map processed by the backbone feature extraction module into a high-dimensional nonlinear feature space; The target detection module is used to obtain the detected infrared weak target according to the mapping result.

4. The infrared small target detection method according to claim 3, characterized in that: The relationship between long-distance features is obtained by establishing a global dependency relationship at all positions of the infrared small target feature map and calculating the similarity weights between different positions of the infrared small target feature map, which is specifically: Input infrared weak target feature map; The input infrared weak target feature map is linearly mapped to obtain the feature θ, feature Feature g; On the feature θ branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a first infrared dim small target feature map after dimensionality reduction; In Features On the branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a second infrared dim small target feature map after dimensionality reduction; On the feature g branch, a convolution operation is performed on the input infrared dim small target feature map to obtain a third infrared dim small target feature map after dimensionality reduction, wherein the sizes of the first infrared dim small target feature map, the second infrared dim small target feature map and the third infrared dim small target feature map after dimensionality reduction are all consistent; The feature θ branch and the feature The output results of the branches are matrix multiplied to obtain the similarity relationship between all positions of the infrared weak target feature map; Normalize the similarity relationship and generate the attention weight matrix; Multiply the attention weight matrix with the output of the feature g branch to obtain a weighted global feature representation to construct global dependencies; The weighted global feature representation is restored to the original number of channels through a convolution operation, and residual processing is performed with the input infrared dim target feature map to obtain the infrared dim target feature map after residual processing, wherein the infrared dim target feature map after residual processing includes the relationship between long-distance features.

5. The infrared small target detection method according to claim 3, characterized in that: The infrared dim small target feature map processed by the backbone feature extraction module is mapped to a high-dimensional nonlinear feature space, which is specifically: Use a convolutional layer as the input layer and construct star blocks, where each star block contains two linear transformations and one star operation; According to the infrared small target feature map processed by the backbone feature extraction module, linear processing is performed by the weight matrix W1 and the weight matrix W2 respectively, and the infrared small target feature map W1X and the infrared small target feature map W2X after linear processing are obtained one by one; The star operation is realized by performing element-wise multiplication processing on the infrared dim small target feature map W1X after applying the activation function and the infrared dim small target feature map W2X; The result of the star operation is fused with the infrared dim small target feature map processed by the backbone feature extraction module to obtain a fused infrared dim small target feature map; The fused infrared dim target feature map is subjected to feature extraction, wherein multiple star blocks are repeatedly used in each stage to extract high-level features, and at the end of each stage, depthwise separable convolution is used for downsampling. In the last stage, global average pooling is used to convert the extracted feature map into a one-dimensional feature vector; The fully connected layer is used to map the one-dimensional feature vector to the number of categories, completing the mapping process of the high-dimensional nonlinear feature space.

6. The infrared small target detection method according to claim 3, characterized in that: The expression of the loss function of the improved YOLOv8 model is as follows: Among them, MPDIoU represents the loss function, IoU represents the ratio of the intersection and union of the predicted box and the real box, h represents the height of the box, w represents the width of the box, d1 represents the straight-line distance between the upper left corner coordinates of the predicted box and the real box, d2 represents the straight-line distance between the lower right corner coordinates of the predicted box and the real box, B represents the bounding box, B gt represents the ground-truth bounding box, Indicates the horizontal coordinate of the upper left corner of the prediction box, Indicates the horizontal coordinate of the upper left corner of the real box, Indicates the ordinate of the upper left corner of the prediction box, Indicates the ordinate of the upper left corner of the real box, Indicates the horizontal coordinate of the lower right corner of the prediction box, Indicates the horizontal coordinate of the lower right corner of the real box, Indicates the ordinate of the lower right corner of the prediction box, It represents the ordinate of the lower right corner of the real box, prd represents the predicted value, and gt represents the real value.

7. The infrared small target detection method according to claim 1, characterized in that: The step S2 comprises the following steps: S201, obtaining an infrared data set, and dividing the infrared data set, wherein the training set is used to train the improved YOLOv8 model, the test set is used to evaluate the generalization ability of the trained YOLOv8 on unknown data, and the validation set is used to evaluate the YOLOv8 during the training process and adjust the hyperparameters; S202, using the divided data set to train the YOLOv8 model, and selecting the optimal weight parameters in the training process; S203, using the YOLOv8 model with optimal weight parameters to detect small target images in the infrared data set.

8. An infrared small target detection system, used to perform an infrared small target detection method, characterized in that: include: The first processing module is used to improve the YOLOv8 model based on the characteristics of infrared weak target images; The second processing module is used to perform target detection on infrared weak target images using the improved YOLOv8 model.

9. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps of the infrared dim small target detection method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed, the steps of the infrared dim small target detection method as described in any one of claims 1 to 7 are implemented.

Images