An infrared weak and small target detection method of line-by-line detection

Abstract

The application discloses an infrared weak small target detection method based on line-by-line detection, and solves the problems of high detection delay and large resource consumption caused by global image caching in the prior art. The method skips the traditional "read-out-caching-global detection" process and performs real-time detection while reading out the infrared image data line by line. The method comprises the following steps: performing first-order and second-order differential calculation on a single-line image vector, extracting and fusing the intra-line spatial features; inputting the continuous multi-line features into an inter-line fusion module based on a self-attention mechanism in parallel, and restoring the high-dimensional features of the target; then, completing target detection through a U-Net network, and adapting the dimension through a line expansion and line compression module; and in the training, adaptively selecting a loss function according to whether the current line block contains a target, so as to improve the training efficiency. The application realizes parallel processing of detection and data reading, significantly improves the timeliness of detection, and reduces the demand for cache and computing resources of edge devices.

Description

Technical Field

[0001] This invention relates to on-orbit data processing and infrared weak target detection technology, and in particular to a line-by-line detection method for infrared weak targets. Background Technology

[0002] Infrared small target detection is one of the core technologies of Infrared Search and Track (IRST) systems and has significant strategic importance in the military field. In long-range imaging, small targets are easily affected by complex backgrounds, exhibiting varied shapes and lacking specific features, and may be completely obscured by background clutter. Therefore, infrared small target detection has always been a highly challenging technical problem. Many researchers have focused on infrared small target detection, especially for images with complex backgrounds. Currently, scholars have proposed numerous infrared small target detection algorithms that can achieve high-precision detection of infrared small targets in various complex backgrounds, but most of them use mature infrared image data after preprocessing as input. However, most edge devices (especially on-board processing platforms) have limited storage and computing resources. Given the rapid increase in infrared image data throughput, infrared small target detection methods based on global infrared images struggle to achieve high-throughput, high-time-efficiency detection of infrared data.

[0003] Spatial domain methods are the most numerous for single-frame infrared small target detection, mainly including filtering methods, local contrast-based methods (LCM), data structure-based methods, and deep learning-based methods. The earliest and most classic infrared small target detection method is the filtering method, proposed at the end of the last century. Filtering detection methods are advantageous due to their simple structure and low computational cost, offering advantages such as strong real-time performance and ease of implementation. However, their performance is significantly limited in complex backgrounds, limiting their application to target detection only in simple backgrounds. Currently, they are generally used as a preprocessing module for other methods.

[0004] Inspired by the contrast mechanism of human vision, Chen C et al. proposed the LCM detection algorithm in 2013. This type of method constructs a local contrast operator by extracting image data of the target region and its neighborhood to enhance infrared small targets and suppress background. Finally, it detects infrared small targets through global adaptive thresholding.

[0005] In infrared images, small infrared targets often exhibit sparsity while the background exhibits low-rank characteristics. Based on this difference, in 2013, Gao et al. proposed a data structure-based method for detecting small infrared targets. This type of method assumes that all background image patches originate from one or more low-rank subspaces, defining the background as the low-rank component and the small target as the sparse component in the model. This transforms the problem of detecting small infrared targets into a low-rank sparse matrix recovery problem, and finally determines the target location through global adaptive thresholding.

[0006] In recent years, deep learning has achieved remarkable results in image recognition, object detection, and other fields, promoting its application in infrared small target detection. Deep learning methods perform well in complex backgrounds and high-noise environments, thus becoming a primary tool for detecting weak infrared targets. Most deep learning methods build object detection networks on traditional deep learning architectures, such as the early CNN architecture and the currently popular YOLO, U-Net, and Transform architectures. Some researchers have proposed new convolutional models, such as windmill-shaped convolutional modules, to address the characteristics of target intensity distribution. Overall, deep learning-based infrared small target detection methods exhibit stronger target feature extraction capabilities and clutter suppression performance, and demonstrate high robustness in complex and variable background environments.

[0007] Furthermore, under low target signal-to-clutter ratio conditions, especially when strong clutter exists in the background, the performance of single-frame infrared small target detection methods is still insufficient. Therefore, researchers have proposed multi-frame detection methods that incorporate temporal information. Multi-frame detection methods mainly utilize the temporal characteristic differences between infrared small targets and the background in a continuous infrared image sequence to extract the target from the background. However, this method has significant limitations: firstly, it requires high background stability and is only applicable to staring mode, making it difficult to meet the application requirements of wide-span scanning scenarios; secondly, the multi-frame algorithm requires pre-caching multiple frames of images to function, which significantly restricts the real-time performance of infrared weak target detection.

[0008] It can be seen that the aforementioned infrared small target detection algorithms mainly use mature infrared images as input. Single-frame infrared small target detection methods have good timeliness; multi-frame detection methods perform better in background clutter suppression, but are only suitable for scenes with relatively stable backgrounds. Traditional small target detection algorithms (such as filtering methods, local contrast methods, and data structure methods) have weak suppression capabilities for complex backgrounds and strong clutter, resulting in a high false alarm rate; while deep learning algorithms show significant advantages due to their powerful target feature extraction capabilities and non-global dependency characteristics, but they also face technical bottlenecks such as high algorithm complexity and high resource consumption.

[0009] However, in practical infrared imaging equipment, the above methods require reading and caching the image line by line before small target detection. This separation of data reading and processing introduces a delay in the photodetector system. In contrast, Single-Row Detection (SRD) technology only needs to read one or several rows of pixels, performing small target detection simultaneously with the line-by-line reading of the infrared image. This overlap of image reading and processing significantly improves the timeliness of infrared small target detection.

[0010] It should be noted that the information disclosed in the background section above is only for understanding the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0011] The main objective of this invention is to overcome the deficiencies in the aforementioned background technology and provide a method for detecting weak infrared targets by line-by-line detection.

[0012] To achieve the above objectives, the present invention adopts the following technical solution:

[0013] A method for detecting weak infrared targets using line-by-line detection includes the following steps:

[0014] S1. Spatial Differential Feature Extraction: Taking the row vectors of the infrared image as the processing object, the spatial differential features within the row are extracted by calculating the first and second derivatives of the row vectors. The features are then enhanced and fused using convolutional layers and activation functions to obtain mixed features within the row.

[0015] S2. Inter-row feature fusion: The intra-row mixed features of multiple consecutive rows are input into the inter-row attention module in parallel. The features of multiple rows are fused based on the self-attention mechanism to recover the high-dimensional features of the infrared small target. Then, the features are compressed through a convolutional network.

[0016] S3. Line-by-line detection network processing: The features after inter-line feature fusion are input into the U-Net target detection network for target detection. The feature dimensions are adapted through the line expansion module and the line compression module at the input and output stages, respectively.

[0017] S4. Adaptive Loss Function Selection: During training, the loss function is adaptively selected based on whether the labels corresponding to the current input multi-line images contain small infrared targets, in order to improve training efficiency and detection performance.

[0018] Further, in step S1, the extraction of intra-row spatial differential features includes:

[0019] Calculate the first derivative of the row vector. This first derivative is the central difference between the gray values ​​of the current pixel and its left and right neighboring pixels. It is used to amplify the gradient difference between the target and the background in the row direction.

[0020] Calculate the second derivative of the row vector. This second derivative is a Laplacian approximation of the gray values ​​of the current pixel and its left and right neighboring pixels, and is used to characterize the curvature change on the row vector.

[0021] The original row vector, its first-order differential vector, and its second-order differential vector are concatenated and input into a feature extraction network consisting of multiple one-dimensional convolutional layers and nonlinear activation functions connected in sequence. The network then extracts and fuses these features to obtain the intra-row mixed features.

[0022] Furthermore, the first-order and second-order differential calculations can distinguish features based on the different mathematical characteristics exhibited by the target, background, and noise on the row vector: the first-order differential corresponding to the infrared small target presents as a pair of adjacent extreme values ​​with a gradual change in amplitude, and its second-order differential presents as a stable change pattern with a minimum value sandwiched between two maxima; the differential feature corresponding to the background has a gradual change in amplitude; while the differential feature corresponding to the noise presents as a sudden change pattern with a drastic change in amplitude.

[0023] Further, in step S2, the inter-line feature fusion includes:

[0024] Each of the consecutive rows of image data is independently input into its own identical spatial differential feature extraction module to obtain the corresponding multi-row intra-row mixed features;

[0025] The multi-row mixed features and the row number information representing the sequential position of each row in the image are input into the inter-row attention module. The inter-row attention module adopts a feature fusion method based on self-attention mechanism, calculates the attention weight between the features of each row and performs weighted fusion, and outputs the fused high-dimensional feature vector.

[0026] The high-dimensional feature vector is input into a feature compression network consisting of at least one one-dimensional convolutional layer and a nonlinear activation function to further reduce the feature dimension.

[0027] Furthermore, the positional information in the interline attention module is the row number sequence of each row vector in the original image.

[0028] Further, in step S3, the line-by-line detection network processing includes:

[0029] A row expansion module is set between the inter-row feature fusion module and the U-Net network to expand the feature dimensions to match the U-Net input.

[0030] Use the U-Net network to perform target detection on the expanded features;

[0031] A row compression module is set after the U-Net output to compress the output features to the dimension corresponding to the multi-line input, so as to facilitate the output of detection results line by line.

[0032] Further, in step S4, the selection of the adaptive loss function includes:

[0033] If the labels corresponding to the current multi-row images contain small infrared targets, then a combination of the focal loss function and the Dice loss function should be used.

[0034] If the labels corresponding to the current multi-row images do not contain infrared small targets, then the focal loss function with doubled weights is selected.

[0035] Furthermore, the adaptive loss function selection mechanism dynamically determines whether the current input block contains the target during training and adjusts the loss calculation method accordingly to improve the training efficiency of the line-by-line detection network.

[0036] Furthermore, during training and inference, multiple consecutive rows of images are cached as local inputs for processing each time; during inference, only the first row of the network output is taken as the detection result for that time, and all results are output during the last inference.

[0037] A computer program product includes a computer program that, when executed by a processor, implements the line-by-line infrared weak target detection method.

[0038] The present invention has the following beneficial effects:

[0039] This invention proposes a line-by-line infrared target detection method based on image row vectors, which effectively skips the traditional "read-cache-global detection" process and directly completes target detection during the line-by-line reading of image data, thereby significantly improving the timeliness of infrared target detection.

[0040] Specifically, this method targets edge processing scenarios such as on-orbit remote sensing, using row vectors read line by line as the processing object, achieving real-time detection without the need for global image caching. Its core lies in constructing a row-by-row detection network supplemented by an adaptive loss function selection mechanism. This network, through a spatial differential feature extraction module, fully utilizes the differences in differential characteristics of the target, background, and noise in a single row of data to extract and fuse multi-order differential features, effectively enhancing the information in a single row. Furthermore, the inter-row feature fusion module employs an attention mechanism to fuse features from multiple consecutive rows, recovering the high-dimensional features of small infrared targets and overcoming the difficulty of limited features in single-row data. The detection network ultimately achieves accurate target localization through U-Net. Simultaneously, the adaptive loss function selection mechanism dynamically adjusts the loss calculation method based on whether the training samples contain the target, adapting to situations where the pixel proportion of small infrared targets is extremely low and not every input row vector contains a local part of the target, improving the network's training efficiency and robustness.

[0041] In summary, this invention enables detection to begin immediately after reading the first few rows of images, achieving parallel processing of the detection process and image reading. This not only significantly reduces the caching and computing resource requirements of edge devices (especially on-board computing platforms), but also significantly improves the overall timeliness of infrared weak target detection, making it suitable for high-throughput, high-real-time infrared remote sensing data processing scenarios.

[0042] Other beneficial effects of the embodiments of the present invention will be further described below. Attached Figure Description

[0043] Figure 1 This is the overall flowchart of the infrared weak target detection method of the present invention, which involves line-by-line detection.

[0044] Figure 2 This is a schematic diagram of the spatial differentiation module in an embodiment of the present invention.

[0045] Figure 3 This is a schematic diagram of the interline feature extraction module in an embodiment of the present invention.

[0046] Figure 4 This is a diagram showing the overall structure of the infrared small target line-by-line detection network according to an embodiment of the present invention.

[0047] Figure 5 This is the infrared weak target detection result of an embodiment of the present invention. The left side is the original image, and the right side is the detection result. Detailed Implementation

[0048] The embodiments of the present invention will be described in detail below. It should be emphasized that the following description is merely exemplary and not intended to limit the scope and application of the present invention.

[0049] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0050] This invention aims to solve the problems of high latency and high resource consumption caused by the reliance on global image caching in existing infrared weak target detection methods. It proposes a row-by-row detection method based on real-time row vector processing. By simultaneously performing spatial differential feature extraction, inter-row attention fusion and target detection during the image row data reading process, it realizes parallel processing of detection and data reading, significantly improves detection timeliness, and greatly reduces the cache and computing resource requirements of edge devices (especially on-board platforms).

[0051] See Figures 1 to 5This invention provides a method for detecting weak infrared targets by line-by-line detection, comprising the following steps:

[0052] Step S1, Spatial Differential Feature Extraction: Taking the row vectors of the infrared image as the processing object, the spatial differential features within the row are extracted by calculating the first and second derivatives of the row vectors, and feature enhancement and fusion are performed using convolutional layers and activation functions to obtain mixed features within the row.

[0053] In some embodiments, step S1, extracting the intra-row spatial differential features, includes: calculating the first derivative of the row vector, which is the central difference between the gray values ​​of the current pixel and its left and right adjacent pixels, used to amplify the gradient difference between the target and the background in the row direction; calculating the second derivative of the row vector, which is the Laplacian approximation of the gray values ​​of the current pixel and its left and right adjacent pixels, used to characterize the curvature change on the row vector; concatenating the original row vector, its first derivative vector, and its second derivative vector, and inputting them together into a feature extraction network composed of multiple one-dimensional convolutional layers and nonlinear activation functions connected in sequence, to extract and fuse the intra-row mixed features.

[0054] In some embodiments, the first and second derivative calculations can distinguish features based on the different mathematical characteristics exhibited by the target, background, and noise on the row vector: the first derivative of the infrared small target presents as a pair of adjacent extreme values ​​with a gradual change in amplitude, and its second derivative presents as a stable change pattern with a minimum value sandwiched between two maxima; the differential feature of the background has a gradual change in amplitude; while the differential feature of the noise presents as a sudden change pattern with a drastic change in amplitude.

[0055] Step S2, Inter-row feature fusion: The intra-row mixed features of multiple consecutive rows are input into the inter-row attention module in parallel. The features of multiple rows are fused based on the self-attention mechanism to recover the high-dimensional features of the infrared small target. Then, the features are compressed through a convolutional network.

[0056] In some embodiments, step S2, the inter-row feature fusion includes: independently inputting consecutive rows of image data into their respective identical spatial differential feature extraction modules to obtain corresponding multi-row intra-row mixed features; inputting the multi-row intra-row mixed features and row number information representing the sequential position of each row in the image into an inter-row attention module, wherein the inter-row attention module adopts a feature fusion method based on a self-attention mechanism, calculates the attention weights between the features of each row and performs weighted fusion, and outputs a fused high-dimensional feature vector; and inputting the high-dimensional feature vector into a feature compression network composed of at least one one-dimensional convolutional layer and a nonlinear activation function to further reduce the feature dimension.

[0057] In some embodiments, the positional information in the interline attention module is the row number sequence of each row vector in the original image.

[0058] Step S3, line-by-line detection network processing: The features after inter-line feature fusion are input into the U-Net object detection network for object detection. The feature dimensions are adapted through the line expansion module and the line compression module at the input and output stages, respectively.

[0059] In some embodiments, step S3, the line-by-line detection network processing includes: setting a line expansion module between the inter-line feature fusion module and the U-Net network to expand the feature dimension to match the U-Net input; using the U-Net network to perform object detection on the expanded features; and setting a line compression module after the U-Net output to compress the output features to a dimension corresponding to the multi-line input, facilitating the line-by-line output of detection results. For example, the line expansion module may consist of a one-dimensional transposed convolutional layer and a ReLU activation function, expanding the feature dimension to the input size required by the U-Net network. The line compression module may consist of a one-dimensional convolutional layer, compressing the number of output feature channels of the U-Net to a dimension matching the object detection task.

[0060] Step S4, Adaptive Loss Function Selection: During training, the loss function is adaptively selected based on whether the labels corresponding to the current input multi-line images contain small infrared targets, in order to improve training efficiency and detection performance.

[0061] In some embodiments, step S4, the selection of the adaptive loss function includes: if the labels corresponding to the current multi-row images contain infrared small targets, then a combination of the focal loss function and the Dice loss function is selected; if the labels corresponding to the current multi-row images do not contain infrared small targets, then a focal loss function with doubled weights is selected.

[0062] In some embodiments, the adaptive loss function selection mechanism dynamically determines whether the current input block contains the target during training and adjusts the loss calculation method accordingly to improve the training efficiency of the line-by-line detection network.

[0063] In some embodiments, during training and inference, multiple consecutive rows of images are cached as local input for processing each time; during inference, only the first row of the network output is taken as the detection result for that time, and all results are output during the last inference.

[0064] The proposed line-by-line infrared weak target detection method utilizes an innovative line-by-line detection architecture to simultaneously identify weak targets during the image data streaming process. This bypasses the global image caching stage required by traditional methods, achieving parallelization of the detection process and data reading. This significantly improves detection timeliness and reduces the storage and computing resource requirements of edge devices (especially on-board processing platforms). Furthermore, the method incorporates a dedicated spatial differential feature extraction module and an inter-line attention fusion module, effectively overcoming the challenge of limited features in single-line data. It recovers and enhances the high-dimensional features of the target from sparse row vectors, maintaining robust detection performance even in complex backgrounds. Coupled with an adaptive loss function selection mechanism, this scheme further optimizes the training process, adapting to situations where the pixel proportion of infrared small targets is extremely low and the target's local distribution is scattered. This ensures robust detection performance while improving network training efficiency and overall robustness.

[0065] The following further describes specific embodiments of the present invention, algorithm examples, and experimental verification.

[0066] This invention provides a method for detecting weak infrared targets using line-by-line detection. The core of the entire detection scheme is a line-by-line detection network, supplemented by an adaptive loss function selection mechanism. The line-by-line detection network mainly consists of a spatial differential feature extraction module, an inter-line feature fusion module, and a U-Net target detection network connected sequentially.

[0067] Specifically, the spatial differential feature extraction module is responsible for receiving single-row infrared image data (i.e., row vectors) and enhancing and mining their intra-row features. Since the feature information contained in a single row of data is limited, this module amplifies the spatial differences between the target, background, and noise by calculating the first and second derivatives of the row vectors. The first derivative is calculated using the central difference method, reflecting the gradient information of grayscale changes; the second derivative is a Laplace approximation used to characterize the curvature changes on the row vector. The background, target, and noise exhibit different patterns under these two differential transformations: the differential response of the background is usually relatively smooth; the first derivative of a weak infrared target shows a pair of adjacent extreme values ​​with smooth amplitude changes, while the second derivative shows a stable pattern with two maxima sandwiched between a minimum value; the differential response of noise exhibits abrupt changes in amplitude. The extracted original row vectors and their first-order and second-order differential vectors are concatenated and then fed into a sub-network consisting of multiple one-dimensional convolutional layers and Mish activation functions connected in sequence for deep feature extraction and fusion, ultimately outputting a hybrid feature rich in intra-row spatial information.

[0068] To recover more complete high-dimensional features of weak infrared targets from limited single-line information, this method further designs an inter-line feature fusion module. This module receives multiple consecutive lines of image data in parallel, and each line is processed by the same spatial differential feature extraction module described above to obtain corresponding multi-line hybrid features. These features, along with line number information representing the absolute position of each line in the original image, are input into an inter-line attention module based on the self-attention mechanism of the Transformer architecture. This module adaptively weights and fuses the multi-line features by calculating the attention weights between the features of each line, thereby recovering and enhancing the target features and suppressing irrelevant background and noise. The fused high-dimensional features are then compressed by a lightweight network consisting of a one-dimensional convolutional layer and an activation function to obtain a more compact feature representation.

[0069] Subsequently, the compressed features are adjusted in dimension by a row expansion module to match the input requirements of the U-Net object detection network. The U-Net network is responsible for completing the final object detection task. After the network output, the feature dimensions are restored by a row compression module to obtain the detection results corresponding to the multi-row regions of the input. During training and inference, each step processes a local block composed of consecutively cached multi-row images. During inference, typically only the first row of the network output is taken as the valid detection result for the current step, and all results are output when processing the last batch of data, thus achieving true row-by-row output.

[0070] Furthermore, considering the extremely low pixel proportion and sparse distribution of infrared small targets in images, this method introduces an adaptive loss function selection mechanism during the training phase. This mechanism dynamically selects different loss functions based on whether the label data corresponding to the current input multi-row image patches contains infrared small targets. Specifically, if the label contains the target, the focal loss function and the DICE loss function are used together; if the label does not contain the target, the focal loss function with doubled weights is used. This mechanism can adapt to situations where the pixel proportion of infrared small targets is extremely low and not every input row vector contains a local area of ​​the target, thereby improving the overall training efficiency and detection performance of the network.

[0071] This method skips the traditional "read-cache-global detection" process and instead uses the row vectors of the infrared image as the direct processing object. It simultaneously completes the detection of weak targets during the process of reading out the image data row by row, thereby reducing the occupation of cache resources and significantly improving the timeliness of detection.

[0072] The specific implementation methods of each part are further detailed below.

[0073] Figure 2 The spatial differential module structure of an embodiment of the present invention is shown. Figure 3The structure of the interline feature extraction module according to an embodiment of the present invention is shown. Figure 4 The overall structure of the infrared small target line-by-line detection network according to an embodiment of the present invention is shown. Figure 2 The spatial differential module structure of an embodiment of the present invention is shown. Figure 3 The structure of the interline feature extraction module according to an embodiment of the present invention is shown. Figure 4 The overall structure of the infrared small target line-by-line detection network according to an embodiment of the present invention is shown. Figure 2 In this diagram, RowfeatNet represents the spatial differential feature extraction module, FeaFusion represents the multi-order feature fusion module (responsible for fusing the original row vectors with multi-order differential features), ConsNet represents the feature integration module (used to integrate the concatenated multi-dimensional features), GradFun represents the differential calculation function (responsible for first-order and second-order differential operation logic), GradNet represents the multi-order differential calculation module (performs first-order and second-order differential calculations of row vectors), CarvFun represents the intra-row feature extraction function (used in conjunction with convolution and activation functions to mine high-dimensional features), CarvNet represents the intra-row feature extraction module (composed of 3 1D convolution kernels and the Mish function), cat represents the feature concatenation operation, Conv1d represents a one-dimensional convolutional layer (or 1D convolution kernel), and Mish represents the Mish activation function. Figure 3 In the diagram, MultiRowAttenNet represents the interline attention mechanism module, r n , ..., r n+m f represents the row vector of the infrared image. n , ..., f n+m Represents intra-row spatial differential features (or intra-row mixed features), Row wiseAttenNet represents the inter-row attention module (performing self-attention weighted fusion), FushNet represents the feature fusion module, and ReLU represents the ReLU activation function; Figure 4 In the middle, r [n,n+m] RowUNet represents a local input block consisting of row vectors from row n to row (n+m) of an infrared image. RowUNet represents a row-by-row detection network (a U-Net network based on row vector processing). Row Expand represents a row expansion module (adapting to the input dimension of U-Net). U-Net represents a U-Net object detection network. Row Compress represents a row compression module (restoring the output feature dimension to adapt to the row-by-row output). Loss represents the loss calculation stage (used for calculating the loss value and updating parameters during training).

[0074] Spatial Differential Feature Extraction Module:

[0075] The spatial differential feature extraction module is the data receiving module of the proposed line-by-line detection network. Its main function is to extract the spatial differential characteristics within a line. It consists of a multi-order differential calculation module, a differential feature extraction module, and a multi-order feature fusion module. Figure 1 As shown. The input to the spatial differential feature extraction module is the row vector of the infrared image. , , , This represents the total number of rows in the image. This represents the number of columns in the image.

[0076] The multi-order differential calculation module mainly calculates row vectors. The first and second derivative properties amplify the differences between weak targets, background, and noise in the vector space. The first derivative properties of the row vectors are extracted using the following formula:

[0077] ,

[0078] The second-order differential property is extracted using the following formula:

[0079] .

[0080] In the row vector, the differential characteristics of most background targets are relatively stable; the recovery changes of small infrared targets in the row vector conform to the one-dimensional Gaussian property, and its first-order differential characteristics are a pair of adjacent maxima and minima with gradual peak changes, and its second-order differential characteristics are two maxima with an adjacent minima in between and a stable peak change; noise in the row vector is similar to an impulse function, and its first-order differential characteristics are a pair of adjacent maxima and minima with drastic peak changes, and its second-order differential characteristics are similar to those of small infrared targets but with more drastic peak changes.

[0081] After obtaining the second-order differential features of the row vectors, , , Simultaneously, in the inline feature extraction module, high-dimensional features are extracted through a series of 1D convolutions and the Mish function. Finally, the feature dimension is obtained by feature concatenation. Inline blending features. The structures of the inline feature extraction module and the feature blending module are as follows: Figure 2 As shown, each consists of three 1D convolutional kernels and a Mish function.

[0082] Interline feature fusion module:

[0083] Interline feature extraction module, such as Figure 3 As shown, firstly by One (in the experiment) 9) The spatial differentiation module receives adjacent image rows in parallel and extracts intra-row features. A number of adjacent image rows can be generated by The row vectors are represented and input into the inter-row feature map module in a pipelined manner. The aforementioned row vectors are then processed by the spatial differentiation module to extract intra-row spatial differential features. The parallel input interline attention module recovers the high-dimensional features of small infrared targets. Note that the inter-row attention mechanism module in this invention adopts the self-attention mechanism module in the Transformer architecture, and the position information is the row number of the row vector in the image. .at last The feature is then input into a CNN network for feature compression. The CNN network consists of two 1D convolutional kernels connected by a Mish function.

[0084] Overall structure of the line-by-line detection network:

[0085] The overall structure of the line-by-line detection network is as follows Figure 4 As shown, the system consists of an inter-row attention mechanism module (MultiRowAttenNet), a row expansion module, a U-Net network, and a row compression module. The U-Net network is the original version, with its input and output data dimensions consistent with the original version. Therefore, a row expansion module is set in both U-Net and MultiRowAttenNet to change the output of MultiRowAttenNet to the same dimension as the input of U-Net; a row compression module is set after the output of U-Net to change the output of U-Net to the same dimension as the input of U-Net. .

[0086] During training and inference, images are input line by line into the line-by-line detection network, and each time they are cached... The rows form local images for training and inference. During training, the output... The size result is compared with the label data of the corresponding region to calculate the loss (LOSS) and update the network parameters; during the inference process, only the first line of output is taken as the final result each time, and the entire result is taken in the last inference.

[0087] Adaptive loss function selection mechanism:

[0088] Due to the input Neighboring image rows may not necessarily contain infrared small targets, therefore an adaptive loss function selection mechanism is needed to adaptively select the loss function during training. The adaptive function selection mechanism proposed in this invention selects the loss function based on whether the label data contains an infrared small target label. Specifically, if the label contains an infrared small target label, then the focal loss function and the DICE loss function are selected; otherwise, the loss function is twice the focal function.

[0089] In one example, the test dataset is the SIRST dataset, which contains 428 images and corresponding pixel-level labels. This invention selects 407 images as the training set and the remaining 20 images as the test set. The input line length is set to 256, the batch size to 8, and the training epochs to 200. The initial learning rate is 0.01, dynamically adjusted; if the loss function does not decrease after 10 consecutive epochs, the learning rate becomes 0.5 times its original value. The selected optimizer is Adam.

[0090] The specific training steps are as follows: 1. Input the dataset and labeled data into the network in batches for forward propagation. 2. The network outputs the prediction results and calculates the loss value with the labels. 3. The network performs backpropagation based on the loss value and updates the weight parameters through the Adam optimizer. 4. Repeat the above process continuously, and the loss value continuously decreases until the required number of training rounds is reached.

[0091] According to experimental tests, Figure 5 This image illustrates the infrared weak target detection results of an embodiment of the present invention. The left side shows the original image, and the right side shows the detection result. Figure 5 As can be seen from the data, for infrared graphic row vectors input line by line, the technology proposed in this invention can effectively suppress the background while enhancing the target, ultimately achieving infrared weak target detection.

[0092] In summary, the infrared small target detection method of this invention, which performs line-by-line detection, does not require waiting until all infrared images have been read before performing infrared small target detection. Instead, it can start infrared small target detection after reading the first few lines of images, allowing it to be performed in parallel with image reading, thus skipping the traditional "read-cache-global detection" process. This invention effectively saves cache and computing resources at the edge (especially on-board computing platforms), achieving highly efficient infrared small target detection.

[0093] This invention also provides a storage medium for storing a computer program, which, when executed, performs at least the methods described above.

[0094] This invention also provides a control device, including a processor and a storage medium for storing a computer program; wherein the processor executes the computer program by performing at least the method described above.

[0095] This invention also provides a processor that executes a computer program, at least performing the methods described above.

[0096] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc or CD-ROM; magnetic surface memory can be disk storage or magnetic tape storage. The storage media described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0097] In the several embodiments provided by this invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0098] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0099] In addition, in the various embodiments of the present invention, each functional unit can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0100] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0101] Alternatively, if the integrated units of this invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0102] The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0103] The features disclosed in the several product embodiments provided by this invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0104] The features disclosed in the several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0105] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various equivalent substitutions or obvious modifications can be made without departing from the concept of the present invention, and all such modifications, achieving the same performance or application, should be considered within the scope of protection of the present invention.

Claims

1. A method for detecting weak infrared targets using line-by-line detection, characterized in that, Includes the following steps: S1. Spatial Differential Feature Extraction: Taking the row vectors of the infrared image as the processing object, spatial differential features within the row are extracted by calculating the first and second derivatives of the row vectors. Convolutional layers and activation functions are then used for feature enhancement and fusion to obtain mixed features within the row. In S1, the extraction of intra-row spatial differential features includes: calculating the first derivative of the row vector, which is the central difference between the gray values ​​of the current pixel and its left and right adjacent pixels, used to amplify the gradient difference between the target and the background in the row direction; calculating the second derivative of the row vector, which is the Laplacian approximation of the gray values ​​of the current pixel and its left and right adjacent pixels, used to characterize the curvature change on the row vector; concatenating the original row vector, its first derivative vector, and its second derivative vector, and inputting them together into the feature extraction network to extract and fuse the intra-row mixed features; S2, Inter-row Feature Fusion: The intra-row mixed features of multiple consecutive rows are input in parallel into the inter-row attention module. Based on the self-attention mechanism, the multi-row features are fused to recover the high-dimensional features of the infrared small target. Then, feature compression is performed through a convolutional network. In step S2, the inter-row feature fusion includes: inputting the image data of multiple consecutive rows independently into their respective identical spatial differential feature extraction modules to obtain the corresponding multi-row intra-row mixed features; inputting the multi-row mixed features and the row number information representing the sequential position of each row in the image into the inter-row attention module. The inter-row attention module adopts a feature fusion method based on the self-attention mechanism, calculates the attention weights between the features of each row and performs weighted fusion, and outputs the fused high-dimensional feature vector; inputting the high-dimensional feature vector into the feature compression network to further reduce the feature dimension. S3. Line-by-line detection network processing: The features after inter-line feature fusion are input into the U-Net target detection network for target detection. The feature dimensions are adapted through the line expansion module and the line compression module at the input and output stages, respectively. S4. Adaptive loss function selection: During training, the loss function is adaptively selected based on whether the labels corresponding to the current input multi-line images contain small infrared targets, in order to improve training efficiency and detection performance.

2. The infrared weak target detection method by line-by-line detection as described in claim 1, characterized in that, In step S1, the feature extraction network is composed of multiple one-dimensional convolutional layers connected sequentially with nonlinear activation functions.

3. The infrared weak target detection method by line-by-line detection as described in claim 2, characterized in that, The first and second derivative calculations can distinguish features based on the different mathematical characteristics exhibited by the target, background, and noise on the row vector: the first derivative of the infrared small target is presented as a pair of adjacent extreme values ​​with a gradual change in amplitude, and its second derivative is presented as a stable change pattern with a minimum value sandwiched between two maximum values; the differential feature of the background has a gradual change in amplitude; while the differential feature of the noise is presented as a sudden change pattern with a drastic change in amplitude.

4. The infrared weak target detection method by line-by-line detection as described in claim 1, characterized in that, In step S2, the feature compression network consists of at least one one-dimensional convolutional layer and a nonlinear activation function.

5. The infrared weak target detection method by line-by-line detection as described in claim 4, characterized in that, The positional information in the interline attention module is the row number sequence of each row vector in the original image.

6. The infrared weak target detection method by line-by-line detection as described in claim 1, characterized in that, In step S3, the line-by-line detection network processing includes: A row expansion module is set between the inter-row feature fusion module and the U-Net network to expand the feature dimensions to match the U-Net input. Use the U-Net network to perform target detection on the expanded features; A row compression module is set after the U-Net output to compress the output features to the dimension corresponding to the multi-line input, so as to facilitate the output of detection results line by line.

7. The infrared weak target detection method by line-by-line detection as described in claim 1, characterized in that, In step S4, the selection of the adaptive loss function includes: If the labels corresponding to the current multi-row images contain small infrared targets, then a combination of the focal loss function and the Dice loss function should be used. If the labels corresponding to the current multi-row images do not contain infrared small targets, then the focal loss function with doubled weights is selected.

8. The infrared weak target detection method by line-by-line detection as described in claim 1, characterized in that, The adaptive loss function selection mechanism dynamically determines whether the current input block contains the target during training and adjusts the loss calculation method accordingly to improve the training efficiency of the line-by-line detection network.

9. The infrared weak target detection method by line-by-line detection as described in claim 1, characterized in that, During training and inference, multiple consecutive rows of images are cached as local inputs for processing each time; during inference, only the first row of the network output is taken as the detection result for that time, and all results are output during the last inference.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the infrared weak target detection method of line-by-line detection as described in any one of claims 1 to 9.

Images