Aerial infrared dim small moving target detection method

Abstract

The invention provides an aerial infrared dim small moving target detection method. The method comprises the following steps: acquiring a to-be-detected image group; based on a grouping convolution kernel, a Transform module and a multi-frame signal adaptive accumulation module of the convolution kernel, performing adaptive signal accumulation on the to-be-detected image group to obtain an image feature matrix with enhanced representation; performing multi-scale detection on the representation enhanced image feature matrix through a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix; and performing exclusion processing on the multi-scale detection result of the image feature matrix based on a multi-scale detection result unification mechanism, and determining a target detection result of the to-be-detected image group. The multi-frame infrared target detection method can solve the problem that the existing multi-frame infrared target detection method is slow in detection speed and cannot meet the requirements of real-time performance or higher-frequency detection of an infrared detection system, and effectively improves the detection precision of the infrared small target under low contrast.

Description

Technical Field

[0001] The invention relates to the technical field of infrared image data processing, and in particular to a method for detecting infrared dim and weak small moving targets in the air. Background Art

[0002] Infrared imaging can meet the conditions of long distance, multi-band, and all-weather, and is widely used in various detection systems. Since infrared detectors can reflect the infrared radiation characteristics of targets at a long distance, the detection distance of infrared detection systems is usually required to be high, which leads to three difficulties: ① The target is usually small in the infrared imaging image, occupying only 5 pixels or less; ② The infrared radiation of the target decreases at the entrance pupil of the infrared detection system, resulting in a low contrast between the target and the background, making it a dim target; ③ The target moves a small distance between adjacent frames, making it difficult to form an effective trajectory in a short time. In addition, the diffraction phenomenon makes the target lack detailed features such as shape and texture, and there are many environmental noises and random noises in the infrared image that are similar to the target representation, which brings strong interference to the detection of infrared dim small targets. Under difficult detection conditions, the infrared detection system also has high requirements for the real-time detection. The above problems have brought great challenges to the detection of infrared dim small moving targets in the air at long distances.

[0003] Traditional single-frame infrared target detection methods can be divided into filtering-based methods, methods based on the human visual system, and low-rank methods. Among them, filtering-based methods such as Tophat are usually limited to specific and uniform scenes; methods based on the human visual system such as LCM are only applicable to scenes with large targets and strong background distinction; low-rank methods such as DeIPI can handle rapidly changing complex backgrounds, but their real-time performance is poor. Although traditional single-frame infrared target detection methods are effective in specific scenarios, they are easily interfered by clutter and noise, which makes them perform poorly in the task of detecting small infrared targets in the long-distance air with strong interference. The multi-frame infrared target detection method of tracking first and then detecting uses multiple frames in the time domain to increase the energy of the target, which can be divided into methods based on three-dimensional matched filters, methods based on Hough transform, methods based on dynamic programming, and methods based on particle filters. Among them, the three-dimensional matched filter accumulates the target signal in the space-time transform domain, but due to the large amount of computation of the three-dimensional Fourier transform, it is difficult to be realized digitally; the Hough transform transforms the problem into finding the peak point in the parameter space, but it is more suitable for the detection of straight targets; the dynamic programming method describes the possible location of the target by constructing a Gaussian template, but it is easily affected by the energy diffusion problem; the particle filter method can handle nonlinear and non-Gaussian noise problems by simulating probability distribution, and is suitable for complex target tracking and detection scenarios, but its accuracy requires a sufficient number of particles to ensure, and the computation is large. The multi-frame infrared target detection method is better than the single-frame algorithm in detection accuracy, but the detection speed is slow and cannot meet the requirements of real-time or higher-frequency detection of infrared detection systems. Summary of the invention

[0004] The embodiment of the present invention provides an aerial infrared dim small moving target detection method. The method can adaptively accumulate signals for a to-be-detected image group through a multi-frame signal adaptive accumulation module based on a grouped convolution kernel, a Transformer module and a convolution kernel to obtain a representation-enhanced image feature matrix, and perform multi-scale detection on the representation-enhanced image feature matrix through a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix, and exclude the multi-scale detection result of the image feature matrix based on a unified mechanism for multi-scale detection results to determine the target detection result of the to-be-detected image group. The problem of the existing multi-frame infrared target detection method being slow in detection speed and unable to meet the requirements of real-time or higher-frequency detection of an infrared detection system is solved, and the detection accuracy of infrared small targets under low contrast is effectively improved.

[0005] An embodiment of the present invention provides a method for detecting an infrared dim small moving target in the air, the method comprising the following steps:

[0006] Obtaining a group of images to be detected;

[0007] Based on the grouped convolution kernel, the Transformer module and the multi-frame signal adaptive accumulation module of the convolution kernel, adaptively accumulating signals of the image group to be detected to obtain an image feature matrix representing the enhancement;

[0008] Performing multi-scale detection on the image feature matrix of the representation enhancement through a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix;

[0009] Based on the multi-scale detection result unification mechanism, the multi-scale detection results of the image feature matrix are excluded to determine the target detection results of the to-be-detected image group.

[0010] Optionally, it is characterized in that the acquiring the image group to be detected includes:

[0011] Acquire multiple consecutive frames of images to be detected;

[0012] The continuous multiple frames of images to be detected are superimposed on the channel dimension to obtain the image set to be detected imageSet∈R H ×W×N :

[0013] imageSet={I i ,I i-1 ,…,I i-N+1} (1)

[0014] Among them, Ii represents the current i-th frame to be detected, where I i ∈R H×W×1 , H is the image length, W is the image width, 1 is the number of image channels; N is the number of frames; when there are no N consecutive frames, use the all-zero matrix 0 H×W×1 Replace insufficient frames:

[0015] I i-j =0 H×W×1 if ij<0 (2)

[0016] Optionally, the multi-frame signal adaptive accumulation module based on the grouped convolution kernel, the Transformer module and the convolution kernel adaptively accumulates signals on the image group to be detected to obtain the image group to be detected with enhanced characterization, including:

[0017] The image group to be detected will first be weighted in the time domain in the multi-frame signal adaptive accumulation module of the convolution kernel, and then superimposed in the spatial domain to generate an image feature matrix representing the enhancement.

[0018] featureSet=F S (F T(imageSet)) (3)

[0019] Among them, F T (·) represents the time domain weighting function, F S (·) represents the spatial superposition function.

[0020] Optionally, a weighting function F in the time domain T (·) In the image set to be detected, imageSet∈R H×W×N First, five group convolution kernels with a step size of 2 are continuously filtered until the size is reduced to The grouped convolution kernel function DWConv(·) performs the same filtering on each image in the image set imageSet to be detected:

[0021] scaleSet={Is i ,…,Is i-N+1}={DWConv(I i )),…,DWConv(I i-N+1 ))} (4)

[0022] Then after the grouped convolution kernel is reduced The correlation feature maps between different frames will be calculated through the Transformer module.

[0023] Optionally, the image set scaleSet to be detected will first pass through three fully connected layers to form a query matrix Answer Matrix Weight Matrix Then the correlation feature graph is obtained through the query mechanism

[0024] relativeSet=Q×K T ×V={Ir i ,…,Ir i-N+1} (5)

[0025] Finally, after the global average pooling operation Avg(·), the pooled correlation feature map relativeSet is obtained;

[0026] Pooling correlation feature map relativeSet to get α∈N gives α for each frame i As weighting factor:

[0027]

[0028] We get α = {α i ,…,α i-N+1}, the spatial superposition function F S(·) The weighted frames are spatially superimposed through the convolution kernel to obtain a variety of feature maps after different filtering:

[0029]

[0030] Among them, C1 is the number of convolution kernel channels, Conv1 represents the first channel of the convolution kernel, that is, the first filtering method; when the number of consecutive frames is insufficient, the weighted image α j ×0 H×W×1 =0 H×W×1 .

[0031] Optionally, the multi-scale detection of the image feature matrix representing the enhancement is performed by using a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix, including:

[0032] The image feature matrix α is enhanced by the preset convolutional neural network YOLOX-Tiny at three scales. j ×0 H×W×1 =0 H×W×1 The features are extracted to obtain the first feature matrix The second characteristic matrix The third characteristic matrix The preset convolutional neural network YOLOX-Tiny classification detection head is used to generate the target geometric center point confidence prediction matrix from the first feature matrix, the second feature matrix, and the third feature matrix:

[0033] featureCls i =Clshead i (featureset i ) (8)

[0034] in, A regression detection head is additionally used for the first feature matrix featureSet1 to generate the offset prediction matrix:

[0035] featureReg=Reghead(featureset1) (9)

[0036] Among them, the offset prediction matrix

[0037] Optionally, before performing multi-scale detection on the image feature matrix representing the enhancement by using the preset convolutional neural network YOLOX-Tiny to obtain the multi-scale detection result of the image feature matrix, the method further includes:

[0038] Get the current frame image to be detected I i H×W×1and the target geometric center coordinates (x ct ,y ct );

[0039] The current frame image to be detected I i H×W×1 and the target geometric center coordinates (x ct ,y ct ) Generate three scales of confidence matrix truth:

[0040]

[0041] in, int(·) means round down; at the same time, I i H×W×1 With (x ct ,y ct ) Generate the true value of the offset matrix:

[0042]

[0043] in,

[0044] Optionally, supervise the target geometric center point confidence prediction matrix featureCls through the loss function i Continuously approaching gtCls i At the same time, the two channels of the offset prediction matrix featureReg continuously approach the true value of the offset matrix gtRegX and the true value of the offset matrix gtRegY respectively.

[0045] Optionally, the multi-scale detection result based on the unified mechanism of multi-scale detection results is excluded from the multi-scale detection results of the image feature matrix to determine the target detection result of the image group to be detected, including:

[0046] Based on the unified mechanism of multi-scale detection results, the multi-scale detection results of the erroneous image feature matrix are eliminated from top to bottom:

[0047] (x cls ,y cls )=argmax(featureCls3↑↑+featureCls2↑+featureCls1) (12)

[0048] Here, argmax(·) indicates the coordinates of the maximum value found in the feature matrix, and ↑ indicates upsampling with a step size of 2.

[0049] Optionally, first, the target geometric center point confidence prediction third matrix featureCls3 determines an 8×8 region containing the target, and then the target geometric center point confidence prediction second matrix featureCls2 selects a 4×4 region from the 8×8 region containing the target, and then the target geometric center point confidence prediction matrix featureCls further determines a 2×2 final region (x cls ,y cls );

[0050] The corresponding offset prediction value can be obtained from the final area:

[0051]

[0052] Finally, the predicted position is obtained by the area value and the offset value:

[0053] (x pre ,y pre )=(x cls +x off ,y cls +y off ) (14)

[0054] By adopting the aerial infrared dim small moving target detection method of the present invention, the signals of the image group to be detected can be adaptively accumulated through a multi-frame signal adaptive accumulation module based on a grouped convolution kernel, a Transformer module and a convolution kernel to obtain an image feature matrix representing enhancement, and multi-scale detection is performed on the image feature matrix representing enhancement through a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix, and the multi-scale detection result of the image feature matrix is ​​excluded by a unified mechanism based on the multi-scale detection result to determine the target detection result of the image group to be detected, thereby solving the problem that the existing multi-frame infrared target detection method has a slow detection speed and cannot meet the requirements of real-time or higher-frequency detection of the infrared detection system, and effectively improves the detection accuracy of small infrared targets under low contrast. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0056] Figure 1 It is a schematic diagram of a process of detecting a dim and small moving infrared target in the air provided by an embodiment of the present invention;

[0057] Figure 2 It is a structural schematic diagram of a method for detecting a dark, weak, and small moving infrared target in the air provided by an embodiment of the present invention;

[0058] Figure 3 is a structural diagram of an adaptive signal accumulation module provided by an embodiment of the present invention;

[0059] Figure 4 It is a schematic diagram of comparing an original infrared image and an image processed by an adaptive signal accumulation module provided by an embodiment of the present invention;

[0060] Figure 5 This is an overall structural diagram of a method for detecting a dark, weak, and small moving infrared target in the air provided by an embodiment of the present invention;

[0061] Figure 6 is a diagram of a unified mechanism for multi-scale detection results provided by an embodiment of the present invention;

[0062] Figure 7 It is a target background contrast map provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] like Figure 1 As shown, Figure 1 The flowchart of a method for detecting a small infrared target in the air provided by an embodiment of the present invention is as follows. The method for detecting a small infrared target in the air includes the following steps:

[0065] 101. Obtain a group of images to be detected.

[0066] In the embodiment of the present invention, the above-mentioned image group to be detected can be understood as a group of continuous multiple frames of infrared images.

[0067] A group of images to be detected can be obtained by acquiring multiple consecutive frames of images to be detected and stacking the multiple consecutive frames of images to be detected on a channel dimension.

[0068] The above-mentioned continuous multiple frames of images to be detected can be understood as several adjacent frames of images in the infrared image sequence.

[0069] Based on the grouped convolution kernel, Transformer module and the multi-frame signal adaptive accumulation module of the convolution kernel, the signals of the image group to be detected are adaptively accumulated to obtain the image feature matrix representing the enhancement.

[0070] In the embodiment of the present invention, the above-mentioned grouped convolution kernel can be understood as grouped convolution, which mainly divides the input feature map into multiple groups and then uses an independent convolution kernel in each group to perform convolution calculation.

[0071] The above Transformer module can be understood as a deep learning model that uses a self-attention mechanism. Through the construction of the encoder and decoder and the existence of the self-attention layer, it can be more efficient and accurate when processing sequence-to-sequence tasks. The encoder is composed of multiple identical large modules stacked together, and each large module consists of three parts: input projection layer, self-attention mechanism layer, and fully connected layer.

[0072] The multi-frame signal adaptive accumulation module of the above-mentioned convolution kernel can effectively eliminate the position difference between target frames, so that the multi-frame signals can be accurately accumulated at the true value position, enhance the target signal strength, and greatly improve the robustness of detection.

[0073] The above-mentioned multi-frame signal adaptive accumulation module based on grouped convolution kernel, Transformer module and convolution kernel can be understood as an adaptive signal accumulation module. The adaptive signal accumulation module can enhance low-contrast targets to a recognizable level by adaptively accumulating signals from consecutive multi-frame images.

[0074] 103. The multi-scale detection of the image feature matrix with enhanced representation is performed through the preset convolutional neural network YOLOX-Tiny to obtain the multi-scale detection result of the image feature matrix.

[0075] In an embodiment of the present invention, the preset convolutional neural network YOLOX-Tiny is a small convolutional neural network model YOLOX-Tiny preset by the system. The small convolutional neural network model YOLOX-Tiny extracts image features through multiple convolutional layers, and then uses these features to perform target detection.

[0076] The above multi-scale detection can be understood as a process of detection through convolutional layers of different depths.

[0077] 104. Based on the unified mechanism of multi-scale detection results, the multi-scale detection results of the image feature matrix are excluded to determine the target detection results of the image group to be detected.

[0078] In an embodiment of the present invention, the above-mentioned multi-scale detection result unification mechanism can be understood as a processing strategy, which suppresses the interference of noise on the detection results from different scales and eliminates erroneous detection results from top to bottom to obtain a unified mechanism with more reliable prediction values.

[0079] The above exclusion process can be understood as a process of suppressing the interference of noise on the detection results from different scales and eliminating erroneous detection results from top to bottom.

[0080] In an embodiment of the present invention, the present invention is based on a multi-frame signal adaptive accumulation module of grouped convolution, convolution and Transformer, which can enhance low-contrast targets to a recognizable level by adaptively accumulating signals from continuous multi-frame images. The present invention performs spatial domain weak target detection on continuous multi-frame infrared images, and can adaptively combine the correlation of the position of the target in continuous frames, remove the influence of noise points in a single frame that are extremely similar to the target, and avoid interference with pixel-level or sub-pixel-level target detection. The present invention uses continuous multi-frame grayscale images to achieve accurate detection of the geometric center of weak targets at the pixel level and sub-pixel level under the condition of contrast ≮ 5%. The present invention adopts a unified mechanism for multi-scale detection results, suppresses the interference of noise on the detection results from different scales, and eliminates erroneous detection results from top to bottom to obtain more reliable prediction values. .

[0081] It should be noted that the whole process of the present invention does not require human intervention or other prior information, and the detection results can be obtained based on single-frame or multi-frame infrared images. The present invention effectively improves the detection accuracy of small infrared targets under low contrast to meet the needs of infrared detection systems for long-distance accurate detection.

[0082] The embodiment of the present invention adaptively accumulates signals of the image group to be detected through a multi-frame signal adaptive accumulation module based on a grouped convolution kernel, a Transformer module and a convolution kernel to obtain an image feature matrix with enhanced characterization, and performs multi-scale detection on the image feature matrix with enhanced characterization through a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix, and excludes the multi-scale detection result of the image feature matrix based on a unified mechanism for multi-scale detection results to determine the target detection result of the image group to be detected, thereby solving the problem that the existing multi-frame infrared target detection method has a slow detection speed and cannot meet the requirements of real-time or higher-frequency detection of the infrared detection system, and effectively improves the detection accuracy of small infrared targets under low contrast.

[0083] like Figure 2 As shown, Figure 2 The present invention provides a schematic diagram of a method for detecting a dark, weak, and small moving target in an aerial infrared environment, which includes: adaptive accumulation of multi-frame signals, multi-scale image feature extraction, a unified mechanism for multi-scale results, and small and weak target locking.

[0084] In an embodiment of the present invention, the above-mentioned multi-frame signal adaptive accumulation is a multi-frame signal adaptive accumulation module based on a grouped convolution kernel, a Transformer module and a convolution kernel, which enhances the target representation by adaptively accumulating signals from continuous multi-frame images.

[0085] The above multi-scale image feature extraction uses a small convolutional neural network YOLOX-Tiny to detect targets at multiple scales.

[0086] The above-mentioned multi-scale detection result unification mechanism eliminates erroneous detection results from top to bottom and gives accurate current detection results.

[0087] The above-mentioned small target locking can be understood as a process of separating small targets from background noise and identifying and tracking them through certain algorithms and technologies.

[0088] In the embodiment of the present invention, the present invention adopts a multi-frame signal adaptive accumulation module based on group convolution, convolution and Transformer, which can enhance low-contrast targets to a recognizable level by adaptively accumulating signals from continuous multi-frame images; adopts a multi-scale detection result unification mechanism to suppress the interference of noise on the detection results from different scales, and eliminates erroneous detection results from top to bottom to obtain a more reliable prediction value. The present invention can solve the problem that the existing multi-frame infrared target detection method has a slow detection speed and cannot meet the requirements of real-time or higher-frequency detection of the infrared detection system, and effectively improves the detection accuracy of small infrared targets under low contrast.

[0089] For example, the present invention can perform target detection on infrared dim and small moving targets on a board based on the Huawei Ascend 310B module. The detection delay of the present invention is less than 10ms, which can effectively improve the detection accuracy of small infrared targets under low contrast.

[0090] Optionally, the step of obtaining the image set to be detected includes: obtaining a plurality of consecutive frames of images to be detected; stacking the plurality of consecutive frames of images to be detected on the channel dimension to obtain the image set to be detected imageSet∈R H×W×N :

[0091] imageSet={I i ,I i-1 ,…,I i-N+1} (1)

[0092] Among them, Ii represents the current i-th frame to be detected, where I i ∈R H×W×1 ; H is the image length, W is the image width, 1 is the number of image channels, and N is the number of frames; when there are no N consecutive images, use the all-zero matrix 0 H×W×1 Replace insufficient frames:

[0093] I i-j =0 H×W×1 if ij<0 (2)

[0094] In the embodiment of the present invention, the above-mentioned continuous multiple frames of images to be detected can be understood as several adjacent frames of images in the infrared image sequence.

[0095] The above all-zero matrix can be understood as a matrix with all elements being zero. An end-to-end model can be constructed, which can accept input of any number of frames from a single frame to a set maximum frame, that is, even if there are not enough continuous frames, the target geometric center can be directly predicted based on a small number of frames or a single frame.

[0096] It should be noted that each frame of image has a matrix representing its features, and the matrix size is H×W×1.

[0097] Optionally, based on the grouped convolution kernel, the Transformer module and the convolution kernel multi-frame signal adaptive accumulation module, the step of adaptively accumulating signals for the image group to be detected to obtain the image group to be detected with enhanced representation includes: the image group to be detected is firstly weighted in the time domain in the multi-frame signal adaptive accumulation module of the convolution kernel, and then superimposed in the spatial domain to generate an image feature matrix featureSet∈R representing the enhanced representation H×W×C1 :

[0098] featureSet=F S (F T (imageSet)) (3)

[0099] Among them, F T (·) represents the time domain weighting function, F S (·) represents the spatial superposition function.

[0100] In the embodiment of the present invention, the above-mentioned time domain weighting can be understood as weighted summing of different pixel points according to their time weights. Time domain weighting can make adjacent frames of images in an image sequence more similar, thereby reducing the influence of interference factors such as motion noise.

[0101] The above-mentioned spatial domain superposition can be understood as integrating multiple signals together to form a new and more complex signal. Spatial domain superposition can enhance the features of the image and improve the classification accuracy.

[0102] The above-mentioned time domain weighting function can be understood as a function that can weight the signal at different time points.

[0103] The multi-frame signal adaptive accumulation module of the above convolution kernel can effectively eliminate the position difference between target frames, so that the multi-frame signals can be accurately accumulated at the true value position, enhance the target signal strength, and greatly improve the robustness of detection.

[0104] Optionally, a weighting function F in the time domain T (·) In the image set to be detected, imageSet∈R H×W×N First, five group convolution kernels with a step size of 2 are continuously filtered until the size is reduced to The grouped convolution kernel function DWConv(·) performs the same filtering on each image in the image set imageSet to be detected:

[0105] scaleSet={Is i ,…,Is i-N+1}={DWConv(I i )),…,DWConv(I i-N+1 ))} (4)

[0106] Then after the grouped convolution kernel is reduced The correlation feature maps between different frames will be calculated through the Transformer module.

[0107] In the embodiment of the present invention, the above filtering can be understood as a filtering process of extracting useful information from an input signal while removing useless or interfering information.

[0108] It should be noted that the grouped convolution kernel function DWConv(·) performs the same filtering on each image of the image group imageSet to be detected, but does not superimpose the filtering results at the end, that is, the same filtering and scaling are performed on each frame of the image.

[0109] Optionally, the image set scaleSet to be detected will first pass through three fully connected layers to form a query matrix Answer Matrix Weight Matrix Then the correlation feature graph is obtained through the query mechanism:

[0110] relativeSet=Q×K T ×V={Ir i ,…,Ir i-N+1} (5)

[0111] Finally, after the global average pooling operation Avg(·), the pooled correlation feature map relativeSet is obtained;

[0112] Pooling correlation feature map relativeSet to get α∈N gives α for each frame iAs weighting factor:

[0113]

[0114] We get α = {α i ,…,α i-N+1}, the spatial superposition function F S (·) The weighted N frames are spatially superimposed through the convolution kernel to obtain a variety of feature maps after different filtering:

[0115]

[0116] Among them, C1 is the number of convolution kernel channels, Conv1 represents the first channel of the convolution kernel, that is, the first filtering method; when the number of consecutive frames is insufficient, the weighted image α j ×0 H×W×1 =0 H×W×1 .

[0117] In an embodiment of the present invention, the above-mentioned fully connected layer plays the role of a "classifier" in the convolutional neural network model.

[0118] The above query mechanism can be understood as an operation method for data retrieval and acquisition.

[0119] It should be noted that when the number of consecutive frames is insufficient, the weighted image α j ×0 H×W×1 =0 H×W×1 , due to 0 H×W×1 In the spatial domain superposition, there is no impact on the various feature maps featureSet after different filtering, and a small number of frames or a single frame input has no negative impact on signal accumulation.

[0120] like Figure 3 As shown, Figure 3 It is a structural diagram of an adaptive signal accumulation module provided by an embodiment of the present invention. Specifically, the adaptive signal accumulation module includes a grouped convolution kernel, a Transformer module and a multi-frame signal adaptive accumulation module of the convolution kernel. The input image group will first be weighted in the time domain in the signal accumulation module to exclude possible interference frames, and then superimposed in the spatial domain to generate an image feature matrix with stronger signal representation. The adaptive signal accumulation module can effectively eliminate the position difference between target frames, so that multi-frame signals can be accurately accumulated at the true value position, enhance the target signal strength, and greatly improve the robustness of detection.

[0121] like Figure 4 As shown, Figure 4 1 is a schematic diagram of comparing an original infrared image and an image processed by an adaptive signal accumulation module provided by an embodiment of the present invention. Specifically, Figure 4 The left image in the middle is the original infrared image, and the black frame is the weak target to be detected; Figure 4 The middle right picture is the image processed by the adaptive signal accumulation module. After being processed by the multi-frame signal adaptive accumulation module of the convolution kernel, the target is clearly visible.

[0122] Optionally, the step of performing multi-scale detection on the image feature matrix representing the enhancement by using a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix includes: extracting features of the image feature matrix featureSet representing the enhancement at three scales by using a preset convolutional neural network YOLOX-Tiny to obtain a first feature matrix The second characteristic matrix The third characteristic matrix And use YOLOX-Tiny's classification detection head to generate the target geometric center point confidence prediction matrix from the first feature matrix, the second feature matrix, and the third feature matrix:

[0123] featureCls i =Clshead i (featureset i ) (8)

[0124] in, For featureSet1, an additional regression detection head is used to generate the offset prediction matrix:

[0125] featureReg=Reghead(featureset1) (9)

[0126] in,

[0127] In an embodiment of the present invention, the preset convolutional neural network YOLOX-Tiny is a small convolutional neural network model YOLOX-Tiny preset by the system. The small convolutional neural network model YOLOX-Tiny extracts image features through multiple convolutional layers, and then uses these features to perform target detection.

[0128] The above three scales can be understood as different image resolutions or different levels of the image pyramid.

[0129] The above regression detection head is a structure used to predict the size of the target bounding box when performing target detection. Additional convolutional layers can be added to the tail of the convolutional neural network to perform offset regression prediction.

[0130] like Figure 5 As shown, Figure 5It is an overall structural diagram of an aerial infrared dim and small moving target detection method in an embodiment of the present invention. Specifically, the dotted box on the left side of the figure is an adaptive signal accumulation module, which includes a grouped convolution kernel, a Transformer module, and a multi-frame signal adaptive accumulation module of the convolution kernel; the adaptive signal accumulation module can effectively eliminate the position difference between target frames, so that the multi-frame signals are accurately accumulated at the true value position, enhance the target signal strength, and greatly improve the robustness of the detection. The dotted box on the right side of the figure is a multi-scale detection result unification mechanism. The multi-scale detection result unification mechanism suppresses the interference of noise on the detection results from different scales, eliminates erroneous detection results from top to bottom, and obtains a more reliable prediction value. The one without a dotted box in the figure is a small convolutional neural network YOLOX-Tiny. The present invention uses a small convolutional neural network YOLOX-Tiny to extract features of a plurality of feature maps featureSet after different filtering at three scales to obtain a first feature matrix The second characteristic matrix The third characteristic matrix The classification detection head of the small convolutional neural network YOLOX-Tiny is used to generate the target geometric center point confidence prediction matrix from these three feature matrices.

[0131] Optionally, before performing multi-scale detection on the image feature matrix representing the enhancement by using a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix, the method further includes:

[0132] Get the current frame image to be detected I i H×W×1 and the target geometric center coordinates (x ct ,y ct );The image to be detected in the current frame I i H×W×1 and the target geometric center coordinates (x ct ,y ct ) Generate three scales of confidence matrix truth:

[0133]

[0134] in, int(·) means round down; at the same time, I i H×W×1 With (x ct ,y ct ) Generate the true value of the offset matrix:

[0135]

[0136] in,

[0137] In the embodiment of the present invention, when training the convolutional neural network YOLOX-Tiny, firstly, the current frame to be detected image I i H×W×1 and the target geometric center coordinates (x ct ,y ct ) Generate three scales of true confidence matrix.

[0138] Optionally, supervise the target geometric center point confidence prediction matrix featureCls through the loss function i Continuously approaching gtCls i At the same time, the two channels of the offset prediction matrix featureReg continue to approach gtRegX and gtRegY respectively.

[0139] In an embodiment of the present invention, the above loss function can be understood as a measurement method for measuring the gap between the predicted value and the actual value of the machine learning model. During the training process, the convolutional neural network model reduces the error between the model predicted value and the actual value by optimizing the loss function, thereby improving the prediction performance of the model. The above loss function can be mean square error (MSE), cross entropy loss (Cross-Entropy Loss), mean absolute error (MAE), etc.

[0140] It should be noted that in the training of the convolutional neural network YOLOX-Tiny, the target geometric center point confidence prediction matrix featureCls is supervised by the loss function i Continuously approaching the true value of the confidence matrix gtCls i At the same time, the two channels of the offset prediction matrix featureReg are required to continuously approach the true value of the offset matrix gtRegX and the true value of the offset matrix gtRegY respectively.

[0141] Optionally, the step of performing exclusion processing on the multi-scale detection results of the image feature matrix based on the multi-scale detection result unification mechanism to determine the target detection results of the image group to be detected includes: based on the multi-scale detection result unification mechanism, excluding the multi-scale detection results of the erroneous image feature matrix from top to bottom:

[0142] (x cls ,y cls )=argmax(featureCls3↑↑+featureCls2↑+featureCls1) (12)

[0143] Here, argmax(·) indicates the coordinates of the maximum value found in the feature matrix, and ↑ indicates upsampling with a step size of 2.

[0144] In an embodiment of the present invention, the above-mentioned multi-scale detection result unification mechanism can be understood as a processing strategy, which suppresses the interference of noise on the detection results from different scales and eliminates erroneous detection results from top to bottom to obtain a more reliable prediction value.

[0145] Optionally, first, the target geometric center point confidence prediction third matrix featureCls3 determines an 8×8 region containing the target, and then the target geometric center point confidence prediction second matrix featureCls2 selects a 4×4 region from the 8×8 region containing the target, and then the target geometric center point confidence prediction matrix featureCls further determines a 2×2 final region (x cls ,y cls );

[0146] By the final area (x cls ,y cls ) can get the corresponding offset prediction value:

[0147]

[0148] Finally, the predicted position is obtained by the area value and the offset value:

[0149] (x pre ,y pre )=(x cls +x off ,y cls +y off ) (14)

[0150] like Figure 6 As shown, Figure 6 : This is a diagram of a unified mechanism for multi-scale detection results provided by an embodiment of the present invention. Specifically, first, the third matrix featureCls3 of the target geometric center point confidence prediction determines an 8×8 region containing the target, and then the second matrix featureCls2 of the target geometric center point confidence prediction selects a 4×4 region from the 8×8 region containing the target, and then the target geometric center point confidence prediction matrix featureCls further determines a 2×2 final region (x cls ,y cls ).

[0151] like Figure 7 As shown, Figure 7 An ancient image of target-background contrast provided by an embodiment of the present invention is based on Figure 7 The calculated contrast between the target and the background range shown can reflect the detection difficulty to a certain extent. The calculation method is: Where Avg(·) represents the average value of pixels in the calculation area, and targets with contrast <100.0% are usually defined as extremely weak targets. Table 1 shows the detection results of the single-frame detection algorithm YOLOX-tiny and the present invention at different contrasts. The results show that when the contrast is greater than 15.0%, YOLOX-tiny and the present invention present similar results; however, when the contrast decreases further, the results of the present invention are much better than YOLOX-tiny, which illustrates the detection reliability of the present invention at long distances and extremely low contrast. The present invention was used to detect 1268 images of small, weak infrared targets with a size of and an average contrast of 11.7%. The detection rate was 95.3%, the false alarm rate was 0.7%, and the average offset value of the geometric center was only 0.8, which is beneficial to further improve the detection distance and anti-interference capability of the infrared detection system.

[0152] Table 1 Detection results of the present invention and YOLOX-tiny at different contrasts

[0153]

[0154] In an embodiment of the present invention, the present invention performs spatial domain weak small target detection on continuous multi-frame infrared image virtual images, and can adaptively combine the correlation of the target's position in continuous frames, remove the influence of noise points in a single frame that are extremely similar to the target, and avoid interference with pixel-level or sub-pixel-level target detection. The present invention uses continuous multi-frame grayscale images to achieve accurate detection of the geometric center of weak small targets at the pixel level and sub-pixel level under the condition of contrast ≮ 5%; the present invention can accept input of any number of frames from a single frame to a set maximum frame, that is, in the absence of sufficient continuous frames, the target geometric center can also be directly predicted based on a small number of frames or single-frame images. The present invention can effectively improve the detection accuracy of small infrared targets under low contrast to meet the needs of infrared detection systems for long-distance accurate detection.

[0155] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the computer-readable storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0156] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A method for detecting dark and weak small moving infrared targets in the air, characterized in that: The method comprises: Obtaining a group of images to be detected; Based on the grouped convolution kernel, the Transformer module and the multi-frame signal adaptive accumulation module of the convolution kernel, adaptively accumulating signals of the image group to be detected to obtain an image feature matrix representing the enhancement; Performing multi-scale detection on the image feature matrix of the representation enhancement through a preset convolutional neural network YOLOX-Tiny to obtain a multi-scale detection result of the image feature matrix; Based on the multi-scale detection result unification mechanism, the multi-scale detection results of the image feature matrix are excluded to determine the target detection results of the to-be-detected image group.

2. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 1, characterized in that: The step of obtaining the image group to be detected includes: Acquire multiple consecutive frames of images to be detected; The continuous multiple frames of images to be detected are superimposed on the channel dimension to obtain the image set to be detected imageSet∈R H×W×N : imageSet={I i ,I i-1 ,…,I i-N+1 } (1) Among them, Ii represents the current i-th frame to be detected, where I i ∈R H×W×1 , H is the image length, W is the image width, 1 is the number of image channels; N is the number of frames; when there are no N consecutive frames, use the all-zero matrix 0 H×W×1 Replace insufficient frames: I i-j =0 H×W×1 if i-j<0 (2) 3. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 2, characterized in that: The multi-frame signal adaptive accumulation module based on the grouped convolution kernel, the Transformer module and the convolution kernel adaptively accumulates signals on the image group to be detected to obtain the image group to be detected with enhanced characterization, including: The image group to be detected will first be weighted in the time domain in the multi-frame signal adaptive accumulation module of the convolution kernel, and then superimposed in the spatial domain to generate an image feature matrix representing the enhancement. featureSet=F S (F T (imageSet)) (3) Among them, F T (·) represents the time domain weighting function, F S (·) represents the spatial superposition function.

4. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 3, characterized in that: In the time domain, the weighting function F T (·) In the image set to be detected, imageSet∈R H×W×N First, five group convolution kernels with a step size of 2 are continuously filtered until the size is reduced to The grouped convolution kernel function DWConv(·) performs the same filtering on each image in the image set imageSet to be detected: scaleSet={Is i ,…,Is i-N+1 }={DWConv(I i )),…,DWConv(I i-N+1 ))} (4) Then after the grouped convolution kernel is reduced The correlation feature maps between different frames will be calculated through the Transformer module.

5. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 4, characterized in that: The image group scaleSet to be detected will first pass through three fully connected layers to form a query matrix Answer Matrix Weight Matrix Then the correlation feature graph is obtained through the query mechanism relativeSet=Q×K T ×V={And i ,…,And i-N+1 } (5) Finally, after the global average pooling operation Avg(·), the pooled correlation feature map relativeSet is obtained; Pooling correlation feature map relativeSet to get α∈N gives α for each frame i As weighting factor: We get α = {α i ,…,α i-N+1 }, the spatial superposition function F S (·) The weighted N frames are spatially superimposed through the convolution kernel to obtain a variety of feature maps after different filtering: Among them, C1 is the number of convolution kernel channels, Conv1 represents the first channel of the convolution kernel, that is, the first filtering method; when the number of consecutive frames is insufficient, the weighted image α j ×0 H×W×1 =0 H×W×1 .

6. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 5, characterized in that: The multi-scale detection of the image feature matrix characterized by enhancement is performed by using the preset convolutional neural network YOLOX-Tiny to obtain the multi-scale detection result of the image feature matrix, including: The features of the image feature matrix featureSet representing the enhancement are extracted at three scales through the preset convolutional neural network YOLOX-Tiny to obtain the first feature matrix The second characteristic matrix The third characteristic matrix The preset convolutional neural network YOLOX-Tiny classification detection head is used to generate the target geometric center point confidence prediction matrix from the first feature matrix, the second feature matrix, and the third feature matrix: featureCls i =Clshead i (featureset i ) (8) in, A regression detection head is additionally used for the first feature matrix featureSet1 to generate the offset prediction matrix: featureReg=Reghead(featureset1) (9) Among them, the offset prediction matrix 7. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 6, characterized in that: Before performing multi-scale detection on the image feature matrix representing the enhancement by using the preset convolutional neural network YOLOX-Tiny to obtain the multi-scale detection result of the image feature matrix, the method further includes: Get the current frame image to be detected I i H×W×1 and the target geometric center coordinates (x ct ,y ct ); The current frame image to be detected I i H×W×1 and the target geometric center coordinates (x ct ,y ct ) Generate three scales of confidence matrix truth: in, int(·) means round down; at the same time, I i H×W×1 With (x ct ,y ct ) Generate the true value of the offset matrix: in, 8. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 7, characterized in that: Supervise the target geometric center point confidence prediction matrix featureCls through the loss function i Continuously approaching gtCls i At the same time, the two channels of the offset prediction matrix featureReg continuously approach the true value of the offset matrix gtRegX and the true value of the offset matrix gtRegY respectively.

9. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 8, characterized in that: The multi-scale detection result unified mechanism based on the multi-scale detection result performs exclusion processing on the multi-scale detection result of the image feature matrix to determine the target detection result of the image group to be detected, including: Based on the unified mechanism of multi-scale detection results, the multi-scale detection results of the erroneous image feature matrix are eliminated from top to bottom: (x cls ,y cls )=argmax(featureCls3↑↑+featureCls2↑+featureCls1) (12) Here, argmax(·) indicates the coordinates of the maximum value found in the feature matrix, and ↑ indicates upsampling with a step size of 2.

10. The method for detecting a dark, weak, and small moving infrared target in the air as claimed in claim 9, characterized in that: First, the third matrix featureCls3 of the target geometric center point confidence prediction determines an 8×8 region containing the target. Then, the second matrix featureCls2 of the target geometric center point confidence prediction selects a 4×4 region from the 8×8 region containing the target. Then, the target geometric center point confidence prediction matrix featureCls further determines a 2×2 final region (x cls ,y cls ); The corresponding offset prediction value can be obtained from the final area: Finally, the predicted position is obtained by the area value and the offset value: (x pre ,and pre )=(x cls +x off ,and cls +y off ) (14)。

Images