Weak small target detection method combining structure tensor of image and self-attention mechanism

Abstract

This invention relates to the field of object tracking and discloses a weak target detection method that combines image structure tensor and self-attention mechanism. The method includes establishing a background suppression function model based on the characteristics of the structure tensor to suppress strong edge contours in the image; and establishing a new mathematical model for background suppression based on the self-attention mechanism to further constrain the background, thus obtaining the final detection result. The background suppression function model based on the characteristics of the structure tensor established in this invention can suppress the background based on the information features of feature values ​​under different backgrounds, and it can better match the suppression effect of strong edge contours compared with existing background suppression functions. Furthermore, the algorithm of this invention achieves excellent results in terms of signal-to-noise ratio, structural similarity, and background suppression factor.

Description

Technical Field

[0001] This invention relates to the field of object tracking, and more particularly to a method for detecting weak targets using a combined image structure tensor and a self-attention mechanism. Background Technology

[0002] The detection of small infrared targets has wide applications in the national economy and military. However, due to factors such as atmospheric radiation, operating distance, and photoelectric interference, the targets in the image are small in size, appear as dots, and have no contour texture features. Sometimes they are even submerged in complex backgrounds. Strong noise and clutter can also affect the algorithm's judgment, making the detection of small targets very difficult.

[0003] To effectively suppress most of the background while enhancing the target signal, this invention provides a weak target detection method that combines the structural tensor of the image with a self-attention mechanism. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a weak target detection method that combines the structural tensor of the image with a self-attention mechanism, including:

[0005] A background suppression function model based on the properties of the structural tensor is established to initially suppress strong edge contours in the image;

[0006] A mathematical model of the self-attention mechanism is established to further constrain the background of the image based on the background suppression function model based on the structural tensor properties.

[0007] Furthermore, the specific steps for establishing the background suppression function model based on the structural tensor properties are as follows:

[0008] Step 1: Obtain the feature values ​​of the image using the structure tensor;

[0009] Step 2: Establish a strong edge contour suppression function based on the characteristics of the eigenvalues ​​obtained from the structure tensor, wherein the strong edge contour suppression function is used to suppress strong edge contours in the image;

[0010] Step 3: Combine the strong edge contour suppression function and the corner description function to suppress the strong edge contour of the background, wherein the corner description function is used to calculate the corner position of the image.

[0011] Furthermore, in step 1, assuming the original image is D, the classical formula for calculating the structure tensor is:

[0012] K ρ This represents a Gaussian kernel function with variance ρ, and * represents convolution operation. Represents the Kronecker product. This indicates calculating the gradient, D. σ This indicates that a Gaussian smoothing filter with variance σ is applied to the original image, where J11, J12, J21, and J22 are J P The four elements of the structure tensor matrix, where,

[0013]

[0014] I x ,I y D σ The gradient in the x and y directions. D σ The partial derivatives of .

[0015] Furthermore, the structure tensor is positive definite and symmetric. At each pixel location in the image, the structure tensor possesses two non-negative eigenvalues. For J in the image p Specifically, its two eigenvalues ​​λ1 and λ2 are calculated using the following formulas:

[0016]

[0017] Among them, eigenvalues It is a matrix-vector space of size m*n.

[0018] Furthermore, in step 2, a strong edge contour suppression function is established based on the behavior of the eigenvalues ​​λ1(x,y) and λ2(x,y) of the image's structural tensor matrix in different regions: when λ1(x,y)≈λ2(x,y)≈0, it is a flat region; when λ1(x,y)≥λ2(x,y)>>0, it is a corner region; and when λ1(x,y)>>λ2(x,y)≈0, it is an edge region. The expression of this function is as follows:

[0019]

[0020] In the above formula, W q Let λ1(x,y) and λ2(x,y) represent the strong edge contour suppression function, and let λ1(x,y) and λ2(x,y) represent the pixel values ​​of the feature matrix of the image at (x,y).

[0021] Furthermore, in step 3, the background suppression result obtained by combining the corner description function is as follows:

[0022]

[0023] W in the above formula q W t These are the edge contour suppression function and the corner description function, respectively. qtThis indicates the initial background suppression result obtained by the present invention.

[0024] Furthermore, the process of establishing the mathematical model of the self-attention mechanism is as follows:

[0025] First, the image matrix is ​​divided into blocks using a sliding window to obtain the block image matrix X;

[0026] Then we take the block image matrix X and its transpose X. T Multiplication is used to enhance the energy of the target and suppress its edge contours;

[0027] Finally, we will X*X T The normalized result is multiplied by the original block image matrix X as a weight matrix to constrain the original block image matrix X and achieve the effect of suppressing background edge residuals.

[0028] Furthermore, the X*X T The formula for multiplying the normalized result, which is used as the weight matrix, by the original block image matrix X is as follows:

[0029] A formula for a self-attention mechanism based on non-overlapping infrared image patches:

[0030] Attention(X) = Softmax(X*X) T )⊙X (5)

[0031] In the above formula, X represents the non-overlapping infrared block image matrix, X T The transpose of the block image matrix X is represented by , * and ⊙, which represent matrix multiplication and the Hadamard product, respectively. Softmax() is the normalization function.

[0032] A computer device includes a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method as described in any of the preceding claims.

[0033] A computer-readable storage medium storing computer code that, when executed, is performed as described in any of the preceding methods.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] (1) The background suppression function model based on the structural tensor properties established in this invention can suppress the background according to the information features of the feature value under different backgrounds. Compared with the existing background suppression function, it can better match the suppression effect of strong edge contours.

[0036] (2) The mathematical model based on the self-attention mechanism established in this invention can analyze the representation of edge contours and target points in the image matrix. By using the mathematical model based on the self-attention mechanism, the continuous edge contours of the image are further constrained, and the target point information is preserved, resulting in a better background suppression effect. Attached Figure Description

[0037] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0038] In the attached diagram:

[0039] Figure 1 This is a diagram of a background constraint algorithm model based on a self-attention mechanism for weak target detection, which combines the structural tensor of the image and the self-attention mechanism of the present invention.

[0040] Figure 2 The figure shows the experimental results of the algorithm for weak target detection method that combines image structure tensor and self-attention mechanism according to the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only for explaining this invention and are not intended to limit this invention.

[0042] Example 1

[0043] This invention provides a method for weak target detection based on a joint image structure tensor and a self-attention mechanism, comprising:

[0044] A background suppression function model based on the properties of the structural tensor is established to initially suppress strong edge contours in the image;

[0045] A mathematical model of the self-attention mechanism is established to further constrain the background of the image based on the background suppression function model based on the properties of the structure tensor.

[0046] Specifically, the steps for establishing a background suppression function model based on the properties of the structure tensor are as follows:

[0047] Step 1: Obtain the feature values ​​of the image using the structure tensor;

[0048] Step 2: Based on the characteristics of the eigenvalues ​​obtained from the structure tensor, establish a strong edge contour suppression function to suppress strong edge contours in the image;

[0049] Step 3: Combine the strong edge contour suppression function and the corner point description function to suppress the strong edge contour of the background.

[0050] Specifically, in step 1, assuming the original image is D, the classic formula for calculating the structure tensor is:

[0051]

[0052] K ρ This represents a Gaussian kernel function with variance ρ, and * represents convolution operation. Represents the Kronecker product. This indicates calculating the gradient, D. σ This indicates that a Gaussian smoothing filter with variance σ is applied to the original image, where J11, J12, J21, and J22 are J P The four elements of the structure tensor matrix, where,

[0053]

[0054] I x ,I y D σ The gradient in the x and y directions. D σ The partial derivatives of .

[0055] Specifically, the structure tensor is positive definite and symmetric. At each pixel location in the image, the structure tensor possesses two non-negative eigenvalues. For J in the image p Specifically, its two eigenvalues ​​λ1 and λ2 are calculated using the following formulas:

[0056]

[0057] Among them, eigenvalues It is a matrix-vector space of size m*n.

[0058] Specifically, in step 2, a strong edge contour suppression function is established based on the behavior of the eigenvalues ​​λ1(x,y) and λ2(x,y) of the image's structure tensor matrix in different regions: when λ1(x,y)≈λ2(x,y)≈0, it is a flat region; when λ1(x,y)≥λ2(x,y)>>0, it is a corner region; and when λ1(x,y)>>λ2(x,y)≈0, it is an edge region. The expression of this function is as follows:

[0059]

[0060] In the above formula, W qLet λ1(x,y) and λ2(x,y) represent the strong edge contour suppression function, and let λ1(x,y) and λ2(x,y) represent the pixel values ​​of the feature matrix of the image at (x,y).

[0061] Specifically, in step 3, the background suppression result obtained by combining the corner description function is as follows:

[0062]

[0063] W in the above formula q W t These are the strong edge contour suppression function and the corner description function, respectively. qt This indicates the initial background suppression result obtained by the present invention.

[0064] Specifically, the process of establishing a mathematical model of the self-attention mechanism is as follows:

[0065] First, the image matrix is ​​divided into blocks using a sliding window to obtain the block image matrix X;

[0066] Then we take the block image matrix X and its transpose X. T Multiplication is used to enhance the energy of the target and suppress its edge contours;

[0067] Finally, we will X*X T The normalized result is multiplied by the original block image matrix X as a weight matrix to constrain the original block image matrix X and achieve the effect of suppressing background edge residuals.

[0068] Specifically, X*X T The formula for multiplying the normalized result, which is used as the weight matrix, with the original block image matrix X is as follows:

[0069] A formula for a self-attention mechanism based on non-overlapping infrared image patches:

[0070] Attention(X) = Softmax(X*X) T )⊙X (5)

[0071] In the above formula, X represents the non-overlapping infrared block image matrix, X T The transpose of the block image matrix X is represented by , * and ⊙, which represent matrix multiplication and the Hadamard product, respectively. Softmax() is the normalization function.

[0072] In the specific implementation process, such as Figure 1 As shown, the input is the background suppression function model result based on the structural tensor properties. The image matrix is ​​divided into blocks using a sliding window to obtain the block image matrix X. t ; the block image matrix X tand its transpose X t T Multiply to get X t ', will X t 'Normalization result X' t "As the weight matrix and the original block image matrix X" t After multiplication, the image matrix of each block is calculated using formula (5) to obtain the result of the self-attention mechanism operation.

[0073] like Figure 2 As shown, the experimental results of the weak target detection method based on the combined image structure tensor and self-attention mechanism of this invention are presented. (a), (b), and (c) represent the original image, the background suppression result image, and the final background constraint result, respectively. The evaluation metrics for the three scenarios are as follows:

[0074] Scene SNR (Signal-to-Noise Ratio) BSF (Background Suppression Factor) SSIM (Structural Similarity) 1 10.07 218.77 0.999 2 13.4 263.04 0.9993 3 11.86 324.01 0.9995

[0075] Among them, the larger the signal-to-noise ratio (SNR) value, the better the image quality. The larger the background suppression factor (PSF), the better the background suppression effect. The range of SSIM value is [0,1]. The larger the value, the more similar the images are. As can be seen from the data in the table, the present invention has achieved good results in terms of signal-to-noise ratio, structural similarity and background suppression factor.

[0076] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting weak targets by combining the structural tensor of an image and a self-attention mechanism, characterized in that, include: A background suppression function model based on the properties of the structure tensor is established to initially suppress strong edge contours in the image. The specific steps are as follows: Step 1: Obtain the feature values ​​of the image using the structure tensor; Step 2: Establish a strong edge contour suppression function based on the characteristics of the eigenvalues ​​obtained from the structure tensor, wherein the strong edge contour suppression function is used to suppress strong edge contours in the image; In step 2, the eigenvalues ​​of the image's structural tensor matrix are used. Performance in different regions, i.e., when When it is a flat area; when When λ1(x,y) >> λ2(x,y) ≈ 0, it represents the corner region; when λ1(x,y) >> λ2(x,y) ≈ 0, it represents the edge region. The strong edge contour suppression function is established, and its expression is as follows: （3） In the above formula This represents the strong edge contour suppression function. The feature matrix of the image is represented in Pixel value at; Step 3: Combine the strong edge contour suppression function and the corner description function to suppress the strong edge contour of the background, wherein the corner description function is used to calculate the corner position of the image; A mathematical model of the self-attention mechanism is established to further constrain the background of the image based on the background suppression function model based on the properties of the structure tensor.

2. The weak target detection method based on the structural tensor and self-attention mechanism of the joint image as described in claim 1, characterized in that: In step 1, it is assumed that the original image is The classic formula for calculating the structure tensor is: （1） The variance is expressed as Gaussian kernel function, This represents the convolution operation. Represents the Kronecker product. This indicates calculating the gradient. This indicates that the variance of the original image is... Gaussian smoothing filter, J11, J12, J21, J22 are The four elements of the structure tensor matrix, where, , They represent The gradient in the x and y directions. for The partial derivatives of .

3. The weak target detection method based on the structural tensor and self-attention mechanism of the joint image as described in claim 2, characterized in that: The structure tensor is positive definite and symmetric. At each pixel location in the image, the structure tensor possesses two non-negative eigenvalues. For the image In other words, its two eigenvalues , The calculation formula is as follows: （2） in, , , It is a matrix-vector space of size m*n.

4. The weak target detection method based on the structural tensor and self-attention mechanism of the joint image as described in claim 2, characterized in that: In step 3, the background suppression result obtained by combining the corner description function is as follows: （4） In the above formula These are the strong edge contour suppression function and the corner point description function, respectively. This indicates the initial background suppression result.

5. The weak target detection method based on the structural tensor and self-attention mechanism of the joint image as described in claim 1, characterized in that: The process of establishing the mathematical model of the self-attention mechanism is as follows: First, the image matrix is ​​divided into blocks using a sliding window to obtain a block image matrix. ; Then we take the block image matrix X and its transpose X. T Multiplication is used to enhance the energy of the target and suppress its edge contours; Finally we will The normalized result is used as the weight matrix and the original block image matrix. Multiply to achieve the same effect on the original block image matrix. The constraints achieve the effect of suppressing background edge residuals.

6. The weak target detection method based on the structural tensor and self-attention mechanism of the joint image as described in claim 5, characterized in that: The The normalized result is used as the weight matrix and the original block image matrix. The formula for multiplication is: A formula for a self-attention mechanism based on non-overlapping infrared image patches: （5） In the above formula X represents the block image matrix. T This represents the transpose of the block image matrix X. and Let these represent matrix multiplication and the Hadamard product, respectively. This is the normalization function.

7. A computer device comprising a memory and one or more processors, the memory storing computer code that, when executed by the one or more processors, causes the one or more processors to perform the method as described in any one of claims 1 to 6.

8. A computer-readable storage medium storing computer code, wherein when the computer code is executed, the method of any one of claims 1 to 6 is performed.

Images