An infrared sea surface target detection method based on long-short channel features and non-original state decomposition

Abstract

The application provides an infrared sea surface target detection method based on long-short channel features and non-original state decomposition, which comprises the following steps: long channel time sequence information is extracted, and long channel structure tensor weight is designed to improve the prominence of the target; pixel coding is performed on the previous frame result as a short channel to obtain short channel pixel coding weight to suppress sea wave interference; an optimization method of non-original state pre-decomposition is designed; long-short channel weight is used to preliminarily separate the target and the background, so that the iteration number of the model is reduced; the preliminarily separated target and the background are substituted into the alternating direction multiplier method for iterative solution, and the original image is completely decomposed into the target and the background. The method provided by the application is based on a sparse low-rank decomposition method, improves the target detection accuracy in a strong sea wave interference background, reduces the iteration number in the iterative solution process, reduces the sea wave clutter interference under the premise of ensuring the detection rate, and effectively improves the running time of the sparse low-rank decomposition method.

Description

Technical Field

[0001] This invention relates to the field of infrared remote sensing image processing technology, and more particularly to an infrared sea surface target detection method based on long and short channel features and non-original state decomposition. Background Technology

[0002] Infrared sensors play a crucial role in special scenarios such as nighttime and foggy weather due to their imaging characteristics, and are therefore widely used in maritime search and rescue. Distant targets on the sea surface typically occupy only a small number of pixels in an image, appearing as a few small bright spots that are easily lost in complex backgrounds. Especially when the camera lens is at a specific angle to the sun, the sunlight reflected from the waves will create numerous small bright spots in the image; these spots share similar characteristics with the target and are easily misdetected.

[0003] To improve detection accuracy and enhance the search capabilities for distressed vessels at sea, researchers have proposed various detection methods. Single-frame detection methods have relatively short runtimes, but because they do not consider multi-frame information, their ability to suppress strong wave interference is relatively poor.

[0004] Multi-frame detection methods include: filtering-based methods such as 3D matched filtering, particle filtering, and attenuated sine wave filtering; methods based on human visual saliency such as spatiotemporal local contrast, spatiotemporal local difference, and spatiotemporal saliency; and methods based on low-rank sparse decomposition such as spatiotemporal weighted variation and spatiotemporal tensor. Among these, filtering-based methods often have poor detection performance. Methods based on human visual saliency often assume that the background changes slowly while the target moves quickly, which does not match the assumption of slow-moving small targets on the infrared sea surface. Methods based on low-rank sparse decomposition often directly stack multiple images into a tensor and then use it in optimization methods, which greatly increases the running time. Although many proposed methods have improved the detection accuracy of infrared target detection, there is still considerable room for research on the problem of small target detection on the infrared sea surface. In particular, how to suppress a large amount of strong sea wave interference without losing the true target is of great research significance for the search for distressed targets on the sea surface. Summary of the Invention

[0005] To address the aforementioned technical problem of suppressing background interference in infrared sea surface target detection under strong wave interference, this invention provides an infrared sea surface target detection method based on long and short channel features and non-original state decomposition. This invention utilizes multi-frame input images to design long-channel structural tensor weights and uses the results of the previous frame to design short-channel pixel encoding weights. These weights are used to initially decompose the current frame input image into target and background parts. Then, the alternating direction multiplier method is used for iterative solution to completely separate the pre-decomposed target and background, obtaining the final detection result.

[0006] The technical means employed in this invention are as follows:

[0007] An infrared sea surface target detection method based on long and short channel features and non-original state decomposition includes:

[0008] S1. Construct the objective function for the sparse low-rank model;

[0009] S2. Based on the temporal consistency of infrared sea surface targets and the temporal variability of wave interference, construct long-channel structural tensor weights.

[0010] S3. Construct short-channel pixel encoding weights;

[0011] S4. Based on the long-channel structural tensor weights and the short-channel pixel encoding weights, construct the target component weights, and use a sliding window to stack the original image and the target component weights into a three-dimensional tensor respectively.

[0012] S5. Using the target component weight tensor, perform a non-original state pre-decomposition operation on the three-dimensional tensor of the original image, and pre-decompose the three-dimensional tensor of the original image into preliminary target components and background components.

[0013] S6. Using the alternating direction multiplier method, the initial target components and background components are solved iteratively.

[0014] S7. Decompose the three-dimensional tensor of the solution into matrix form to obtain the final result.

[0015] Further, step S1 specifically includes:

[0016] S11. Let the original input image matrix be D, then the patch tensor of D is defined as... If we treat the patch tensor of an original image as a linear sum of the target and background components, then we have:

[0017]

[0018] in, Indicates background components, Indicates the target component;

[0019] S12. Considering the low-rank characteristics of the background and the sparsity of the target, the formula in step S11 is transformed into the following minimization problem:

[0020]

[0021] in, Let λ represent the estimate of the tensor rank of the background patch, and let λ represent the trade-off parameter. ⊙ represents the weight of the target component, and ⊙ represents the Hadamard product.

[0022] Further, step S2 specifically includes:

[0023] S21. Based on the temporal consistency of infrared sea surface targets and the temporal variability of wave interference, let the channel length be l and the current frame be D. l The first l-1 frames are D1 to D l-1 Long channel structure tensor weight W LST Designed as:

[0024]

[0025] Where S represents the saliency map obtained by replacing the original image with the Hadamard product of the original image and the average of the previous l-1 frames. ST represents the calculation of the structure tensor;

[0026] S22. Calculate the structure tensor of a matrix X using the following formula:

[0027]

[0028] Among them, K ρ D represents a Gaussian kernel with variance ρ. σ D represents Gaussian filtering smoothing with variance σ. σ The horizontal and vertical gradients are respectively:

[0029] S23. Calculate the eigenvalue matrix of the structure tensor ST(X) of matrix X. The calculation formula is as follows:

[0030]

[0031] Further, step S3 specifically includes:

[0032] S31. Perform dilation processing on the result of the previous frame of the current frame. Let the binarized result of the previous frame be R. l-1 The expansion operation can be represented as:

[0033]

[0034] in, Indicates the expansion operation, E s represents a square structuring element, and s represents the size of the square structuring element;

[0035] S32、R' l-1 The pixels in the array are assigned different values ​​based on their relative positions, and this value is named the short-channel pixel encoding weight W. SPE :

[0036]

[0037] Where i represents the i-th row of the matrix, rmin R' l-1 The minimum number of rows containing non-zero elements, r max R' l-1 The maximum number of rows in the array, where α represents a small positive constant.

[0038] Further, step S4 specifically includes:

[0039] After obtaining the weights of the long and short channels, the target component weights are constructed by combining the two. The corresponding matrix W is represented as:

[0040]

[0041] Where J is a matrix of all ones. This indicates element-wise division.

[0042] Further, step S5 specifically includes:

[0043] Before iteratively solving, the original image Pre-decomposed into target tensor and background tensor as follows:

[0044]

[0045]

[0046] Further, step S6 specifically includes:

[0047] S61. Design the augmented Lagrange function as follows:

[0048]

[0049] in, represents the Lagrange multiplier, <·> represents the inner product, and μ represents the penalty factor;

[0050] S62. The designed augmented Lagrangian function is divided into several subproblems. Let the number of iterations be k. The target component is iterated to the (k+1)th iteration and updated as follows:

[0051]

[0052] Where S represents the soft threshold operator;

[0053] S63, Set There exists a minimization problem, as follows:

[0054]

[0055] S64. Solve the minimization problem in step S63 using the soft threshold operator. The solution formula is as follows:

[0056] S τ (x)-sign(x)×max(|x|-τ,0)

[0057] S65, the background component is updated to the (k+1)th iteration as follows:

[0058]

[0059] Where N represents the first N protected singular values, and P represents the partial singular value threshold operator;

[0060] S66, Let If Y = Y1 + Y2, then there exists a minimization problem, as follows:

[0061]

[0062] S67. Solve the minimization problem in step S66 using the singular value threshold operator. The solution formula is as follows:

[0063]

[0064] in, S represents the singular value threshold operator;

[0065] S68, will and μ k+1 They were updated to:

[0066]

[0067] μ k+1 =ρμ k

[0068] Where ρ is a constant, the iteration stops when the following condition is met:

[0069]

[0070] Further, step S7 specifically includes:

[0071] The target tensor obtained after the iteration stops is reversed to restore it into a matrix, thus obtaining the final detection result.

[0072] Compared with the prior art, the present invention has the following advantages:

[0073] 1. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition provided by this invention, compared with the traditional multi-frame sparse low-rank method, does not directly input the original sequence image into the sparse low-rank tensor model, but instead pre-constructs a single matrix weight using multi-frame information. This design can both use multi-frame information to suppress sea wave interference and make the complexity of this invention comparable to that of a single-frame model.

[0074] 2. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition provided by this invention adds additional positional information to the pixel encoding weight designed by utilizing the positional difference between the distant small target and the sea wave interference at the sea-line, which is more significantly suppressed by the influence of sea wave interference compared to general methods.

[0075] 3. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition provided by the present invention pre-divides the original image into incompletely decomposed target components and background components using weights before solving the model. This design can significantly reduce the number of iterations compared to the general solution method starting from scratch, thereby reducing the model's running time.

[0076] In summary, this invention can be widely applied to fields such as target detection. Besides effectively suppressing strong wave interference in infrared sea surface images, this invention utilizes a pre-decomposition strategy to reduce the number of iterations in the solution process of sparse low-rank models, which can be applied to many models solved using the alternating direction multiplier method. Attached Figure Description

[0077] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0078] Figure 1 This is a flowchart of the method of the present invention.

[0079] Figure 2 This is a schematic diagram of the long-channel structure tensor weights provided in an embodiment of the present invention.

[0080] Figure 3 This is a schematic diagram of short-channel pixel encoding weights provided in an embodiment of the present invention.

[0081] Figure 4 This is a schematic diagram of a matrix-constructed three-dimensional patch tensor provided in an embodiment of the present invention.

[0082] Figure 5A comparison between the conventional alternating direction multiplier method (ADMM) provided for embodiments of the present invention and the ADMM based on non-original state pre-decomposition (NOSD) in the present invention.

[0083] Figure 6 The image shows a comparison of the detection results of the present invention with those of various comparative methods, provided for embodiments of the present invention.

[0084] Figure 7 ROC curves of the present invention and various comparative methods provided for embodiments of the present invention.

[0085] Figure 8 A comparison chart of the average running time of the present invention and various other methods provided for embodiments of the present invention. Detailed Implementation

[0086] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0087] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0088] like Figure 1 As shown, this invention provides an infrared sea surface target detection method based on long and short channel features and non-original state decomposition, comprising:

[0089] S1. Construct the objective function for the sparse low-rank model;

[0090] S2. Based on the temporal consistency of infrared sea surface targets and the temporal variability of wave interference, construct long-channel structural tensor weights.

[0091] S3. Construct short-channel pixel encoding weights;

[0092] S4. Based on the long-channel structural tensor weights and the short-channel pixel encoding weights, construct the target component weights, and use a sliding window to stack the original image and the target component weights into a three-dimensional tensor respectively.

[0093] S5. Using the target component weight tensor, perform a non-original state pre-decomposition operation on the three-dimensional tensor of the original image, and pre-decompose the three-dimensional tensor of the original image into preliminary target components and background components.

[0094] S6. Using the alternating direction multiplier method, the initial target components and background components are solved iteratively.

[0095] S7. Decompose the three-dimensional tensor of the solution into matrix form to obtain the final result.

[0096] In a specific implementation, as a preferred embodiment of the present invention, step S1 specifically includes:

[0097] S11. Stacking matrices into a 3D patch tensor using a sliding window approach and then solving it using sparse low-rank decomposition is considered an efficient processing method. Let the original input image matrix be D, then the patch tensor of D is defined as... If we treat the patch tensor of an original image as a linear sum of the target and background components, then we have:

[0098]

[0099] in, Indicates background components, Indicates the target component;

[0100] S12. Considering the low-rank characteristics of the background and the sparsity of the target, the formula in step S11 is transformed into the following minimization problem:

[0101]

[0102] in, The tensor rank of the background patch is estimated using the tensor nuclear norm part as the tensor rank estimation method. λ represents the trade-off parameter. This represents the weights of the target components, guiding the model to obtain more accurate results. ⊙ represents the Hadamard product.

[0103] In a specific implementation, as a preferred embodiment of the present invention, step S2 involves constructing the long-channel structure tensor weights as follows: Figure 2 As shown, it specifically includes:

[0104] S21. Based on the temporal consistency of infrared sea surface targets and the temporal variability of wave interference, let the channel length be l and the current frame be D. l The first l-1 frames are D1 to Dl-1 Long channel structure tensor weight W LST Designed as:

[0105]

[0106] Where S represents the saliency map obtained by replacing the original image with the Hadamard product of the original image and the average of the previous l-1 frames. ST represents the calculation of the structure tensor; Figure 2 W can be seen in the middle LST It can significantly improve the salience of the target.

[0107] S22. Calculate the structure tensor of a matrix X using the following formula:

[0108]

[0109] Among them, K ρ D represents a Gaussian kernel with variance ρ. σ D represents Gaussian filtering smoothing with variance σ. σ The horizontal and vertical gradients are respectively:

[0110] S23. Calculate the eigenvalue matrix of the structure tensor ST(X) of matrix X. The calculation formula is as follows:

[0111]

[0112] In a specific implementation, as a preferred embodiment of the present invention, step S3 involves constructing short-channel pixel coding weights as follows: Figure 3 As shown, it specifically includes:

[0113] S31. Dilate the result of the previous frame in the current frame to prevent target suppression due to inconsistent target positions between the two frames. Let R be the binarized result of the previous frame. l-1 The expansion operation can be represented as:

[0114]

[0115] in, Indicates the expansion operation, E s represents a square structuring element, and s represents the size of the square structuring element;

[0116] S32、R' l-1 The pixels in the array are assigned different values ​​based on their relative positions, and this value is named the short-channel pixel encoding weight W. SPE :

[0117]

[0118] Where i represents the i-th row of the matrix, r min R' l-1 The minimum number of rows containing non-zero elements, r max R' l-1 The maximum number of rows in the array, where α represents a small positive constant. If α is 0, it will completely suppress the 0-value region, which may affect the detection of newly emerging targets. Figure 3 W can be seen in the middle SPE It can significantly suppress strong wave interference.

[0119] In a specific implementation, as a preferred embodiment of the present invention, step S4 specifically includes:

[0120] After obtaining the weights of the long and short channels, the target component weights are constructed by combining the two. The corresponding matrix W is represented as:

[0121]

[0122] Where J is a matrix of all ones. This indicates element-wise division.

[0123] In a specific implementation, as a preferred embodiment of the present invention, step S5 specifically includes:

[0124] like Figure 4 As shown, before iteratively solving, the original image is... Pre-decomposed into target tensor and background tensor as follows:

[0125]

[0126]

[0127] This method helps reduce the number of iterations and speeds up the model's execution.

[0128] In a specific implementation, as a preferred embodiment of the present invention, step S6 specifically includes:

[0129] S61. Design the augmented Lagrange function as follows:

[0130]

[0131] in, represents the Lagrange multiplier, <·> represents the inner product, and μ represents the penalty factor;

[0132] S62. The designed augmented Lagrangian function is divided into several subproblems. Let the number of iterations be k. The target component is iterated to the (k+1)th iteration and updated as follows:

[0133]

[0134] Where S represents the soft threshold operator;

[0135] S63, Set There exists a minimization problem, as follows:

[0136]

[0137] S64. Solve the minimization problem in step S63 using the soft threshold operator. The solution formula is as follows:

[0138] S τ (x)-sign(x)×max(|x|-τ,0)

[0139] S65, the background component is updated to the (k+1)th iteration as follows:

[0140]

[0141] Where N represents the first N protected singular values, and P represents the partial singular value threshold operator;

[0142] S66, Let If Y = Y1 + Y2, then there exists a minimization problem, as follows:

[0143]

[0144] S67. Solve the minimization problem in step S66 using the singular value threshold operator. The solution formula is as follows:

[0145]

[0146] in, S represents the singular value threshold operator;

[0147] S68, will and μ k+1 They were updated to:

[0148]

[0149] μ k+1 =ρμ k

[0150] Where ρ is a constant, the iteration stops when the following condition is met:

[0151]

[0152] In this embodiment, before solving using the alternating direction multiplier method, the matrix substituted into the solution model needs to be converted into a patch tensor. The patch tensor is constructed as follows: Figure 5 As shown.

[0153] In a specific implementation, as a preferred embodiment of the present invention, step S7 specifically includes:

[0154] The target tensor obtained after the iteration stops is reversed to restore it into a matrix, thus obtaining the final detection result.

[0155] Example

[0156] In this embodiment, the detection performance of the present invention and its comparison with mainstream methods are demonstrated in four different strong wave sequence images: FKRW, RLCM, PSTNN, NTFRA, STLDM, WSNMSTIPT, and MFSTPT. The first four methods are single-frame methods, while the latter four are multi-frame methods. FKRW is a filtering-based method. RLCM and STLDM are based on local saliency. The remaining methods are based on sparse low-rank methods.

[0157] The comparison results are as follows Figure 6 and Figure 7 As shown, this invention can significantly suppress strong wave interference and obtain more accurate detection results compared to other methods.

[0158] Figure 8 The average runtime comparison of several methods is presented. The filter-based FKRW method has the shortest runtime. However, the detection accuracy of FKRW is unacceptable. Compared with single-frame sparse low-rank methods, RLCM and STLDM have longer runtimes. Multi-frame sparse low-rank methods exhibit considerably long runtimes due to directly merging multiple frames into the model. Compared with the comparative methods, the runtime of the proposed method ranks second. Both PSTNN and the proposed method use the same rank estimation method. However, due to the introduction of a non-primitive state decomposition strategy, the proposed method achieves higher computational efficiency than PSTNN.

[0159] In summary, this invention offers higher detection accuracy compared to other comparison methods while maintaining a shorter runtime. This fully demonstrates the effectiveness of the invention. The results of this invention can be applied to the field of infrared small target detection and have broad application prospects.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting infrared sea surface targets based on long and short channel features and non-original state decomposition, characterized in that, include: S1. Construct the objective function for the sparse low-rank model; S2. Based on the temporal consistency of infrared sea surface targets and the temporal variability of wave interference, construct long-channel structural tensor weights. S3. Construct short-channel pixel encoding weights; S4. Based on the long-channel structural tensor weights and the short-channel pixel encoding weights, construct the target component weights, and use a sliding window to stack the original image and the target component weights into a three-dimensional tensor respectively. S5. Using the target component weight tensor, perform a non-original state pre-decomposition operation on the three-dimensional tensor of the original image, and pre-decompose the three-dimensional tensor of the original image into preliminary target components and background components. S6. Using the alternating direction multiplier method, the initial target components and background components are solved iteratively. S7. Decompose the three-dimensional tensor of the solution into matrix form to obtain the final result.

2. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S1 specifically includes: S11. Let the original input image matrix be D, then the patch tensor of D is defined as... If we treat the patch tensor of an original image as a linear sum of the target and background components, then we have: in, Indicates background components, Indicates the target component; S12. Considering the low-rank characteristics of the background and the sparsity of the target, the formula in step S11 is transformed into the following minimization problem: in, Let λ represent the estimate of the tensor rank of the background patch, and let λ represent the trade-off parameter. Indicates the weight of the target component. This represents the Hadamard product.

3. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S2 specifically includes: S21. Based on the temporal consistency of infrared sea surface targets and the temporal variability of wave interference, let the channel length be l and the current frame be D. l The first l-1 frames are D1 to D l-1 Long channel structure tensor weight W LST Designed as: Where S represents the saliency map obtained by replacing the original image with the Hadamard product of the original image and the average of the previous l-1 frames. ST represents the calculation of the structure tensor; S22. Calculate the structure tensor of a matrix X using the following formula: Among them, K ρ D represents a Gaussian kernel with variance ρ. σ D represents Gaussian filtering smoothing with variance σ. σ The horizontal and vertical gradients are respectively: S23. Calculate the eigenvalue matrix of the structure tensor ST(X) of matrix X. The calculation formula is as follows:

4. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S3 specifically includes: S31. Perform dilation processing on the result of the previous frame of the current frame. Let the binarized result of the previous frame be R. l-1 The expansion operation can be represented as: in, Indicates the expansion operation, E s represents a square structuring element, and s represents the size of the square structuring element; S32、R' l-1 The pixels in the array are assigned different values ​​based on their relative positions, and this value is named the short-channel pixel encoding weight W. SPE : Where i represents the i-th row of the matrix, r min R' l-1 The minimum number of rows containing non-zero elements, r max R' l-1 The maximum number of rows in the array, where α represents a small positive constant.

5. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S4 specifically includes: After obtaining the weights of the long and short channels, the target component weights are constructed by combining the two. The corresponding matrix W is represented as: Where J is a matrix of all ones. This indicates element-wise division.

6. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S5 specifically includes: Before iteratively solving, the original image Pre-decomposed into target tensor and background tensor as follows:

7. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S6 specifically includes: S61. Design the augmented Lagrange function as follows: in, represents the Lagrange multiplier, <·> represents the inner product, and μ represents the penalty factor; S62. Divide the designed augmented Lagrangian function into several subproblems. Let the number of iterations be k. The target component is iterated to the (k+1)th iteration and updated as follows: Where S represents the soft threshold operator; S63, Set There exists a minimization problem, as follows: S64. Solve the minimization problem in step S63 using the soft threshold operator. The solution formula is as follows: S τ (x)-sign(x)×max(|x|-τ,0) S65, the background component is updated to the (k+1)th iteration as follows: Where N represents the first N protected singular values, and P represents the partial singular value threshold operator; S66, Let If Y = Y1 + Y2, then there exists a minimization problem, as follows: S67. Solve the minimization problem in step S66 using the singular value threshold operator. The solution formula is as follows: in, S represents the singular value threshold operator; S68, will and μ k+1 Updated to: m k+1 =rm k Where ρ is a constant, the iteration stops when the following condition is met:

8. The infrared sea surface target detection method based on long and short channel features and non-original state decomposition according to claim 1, characterized in that, Step S7 specifically includes: The target tensor obtained after the iteration stops is reversed to restore it into a matrix, thus obtaining the final detection result.

Images