Sea bumping blurred image denoising method and system oriented to complex sea conditions

Abstract

The invention relates to the technical field of image processing, in particular to a sea bumping blurred image denoising method and system oriented to complex sea conditions, and the method comprises the steps: firstly setting a reference long-range window in a target image, and then obtaining image content motion data according to bumping motion parameters; and then performing Gabor filtering based on the picture content motion data to obtain a direction feature graph, finally optimizing the attention score according to the direction feature graph in a deblurring model operation process, and obtaining a de-noised image based on a deblurring model. According to the method, the direction feature map is obtained through Gabor filtering, the direction feature map is used for representing the pixels which are generated in the reference long-range window and have the small fuzzy degree, the attention score calculated during deblurring is optimized through the direction feature map, the influence of the pixels with the small fuzzy degree on correction of other pixels is enhanced, and the correction accuracy of the attention score is improved. Therefore, the image deblurring effect is improved, and the problem that the deblurring technology in the prior art is poor in marine image effect is solved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for denoising a turbulent and blurred image at sea in complex sea conditions. Background Art

[0002] Bump-induced blur is a common image degradation phenomenon, typically caused by relative motion of the object or imaging system during exposure. This motion blur not only impairs the visual quality of the image but also complicates subsequent image analysis and processing. To overcome this challenge, researchers have developed a variety of image deblurring techniques to restore image clarity and detail.

[0003] However, unlike the blur caused by vibrations on land, the ocean's turbulence is lower in frequency and greater in amplitude, and is accompanied by swaying and rotational motions. Furthermore, ocean images inherently have a wide viewing angle and a large viewing distance. These factors combine to make the blur pattern in ocean images more complex, nonlinear, and dynamic. Consequently, existing image deblurring techniques are less than ideal when applied to ocean images, potentially exhibiting both local overfitting and underfitting.

[0004] Therefore, people need a technology that can effectively handle the blurred images caused by sea turbulence. Summary of the Invention

[0005] Therefore, the present invention provides a method and system for denoising a turbulent and blurred image at sea in complex sea conditions, so as to solve the problem that the deblurring technology in the prior art has a poor effect when applied to marine images.

[0006] The present invention provides a method for denoising a turbulent blurred image at sea in complex sea conditions, comprising:

[0007] (1) Acquire a target image and set a reference perspective window in the target image;

[0008] (2) obtaining the jolting motion parameters and obtaining the picture content motion data according to the jolting motion parameters;

[0009] (3) Based on the motion data of the picture content, the image in the reference perspective window is subjected to Gabor filtering to obtain a directional feature map;

[0010] (4) The target image is input into the deblurring model based on the attention mechanism, and the attention score is optimized according to the directional feature map during the operation of the deblurring model, and the denoised image is obtained based on the deblurring model.

[0011] In a preferred implementation, the deblurring model includes a blocking layer, an embedding layer, multiple coding layers, a deblurring layer, and an optimization layer, wherein the blocking layer, the embedding layer, the multiple coding layers, and the deblurring layer are connected in sequence, and the optimization layer is connected to the input end of the coding layer, wherein:

[0012] The block layer is used to divide the target image into blocks to obtain multiple image blocks;

[0013] The embedding layer is used to expand the image block by pixel and perform position embedding to obtain the image block vector;

[0014] The coding layer is used to perform linear transformation on the image block vector to obtain the Q matrix, K matrix and V matrix corresponding to the image block; according to the Q matrix and K matrix, the correlation matrix is ​​obtained;

[0015] The optimization layer is used to optimize the correlation matrix based on the directional feature map;

[0016] The encoding layer is also used to obtain the attention score of the image block based on the optimized correlation matrix and V matrix; and obtain the context vector based on the attention score;

[0017] The deblurring layer is used to obtain a denoised image based on the context vector.

[0018] In a preferred implementation, optimizing the correlation matrix according to the directional feature map includes:

[0019] According to the directional feature map, the stable feature value of each pixel is obtained;

[0020] According to the correlation matrix and the stable eigenvalue of each pixel, an optimization matrix is ​​established. The correspondence between each element and pixel in the optimization matrix is ​​the same as the correspondence between each element and pixel in the correlation matrix. The value of each element in a column corresponding to a pixel in the optimization matrix is ​​the stable eigenvalue corresponding to the pixel.

[0021] The incidence matrix and the optimization matrix are added bit by bit to obtain the optimized incidence matrix.

[0022] In a preferred implementation, obtaining a stable eigenvalue of each pixel according to the directional feature map includes:

[0023] Get the maximum grayscale value and the minimum grayscale value in the directional feature map;

[0024] Obtaining the grayscale value of the target pixel in the directional feature map as the first grayscale value;

[0025] Obtaining the grayscale values ​​of the adjacent pixels of the target pixel in the directional feature map as the second grayscale value;

[0026] A stable characteristic value of the target pixel is obtained according to the maximum grayscale value, the minimum grayscale value, the first grayscale value and the second grayscale value.

[0027] In a preferred implementation, obtaining a stable characteristic value of a target pixel according to the maximum grayscale value, the minimum grayscale value, the first grayscale value, and the second grayscale value includes:

[0028] The stable eigenvalue of the target pixel is obtained by the following formula:

[0029]

[0030] Among them, S is the stable eigenvalue, p is the first gray value, p max is the maximum gray value, p min is the minimum grayscale value, x is the number of second grayscale values ​​whose difference with the first grayscale value is less than the preset threshold, n is the neighborhood size when counting the second grayscale values, and k1 and k2 are both preset weight coefficients.

[0031] In a preferred implementation, obtaining the attention score of the image block according to the optimized correlation matrix and V matrix includes:

[0032] The attention score of the image patch is calculated by the following formula:

[0033]

[0034] Among them, A represents the attention score, softmax is the normalized exponential function, Q is the Q matrix, K is the K matrix, QK T That is, it represents the correlation matrix, O is the optimization matrix, QK T +O represents the optimized correlation matrix, d k is the key value dimension in the K matrix, and V is the V matrix.

[0035] In a preferred implementation, the optimization layer is only connected to the input end of the first coding layer.

[0036] In a preferred implementation, step (1) includes:

[0037] Acquire a target image and identify a sea level line in the target image;

[0038] Based on the preset width, a strip area is established in the target graphic along the extended trajectory of the coastal plane line in the target image, and the strip area is set as the reference perspective window.

[0039] In a preferred implementation, step (2) includes:

[0040] Obtaining bump motion parameters, which include displacement parameters and rotation parameters;

[0041] The imaging plane of the target image is obtained, and the displacement parameters and rotation parameters are projected onto the imaging plane to obtain the picture content motion data.

[0042] The present invention also provides a denoising system for marine turbulence blurred images in complex sea conditions, comprising:

[0043] An image marking module is used to obtain a target image and set a reference perspective window in the target image;

[0044] A direction mapping module is used to obtain the bump motion parameters and obtain the picture content motion data according to the bump motion parameters;

[0045] A feature analysis module is used to perform Gabor filtering on the image in the reference perspective window based on the motion data of the picture content to obtain a directional feature map;

[0046] The blur denoising module is used to input the target image into the deblurring model based on the attention mechanism, optimize the attention score according to the directional feature map during the deblurring model operation, and obtain the denoised image based on the deblurring model.

[0047] The beneficial effects of adopting the above embodiment are:

[0048] The present invention provides a method and system for denoising a turbulent, blurred image at sea in complex sea conditions. The method first acquires a target image and sets a reference distant view window within the target image. Then, turbulence motion parameters are obtained. Based on the turbulence motion parameters, image content motion data is obtained. Then, based on the image content motion data, the image within the reference distant view window is subjected to Gabor filtering to obtain a directional feature map. Finally, the target image is input into a deblurring model based on an attention mechanism. During the deblurring model operation, the attention score is optimized based on the directional feature map, and a denoised image is obtained based on the deblurring model. In the present invention, the reference distant view window marks the distant portion of the target image. This portion is considered to be less affected by sea turbulence and less blurred. Based on this characteristic, the present invention first obtains a directional feature map through Gabor filtering. The directional feature map is used to identify pixels within the reference distant view window that have less blur along the image content motion data. The directional feature map is then used to optimize the attention score calculated during deblurring, enhancing the influence of these less blurred pixels on the correction of other pixels. This improves the image deblurring effect and addresses the problem of poor deblurring performance in conventional deblurring techniques applied to marine images. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A flow chart of a method provided by an embodiment of the present invention;

[0050] Figure 2 A schematic diagram of motion blur described in the invention;

[0051] Figure 3 A model structure diagram of a defuzzification model provided by an embodiment of the present invention;

[0052] Figure 4 The embodiment of the present invention provides Figure 3 A diagram of the specific steps executed by the middle optimization layer;

[0053] Figure 5 The embodiment of the present invention provides Figure 4 Specific step diagram of step S401;

[0054] Figure 6 Schematic diagrams of several situations when calculating stable eigenvalues ​​in the invention;

[0055] Figure 7 This is a system structure diagram of the marine turbulence blurred image denoising system for complex sea conditions of the intelligent reconnaissance equipment provided by the present invention. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0057] Combine Figure 1 As shown, a specific embodiment of the present invention discloses a method for denoising a turbulent blurred image at sea in complex sea conditions, comprising the following steps:

[0058] S101, acquiring a target image and setting a reference perspective window in the target image;

[0059] S102, obtaining a bumping motion parameter, and obtaining picture content motion data according to the bumping motion parameter;

[0060] S103, performing Gabor filtering on the image in the reference perspective window based on the picture content motion data to obtain a directional feature map;

[0061] S104: input the target image into a deblurring model based on an attention mechanism, optimize the attention score according to the directional feature map during the operation of the deblurring model, and obtain a denoised image based on the deblurring model.

[0062] In the above process, the target image is the image currently being deblurred, acquired by the intelligent reconnaissance equipment. The reference distant view window marks the area of ​​distant content within the target image, such as the sea horizon, clouds, and other distant objects. Unlike turbulence on land, turbulence at sea manifests primarily as slow displacement and small rotations caused by the undulations of waves. Due to the characteristics of turbulence at sea, the image within the reference distant view window is assumed to be less blurred due to turbulence. Therefore, the pixels within the reference distant view window can be used as a reference for restoring other pixels within the image.

[0063] The turbulence motion parameters are parameters that describe the turbulence motion of the sea surface, and they can be collected using any existing method. For example, if the intelligent reconnaissance equipment is a device that moves on the sea surface, then the turbulence motion parameters are the motion parameters of the camera when shooting the target image. If the intelligent reconnaissance equipment is a device that does not contact the sea surface, such as a drone, then the turbulence motion parameters can be collected by sensors equipped on the intelligent reconnaissance equipment, ships on the sea surface, buoys and other equipment. Specifically, according to different collection designs, the turbulence motion parameters can be collected by IMU units such as accelerometers (measuring linear acceleration) and gyroscopes (measuring angular velocity), or they can be obtained by high-precision position differential (RTK) using satellite remote sensing positioning systems such as GPS and Beidou. They can also be obtained by using laser radars or radar rangefinders to emit lasers or microwaves to detect sea surface fluctuations and invert wave height and phase to complete the acquisition.

[0064] The picture content motion data is data representing the motion direction of the image content in the target image. It can be understood that the picture content motion data is essentially the projection of the direction of the sea level bumpy motion in the image.

[0065] The Gabor filter is a linear filter widely used in image processing and computer vision. Based on the Gabor function, it effectively simulates the human visual system's response to specific spatial frequencies and directions. Therefore, it can extract features in an image at specific directions and frequencies. Once the motion data of the image content is known, the Gabor filter can be used to extract the content features of the target image extending in the direction of the image content's motion.

[0066] Similarly, the features of the motion data of the image content can also be considered to be features that are less affected by the motion of the image content itself, resulting in less blur and being able to reflect the true information of the image. Figure 2As shown, if an object is moving vertically when photographed, if the object is a point, the image will appear as a vertically extending straight line due to motion blur. If the object is a horizontally extending straight line, the image will appear as a square area. However, it is conceivable that if the object is a vertically extending straight line, the image will also be slightly distorted, but will still appear as a vertically extending straight line. Therefore, the pixels corresponding to the features in the image content motion data can also be used as a reference for restoring other pixels in the image.

[0067] The attention mechanism is a technique that mimics human visual attention in deep learning models, allowing the model to dynamically focus on the most relevant parts of the input data for the task at hand. In deblurring models, the attention mechanism helps the model identify and restore key features in an image, such as edges and texture, thereby improving deblurring effectiveness. The attention score is a quantitative metric that measures the importance of different parts of the input information to the task at hand. In the attention mechanism, the model determines the level of attention to be paid to different parts of the input information based on the attention score. Specifically, for image deblurring, the attention mechanism calculates the level of attention paid to each pixel.

[0068] The present invention extracts pixels with smaller blur scales in the target image through a reference perspective window and Gabor filtering, and then uses these pixels to optimize the attention score calculated during deblurring, thereby improving the deblurring effect and solving the problem that the deblurring technology in the prior art has poor effect when applied to marine images.

[0069] It is understood that, in actual implementation, the specific position and range of the reference perspective window can be set manually or automatically by a computer using any existing target recognition algorithm. The present invention provides a preferred embodiment for automatically setting the reference perspective window:

[0070] In a preferred embodiment, the above step S101 of acquiring a target image and setting a reference perspective window in the target image specifically includes:

[0071] Acquire a target image and identify a sea level line in the target image;

[0072] Based on the preset width, a strip area is established in the target graphic along the extended trajectory of the coastal plane line in the target image, and the strip area is set as the reference perspective window.

[0073] In the above process, the sea level line in the target image (specifically, the line separating the sea and sky in the image) can be detected using any existing method. For example, the target image is first converted to grayscale, then the Canny algorithm is applied to detect edges in the grayscale image. A Hough transform is then used to detect lines. An angle threshold is set to filter horizontal lines. The longest horizontal line is then selected from the detected results to determine the sea level line. It is understood that the long-range view is defined as the band extending along the sea level line, which serves as the baseline long-range view window.

[0074] Similarly, the specific method of converting the bump motion parameters into the picture content motion data can also adopt any existing technology. In a preferred embodiment, the above step S102, obtaining the bump motion parameters and obtaining the picture content motion data based on the bump motion parameters, specifically includes:

[0075] Obtaining bump motion parameters, which include displacement parameters and rotation parameters;

[0076] The imaging plane of the target image is obtained, and the displacement parameters and rotation parameters are projected onto the imaging plane to obtain the picture content motion data.

[0077] It is understandable that the sea surface turbulence and motion state may be different in different areas of the same image, so the same target image can also contain multiple different image content motion data. When performing Gabor filtering, local filtering can be performed simultaneously on different image areas based on different image content motion data.

[0078] Furthermore, the deblurring model based on the attention mechanism can also be implemented using any existing model, for example, it can be implemented using the existing Vision Transformer (ViT) model combined with the existing CNN neural network model. The purpose of the present invention is to correct the attention score calculated in the process, and the attention score can also be corrected in any way. For example, after calculating the attention score, the attention score corresponding to each pixel representing the directional feature in the directional feature map is increased or decreased. For another example, the directional feature map is also used as an input, so that it and the target image participate in the same linear transformation at the same time to affect the final calculated attention score. However, in most cases, the attention score is not a specific numerical value but a vector form, which has poor interpretability. This high-dimensional characteristic makes it difficult to derive the optimization amount of the attention score through a certain mathematical model, which in turn affects the accuracy.

[0079] Based on the above defects, the present invention provides a method for correction in the process of attention score calculation. In a preferred embodiment, in the above step S104, the deblurring model includes a blocking layer, an embedding layer, multiple encoding layers, a deblurring layer and an optimization layer, wherein the blocking layer, the embedding layer, the multiple encoding layers and the deblurring layer are connected in sequence, and the optimization layer is connected to the input end of the encoding layer, wherein:

[0080] The block layer is used to divide the target image into blocks to obtain multiple image blocks;

[0081] The embedding layer is used to expand the image block by pixel and perform position embedding to obtain the image block vector;

[0082] The coding layer is used to perform linear transformation on the image block vector to obtain the Q matrix, K matrix and V matrix corresponding to the image block; according to the Q matrix and K matrix, the correlation matrix is ​​obtained;

[0083] The optimization layer is used to optimize the correlation matrix based on the directional feature map;

[0084] The encoding layer is also used to obtain the attention score of the image block based on the optimized correlation matrix and V matrix; and obtain the context vector based on the attention score;

[0085] The deblurring layer is used to obtain a denoised image based on the context vector.

[0086] This embodiment is essentially an improvement to the existing VIT model. In this embodiment, the blocking layer, embedding layer, and multiple encoding layers are all functional layers already present in the existing VIT model. The deblurring layer is also an existing model for performing deblurring functions and will not be described in detail here. The optimization layer is the innovative aspect of this invention.

[0087] In the attention mechanism, the Q matrix, K matrix, and V matrix represent the query, key, and value matrices, respectively. They are obtained by linear transformation of the input features (which can be regarded as each pixel in this invention). Specifically: the Q matrix represents the query vector of each pixel in the current input, which is used to find other pixels related to itself from the image. The K matrix represents the key vector of each pixel in the input, which is used to characterize the feature information of each pixel. The V matrix represents the value vector of each position in the input sequence, which is used to calculate the output after attention.

[0088] The QKV matrix operations described above are typically performed in the VIT model. In the multi-head attention layer within the coding layer, Q, K, and V are divided into multiple heads, each of which is transformed using an independent set of weight matrices to further enhance differentiation. The optimization layer of the present invention is actually connected to each head in the multi-head attention within the coding layer. The coding layer also includes feedforward neural network layers and residual layers, but these are all prior art and are not described in detail in this embodiment. The accompanying figures also simplify the layers appropriately.

[0089] The new matrix obtained by multiplying the transposed matrices of the Q matrix and the K matrix can initially represent the relationship between each pixel and other pixels, which is the correlation matrix in this embodiment. The present invention optimizes the attention score by optimizing the correlation matrix.

[0090] The advantages of this embodiment are that the correlation matrix is ​​highly interpretable, and the relationship between two pixels is represented by only one element in the correlation matrix. This makes the optimization strategy for the correlation matrix easier to quantify, and the optimization difficulty is lower than directly optimizing the attention score, while the accuracy is higher than optimizing the attention score. In addition, this embodiment minimizes the modification of the existing deep learning model, requiring only the addition of an optimization layer and simple adjustments to the encoding layer to significantly improve the image deblurring effect. This improvement not only reduces the modification to the existing technology but also maintains the reusability and modularity of the model, making subsequent debugging and function expansion more convenient and flexible. Due to the small modifications, the model can be quickly integrated into the existing image processing process without large-scale system architecture adjustments, thereby reducing development and deployment costs. In addition, this lightweight modification also makes the model easier to maintain and upgrade, improving development efficiency. In terms of performance, by introducing the optimization layer, the model can more effectively capture image details and restore the quality of blurred images while maintaining computational efficiency.

[0091] Furthermore, in one embodiment, the optimization layer is only connected to the input end of the first coding layer. The specific structure of the deblurring model in this embodiment is as follows: Figure 3 shown.

[0092] The optimization layer is only connected to the first encoding layer without modifying the subsequent encoding layers. The advantage of this design is that it can maintain the model's self-attention analysis capability. By limiting the addition of optimization layers, the risk of overfitting is reduced, and the model can better generalize to unseen data during training.

[0093] Furthermore, the correlation matrix can be modified using any existing method. For example, the values ​​of the elements in the correlation matrix corresponding to the directional characteristics in the directional characteristic map can be increased by a preset value, while the values ​​of other elements can be decreased by a preset value. Another example is that the directional characteristic map can be subjected to the same linear transformation as that used to calculate the Q, K, and V matrices to obtain a matrix representing the directional characteristics, and then superimposed. However, the present invention provides a more refined adjustment method.

[0094] Combine Figure 4 As shown, in a preferred embodiment, the optimization layer performs the following operations: optimizing the correlation matrix according to the directional feature map, specifically including:

[0095] S401, obtaining a stable eigenvalue of each pixel according to the directional feature map;

[0096] S402. Establish an optimization matrix based on the incidence matrix and the stable eigenvalue of each pixel. The correspondence between each element and pixel in the optimization matrix is ​​the same as the correspondence between each element and pixel in the incidence matrix. Each element value in a column corresponding to a pixel in the optimization matrix is ​​the stable eigenvalue corresponding to the pixel.

[0097] S403: Add the incidence matrix and the optimization matrix bit by bit to obtain an optimized incidence matrix.

[0098] Unlike other methods, the present invention first establishes an additional stable eigenvalue, and then uses the stable eigenvalue to optimize the correlation matrix. The stable eigenvalue represents the "stability" of the content in a pixel against blur in a bumpy environment. This embodiment only needs to debug the calculation method of the stable eigenvalue during model training, which improves the accuracy while ensuring the adjustability of the model. The optimization matrix is ​​a matrix composed of stable eigenvalues. Like the correlation matrix, each element in the optimization matrix can also be regarded as the initial attention of a pixel to another pixel. Therefore, in the optimization matrix, a column of elements corresponding to a pixel are all stable eigenvalues ​​corresponding to the pixel. The optimization matrix and the correlation matrix are added bit by bit, which realizes the preliminary adjustment of the attention score. After that, the normalized sum calculation is performed to obtain the attention score of each pixel.

[0099] Specifically, in a preferred embodiment, the steps performed by the encoding layer include: obtaining the attention score of the image block according to the optimized correlation matrix and the V matrix, specifically including:

[0100] The attention score of the image patch is calculated by the following formula:

[0101]

[0102] Among them, A represents the attention score, softmax is the normalized exponential function, Q is the Q matrix, K is the K matrix, QK T That is, it represents the correlation matrix, O is the optimization matrix, QK T +O represents the optimized correlation matrix, d k is the key value dimension in the K matrix, and V is the V matrix.

[0103] More specifically, when calculating the optimization matrix, the specific calculation method of the stable eigenvalue can also be flexibly set according to the actual situation. For example, the grayscale value in the directional feature map can be directly used for a simple linear transformation as the stable eigenvalue of each pixel. The present invention also provides a more accurate method, combined with Figure 5 As shown, in a preferred embodiment, the above step S401, obtaining the stable eigenvalue of each pixel according to the directional feature map, specifically includes:

[0104] S501, obtaining the maximum grayscale value and the minimum grayscale value in the directional feature map;

[0105] S502, obtaining a grayscale value of a target pixel in a directional feature map as a first grayscale value;

[0106] S503, obtaining the grayscale values ​​of the adjacent pixels of the target pixel in the directional feature map as the second grayscale value;

[0107] S504 : Obtain a stable characteristic value of the target pixel according to the maximum grayscale value, the minimum grayscale value, the first grayscale value, and the second grayscale value.

[0108] It is understood that the target pixel is the pixel whose stable eigenvalue is currently being analyzed. The first grayscale value, combined with the maximum grayscale value and the minimum grayscale value, can be used to calculate the relative level of the target pixel within the reference perspective window. However, this embodiment also incorporates the second grayscale value for consideration. The purpose is to consider not only the target pixel itself but also other pixels surrounding the target pixel when calculating the stable eigenvalue, thereby obtaining a more accurate result. For example, a relative value is calculated using the maximum grayscale value, the minimum grayscale value, and the first grayscale value, and then added to the average of all second grayscale values ​​to obtain the stable eigenvalue.

[0109] In the above process, the specific definition of "adjacent pixels," that is, how two pixels are considered adjacent, can be flexibly set based on actual circumstances. For example, two pixels can be defined as adjacent if there are no other pixels between them. Alternatively, all pixels within a neighborhood centered on the target pixel can be defined as adjacent to the target pixel.

[0110] Specifically, in a preferred embodiment, the above step S504, obtaining the stable characteristic value of the target pixel according to the maximum grayscale value, the minimum grayscale value, the first grayscale value, and the second grayscale value, specifically includes:

[0111] The stable eigenvalue of the target pixel is obtained by the following formula:

[0112]

[0113] Among them, S is the stable eigenvalue, p is the first gray value, p max is the maximum gray value, p min is the minimum grayscale value, x is the number of second grayscale values ​​whose difference with the first grayscale value is less than the preset threshold, n is the neighborhood size when counting the second grayscale values, and k1 and k2 are both preset weight coefficients.

[0114] The meaning of the above formula is that the target element is centered in a square area consisting of n×n pixels, which is the neighborhood of the target element. The pixels within this area are the neighbors of the target pixel. In this case, the stable eigenvalue is related to the number of pixels in the neighborhood that are close to the target pixel.

[0115] Combine Figure 6 As shown in the figure, if the number of pixels close to the target pixel in the neighborhood is small, the target pixel can be considered to be noise, and the stability represented by the stable eigenvalue should be low. Conversely, if the number of pixels close to the target pixel in the neighborhood is large, it can be considered that the target pixel has a bumpy blur and is not a scale-invariant directional feature. When the number of pixels close to the target pixel in the neighborhood is within a certain range, the pixel can be considered to have good stability. Similarly, how to define the grayscale values ​​of two pixels as "close" can also be flexibly set according to the specific situation.

[0116] Combine Figure 7 As shown, the present invention also provides a turbulent and blurred image denoising system for complex sea conditions of intelligent reconnaissance equipment, comprising:

[0117] An image marking module 710 is used to obtain a target image and set a reference perspective window in the target image;

[0118] Direction mapping module 720, used to obtain jolting motion parameters and obtain picture content motion data based on the jolting motion parameters;

[0119] A feature analysis module 730 is configured to perform Gabor filtering on the image within the reference perspective window based on the motion data of the image content to obtain a directional feature map;

[0120] The blur denoising module 740 is used to input the target image into the deblurring model based on the attention mechanism, optimize the attention score according to the directional feature map during the operation of the deblurring model, and obtain a denoised image based on the deblurring model.

[0121] It should be noted here that the corresponding system provided in the above embodiments can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can be found in the corresponding contents in the above method embodiments, which will not be repeated here.

[0122] The present invention provides a method and system for denoising a turbulent, blurred image at sea in complex sea conditions. The method first acquires a target image and sets a reference distant view window within the target image. Then, turbulence motion parameters are obtained. Based on the turbulence motion parameters, image content motion data is obtained. Then, based on the image content motion data, the image within the reference distant view window is subjected to Gabor filtering to obtain a directional feature map. Finally, the target image is input into a deblurring model based on an attention mechanism. During the deblurring model operation, the attention score is optimized based on the directional feature map, and a denoised image is obtained based on the deblurring model. In the present invention, the reference distant view window marks the distant portion of the target image. This portion is considered to be less affected by sea turbulence and less blurred. Based on this characteristic, the present invention first obtains a directional feature map through Gabor filtering. The directional feature map is used to identify pixels within the reference distant view window that have less blur along the image content motion data. The directional feature map is then used to optimize the attention score calculated during deblurring, enhancing the influence of these less blurred pixels on the correction of other pixels. This improves the image deblurring effect and addresses the problem of poor deblurring performance in conventional deblurring techniques applied to marine images.

[0123] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referenced to each other.

[0124] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for denoising blurred images of turbulent sea conditions in complex sea conditions, characterized by: The following steps are involved: (1) Acquire a target image and set a reference perspective window in the target image; (2) obtaining the jolting motion parameters and obtaining the picture content motion data according to the jolting motion parameters; (3) Based on the motion data of the picture content, the image in the reference perspective window is subjected to Gabor filtering to obtain a directional feature map; (4) The target image is input into the deblurring model based on the attention mechanism, and the attention score is optimized according to the directional feature map during the operation of the deblurring model, and the denoised image is obtained based on the deblurring model.

2. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 1 is characterized in that: The deblurring model described in step (4) includes a blocking layer, an embedding layer, multiple coding layers, a deblurring layer, and an optimization layer, wherein the blocking layer, the embedding layer, the multiple coding layers, and the deblurring layer are connected in sequence, and the optimization layer is connected to the input end of the coding layer, wherein: The block layer is used to divide the target image into blocks to obtain multiple image blocks; The embedding layer is used to expand the image block by pixel and perform position embedding to obtain the image block vector; The coding layer is used to perform linear transformation on the image block vector to obtain the Q matrix, K matrix and V matrix corresponding to the image block; according to the Q matrix and K matrix, the correlation matrix is ​​obtained; The optimization layer is used to optimize the correlation matrix based on the directional feature map; The encoding layer is also used to obtain the attention score of the image block based on the optimized correlation matrix and V matrix; and obtain the context vector based on the attention score; The deblurring layer is used to obtain a denoised image based on the context vector.

3. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 2, characterized in that: Optimize the correlation matrix based on the directional feature map, including: According to the directional feature map, the stable feature value of each pixel is obtained; According to the correlation matrix and the stable eigenvalue of each pixel, an optimization matrix is ​​established. The correspondence between each element and pixel in the optimization matrix is ​​the same as the correspondence between each element and pixel in the correlation matrix. The value of each element in a column corresponding to a pixel in the optimization matrix is ​​the stable eigenvalue corresponding to the pixel. The incidence matrix and the optimization matrix are added bit by bit to obtain the optimized incidence matrix.

4. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 3 is characterized in that: According to the directional feature map, the stable feature value of each pixel is obtained, including: Get the maximum grayscale value and the minimum grayscale value in the directional feature map; Obtaining the grayscale value of the target pixel in the directional feature map as the first grayscale value; Obtaining the grayscale values ​​of the adjacent pixels of the target pixel in the directional feature map as the second grayscale value; A stable characteristic value of the target pixel is obtained according to the maximum grayscale value, the minimum grayscale value, the first grayscale value and the second grayscale value.

5. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 4 is characterized in that: Obtaining a stable characteristic value of a target pixel according to the maximum grayscale value, the minimum grayscale value, the first grayscale value, and the second grayscale value, including: The stable eigenvalue of the target pixel is obtained by the following formula: Among them, S is the stable eigenvalue, p is the first gray value, p max is the maximum gray value, p min is the minimum grayscale value, x is the number of second grayscale values ​​whose difference with the first grayscale value is less than the preset threshold, n is the neighborhood size when counting the second grayscale values, and k1 and k2 are both preset weight coefficients.

6. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 2, characterized in that: According to the optimized correlation matrix and V matrix, the attention score of the image block is obtained, including: The attention score of the image patch is calculated by the following formula: Among them, A represents the attention score, softmax is the normalized exponential function, Q is the Q matrix, K is the K matrix, QK T That is, it represents the correlation matrix, O is the optimization matrix, QK T +O represents the optimized correlation matrix, d k is the key value dimension in the K matrix, and V is the V matrix.

7. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 2, characterized in that: The optimization layer is only connected to the input of the first encoding layer.

8. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 1, characterized in that: Step (1) specifically includes: Acquire a target image and identify a sea level line in the target image; Based on the preset width, a strip area is established in the target graphic along the extended trajectory of the coastal plane line in the target image, and the strip area is set as the reference perspective window.

9. The method for denoising a turbulent blurred image at sea in complex sea conditions according to claim 1, characterized in that: Step (2) specifically includes: Obtaining bump motion parameters, which include displacement parameters and rotation parameters; The imaging plane of the target image is obtained, and the displacement parameters and rotation parameters are projected onto the imaging plane to obtain the picture content motion data.

10. A turbulent and blurred image denoising system for complex sea conditions, characterized by: For implementing the method according to any one of claims 1 to 9, the system comprises: An image marking module is used to obtain a target image and set a reference perspective window in the target image; A direction mapping module is used to obtain the bump motion parameters and obtain the picture content motion data according to the bump motion parameters; A feature analysis module is used to perform Gabor filtering on the image in the reference perspective window based on the motion data of the picture content to obtain a directional feature map; The blur denoising module is used to input the target image into the deblurring model based on the attention mechanism, optimize the attention score according to the directional feature map during the deblurring model operation, and obtain the denoised image based on the deblurring model.

Images