A four-dimensional block matching polarimetric image denoising method based on polarization constraint

Abstract

The application discloses a four-dimensional block matching polarized image denoising method based on polarization constraint, wherein the method comprises a polarized image expansion step, a four-dimensional block matching step, a four-dimensional filtering step and a polarized image estimation step; the polarized image expansion is used to make full use of the polarization information among image channels to generate three-dimensional polarization data; the four-dimensional block matching integrates three-dimensional polarized image blocks with the same polarization information into a four-dimensional matrix for filtering; the four-dimensional filtering step performs four-dimensional transformation on the four-dimensional matrix, and then filters and denoises the image blocks in the transformed domain; and the image estimation step restores the image blocks to generate noise-free estimation of the noise polarized image. The method constrains the polarization information through the polarized image generation step and the four-dimensional block matching step, and can not only efficiently remove the noise in the polarized image after filtering processing, but also accurately recover the polarization information of the target from the noise image, thereby being beneficial to the practical application scene and range of the polarized imaging.

Description

Technical Field

[0001] This invention relates to the field of polarization imaging technology, and in particular to a four-dimensional block matching polarization image denoising method based on polarization constraints. Background Technology

[0002] Polarization imaging, building upon traditional imaging techniques, acquires polarization images of the scene being probed by obtaining light intensity images modulated by polarizing optical devices. These polarization images provide richer information about the material, geometry, and surface roughness of objects within the scene, playing a crucial role in filtering out background and other invalid information, enhancing image quality, and effectively improving target detection and identification capabilities. Therefore, polarization imaging has potential applications in detecting stealth, camouflage, and false targets in harsh environments such as fog, smoke, and complex backgrounds. However, in practical applications, noise can significantly interfere with polarization imaging. The nonlinear calculations in the polarization information processing amplify noise in the image, making polarization information sensitive to noise and easily submerged in it.

[0003] Existing image denoising techniques mainly fall into two categories: data-driven and non-data-driven. Thanks to the development of convolutional neural networks, data-driven learning methods have shown excellent denoising performance; however, these methods rely on training on large datasets, resulting in high training costs. In contrast, non-data-driven methods are easier to implement and can achieve good denoising results under certain noise types and levels. For example, the 3D block matching method, based on nonlocality and transform domain filtering, can quickly obtain high-quality denoised images. While these methods can remove noise from images, they lack design considerations for polarization images. In fact, there is physical correlation between the channels of a polarization image; directly applying ordinary digital image denoising methods to polarization images will fail to fully utilize polarization information. Therefore, an effective polarization image denoising method is needed that can not only remove noise but also effectively restore polarization information. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and solve the problem that polarization image denoising methods are difficult to restore polarization information while removing noise, and to provide a four-dimensional block matching polarization image denoising method based on polarization constraints.

[0005] The objective of this invention is achieved through the following technical solution:

[0006] A four-dimensional block matching polarization image denoising method based on polarization constraints includes the following steps:

[0007] Step 1. A polarization image data augmentation method based on polarization Stokes vector relationship is used to augment three or more polarization images into any number of polarization images, and stack all polarization images along the polarization dimension into a three-dimensional polarization image to enhance the utilization of the inherent physical relationship between polarization image channels.

[0008] Step 2. Four-dimensional block matching: Select a three-dimensional image block as reference block A in the three-dimensional polarization image in step 1, traverse and match other image blocks in the three-dimensional polarization image, and combine the image blocks that match reference block A as similar blocks A' in the fourth dimension to form a four-dimensional similar block matrix. Then, reselect reference block A to perform similar block matching until reference block A has traversed the entire three-dimensional polarization image.

[0009] Step 3. Four-dimensional collaborative filtering: Perform a four-dimensional transformation A” on each group of four-dimensional similar block matrices in Step 2 to obtain a four-dimensional similar block matrix in the transform domain. Then, filter the four-dimensional similar block matrix in the transform domain using hard thresholding. Finally, perform a four-dimensional inverse transformation A''' on the four-dimensional similar block matrix in the transform domain to obtain the filtered four-dimensional similar block matrix.

[0010] Step 4. Basic estimation: Reconstruct the three-dimensional similar blocks in each group of filtered four-dimensional similar block matrices from Step 3 according to their positions in the three-dimensional polarization image in Step 1. Apply a weighted average to the overlapping areas of similar blocks in different groups of filtered four-dimensional similar block matrices to obtain the basic denoised image.

[0011] Step 5. Secondary four-dimensional block matching: Select a three-dimensional image block as reference block B from the base denoised image obtained in step 4, traverse and match other three-dimensional image blocks in the base denoised image, and combine blocks similar to reference block B as similar blocks B' in the fourth dimension to estimate the four-dimensional similarity block matrix of the image. Then, reselect reference block B for matching until reference block B traverses the entire base denoised image. At the same time, extract the corresponding four-dimensional similarity block matrix of the noise image in the three-dimensional polarization image according to the position of the matched similar block B'.

[0012] Step 6. Four-dimensional Wiener filtering: Perform a four-dimensional transformation B” on the two types of four-dimensional similar block matrices obtained in Step 5 to obtain the four-dimensional similar block matrix of the basic estimated image in the transform domain and the four-dimensional similar block matrix of the noisy image in the transform domain. Then, use the transform coefficients of the four-dimensional similar block matrix of the basic estimated image as Wiener filtering coefficients, and perform Wiener filtering and four-dimensional inverse transformation B’’’ on the four-dimensional similar block matrix of the noisy image in the transform domain in sequence to obtain the corrected four-dimensional similar block matrix of the noisy image.

[0013] Step 7. Final estimation: Restore the three-dimensional similar blocks in the four-dimensional similar block matrix of each group of corrected noise images according to their positions in the basic denoised image in step 5. Apply weighted average processing to the overlapping areas of similar blocks in the four-dimensional similar block matrix of different groups of corrected noise images to obtain the final estimated image.

[0014] Furthermore, the polarization image data augmentation method calculates the Stokes vector using three or more polarization images at arbitrary angles, and then calculates the polarization image at arbitrary angles from the Stokes vectors according to the polarization relationship, thereby augmenting three or more polarization images into any number of polarization images. All polarization images are stacked in order of angle size to form three-dimensional data, where the three dimensions are image length, image width, and polarization angle.

[0015] Furthermore, each reference block is three-dimensional data with a size of L×L×L, where L≥3. This is because complete polarization information can only be constructed from polarization images at three angles. To better utilize polarization information, the matching block contains information from polarization images at least three angles. The similarity between each reference block and each similar block is measured using the L2 norm. The similar blocks are arranged from high to low similarity, and to improve algorithm efficiency, the top N similar blocks are combined in the fourth dimension to form a four-dimensional similarity block matrix.

[0016] Furthermore, to improve algorithm efficiency, the four-dimensional transformation A” is split into a three-dimensional transformation and a one-dimensional transformation. The three-dimensional discrete Haar transform is applied to the three dimensions of image length, image width, and polarization angle, and the one-dimensional Haar wavelet transform is applied to the fourth dimension. The corresponding four-dimensional inverse transformation A’’’ is the three-dimensional discrete Haar inverse transform and the one-dimensional Haar wavelet inverse transform.

[0017] Furthermore, to improve algorithm efficiency, the four-dimensional transformation B” is split into a three-dimensional transformation and a one-dimensional transformation. The three-dimensional discrete cosine transform is applied to the three dimensions of image length, image width, and polarization angle, and the one-dimensional Haar wavelet transform is applied to the fourth dimension. The corresponding four-dimensional inverse transformation B’’’ is the three-dimensional discrete cosine inverse transform and the one-dimensional Haar wavelet inverse transform.

[0018] Furthermore, the images obtained from the basic estimation and the final estimation are three-dimensional polarization images, with the three dimensions being image length, image width, and polarization angle, respectively. In the final estimated image, the polarization image corresponding to the angle of the polarization image before image amplification is extracted as the output.

[0019] The present invention also provides a four-dimensional block matching polarization image denoising device based on polarization constraints, comprising:

[0020] The image augmentation unit is used to augment three or more polarization images into any number of polarization images and stack all polarization images into a three-dimensional polarization image.

[0021] The four-dimensional block matching unit is used to select a three-dimensional image block as a reference block A in the three-dimensional polarization image, traverse and match other image blocks in the three-dimensional polarization image, and combine the image blocks that match the reference block A as similar blocks A' in the fourth dimension into a set of four-dimensional similar block matrices. Then, the reference block A is selected again to perform similar block matching until the reference block A traverses the entire three-dimensional polarization image.

[0022] The four-dimensional collaborative filtering unit is used to perform a four-dimensional transformation A” on each group of four-dimensional similar block matrices to obtain a four-dimensional similar block matrix in the transform domain. Then, the four-dimensional similar block matrix in the transform domain is filtered by hard threshold filtering. Finally, the four-dimensional similar block matrix in the transform domain is subjected to a four-dimensional inverse transformation A''' to obtain the filtered four-dimensional similar block matrix.

[0023] The basic estimation unit is used to restore the three-dimensional similar blocks in each group of filtered four-dimensional similar block matrices according to their positions in the three-dimensional polarization image in the image augmentation unit. The overlapping areas of similar blocks in different groups of filtered four-dimensional similar block matrices are processed by weighted averaging to obtain the basic denoised image.

[0024] The secondary four-dimensional block matching unit is used to select a three-dimensional image block as a reference block B in the basic denoised image, traverse other image blocks in the matching image, and combine blocks similar to the reference block B as similar blocks B' in the fourth dimension to estimate the four-dimensional similar block matrix of the image. Then, the reference block B is reselected for matching until the reference block B traverses the entire basic denoised image. At the same time, the corresponding four-dimensional similar block matrix of the noise image is extracted in the three-dimensional polarization image according to the position of the matched similar block B'.

[0025] The four-dimensional Wiener filtering unit is used to perform a four-dimensional transformation B” on the two types of four-dimensional similar block matrices obtained by the second four-dimensional block matching unit to obtain the four-dimensional similar block matrix of the basic estimated image in the transform domain and the four-dimensional similar block matrix of the noisy image in the transform domain. Then, the transform coefficients of the four-dimensional similar block matrix of the basic estimated image are used as Wiener filtering coefficients. After performing Wiener filtering and four-dimensional inverse transformation B''' on the four-dimensional similar block matrix of the noisy image in the transform domain, the corrected four-dimensional similar block matrix of the noisy image is obtained.

[0026] The final estimation unit is used to restore the three-dimensional similar blocks in the four-dimensional similar block matrix of each group of corrected noise images according to their positions in the basic denoised image in the secondary four-dimensional block matching unit. The overlapping areas of similar blocks in the four-dimensional similar block matrix of different groups of corrected noise images are processed by weighted averaging to obtain the final estimated image.

[0027] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the polarization constraint-based four-dimensional block matching polarization image denoising method.

[0028] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the polarization constraint-based four-dimensional block matching polarization image denoising method.

[0029] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0030] 1. The image denoising method of this invention is mainly based on the idea of ​​nonlocal means, and uses four-dimensional block matching to organically combine regions with similar polarization characteristics in the image for transform domain filtering. Four-dimensional block matching can maintain the inherent physical relationship between the dimensions of polarized images during the denoising process. Therefore, the network can not only further improve the image denoising effect, but also recover the polarization information of the image.

[0031] 2. This invention relates to a polarization image augmentation method, which mainly calculates the Stokes vector by arbitrarily selecting three polarization angles, then calculates the polarization image at any angle based on polarization physical relationships, and finally packages the polarization images into three-dimensional data along the polarization dimension. Data augmentation can strengthen the polarization correlation between polarization images and fully utilize polarization information during the denoising process.

[0032] 3. The method of the present invention constrains the polarization information through the polarization image generation step and the four-dimensional block matching step. After filtering, it can not only efficiently remove noise in the polarization image, but also accurately recover the polarization information of the target from the noisy image, thereby helping to expand the practical application scenarios and scope of polarization imaging. Attached Figure Description

[0033] Figure 1 This is a flowchart of a polarization image denoising method based on polarization information according to an embodiment of the present invention;

[0034] Figure 2a and Figure 2b These are schematic diagrams illustrating the denoising and polarization information restoration effects on polarized images of a metal coin and a plastic doll, respectively. Detailed Implementation

[0035] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The present invention is applicable to different color polarization imaging systems (amplitude division, focal plane division, etc.) and different types of polarization information imaging methods (Stokes, Mueller, etc.). The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0036] The following describes, with reference to the accompanying drawings, a four-dimensional block matching polarization image denoising method based on polarization information proposed according to an embodiment of the present invention, taking the Stokes vector imaging method based on focal plane polarization image as an example.

[0037] Figure 1 This is a flowchart of a four-dimensional block matching polarization image denoising method based on polarization constraints provided in this embodiment. The image denoising method of this embodiment includes:

[0038] S1, a polarization image augmentation method based on polarization Stokes vectors. In this embodiment, the acquired focal plane polarization image includes... Polarization images at four angles, these four images are represented as follows: Then, the Stokes vector is calculated based on the polarization images from these four angles. The calculation method is as follows:

[0039] (1)

[0040] Then, using the inherent physical relationships of polarization information, the Stokes vector is used to calculate the polarization image at any angle:

[0041] (2)

[0042] Formula (2) can be used to amplify three or more polarization images into any number of images. Then, the amplified images are stacked together with the original images in the polarization dimension to form three-dimensional polarization data, denoted as . Its shape is W×H×P, where W is the image width, H is the image length, and P is the polarization image angle.

[0043] S2, the polarization four-dimensional block matching method, firstly, in the three-dimensional polarization image We select an 8×8×8 image block as a reference block, denoted as Then, iterate through and search the three-dimensional polarization image. Other 3D image blocks in the image will be compared with the reference block. Similar image patches as matching blocks The matching method involves calculating the similarity between the two:

[0044] (3)

[0045] in It is the size of the image patch. yes Norm. When the similarity between the matching block and the reference block is less than a threshold. When the matching block is added as a similar block to the similar block set, the matching block is added as a similar block. In, it is represented as:

[0046] (4)

[0047] Similar block set The similar blocks in the matrix are sorted in descending order according to the similarity calculated by formula (3), and then the first 32 similar blocks are used to form a four-dimensional similar block matrix in the additional fourth dimension. Its shape is 8×8×8×32.

[0048] S3, polarization four-dimensional collaborative filtering. First, the four-dimensional similar block matrix... Perform a four-dimensional transformation A", denoted as The transform domain four-dimensional similarity block matrix is ​​obtained. Specifically, the four-dimensional transform A” can be decomposed into a three-dimensional discrete Haar transform on the three dimensions of (image width, image length, polarization angle) and a one-dimensional Haar wavelet transform on the fourth dimension. Then, a hard thresholding filter operation is performed on the transform domain four-dimensional similarity block matrix. Its filter function is defined as:

[0049] (5)

[0050] in The filtering threshold is used. The final filtered four-dimensional similarity block matrix is ​​then obtained. After undergoing a four-dimensional inverse transformation A''', denoted as Then you can get the result, which is represented as:

[0051] (6)

[0052] S4, Basic estimation for polarization image denoising, which involves filtering the four-dimensional similarity block matrix. Based on the initial position of the internal similar blocks (3D similar blocks), each filtered 3D similar block is... Returning the image to its original location and restoring and recombining the image is necessary. Since the same filtered three-dimensional similar block may be located in different filtered four-dimensional similar block matrices, and different similar blocks may contain the same elements, it is necessary to integrate the duplicate image blocks.

[0053] Specifically, in one embodiment of the invention, a weighted average is applied to the overlapping regions of three-dimensional similar blocks in different groups of four-dimensional blocks to obtain a basic estimate of the image. That is, the basic denoised image The formula is:

[0054] (7)

[0055] in It is the characteristic function of each 3D similar block. The weight of each block is expressed as:

[0056] (8)

[0057] in It represents the number of non-zero elements in hard threshold filtering.

[0058] S5, secondary polarization four-dimensional block matching. Using the base denoised image. As input, improve the accuracy of similar block matching. Using four-dimensional block matching in the base denoised image, find the reference block of the base denoised image according to formula (3). Similar blocks Then these similar blocks are added to the base set of similar blocks for the denoised image. , is represented as:

[0059] (9)

[0060] In the formula, the threshold Then, similar blocks of the base denoised image are... Sort by similarity and select the top 32 similar blocks. Utilizing these similar blocks The location yields two four-dimensional similarity block matrices. One of these matrices originates from the three-dimensional polarization image. Another one comes from the base denoised image. These two four-dimensional similarity block matrices are represented as follows: and .

[0061] S6, polarization four-dimensional Wiener filtering. After obtaining two four-dimensional similar block matrices, ... and The transformation domain is obtained by performing a four-dimensional transformation. and transform domain In the transform domain The four-dimensional transform coefficients are used as Wiener filter coefficients. Wiener filtering is an optimal estimator based on the minimum mean square error criterion. This filter minimizes the mean square error between the actual output and the expected output. Therefore, Wiener filtering is an optimal filtering system that can be used to extract signals contaminated by stationary noise. In this embodiment of the invention, the empirical filter coefficients are defined in the following form:

[0062] (10)

[0063] in Representing a three-dimensional polarization image Standard deviation of noise It is a four-dimensional transformation operation, which is different from the transformation in the polarization four-dimensional collaborative filtering step. It consists of a three-dimensional discrete cosine transform and a one-dimensional Haar wavelet transform.

[0064] Therefore, the four-dimensional similarity block matrix of the noisy image obtained after Wiener filtering and subsequent four-dimensional inverse transform with quadratic matching correction is... The estimated value can be expressed by the following formula:

[0065] (11)

[0066] in It is a four-dimensional inverse transform operation, which consists of a three-dimensional discrete cosine inverse transform and a one-dimensional Haar wavelet inverse transform.

[0067] S7, Final estimation of polarization image, similar to the basic estimation, involves using the corrected four-dimensional similarity block matrix of the noisy image. Each corrected similar block is arranged according to its initial position. After restoring the image to its original position and reconstructing it, there will inevitably be many overlapping parts between different similar blocks. Therefore, the pixels in the overlapping parts of different similar blocks may have multiple estimated values. The weighted average of these estimated values ​​is used as the final estimated value for these pixels. After restoring all the four-dimensional similar block matrices, the final estimated image of the polarization image is obtained. Its expression is as follows:

[0068] (12)

[0069] in It is each corrected similar block , characteristic function It is the weight of similar blocks, expressed as:

[0070] (13)

[0071] In embodiments of the present invention, a four-dimensional block matching denoising method based on polarization information is used to remove noise from the polarization image, and then the light intensity image is reconstructed based on the recovered polarization information. Linear polarization degree map (DoLP) and polarization angle map (AoP) are used. Furthermore, mean filtering, median filtering, wavelet transform-based denoising, singular value decomposition-based denoising, and 3D block matching-based denoising methods are compared. Figure 2a and Figure 2b As shown. Figure 2a and Figure 2b The first column shows the noisy image, the last column shows the noise-free ground truth image, and the other columns represent various denoising methods. Figure 2a and Figure 2b The first row is the light intensity diagram, the second row is the linear skewness diagram, and the third row is the polarization angle diagram.

[0072] from Figure 2a and Figure 2b As can be seen from the above, the four-dimensional block matching polarization image denoising algorithm based on polarization information proposed in this invention can effectively remove noise from polarization images. Furthermore, it can effectively recover polarization information from polarization degree images and polarization angle images, which are extremely sensitive to noise. Especially for polarization angle images, a comparison of detail images clearly shows that the method of this invention recovers clearer detail information and is closer to the true image. Therefore, this invention has better polarization image denoising and polarization information recovery effects. In addition, this invention has a certain generalization ability for objects with different polarization characteristics. Figure 2a The middle one is a metal coin. Figure 2b The figure in the middle is a plastic doll; these two objects have different polarization characteristics.

[0073] Preferably, embodiments of this application also provide a four-dimensional block matching polarization image denoising device based on polarization constraints, including...

[0074] The image augmentation unit is used to augment three or more polarization images into any number of polarization images and stack all polarization images into a three-dimensional polarization image.

[0075] The four-dimensional block matching unit is used to select a three-dimensional image block as a reference block A in the three-dimensional polarization image, traverse and match other image blocks in the three-dimensional polarization image, and combine the image blocks that match the reference block A as similar blocks A' in the fourth dimension into a set of four-dimensional similar block matrices. Then, the reference block A is selected again to perform similar block matching until the reference block A traverses the entire three-dimensional polarization image.

[0076] The four-dimensional collaborative filtering unit is used to perform a four-dimensional transformation on each group of four-dimensional similar block matrices to obtain a four-dimensional similar block matrix in the transform domain. Then, the four-dimensional similar block matrix in the transform domain is filtered by hard thresholding. Finally, the four-dimensional similar block matrix in the transform domain is subjected to a four-dimensional inverse transformation to obtain the filtered four-dimensional similar block matrix.

[0077] The basic estimation unit is used to restore the three-dimensional similar blocks in each group of filtered four-dimensional similar block matrices according to their positions in the three-dimensional polarization image in the image augmentation unit. The overlapping areas of similar blocks in different groups of filtered four-dimensional similar block matrices are processed by weighted averaging to obtain the basic denoised image.

[0078] The secondary four-dimensional block matching unit is used to select a three-dimensional image block as a reference block B in the basic denoised image, traverse other image blocks in the matching image, and combine blocks similar to the reference block B as similar blocks B' in the fourth dimension to estimate the four-dimensional similar block matrix of the image. Then, the reference block B is reselected for matching until the reference block B traverses the entire basic denoised image. At the same time, the corresponding four-dimensional similar block matrix of the noise image is extracted in the three-dimensional polarization image according to the position of the matched similar block B'.

[0079] The four-dimensional Wiener filtering unit is used to perform four-dimensional transformation on the two types of four-dimensional similar block matrices obtained by the two-dimensional four-dimensional block matching unit to obtain the four-dimensional similar block matrix of the basic estimated image in the transform domain and the four-dimensional similar block matrix of the noisy image in the transform domain. Then, the transform coefficients of the four-dimensional similar block matrix of the basic estimated image are used as Wiener filtering coefficients. After performing Wiener filtering and four-dimensional inverse transformation on the four-dimensional similar block matrix of the noisy image in the transform domain, the corrected four-dimensional similar block matrix of the noisy image is obtained.

[0080] The final estimation unit is used to restore the three-dimensional similar blocks in the four-dimensional similar block matrix of each group of corrected noise images according to their positions in the basic denoised image in the secondary four-dimensional block matching unit. The overlapping areas of similar blocks in the four-dimensional similar block matrix of different groups of corrected noise images are processed by weighted averaging to obtain the final estimated image.

[0081] Preferably, embodiments of this application also provide a specific implementation of an electronic device capable of implementing all steps in the polarization constraint-based four-dimensional block matching polarization image denoising method described in the above embodiments. The electronic device specifically includes the following:

[0082] Processor, memory, communications interface, and bus;

[0083] The processor, memory, and communication interface communicate with each other via a bus; the communication interface is used to realize information transmission between server-side devices, metering devices, and user-side devices.

[0084] The processor is used to call the computer program in memory. When the processor executes the computer program, it implements all the steps in the polarization constraint-based four-dimensional block matching polarization image denoising method in the above embodiments.

[0085] Embodiments of this application also provide a computer-readable storage medium capable of implementing all steps of the polarization constraint-based four-dimensional block matching polarization image denoising method in the above embodiments. The computer-readable storage medium stores a computer program that, when executed by a processor, implements all steps of the polarization constraint-based four-dimensional block matching polarization image denoising method in the above embodiments.

[0086] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0087] While this application provides method operation steps as shown in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the method can be executed in the order shown in the embodiments or drawings or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0088] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0090] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0091] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0092] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for denoising four-dimensional block matching polarization images based on polarization constraints, characterized in that, Includes the following steps: Step 1. A polarization image data augmentation method based on polarization Stokes vector relationship is used to augment three or more polarization images into any number of polarization images, and stack all polarization images along the polarization dimension into a three-dimensional polarization image to enhance the utilization of the inherent physical relationship between polarization image channels. Step 2. Four-dimensional block matching: Select a three-dimensional image block as reference block A in the three-dimensional polarization image in step 1, traverse and match other image blocks in the three-dimensional polarization image, and combine the image blocks that match reference block A as similar blocks A' in the fourth dimension to form a four-dimensional similar block matrix. Then, reselect reference block A to perform similar block matching until reference block A has traversed the entire three-dimensional polarization image. Step 3. Four-dimensional collaborative filtering: Perform a four-dimensional transformation A” on each group of four-dimensional similar block matrices in Step 2 to obtain a four-dimensional similar block matrix in the transform domain. Then, filter the four-dimensional similar block matrix in the transform domain using hard thresholding. Finally, perform a four-dimensional inverse transformation A''' on the four-dimensional similar block matrix in the transform domain to obtain the filtered four-dimensional similar block matrix. Step 4. Basic estimation: Reconstruct the three-dimensional similar blocks in each group of filtered four-dimensional similar block matrices from Step 3 according to their positions in the three-dimensional polarization image in Step 1. Apply a weighted average to the overlapping areas of similar blocks in different groups of filtered four-dimensional similar block matrices to obtain the basic denoised image. Step 5. Secondary four-dimensional block matching: Select a three-dimensional image block as reference block B from the base denoised image obtained in step 4, traverse and match other three-dimensional image blocks in the base denoised image, and combine blocks similar to reference block B as similar blocks B' in the fourth dimension to estimate the four-dimensional similarity block matrix of the image. Then, reselect reference block B for matching until reference block B traverses the entire base denoised image. At the same time, extract the corresponding four-dimensional similarity block matrix of the noise image in the three-dimensional polarization image according to the position of the matched similar block B'. Step 6. Four-dimensional Wiener filtering: Perform four-dimensional transformation B” on the four-dimensional similar block matrix of the base estimated image and the four-dimensional similar block matrix of the noise image obtained in Step 5 to obtain the four-dimensional similar block matrix of the base estimated image and the four-dimensional similar block matrix of the noise image in the transform domain. Then, use the transform coefficients of the four-dimensional similar block matrix of the base estimated image as Wiener filtering coefficients, and perform Wiener filtering and four-dimensional inverse transformation B’’’ on the four-dimensional similar block matrix of the noise image in the transform domain to obtain the corrected four-dimensional similar block matrix of the noise image. Step 7. Final estimation: Restore the three-dimensional similar blocks in the four-dimensional similar block matrix of each group of corrected noise images according to their positions in the basic denoised image in step 5. Apply weighted average processing to the overlapping areas of similar blocks in the four-dimensional similar block matrix of different groups of corrected noise images to obtain the final estimated image.

2. The four-dimensional block matching polarization image denoising method based on polarization constraints according to claim 1, characterized in that, The polarization image data augmentation method calculates the Stokes vector from three or more polarization images at arbitrary angles, and then calculates the polarization image at arbitrary angles from the Stokes vector based on the polarization relationship, thereby augmenting three or more polarization images into any number of polarization images. All polarization images are stacked in order of angle size to form three-dimensional data, where the three dimensions are image length, image width, and polarization angle.

3. The four-dimensional block matching polarization image denoising method based on polarization constraints according to claim 1, characterized in that, Each reference block is three-dimensional data with a size of L×L×L, where L≥3. This is because complete polarization information can only be constructed from polarization images at three angles. To better utilize polarization information, the matching block contains information from polarization images at least three angles. The similarity between each reference block and each similar block is measured using the L2 norm. The similar blocks are arranged from high to low similarity, and to improve algorithm efficiency, the top N similar blocks are combined in the fourth dimension to form a four-dimensional similarity block matrix.

4. The four-dimensional block matching polarization image denoising method based on polarization constraints according to claim 1, characterized in that, To improve algorithm efficiency, the four-dimensional transformation A” is split into a three-dimensional transformation and a one-dimensional transformation. The three-dimensional discrete Haar transform is applied to the three dimensions of image length, image width, and polarization angle, and the one-dimensional Haar wavelet transform is applied to the fourth dimension. The corresponding four-dimensional inverse transformation A’’’ is the three-dimensional discrete Haar inverse transform and the one-dimensional Haar wavelet inverse transform.

5. The four-dimensional block matching polarization image denoising method based on polarization constraints according to claim 1, characterized in that, To improve algorithm efficiency, the four-dimensional transformation B” is split into a three-dimensional transformation and a one-dimensional transformation. The three-dimensional discrete cosine transform is applied to the three dimensions of image length, image width, and polarization angle, and the one-dimensional Haar wavelet transform is applied to the fourth dimension. The corresponding four-dimensional inverse transformation B’’’ is the three-dimensional discrete cosine inverse transform and the one-dimensional Haar wavelet inverse transform.

6. The four-dimensional block matching polarization image denoising method based on polarization constraints according to claim 1, characterized in that, The images obtained from the basic estimation and the final estimation are three-dimensional polarization images, with the three dimensions being image length, image width, and polarization angle, respectively. In the final estimated image, the polarization image corresponding to the angle of the polarization image before image amplification is extracted as the output.

7. A four-dimensional block matching polarization image denoising device based on polarization constraints, characterized in that, include The image augmentation unit is used to augment three or more polarization images into any number of polarization images and stack all polarization images into a three-dimensional polarization image. The four-dimensional block matching unit is used to select a three-dimensional image block as a reference block A in the three-dimensional polarization image, traverse and match other image blocks in the three-dimensional polarization image, and combine the image blocks that match the reference block A as similar blocks A' in the fourth dimension into a set of four-dimensional similar block matrices. Then, the reference block A is selected again to perform similar block matching until the reference block A traverses the entire three-dimensional polarization image. The four-dimensional collaborative filtering unit is used to perform a four-dimensional transformation A” on each group of four-dimensional similar block matrices to obtain a four-dimensional similar block matrix in the transform domain. Then, the four-dimensional similar block matrix in the transform domain is filtered by hard threshold filtering. Finally, the four-dimensional similar block matrix in the transform domain is subjected to a four-dimensional inverse transformation A''' to obtain the filtered four-dimensional similar block matrix. The basic estimation unit is used to restore the three-dimensional similar blocks in each group of filtered four-dimensional similar block matrices according to their positions in the three-dimensional polarization image in the image augmentation unit. The overlapping areas of similar blocks in different groups of filtered four-dimensional similar block matrices are processed by weighted averaging to obtain the basic denoised image. The secondary four-dimensional block matching unit is used to select a three-dimensional image block as a reference block B in the basic denoised image, traverse other image blocks in the matching image, and combine blocks similar to the reference block B as similar blocks B' in the fourth dimension to estimate the four-dimensional similar block matrix of the image. Then, the reference block B is reselected for matching until the reference block B traverses the entire basic denoised image. At the same time, the corresponding four-dimensional similar block matrix of the noise image is extracted in the three-dimensional polarization image according to the position of the matched similar block B'. The four-dimensional Wiener filtering unit is used to perform a four-dimensional transformation B” on the two types of four-dimensional similar block matrices obtained by the second four-dimensional block matching unit to obtain the four-dimensional similar block matrix of the basic estimated image in the transform domain and the four-dimensional similar block matrix of the noisy image in the transform domain. Then, the transform coefficients of the four-dimensional similar block matrix of the basic estimated image are used as Wiener filtering coefficients. After performing Wiener filtering and four-dimensional inverse transformation B''' on the four-dimensional similar block matrix of the noisy image in the transform domain, the corrected four-dimensional similar block matrix of the noisy image is obtained. The final estimation unit is used to restore the three-dimensional similar blocks in the four-dimensional similar block matrix of each group of corrected noise images according to their positions in the basic denoised image in the secondary four-dimensional block matching unit. The overlapping areas of similar blocks in the four-dimensional similar block matrix of different groups of corrected noise images are processed by weighted averaging to obtain the final estimated image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the polarization constraint-based four-dimensional block matching polarization image denoising method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the polarization constraint-based four-dimensional block matching polarization image denoising method according to any one of claims 1 to 6.

Images