Sparse image stripe defect complementing and repairing method and device

Abstract

The invention discloses a sparse image stripe defect complementing and repairing method and device, and relates to the technical field of image repairing. The method comprises the following steps: acquiring a sparse image with fringe defects; performing two-dimensional difference sparsification on the sparse image with the fringe defect to obtain a sparse difference domain; performing two-dimensional Fourier transform on the sparse difference domain to obtain a difference frequency domain; based on the difference frequency domain, constructing a difference frequency domain binary Hankel structure matrix; and constructing a sparse image completion model based on the differential frequency domain bifolding Hankel structure matrix, and solving the sparse image completion model in a distributed manner by using an alternating iteration ADMM algorithm to obtain a repaired sparse image. According to the method, stripe defects of the sparse image can be effectively processed, the repaired image does not have obvious stripe traces, a new theoretical method and an effective way are provided for sparse image repair, and the method has important significance on image processing in the fields of biomedical imaging, aerospace, astronomy and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image inpainting, in particular to a sparse image stripe missing completion and inpainting method and device. BACKGROUND

[0002] Sparse images have unique value in the fields of medical imaging, aerospace and astronomy. Obtaining accurate, clear and easy-to-process images is crucial for completing image analysis, target detection and recognition tasks. Therefore, it is particularly necessary to perform image inpainting completion on damaged sparse images.

[0003] At present, image inpainting technology generally aims at visible light natural image inpainting completion, and most natural images have smoothness features. Therefore, the repair method usually uses the smoothness features of natural images to construct an image inpainting model. The effect of repair for sparse images is not good. For example, night infrared images have sparsity, and the use of an image matrix low-rank completion method based on the construction of natural image smoothness features for stripe missing repair still has stripe traces in the repaired infrared image.

[0004] Low-rank matrix completion method is to recover a large number of elements by sampling part of the elements of the matrix, which is an effective generalization of the compression sensing theory from vector space to matrix space. The image inpainting technology based on the principle of matrix low-rankness generally models the repair completion algorithm by constraining the low-rank matrix constructed in the time domain of the image. A classic matrix low-rank image inpainting method is based on the relationship of the filter reduction, which constructs an image inpainting completion algorithm model by establishing a two-fold Hankel low-rank structure matrix in the time domain. However, this method is based on the general characteristics of the image time domain smoothness corresponding to the frequency domain sparsity, and is not suitable for images with time domain sparsity, especially when repairing stripe missing sparse images. The repair effect of this method is not good, and the repaired sparse image still has obvious stripe traces. SUMMARY

[0005] In order to solve the problems in the prior art, the present application provides the following technical scheme.

[0006] The first aspect of the present application provides a sparse image stripe missing completion and inpainting method, comprising: acquiring a sparse image with stripe missing; performing two-dimensional difference sparsification on the sparse image with stripe missing to obtain a sparse difference field; performing two-dimensional Fourier transform on the sparse difference field to obtain a difference frequency domain; based on the difference frequency domain, constructing a difference frequency domain two-fold Hankel structure matrix; based on the difference frequency domain two-fold Hankel structure matrix, constructing a sparse image completion model as shown in the following formula: ; ; in, This represents the optimal solution for repairing the image; Represents a two-dimensional Fourier transform operator; This represents a two-dimensional difference operator; Represents a two-dimensional Hankel matrix transformation operator; A sparse image representing stripe defects; Represents a sparse image without defects; Represents the set of undamaged pixels; The sparse image completion model is solved in a distributed manner using the alternating iterative ADMM algorithm to obtain the repaired sparse image.

[0007] Preferably, constructing a differential frequency domain bifold Hankel structure matrix based on the differential frequency domain includes: Obtain the differential frequency domain matrix, and then flip the differential frequency domain matrix by first turning it down 180 degrees and then to the right 180 degrees to obtain the flipped differential frequency domain matrix. A square filter matrix is ​​randomly generated, and the length of the filter matrix is ​​smaller than the length and width of the flipped differential frequency domain matrix; The filter matrix is ​​overlaid on the flipped differential frequency domain matrix to obtain the overlay block of the flipped differential frequency domain matrix; the filter matrix is ​​multiplied by the corresponding points of the overlay block, and the product values ​​are added to obtain an element value; The filter matrix slides along rows and columns starting from the bottom right corner of the flipped differential frequency domain matrix. It slides along rows from right to left until the left edge of the filter matrix covers the left boundary of the flipped differential frequency domain matrix, sliding one pixel at a time and calculating an element value. After reaching the left boundary, it slides along columns from bottom to top one pixel, repeating the sliding along rows from right to left, calculating an element value for each pixel slide, until the top edge of the filter matrix covers the top boundary of the flipped differential frequency domain matrix. Each time the element value is calculated by sliding from right to left, it is placed into a row of the matrix, and the elements are placed in the order from the first row to the last row according to the sliding order. The resulting matrix is ​​the constructed differential frequency domain two-fold Hankel structure matrix.

[0008] Preferably, the step of using the Alternating Iterative Model (ADMM) algorithm to solve the sparse image completion model in a distributed manner to obtain the repaired sparse image includes: Construct the improved sparse image completion model as shown in the following equation: ; ; in, As an auxiliary variable, ; The optimal solution for auxiliary variables; For residual variables, The optimal solution for the residual variable; For penalty parameters, ; Set the iteration conditions and perform the following initialization: use the solution from the previous iteration as a known variable, and initialize... , , It is a matrix of all zeros. , Indicates the first The next iteration; renew : ; in, ; = ; ; These represent sparse images without defects. Length and width; , , They represent respectively to , , The inverse operation; , , It is a matrix of all ones.

[0009] renew : ; Among them, for the matrix Singular value decomposition yields unitary orthogonal matrices. diagonal matrix unitary orthogonal matrix The upper right corner mark Indicates matrix transpose; The elements on the main diagonal are singular values. ,Right now , ; express .

[0010] renew : ; If the iteration condition is met, then the result obtained when the iteration stops is... As the restored sparse image.

[0011] Preferably, the iteration condition is: reaching a preset maximum number of iterations or the relative difference between consecutive iterations. Less than or equal to the set threshold; in, .

[0012] Preferably, the following method is used for two-dimensional differential sparsity: ; Indicates horizontal pair Perform a difference operation; Indicates vertical pair Perform a difference operation; Indicates to Perform two-dimensional difference sparsification operation; = ; = ; These represent sparse images without defects. Length and width.

[0013] right Inverse operation express: .

[0014] A second aspect of the present invention provides a sparse image stripe defect repair device, comprising: The image acquisition module is used to acquire sparse images with stripe defects; The difference module is used to perform two-dimensional difference sparsification on the sparse image with stripe defects to obtain a sparse difference domain. The transformation module is used to perform a two-dimensional Fourier transform on the sparse difference domain to obtain the difference frequency domain. A matrix construction module is used to construct a differential frequency domain bifold Hankel structure matrix based on the differential frequency domain. The model building module is used to construct a sparse image completion model based on the differential frequency domain bifold Hankel structure matrix, as shown in the following equation: ; ; in, This represents the optimal solution for repairing the image; Represents a two-dimensional Fourier transform operator; This represents a two-dimensional difference operator; Represents a two-dimensional Hankel matrix transformation operator; A sparse image representing stripe defects; Represents a sparse image without defects; Represents the set of undamaged pixels; The model solving module is used to solve the sparse image completion model in a distributed manner using the alternating iterative ADMM algorithm to obtain the repaired sparse image.

[0015] Preferably, the matrix construction module constructs the differential frequency domain bifold Hankel structure matrix using the following method: Obtain the differential frequency domain matrix, and then flip the differential frequency domain matrix by first turning it down 180 degrees and then to the right 180 degrees to obtain the flipped differential frequency domain matrix. A square filter matrix is ​​randomly generated, and the length of the filter matrix is ​​smaller than the length and width of the flipped differential frequency domain matrix; The filter matrix is ​​overlaid on the flipped differential frequency domain matrix to obtain the overlay block of the flipped differential frequency domain matrix; the filter matrix is ​​multiplied by the corresponding points of the overlay block, and the product values ​​are added to obtain an element value; The filter matrix slides along rows and columns starting from the bottom right corner of the flipped differential frequency domain matrix. It slides along rows from right to left until the left edge of the filter matrix covers the left boundary of the flipped differential frequency domain matrix, sliding one pixel at a time and calculating an element value. After reaching the left boundary, it slides along columns from bottom to top one pixel, repeating the sliding along rows from right to left, calculating an element value for each pixel slide, until the top edge of the filter matrix covers the top boundary of the flipped differential frequency domain matrix. Each time the element value is calculated by sliding from right to left, it is placed into a row of the matrix, and the elements are placed in the order from the first row to the last row according to the sliding order. The resulting matrix is ​​the constructed differential frequency domain two-fold Hankel structure matrix.

[0016] Preferably, the model solving module obtains the repaired sparse image using the following method: Construct the improved sparse image completion model as shown in the following equation: ; ; in, As an auxiliary variable, ; The optimal solution for auxiliary variables; For residual variables, The optimal solution for the residual variable; For penalty parameters, ; Set the iteration conditions and perform the following initialization: use the solution from the previous iteration as a known variable, and initialize... , , It is a matrix of all zeros. , Indicates the first The next iteration; renew : ; in, ; = ; ; These represent sparse images without defects. Length and width; , , They represent respectively to , , The inverse operation; , , It is a matrix of all ones.

[0017] renew : ; in, ; renew : ; Among them, for the matrix Singular value decomposition yields unitary orthogonal matrices. diagonal matrix unitary orthogonal matrix The upper right corner mark Indicates matrix transpose; The elements on the main diagonal are singular values. ,Right now , ; express .

[0018] If the iteration condition is met, then the result obtained when the iteration stops is... As the restored sparse image.

[0019] Preferably, the iteration condition is: reaching a preset maximum number of iterations or the relative difference between consecutive iterations. Less than or equal to the set threshold; in, .

[0020] Preferably, the difference module performs two-dimensional difference sparsity using the following method: ; Indicates horizontal pair Perform a difference operation; Indicates vertical pair Perform a difference operation; Indicates to Perform two-dimensional difference sparsification operation; = ; = ; These represent sparse images without defects. Length and width.

[0021] right Inverse operation express: .

[0022] The beneficial effects of this invention are as follows: This invention provides a method and apparatus for repairing sparse image stripe defects, which fully utilizes sparse representation and compressed sensing completion theory to handle the problem of sparse image repair. First, to fully utilize the compressed sensing completion principle based on sparsity, this invention improves the sparsity of the image to be processed through two-dimensional differential transform domain processing after acquiring the damaged sparse image. Then, considering the correspondence between temporal sparsity and frequency domain smoothness, a two-fold Hankel low-rank structured matrix is ​​constructed in the frequency domain of the differential image according to the principle of reduction filters. Next, an image repair model based on minimizing the rank of the differential frequency domain two-fold Hankel low-rank structured matrix is ​​constructed. Finally, the ADMM iterative algorithm is derived to solve the model. Experiments show that this invention can effectively handle stripe defects in sparse images, resulting in repaired images without obvious stripe marks, which is superior to the traditional direct two-fold Hankel low-rank matrix method. The sparse image completion and restoration scheme based on the low-rank property of the differential frequency domain matrix provided by this invention offers a new theoretical method and effective approach for the restoration of sparse images such as nighttime infrared images, which is of great significance for image processing in fields such as biomedical imaging, aerospace and astronomy. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the sparse image stripe defect repair method of the present invention. Figure 2 This is a schematic diagram of the sparse image stripe defect repair process described in this invention; Figure 3This is a schematic diagram illustrating the construction process of the differential frequency domain bifold Hankel structure matrix described in this invention; Figure 4 This is a schematic diagram of the functional structure of the sparse image stripe defect repair device of the present invention. Detailed Implementation

[0024] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0025] The method provided by this invention can be implemented in a terminal environment that may include one or more of the following components: a processor, a memory, and a display screen. The memory stores at least one instruction, which is loaded and executed by the processor to implement the method described in the following embodiments.

[0026] A processor may include one or more processing cores. The processor uses various interfaces and lines to connect various parts of the terminal, and performs various functions and processes data by running or executing instructions, programs, code sets or instruction sets stored in memory, and by calling data stored in memory.

[0027] Memory can include random access memory (RAM) or read-only memory (ROM). Memory can be used to store instructions, programs, code, code sets, or instructions.

[0028] The display screen is used to show the user interface of each application.

[0029] In addition, those skilled in the art will understand that the structure of the terminal described above does not constitute a limitation on the terminal. The terminal may include more or fewer components, or combine certain components, or have different component arrangements. For example, the terminal may also include radio frequency circuits, input units, sensors, audio circuits, power supplies, and other components, which will not be described in detail here.

[0030] To address the issue that traditional low-rank matrix methods are ineffective in repairing sparse images with stripe-like contamination, this invention breaks away from the traditional approach of constructing a low-rank matrix in the image's time domain. Instead, it constructs a low-rank matrix from the image's differential frequency domain, proposing a sparse image completion and repair method based on the low-rank property of the differential frequency domain bifold Hankel matrix. This provides a new theoretical method and effective approach for the repair of sparse images such as nighttime infrared images.

[0031] Example 1 like Figure 1 and Figure 2As shown, this embodiment of the invention provides a method for repairing sparse image stripe defects, which may include the following steps: S101, Obtain the sparse image with stripe defects. A sparse image is generally an image whose pixel values ​​are sorted from smallest to largest, and a curve is plotted showing an exponential decay approaching zero. Visually, most pixels in this type of image are close to gray or black, while a small portion contains bright pixels. Examples include nighttime infrared images and medical MRI images. Compared to regular natural light images, sparse images have less detailed, richer, and more balanced pixel values, and also carry a large amount of redundant information. A sparse image with stripe defects refers to an image to be repaired where the missing pixels are distributed in a stripe-like pattern on the sparse image.

[0032] S102, perform two-dimensional differential sparsification on the sparse image with stripe defects to obtain a sparse difference domain; Two-dimensional difference operations involve subtracting each pixel from its neighboring pixels in both the horizontal and vertical dimensions. Leveraging the characteristic of pixel value variations in sparse images, this operation enhances sparsity, resulting in a sparse difference domain. Sparse images (such as nighttime infrared images) inherently contain a large number of pixels with small values, exhibiting a degree of local smoothness, and have a limited number of pixels with absolutely zero values. Utilizing this characteristic, two-dimensional difference operations can increase the number of pixels in the sparse image that approach or equal to zero, further reducing the number of non-zero elements and achieving sparsity. Furthermore, non-zero elements in the sparse difference domain better represent the true feature variations of the image, thus reducing the influence of large pixel values ​​and further minimizing redundant large pixel values. Additionally, the sparse domain after difference is more suitable for subsequent Fourier transform processing, exhibiting a certain degree of smoothness.

[0033] S103, Perform a two-dimensional Fourier transform on the sparse difference domain to obtain the difference frequency domain; A two-dimensional discrete Fourier transform is performed on the sparse difference domain obtained in the previous step to obtain the difference frequency domain. Based on the symmetry property of the Fourier transform—that sparseness in the time domain corresponds to smoothness in the frequency domain—the sparse difference domain, being highly sparse in the time domain, must necessarily possess smoothness in the frequency domain; that is, the resulting difference frequency domain must also possess smoothness characteristics. This operation is crucial for the subsequent construction of the low-rank matrix, ensuring that the bifold Hankel structure matrix constructed in the smooth difference frequency domain possesses low-rank characteristics. Furthermore, this operation is key to resolving the problem of poor restoration results caused by directly constructing the bifold Hankel matrix in the image domain using traditional methods.

[0034] S104, Based on the differential frequency domain, construct a differential frequency domain two-fold Hankel structure matrix. This step operates according to the principle of reduction filters, that is, the smoothed feature point matrix must have a filter such that convolution with it yields a matrix of all zeros. According to the properties of convolution, multiplication in the time domain corresponds to convolution in the frequency domain, and multiplication in the frequency domain corresponds to convolution in the time domain. For the multiplication domain, it can be constructed by vectorizing the filter matrix and multiplying it with the two-fold Toplitz structure matrix or the two-fold Hankel structure matrix to obtain a vector of all zeros. Since multiplying a matrix with a vector yields a zero vector, it can be deduced that this matrix is ​​not a full-rank matrix. In this invention, this matrix is ​​a sparse differential domain matrix of a sparse image, which has strong sparsity, so it can be deduced that this matrix is ​​a low-rank matrix. A low-rank matrix is ​​one in which, after singular value decomposition, the singular values, sorted from smallest to largest, show a small number of large singular values, with most singular values ​​being very small and tending towards 0. Based on the above, the constructed two-fold Hankel structure matrix has the characteristic of low rank. Compared to the traditional method of directly constructing a bifold Hankel matrix in the time domain, this operation has better low-rank characteristics for sparse images by constructing a bifold Hankel matrix in the difference frequency domain. It can better handle subsequent completion optimization of sparse images based on low-rank matrices and reduce stripe marks after restoration.

[0035] S105, based on the differential frequency domain bifold Hankel structure matrix, constructs the sparse image low-rank matrix completion model as shown in the following equation: ; ; in, This represents the optimal solution for repairing the image; Represents a two-dimensional Fourier transform operator; This represents a two-dimensional difference operator; Represents a two-dimensional Hankel matrix transformation operator; A sparse image representing stripe defects; Represents a sparse image without defects; Represents the set of undamaged pixels; S106. The alternating direction multiplier method (ADMM) algorithm is used to solve the sparse image completion model in a distributed manner, resulting in the repaired sparse image. ADMM is an algorithm for solving decomposable, constrained optimization problems. It excels at handling problems where the objective function can be decomposed into the sum of multiple parts. In this invention, the optimal solution obtained by using the ADMM algorithm to solve the sparse image completion model is the repaired sparse image.

[0036] In one embodiment of the present invention, such as Figure 3 As shown, the construction of the differential frequency domain bifold Hankel structure matrix based on the differential frequency domain can be implemented using the following steps: First, for the difference frequency domain matrix The flipped differential frequency domain matrix is ​​obtained by first rotating it downwards by 180 degrees and then to the right by 180 degrees. Then, randomly generate a size of The filter matrix, where Smaller than the length of the image And smaller than the width of the image. ; The size of the filter matrix after the inversion is... The lower right corner area is completely covered; The two covered together Multiply corresponding points of the matrix blocks of the same size, and then add all the product values ​​to get a calculated value; The filter matrix is ​​slid one point at a time within the flipped differential frequency domain matrix. Each time it slides, a calculated value is obtained as described above. The sliding sequence described above is as follows: slide from right to left and from bottom to top. That is, first slide one point from right to left until the filter matrix is ​​about to exceed the left boundary of the flipped differential frequency domain matrix, then slide one pixel up, and then slide one point from right to left again, and so on, until the filter matrix is ​​about to exceed the upper boundary of the flipped differential frequency domain matrix.

[0037] The calculated values ​​obtained by sliding from right to left are placed into a row of a matrix, sequentially from the first row to the last. The resulting matrix is ​​the constructed differential frequency domain two-fold Hankel structure matrix. Its size is .

[0038] In one embodiment of the present invention, the step of using the Alternating Iterative Modulation Model (ADMM) algorithm to solve the sparse image completion model in a distributed manner to obtain the repaired sparse image may include the following steps: Construct the improved sparse image completion model as shown in the following equation: ; ; in, As an auxiliary variable, ; The optimal solution for auxiliary variables; For residual variables, The optimal solution for the residual variable; For penalty parameters, ; Set the iteration conditions and perform the following initialization: use the solution from the previous iteration as a known variable, and initialize... , , It is a matrix of all zeros. , Indicates the first The next iteration; renew : ; in, ; = ; = ; These represent sparse images without defects. Length and width; , , They represent respectively to , , The inverse operation; , , It is a matrix of all ones.

[0039] renew : ; in, ; renew : ; Among them, for the matrix Singular value decomposition (SVD) yields unitary orthogonal matrices. diagonal matrix unitary orthogonal matrix The upper right corner mark Indicates matrix transpose; The elements on the main diagonal are singular values. ,Right now , ; express .

[0040] If the iteration condition is met, then the result obtained when the iteration stops is... As the restored sparse image.

[0041] Specifically, the following methods can be used for implementation: Using the nuclear norm as a low-rank constraint, the rank minimization model is relaxed to the nuclear norm minimization model to establish an image restoration and completion model.

[0042] A sparse image completion model based on the low-rank property of the difference frequency domain matrix can be constructed as shown in the following equation. (1) in, This represents the optimal solution for repairing the image. Represents the two-dimensional Fourier transform operator. This represents a two-dimensional difference operator. This represents the two-dimensional Hankel matrix transformation operator. Represents a contaminated sparse image. Represents the original sparse image. This represents the set of undamaged pixels.

[0043] Specifically Yes The two-dimensional difference sparsification operation is performed as shown in the following equation. (2) This indicates that a difference operation is performed horizontally. This indicates that a difference operation is performed vertically.

[0044] The model is optimized and solved using a distributed alternating iterative algorithm (1).

[0045] First, we introduce two variables, an auxiliary variable. and residual variables Model (1) can be written in the following form: (3) in, For penalty parameters ( Then model (1) solves for an unknown variable. The problem becomes solving the model (3) to solve for the three unknown variables. Sub-problems. Then, each sub-problem is solved iteratively in a distributed manner.

[0046] Step 1, Initialization. Using the solution from the previous iteration as a known variable, ... Indicates the first Next iteration. Initialization. , , It is a matrix of all zeros. .

[0047] Step 2, set iteration conditions: reach the maximum number of iterations or the relative difference between previous and subsequent iterations. Set the maximum number of iterations. . ,in Let Frobenius be the matrix norm.

[0048] Step 3, Solve Sub-problems: (4) Taking the derivative of equation (4) and setting it to zero, we obtain the extremum condition: in , , They represent respectively to , , The inverse operation. Among them... , , It is an all-one matrix. By Gauss-Seld's method, we can obtain: (5) Introduction By performing variable substitution, According to equation (5), we can obtain: The solution to equation (4) is obtained: ; (6) Step 4, Solve Sub-problems: (7) Will Perform singular value decomposition: in It is a diagonal matrix, and the elements on the main diagonal are singular values. ,Right now , , Representing images respectively Length and width.

[0049] Will Singular value thresholding: Then, by the shrinking singular value thresholding algorithm, equation (7) has a unique solution and can be obtained in one step: .

[0050] Step 5, Solve Sub-problems: It can be solved in one step. .

[0051] Step 6: Output the iteration condition when the iteration stops. This refers to the repaired image data.

[0052] In one embodiment of the present invention, the iteration condition may be: reaching a preset maximum number of iterations or the relative difference between previous and subsequent iterations. Less than or equal to the set threshold; in, , It is the F-norm of the matrix (full name: Frobenius norm).

[0053] Example 2 like Figure 4 As shown, another aspect of the present invention also includes a functional module architecture that is completely consistent with the aforementioned method flow. That is, the embodiments of the present invention also provide a sparse image stripe defect repair device, including: Image acquisition module 401 is used to acquire sparse images with stripe defects; The difference module 402 is used to perform two-dimensional difference sparsification on the sparse image with stripe defects to obtain a sparse difference domain. Transformation module 403 is used to perform a two-dimensional Fourier transform on the sparse differential domain to obtain the differential frequency domain; Matrix construction module 404 is used to construct a differential frequency domain two-fold Hankel structure matrix based on the differential frequency domain; Model building module 405 is used to construct a sparse image completion model based on the differential frequency domain bifold Hankel structure matrix, as shown in the following equation: ; ; in, This represents the optimal solution for repairing the image; Represents a two-dimensional Fourier transform operator; This represents a two-dimensional difference operator; Represents a two-dimensional Hankel matrix transformation operator; A sparse image representing stripe defects; Represents a sparse image without defects; Represents the set of undamaged pixels; The model solving module 406 is used to solve the sparse image completion model in a distributed manner using the alternating iterative ADMM algorithm to obtain the repaired sparse image.

[0054] Furthermore, the matrix construction module constructs the differential frequency domain bifold Hankel structure matrix using the following method: First, for the difference frequency domain matrix The flipped differential frequency domain matrix is ​​obtained by first rotating it downwards by 180 degrees and then to the right by 180 degrees. Then, randomly generate a size of The filter matrix, where Smaller than the length of the image And smaller than the width of the image. ; The size of the filter matrix after the inversion is... The lower right corner area is completely covered; The two covered together Multiply corresponding points of the matrix blocks of the same size, and then add all the product values ​​to get a calculated value; The filter matrix is ​​slid one point at a time within the flipped differential frequency domain matrix. Each time it slides, a calculated value is obtained as described above. The sliding sequence described above is as follows: slide from right to left and from bottom to top. That is, first slide one point from right to left until the filter matrix is ​​about to exceed the left boundary of the flipped differential frequency domain matrix, then slide one pixel up, and then slide one point from right to left again, and so on, until the filter matrix is ​​about to exceed the upper boundary of the flipped differential frequency domain matrix.

[0055] The calculated values ​​obtained by sliding from right to left are placed into a row of a matrix, sequentially from the first row to the last. The resulting matrix is ​​the constructed differential frequency domain two-fold Hankel structure matrix. Its size is .

[0056] Furthermore, the model solving module obtains the repaired sparse image using the following method: Construct the improved sparse image completion model as shown in the following equation: ; ; in, As an auxiliary variable, ; The optimal solution for auxiliary variables; For residual variables, The optimal solution for the residual variable; For penalty parameters, ; Set the iteration conditions and perform the following initialization: use the solution from the previous iteration as a known variable, and initialize... , , It is a matrix of all zeros. , Indicates the first The next iteration; renew : ; in, ; express ; express ; These represent sparse images without defects. Length and width; , They are respectively , ; , , They represent respectively to , , The inverse operation; , , It is a matrix of all ones.

[0057] renew : ; in, ; renew : ; Among them, for the matrix Singular value decomposition (SVD) yields unitary orthogonal matrices. diagonal matrix unitary orthogonal matrix The upper right corner mark Indicates matrix transpose; The elements on the main diagonal are singular values. ,Right now , ; express .

[0058] If the iteration condition is met, then the result obtained when the iteration stops is... As the restored sparse image.

[0059] Furthermore, the iteration condition is: reaching a preset maximum number of iterations or the relative difference between consecutive iterations. Less than or equal to the set threshold; in, , It is the F-norm of the matrix (full name: Frobenius norm).

[0060] Furthermore, in the difference module, two-dimensional difference sparsity is performed using the following method: ; Indicates horizontal pair Perform a difference operation; Indicates vertical pair Perform a difference operation; Indicates to Perform two-dimensional difference sparsification operation; = ; = ; These represent sparse images without defects. Length and width.

[0061] right Inverse operation express: .

[0062] This device can be implemented using the sparse image stripe defect completion and repair method provided in Embodiment 1 above. For the specific implementation method, please refer to the description in Embodiment 1, which will not be repeated here.

[0063] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention. Clearly, those skilled in the art can make various alterations and modifications to the invention without departing from its spirit and scope. Thus, if these modifications and modifications of the invention fall within the scope of the claims and their equivalents, the invention is also intended to include these modifications and modifications.

Claims

1. A method for repairing sparse image stripe defects, characterized in that, include: Obtain sparse images with stripe defects; Two-dimensional differential sparsification is performed on the sparse image with stripe defects to obtain a sparse difference domain; A two-dimensional Fourier transform is performed on the sparse difference domain to obtain the difference frequency domain; Based on the differential frequency domain, a differential frequency domain two-fold Hankel structure matrix is ​​constructed; Based on the differential frequency domain bifold Hankel structure matrix, a sparse image completion model is constructed as shown in the following equation: ； ； in, This represents the optimal solution for repairing the image; Represents a two-dimensional Fourier transform operator; This represents a two-dimensional difference operator; Represents a two-dimensional Hankel matrix transformation operator; A sparse image representing stripe defects; Represents a sparse image without defects; Represents the set of undamaged pixels; The sparse image completion model is solved in a distributed manner using the alternating iterative ADMM algorithm to obtain the repaired sparse image.

2. The sparse image stripe defect repair method as described in claim 1, characterized in that, The construction of a differential frequency domain two-fold Hankel structure matrix based on the differential frequency domain includes: Obtain the differential frequency domain matrix, and then flip the differential frequency domain matrix by first turning it down 180 degrees and then to the right 180 degrees to obtain the flipped differential frequency domain matrix. A square filter matrix is ​​randomly generated, and the length of the filter matrix is ​​smaller than the length and width of the flipped differential frequency domain matrix; The filter matrix is ​​overlaid on the flipped differential frequency domain matrix to obtain the overlay block of the flipped differential frequency domain matrix; the filter matrix is ​​multiplied by the corresponding points of the overlay block, and the product values ​​are added to obtain an element value; The filter matrix slides along rows and columns starting from the bottom right corner of the flipped differential frequency domain matrix. It slides along rows from right to left until the left edge of the filter matrix covers the left boundary of the flipped differential frequency domain matrix, sliding one pixel at a time and calculating an element value. After reaching the left boundary, it slides along columns from bottom to top one pixel, repeating the sliding along rows from right to left, calculating an element value for each pixel slide, until the top edge of the filter matrix covers the top boundary of the flipped differential frequency domain matrix. Each time the element value is calculated by sliding from right to left, it is placed into a row of the matrix, and the elements are placed in the order from the first row to the last row according to the sliding order. The resulting matrix is ​​the constructed differential frequency domain two-fold Hankel structure matrix.

3. The sparse image stripe defect repair method as described in claim 1, characterized in that, The method of using the Alternating Iterative Model (ADMM) algorithm to solve the sparse image completion model in a distributed manner to obtain the repaired sparse image includes: Construct the improved sparse image completion model as shown in the following equation: ； ； in, As an auxiliary variable, ; The optimal solution for auxiliary variables; For residual variables, The optimal solution for the residual variable; For penalty parameters, ; Set the iteration conditions and perform the following initialization: use the solution from the previous iteration as a known variable, and initialize... , , It is a matrix of all zeros. , Indicates the first The next iteration; renew : ； in, ; = ; ; These represent sparse images without defects. Length and width; , , They represent respectively to , , The inverse operation; , , It is a matrix of all ones. renew : ； Among them, for the matrix Singular value decomposition yields unitary orthogonal matrices. diagonal matrix unitary orthogonal matrix The upper right corner mark Indicates matrix transpose; The elements on the main diagonal are singular values. ,Right now , ; express . renew : ； If the iteration condition is met, then the result obtained when the iteration stops is... As the restored sparse image.

4. The sparse image stripe defect repair method as described in claim 3, characterized in that, The iteration condition is: reaching a preset maximum number of iterations or the relative difference between consecutive iterations. Less than or equal to the set threshold; in, .

5. The sparse image stripe defect repair method as described in claim 1, characterized in that, Two-dimensional difference sparsification is performed using the following method: ； Indicates horizontal pair Perform a difference operation; Indicates vertical pair Perform a difference operation; Indicates to Perform two-dimensional difference sparsification operation; = ; = ; These represent sparse images without defects. Length and width. right Inverse operation express: 。 6. A device for repairing sparse image stripe defects, characterized in that, include: The image acquisition module is used to acquire sparse images with stripe defects; The difference module is used to perform two-dimensional difference sparsification on the sparse image with stripe defects to obtain a sparse difference domain. The transformation module is used to perform a two-dimensional Fourier transform on the sparse difference domain to obtain the difference frequency domain. A matrix construction module is used to construct a differential frequency domain bifold Hankel structure matrix based on the differential frequency domain. The model building module is used to construct a sparse image completion model based on the differential frequency domain bifold Hankel structure matrix, as shown in the following equation: ； ； in, This represents the optimal solution for repairing the image; Represents a two-dimensional Fourier transform operator; This represents a two-dimensional difference operator; Represents a two-dimensional Hankel matrix transformation operator; A sparse image representing stripe defects; Represents a sparse image without defects; Represents the set of undamaged pixels; The model solving module is used to solve the sparse image completion model in a distributed manner using the alternating iterative ADMM algorithm to obtain the repaired sparse image.

7. The sparse image stripe defect repair device as described in claim 6, characterized in that, The matrix construction module constructs the differential frequency domain bifold Hankel structure matrix using the following method: Obtain the differential frequency domain matrix, and then flip the differential frequency domain matrix by first turning it down 180 degrees and then to the right 180 degrees to obtain the flipped differential frequency domain matrix. A square filter matrix is ​​randomly generated, and the length of the filter matrix is ​​smaller than the length and width of the flipped differential frequency domain matrix; The filter matrix is ​​overlaid on the flipped differential frequency domain matrix to obtain the overlay block of the flipped differential frequency domain matrix; the filter matrix is ​​multiplied by the corresponding points of the overlay block, and the product values ​​are added to obtain an element value; The filter matrix slides along rows and columns starting from the bottom right corner of the flipped differential frequency domain matrix. It slides along rows from right to left until the left edge of the filter matrix covers the left boundary of the flipped differential frequency domain matrix, sliding one pixel at a time and calculating an element value. After reaching the left boundary, it slides along columns from bottom to top one pixel, repeating the sliding along rows from right to left, calculating an element value for each pixel slide, until the top edge of the filter matrix covers the top boundary of the flipped differential frequency domain matrix. Each time the element value is calculated by sliding from right to left, it is placed into a row of the matrix, and the elements are placed in the order from the first row to the last row according to the sliding order. The resulting matrix is ​​the constructed differential frequency domain two-fold Hankel structure matrix.

8. The sparse image stripe defect repair device as described in claim 6, characterized in that, The model solving module obtains the repaired sparse image using the following method: Construct the improved sparse image completion model as shown in the following equation: ； ； in, As an auxiliary variable, ; The optimal solution for auxiliary variables; For residual variables, The optimal solution for the residual variable; For penalty parameters, ; Set the iteration conditions and perform the following initialization: use the solution from the previous iteration as a known variable, and initialize... , , It is a matrix of all zeros. , Indicates the first The next iteration; renew : ； in, ; = ; ; These represent sparse images without defects. Length and width; , , They represent respectively to , , The inverse operation; , , It is a matrix of all ones. renew : ； in, ； renew : ； Among them, for the matrix Singular value decomposition yields unitary orthogonal matrices. diagonal matrix unitary orthogonal matrix The upper right corner mark Indicates matrix transpose; The elements on the main diagonal are singular values. ,Right now , ; express . If the iteration condition is met, then the result obtained when the iteration stops is... As the restored sparse image.

9. The sparse image stripe defect repair device as described in claim 8, characterized in that, The iteration condition is: reaching a preset maximum number of iterations or the relative difference between consecutive iterations. Less than or equal to the set threshold; in, .

10. The sparse image stripe defect repair device as described in claim 1, characterized in that, The difference module performs two-dimensional difference sparsity using the following method: ； Indicates horizontal pair Perform a difference operation; Indicates vertical pair Perform a difference operation; Indicates to Perform two-dimensional difference sparsification operation; = ; = ; These represent sparse images without defects. Length and width. right Inverse operation express: 。

Images