A remote sensing image stripe noise removal method, device, equipment and storage medium

Abstract

The application discloses a remote sensing image stripe noise removal method and device, equipment and storage medium, and relates to the technical field of remote sensing image processing. The method comprises the following steps: acquiring a to-be-processed remote sensing image, constructing a smoothing constraint term based on an edge weight factor; calculating an average cross-track profile based on gradient information, and constructing a statistical constraint term; constructing a stripe sparse constraint based on an RBS regularization method; combining the smoothing constraint term, the statistical constraint term and the stripe sparse constraint to construct an optimization model for removing stripe noise, calculating the optimization model to remove the stripe noise, and outputting a clean image without stripe noise. The application combines the smoothing constraint term and the statistical constraint term, so that the constructed optimization model can fully consider the prior information of a potential clean image, and the prior information of stripe noise is fully utilized through the stripe sparse constraint, thereby effectively removing the stripe noise in a line array push-broom remote sensing image, and improving the radiation quality of the remote sensing image.

Description

Technical Field

[0001] This application relates to the field of remote sensing image processing technology, and in particular to a method, apparatus, device and storage medium for removing stripe noise from remote sensing images. Background Technology

[0002] Linear pushbroom cameras possess super-resolution imaging capabilities, wide swath and efficient coverage, radiometric stability and anti-interference properties, and flexible multi-spectral configuration. Remote sensing satellites equipped with linear pushbroom cameras can produce high-resolution, high-precision remote sensing images. However, during operation, pushbroom remote sensing satellites equipped with linear pushbroom cameras are susceptible to factors such as temperature variations, space radiation environment, electromagnetic interference, and sensor aging. These factors can lead to uneven sensor imaging response, resulting in a large amount of stripe noise in the images, which severely affects the quality of remote sensing images and limits the practicality of pushbroom remote sensing satellites for further analysis and application.

[0003] Currently, commonly used stripe noise removal methods include: 1) Statistical methods: These methods assume that the statistical information of each sensor is consistent and use the statistical information of a reference detector to correct stripe pixels. This method is theoretically simple, easy to implement, and has high execution efficiency. It can remove regular stripes in images well, but its effect on nonlinear or irregular stripe noise is poor; 2) Filtering methods: These methods use filters to truncate specific stripe components in the transform domain, thereby removing stripe noise. This method is effective for removing periodic stripe noise, but its effect on removing non-periodic stripes or more complex types of stripes is poor; 3) Optimization methods: These methods treat the stripe removal task as an ill-posed inverse problem. They construct and solve an optimization model using prior knowledge of stripe noise and potentially clean images to achieve stripe noise removal. However, this approach ignores the non-uniformity of complex stripe noise in real-world scenarios, which can easily lead to insufficient or excessive smoothing in certain areas. It can also easily ignore the statistical features of the image. 4) Deep learning-based methods: Most existing models are trained end-to-end in a supervised manner. This means that in order to simulate stripe noise in real remote sensing images as much as possible, it is difficult to obtain enough training samples, and thus it is difficult to train a deep learning model that meets the requirements.

[0004] Therefore, there is currently a lack of a stable and reliable method for removing stripe noise from remote sensing images. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for removing stripe noise from remote sensing images, to address the deficiencies in the aforementioned related technologies. The technical solution is as follows:

[0006] In a first aspect, this application provides a method for removing stripe noise from remote sensing images, comprising:

[0007] Acquire the remote sensing image to be processed;

[0008] Determine the edge weight factor for each pixel in the remote sensing image to be processed, and construct a smoothing constraint term based on the edge weight factor;

[0009] The location of stripe noise in the remote sensing image to be processed is determined based on the gradient information of the remote sensing image to be processed. The average cross trajectory profile of the remote sensing image to be processed is calculated based on the location of the stripe noise. Statistical constraint terms are constructed based on the average cross trajectory profile.

[0010] Stripe sparsity constraints are constructed based on the RBS regularization method;

[0011] An optimization model for removing stripe noise is constructed by combining the smoothing constraint term, the statistical constraint term, and the stripe sparsity constraint term.

[0012] The optimization model is calculated to remove stripe noise from the remote sensing image to be processed, and a clean image without stripe noise is output.

[0013] In one alternative to the first aspect, determining the edge weight factor for each pixel in the remote sensing image to be processed includes:

[0014] The remote sensing image to be processed is smoothed by guided filtering to obtain smoothing components and high-frequency components.

[0015] Calculate the standard deviation of the pixel DN value corresponding to the smooth component and the standard deviation of the pixel DN value corresponding to the high-frequency component at each pixel location. Based on the standard deviation of the pixel DN value, calculate the initial weight factor at each pixel location using the following formula:

[0016] ;

[0017] The initial weight factors are normalized, and the edge weight factors at each pixel location are determined based on the comparison between the normalized initial weight factors and the preset normalization threshold, using the following formula:

[0018] ;

[0019] The smoothing constraint term constructed based on the edge weight factor includes:

[0020] ;

[0021] in, The position coordinates of the pixel, subscript The subscript represents the smoothing component. This refers to the high-frequency components. Coordinates in the smooth component Standard deviation of pixel DN value at that location Coordinates in the high-frequency components Standard deviation of pixel DN value at that location The coordinates in the remote sensing image to be processed The initial weighting factor at the location, These are the initial weighting factors after normalization. To preset the normalization threshold, Let W be the edge weight matrix, and let W be the parameter. The coordinates in the remote sensing image to be processed Edge weighting factor at the location; For the smoothing constraint term, For the clean image, The remote sensing image to be processed, For the partial differential operator along the direction of the fringe noise, For partial differential operators across the fringe noise direction, For parameters, Indicates the calculation of internal elements The L1 norm.

[0022] In one alternative embodiment of the first aspect, determining the location of stripe noise in the remote sensing image to be processed based on gradient information of the remote sensing image to be processed includes:

[0023] The gradient difference between the gradient along the stripe noise direction and the gradient across the stripe noise direction in the remote sensing image to be processed is calculated to generate a binary image. The type of each pixel in the remote sensing image to be processed is determined based on the binary image, using the formula:

[0024] ;

[0025] in, m is the number of rows in the remote sensing image to be processed. , where n is the column number of the remote sensing image to be processed. The gradient of the cross-striped noise direction of the pixel in the i-th row and j-th column of the remote sensing image to be processed. The gradient of the pixel in the i-th row and j-th column of the remote sensing image to be processed along the stripe noise direction. For the binary image, A value of 1 indicates that the pixel in the i-th row and j-th column is a pixel with stripe noise. A value of 0 indicates that the pixel in the i-th row and j-th column is a pixel without stripe noise. This indicates retrieving the internal elements. The absolute value;

[0026] Based on the binary image, determine the number of pixels with stripe noise in all columns of the i-th row of the remote sensing image to be processed. If the number is greater than a preset pixel count threshold, then stripe noise is determined to exist in the i-th row, and the location of the stripe noise is obtained by applying the formula:

[0027] ;

[0028] in, To preset the pixel count threshold, For parameters, Used to indicate the location of stripe noise This indicates that there is stripe noise in the i-th row.

[0029] In one alternative of the first aspect, the step of calculating the average cross trajectory profile of the remote sensing image to be processed based on the location of the stripe noise, and constructing a statistical constraint term based on the average cross trajectory profile, includes:

[0030] Construct a one-dimensional variational model for calculating the average cross trajectory profile curve, including:

[0031] ;

[0032] The one-dimensional variational model is solved using the iterative weighted least squares method, applying the formula:

[0033] ;

[0034] Update the fidelity weight matrix used for the k-th iteration by applying the formula:

[0035] ;

[0036] ;

[0037] The average cross trajectory profile curve is obtained by performing the k-th iteration based on the updated fidelity weight matrix, using the formula:

[0038] ;

[0039] If the average cross trajectory profile curve obtained in the k-th iteration is... The average cross trajectory profile curve obtained from the (k-1)th iteration If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, output the average cross trajectory profile curve of the kth iteration.

[0040] The statistical constraint term is constructed based on the output average cross trajectory profile curve, and the formula is applied:

[0041] ;

[0042] in, To estimate the obtained average cross trajectory profile curve, For an element to be all column vectors, D is the Gaussian blur kernel , The parameter used to adjust the smoothness of the average cross trajectory profile curve. Here, k is the parameter, representing the iteration number. Let be the fidelity weight matrix for the k-th iteration. For the statistical constraint term, The average cross trajectory profile curve is calculated based on the remote sensing image to be processed. Let P be a column vector representing the locations of the stripe noise, and T be the transpose of the matrix. The Hadamard product of two matrices. Indicates the calculation of internal elements L2 norm, Indicates the calculation of internal elements of Norm.

[0043] In one alternative to the first aspect, the construction of stripe sparsity constraints based on the RBS regularization method includes:

[0044] The striped image corresponding to the remote sensing image to be processed is divided into multiple image blocks along the cross-striped noise direction; the height of each image block is the same as the height of the striped image, the width of each image block is the same, and the sum of the widths of all image blocks is the same as the width of the striped image.

[0045] The block weight matrix for each image patch is determined, and the stripe sparsity constraint is constructed using RBS regularization, applying the formula:

[0046] ;

[0047] ;

[0048] ;

[0049] in, This represents the z-th image patch. s is the total number of image patches. For image blocks width, , This represents the e-th row of the z-th image patch. , Let z be the block weight matrix corresponding to the z-th image block. This represents the e-th row in the block weight matrix corresponding to the z-th image block. For a non-zero parameter, This is the sparse constraint for the stripes. For parameters, Weighted for block weight matrix Mixed norm.

[0050] In one alternative approach of the first aspect, a numerical fidelity term corresponding to the remote sensing image to be processed is obtained, and an optimization model for removing stripe noise is constructed by combining the smoothing constraint term, the statistical constraint term, and the stripe sparsity constraint, including:

[0051]

[0052] in, This refers to the numerical fidelity term corresponding to the remote sensing image to be processed.

[0053] In one alternative of the first aspect, calculating the optimization model to remove stripe noise from the remote sensing image to be processed and output a clean image includes:

[0054] Define the problem as minimizing a clean image:

[0055]

[0056] By introducing auxiliary variables, the problem of minimizing the clean image is transformed into a constrained minimization problem, and the formula is applied:

[0057] ;

[0058] H= ;

[0059] ;

[0060] Determine the corresponding augmented Lagrangian function and apply the formula:

[0061] ;

[0062] Iterative calculations based on the Fast Two-Dimensional Fourier Transform yield a two-dimensional matrix form of the clean image, applying the formula:

[0063] ;

[0064] Define the problem of minimizing the fringe pattern:

[0065] ;

[0066] By introducing auxiliary variables, we obtain the corresponding augmented Lagrange function, and apply the formula:

[0067] ;

[0068] ;

[0069] ;

[0070] Iterative calculations based on the Fast Two-Dimensional Fourier Transform yield the two-dimensional matrix form of the stripe image, applying the formula:

[0071] ;

[0072] If the iteration termination condition is met, output the clean image and the corresponding stripe image obtained in the last iteration.

[0073] Among them, H, V, and Q are auxiliary variables. , , For Lagrange multipliers, , , As a penalty parameter, the two-dimensional matrix forms of the clean image and the striped image of the same iteration number are calculated simultaneously in the same iteration process.

[0074] Secondly, this application also provides a remote sensing image stripe noise removal device, comprising:

[0075] The data acquisition module is used to acquire remote sensing images to be processed;

[0076] A constraint construction module is used to determine the edge weight factor of each pixel in the remote sensing image to be processed, and to construct a smoothing constraint term based on the edge weight factor.

[0077] The constraint construction module is also used to determine the position of stripe noise in the remote sensing image to be processed based on the gradient information of the remote sensing image to be processed, calculate the average cross trajectory profile of the remote sensing image to be processed based on the position of the stripe noise, and construct statistical constraint terms based on the average cross trajectory profile.

[0078] The constraint construction module is also used to construct stripe sparse constraints based on the RBS regularization method;

[0079] The calculation module is used to combine the smoothing constraint term, the statistical constraint term, and the fringe sparsity constraint to construct an optimization model for removing fringe noise;

[0080] The calculation module is also used to calculate the optimization model based on the alternating direction multiplier method to remove stripe noise in the remote sensing image to be processed and output a clean image.

[0081] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method provided by the first aspect of this application or any implementation thereof.

[0082] Fourthly, this application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method provided by the first aspect of this application or any implementation thereof.

[0083] The beneficial effects of the technical solutions provided in some embodiments of this application include at least the following:

[0084] This application provides a method for removing stripe noise from remote sensing images. It constructs a smoothing constraint term based on the edge weight factor of each pixel in the remote sensing image to be processed, which avoids the defects of related technologies that use only a single scalar as a smoothing coefficient, resulting in insufficient or excessive smoothing in some areas. This method can better protect image details.

[0085] The location of stripe noise in the remote sensing image to be processed is determined by gradient information, and the average cross trajectory profile is calculated by combining the location of the stripe noise. The statistical constraint term constructed in this way can provide more comprehensive information for stripe noise removal.

[0086] The stripe sparsity constraint is constructed by using the RBS regularization method, which can accurately describe the stripe sparsity even for complex non-uniform stripes, making the constructed optimization model applicable to real-world scenarios.

[0087] By combining smoothing and statistical constraints, the constructed optimization model can fully consider the prior information of potentially clean images. Through sparse stripe constraints, the constructed optimization model can make full use of the prior information of stripe noise, thereby effectively removing stripe noise in linear pushbroom remote sensing images and improving the radiometric quality of remote sensing images. Attached Figure Description

[0088] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0089] Figure 1 This is one of the flowcharts illustrating a method for removing stripe noise from remote sensing images provided in this application embodiment;

[0090] Figure 2 This is a second schematic flowchart of a remote sensing image stripe noise removal method provided in the embodiments of this application;

[0091] Figure 3 This is a schematic diagram of a remote sensing image for a method of removing stripe noise from a remote sensing image provided in an embodiment of this application;

[0092] Figure 3 Part (a) is a schematic diagram of the remote sensing image to be processed. Figure 3 Part (b) is a schematic diagram of the image edges. Figure 3 Part (c) is a schematic diagram of the smoothing component. Figure 3 The middle (d) section is a schematic diagram of the high-frequency components. Figure 3 Part (e) is a schematic diagram of the initial weighting factors. Figure 3 Part (f) is a schematic diagram of the edge weighting factor;

[0093] Figure 4 This is a schematic diagram of the structure of a remote sensing image stripe noise removal device provided in an embodiment of this application;

[0094] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0095] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0096] The terms "comprising" and "having," and any variations thereof, in the specification, claims, and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or modules is not limited to the steps or modules listed, but may optionally include steps or modules not listed, or may optionally include other steps or modules inherent to such process, method, product, or apparatus.

[0097] It should be noted that the terms "first" and "second" used in this application are merely to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in an order other than those described or illustrated herein.

[0098] It should be noted that the stripe removal models in related technologies have some problems:

[0099] 1) The relevant technology adopts the norm l 1-norm or l 2,1 The -norm parameter is used to describe the global sparsity of the stripes, but the appearance of stripe noise varies greatly in real-world scenarios. This approach ignores the non-uniformity of complex stripes in real-world scenarios.

[0100] 2) Related techniques use a single scalar as a smoothing coefficient to smooth all stripes in an image. However, stripe noise may exhibit different intensities in different regions of the image. As a result, the smoothing method of related techniques may lead to problems of insufficient or excessive smoothing in some areas.

[0101] 3) In order to reduce computational complexity, related technologies ignore the statistical features and other information of the image, and the established constraint models are too ideal and difficult to apply to stripe noise removal in real-world scenarios.

[0102] The present application will now be described in detail with reference to specific embodiments.

[0103] Next, combine Figure 1-2 This application introduces a method for removing stripe noise from remote sensing images, based on embodiments thereof. For details, please refer to... Figure 1 , Figure 1 This illustration shows a flowchart of a remote sensing image stripe noise removal method provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps:

[0104] S101, acquire the remote sensing image to be processed.

[0105] S102, determine the edge weight factor of each pixel in the remote sensing image to be processed, and construct a smoothing constraint term based on the edge weight factor.

[0106] S103, determine the location of stripe noise in the remote sensing image to be processed based on the gradient information of the remote sensing image to be processed, calculate the average cross trajectory profile of the remote sensing image to be processed based on the location of the stripe noise, and construct a statistical constraint term based on the average cross trajectory profile.

[0107] S104, constructing stripe sparsity constraints based on RBS regularization method.

[0108] S105, combining the smoothing constraint, the statistical constraint, and the fringe sparsity constraint, an optimization model for removing fringe noise is constructed.

[0109] S106, Calculate the optimization model to remove stripe noise in the remote sensing image to be processed, and output a clean image without stripe noise.

[0110] Specifically, in S101, for the acquired pushbroom remote sensing image, the direction of the stripe noise in the remote sensing image can be used to determine whether to perform rotation processing. If the stripe noise is vertically distributed, a 90-degree clockwise rotation is performed; if the stripe noise is horizontally distributed, no rotation is performed, thus obtaining a remote sensing image to be processed where the stripe noise is horizontally distributed. The remote sensing image to be processed is as follows: Figure 2 As shown, for example Figure 2 The example image Y is a noisy image.

[0111] Specifically, in S102, a weighting factor can be defined to describe image edges based on gradient domain optimization and edge-aware weighting theory, using the formula:

[0112] ;

[0113] For image A, pixels can be selected. Centered The standard deviation of the DN values ​​of the pixels within the window is calculated. You can select pixels Centered The standard deviation of the DN values ​​of the pixels within the window is calculated. . Pixels can be measured Relative to the importance of the entire image A, windows of different sizes can be used simultaneously, thus enabling... It can effectively separate the edges of an image.

[0114] It should be noted that the pixel DN value (Digital Number) refers to the numerical value possessed by each pixel in an image during digital image processing. This value typically represents the quantification result of the pixel's reflectance, radiance, or other physical quantities in a specific wavelength band. The DN value is one of the most fundamental data representation forms in remote sensing imagery.

[0115] In some embodiments, taking into account the interference of strip noise, for example... Figure 3 The remote sensing image to be processed is shown in part (a). It may not be able to represent image edges very well (e.g.) Figure 3 As shown in section (b), the step of determining the edge weight factor of each pixel in the remote sensing image to be processed in S102 specifically includes:

[0116] The remote sensing image to be processed is smoothed by guided filtering to obtain the smoothing component g (e.g., ...). Figure 3 (as shown in part (c)) and high-frequency component d (as shown in part (c)) Figure 3 (as shown in part (d)), where the high-frequency components ;

[0117] Calculate the standard deviation of the pixel DN value corresponding to the smooth component and the standard deviation of the pixel DN value corresponding to the high-frequency component at each pixel location, and calculate the initial weight factor (e.g., ...) at each pixel location based on the standard deviation of the pixel DN value. Figure 3 (as shown in part (e)), apply the formula:

[0118] ;

[0119] The initial weight factors are normalized, and the edge weight factors at each pixel location are determined based on the comparison between the normalized initial weight factors and the preset normalization threshold (e.g., ...). Figure 3 (as shown in part (f)), apply the formula:

[0120] ;

[0121] The anisotropic total variational model in related technologies can be represented as follows:

[0122] ;

[0123] Based on the aforementioned edge weight factors, the above heterogeneous total variational models can be improved to construct smooth constraint terms, including:

[0124] ;

[0125] in, The position coordinates of the pixel, subscript The subscript represents the smoothing component. Represents the high-frequency component; Coordinates in the smooth component The standard deviation of the pixel DN value at that location can be based on The data was acquired using a scale window. Coordinates in the high-frequency components The standard deviation of the pixel DN value at a given location can be obtained based on a scale window different from that of the smoothing component;

[0126] The coordinates in the remote sensing image to be processed The initial weighting factor at the location, These are the initial weighting factors after normalization. The threshold for separating edges from smooth regions, i.e., the preset normalization threshold, can be set to 0.1, but this application embodiment does not limit this; For parameters, The smaller the value, the better, such as 0.2. However, this application does not limit this to any particular value.

[0127] W is the edge weight matrix. The coordinates in the remote sensing image to be processed Edge weighting factor at the location; For the smoothing constraint term, For the clean image, The remote sensing image to be processed, For the partial differential operator along the direction of the fringe noise, For partial differential operators across the fringe noise direction, For parameters, Indicates the calculation of internal elements " The L1 norm of “”, more specifically, the L1 norm represents: the sum of the absolute values ​​of the interior elements; understandably, here “” "This is merely a placeholder in the relevant calculation symbols, used to represent the corresponding internal element, and has no actual meaning."

[0128] Understandably, The gradient of the remote sensing image Y to be processed can be mapped onto the potential clean image X along the fringe direction to protect the gradient of the clean image X from fringe noise. This can be achieved by minimizing the gradient across the stripe direction of the potentially clean image X, thus suppressing stripe noise. By introducing an edge weight factor W, high weight values ​​are applied to regions with severe stripe noise, while smaller weight values ​​are applied to regions containing image texture and details.

[0129] Furthermore, in S103, based on the average cross trajectory profile and Hodrick-Prescott (HP) decomposition technique, more accurate statistical information can be obtained while taking into account the stripe position attributes. This information is then embedded into the two-dimensional variational model as guiding information to obtain statistical constraint terms, specifically including:

[0130] Determining the location of stripe noise in the remote sensing image to be processed based on gradient information includes:

[0131] The gradient difference between the gradient along the stripe noise direction and the gradient across the stripe noise direction in the remote sensing image to be processed is calculated to generate a binary image. The type of each pixel in the remote sensing image to be processed is determined based on the binary image, using the formula:

[0132] ;

[0133] in, m is the number of rows in the remote sensing image to be processed. , where n is the column number of the remote sensing image to be processed. The gradient of the cross-striped noise direction of the pixel in the i-th row and j-th column of the remote sensing image to be processed. The gradient of the pixel in the i-th row and j-th column of the remote sensing image to be processed along the stripe noise direction. For the binary image, A value of 1 indicates that the pixel in the i-th row and j-th column is a pixel with stripe noise. A value of 0 indicates that the pixel in the i-th row and j-th column is a pixel without stripe noise. This indicates retrieving the inner element " The absolute value of ";

[0134] Based on the binary image, determine the number of pixels with stripe noise in all columns of the i-th row of the remote sensing image to be processed. If the number of pixels with stripe noise in all columns of the i-th row is greater than a preset pixel count threshold, then stripe noise is determined to exist in the i-th row, and the location of the stripe noise is obtained by applying the formula:

[0135] ;

[0136] in, 'a' is a preset threshold for the number of pixels. It can be set to 0.75, but this embodiment of the application does not limit it; Used to indicate the location of stripe noise This indicates that there is stripe noise in the i-th row.

[0137] Furthermore, the average cross trajectory profile of the remote sensing image to be processed is calculated by combining the location of the stripe noise, and a statistical constraint term is constructed based on the average cross trajectory profile, including:

[0138] Based on the average cross trajectory profile and HP decomposition technique of the image Y to be processed, a one-dimensional variational model for calculating the average cross trajectory profile curve is constructed, including:

[0139] ;

[0140] The one-dimensional variational model is solved using the iterative weighted least squares method, applying the formula:

[0141] ;

[0142] Update the fidelity weight matrix used for the k-th iteration by applying the formula:

[0143] ;

[0144] ;

[0145] The average cross trajectory profile curve is obtained by performing the k-th iteration based on the updated fidelity weight matrix, using the formula:

[0146] ;

[0147] If the average cross trajectory profile curve obtained in the k-th iteration is... The average cross trajectory profile curve obtained from the (k-1)th iteration If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, output the average cross trajectory profile curve of the kth iteration.

[0148] in, To estimate the obtained average cross trajectory profile curve, For an element to be all Column vectors; It can be used to characterize the density of stripes; for sparse stripes, it can be set to... For dense stripes, you can set... This application does not limit the specific implementation details; D is the Gaussian blur kernel. ; To adjust the smoothness of the average cross-trajectory profile curve, the following parameters can be set: ; For the parameter, you can choose the smallest possible value, for example, you can choose 10. -5 However, the embodiments in this application do not limit this.

[0149] Where k represents the iteration number, Let be the fidelity weight matrix for the k-th iteration; P is the column vector representing the locations of the stripe noise, and T represents the transpose of the matrix. The Hadamard product is the element-wise multiplication of two matrices of the same dimension. Indicates the calculation of internal elements " The L2 norm of '', also known as the Euclidean norm, more specifically, represents the square root of the sum of the squares of its internal elements.

[0150] Indicates calculation Norms can be expressed using mathematical formulas:

[0151] .

[0152] L1 norm and L2 norm are both types of Lp norm, where p=1 is the L1 norm;

[0153] P=2, which is the L2 norm.

[0154] It should be noted that the Frobenius Norm Error is a method for measuring the difference between two matrices, which is achieved by calculating the Frobenius norm distance between the two matrices.

[0155] For example, in the iterative calculation of the average cross trajectory profile curve, an initial value for the average cross trajectory profile curve can be given first. initial value Similar to the average cross trajectory profile curve of the remote sensing image to be processed, a preset threshold of 10 is set. -5 The preset number of iterations is set to 50 for iterative calculation:

[0156] Let k=0, based on the initial value The updated value yields the fidelity weight matrix. Calculations yielded ;

[0157] like and If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, then the output is... ;

[0158] Otherwise, continue the iterative calculation, and denote k = k + 1;

[0159] k=1, based on The updated value yields the fidelity weight matrix. Calculations yielded ;

[0160] like and If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, then the output is... ;

[0161] Otherwise, continue the iterative calculation, and denote k = k + 1 = 2;

[0162] ;

[0163] k, based on The updated value yields the fidelity weight matrix. Calculations yielded ;

[0164] like and If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, then the output is... ;

[0165] Otherwise, continue with the iterative calculation;

[0166] The iteration continues until the termination condition is met, at which point the corresponding average cross trajectory profile curve is output.

[0167] Understandably, the calculated average cross trajectory profile curve is in one-dimensional vector form.

[0168] Furthermore, statistical constraint terms are constructed based on the output average cross trajectory profile curve, and the formula is applied:

[0169] ;

[0170] For the statistical constraint term, The average cross trajectory profile curve is calculated based on the remote sensing image to be processed. For parameters.

[0171] Understandably, the embodiments of this application are implemented through... The difference between the penalized estimated average cross trajectory profile and the average cross trajectory profile of the clean image can effectively maintain the consistency between the recovered data and the guiding information, and play a role in further controlling the destriating process to obtain reliable output, thereby improving the effect of removing stripe noise.

[0172] Considering the complexity of stripe noise in real-world remote sensing images, the steps in S104 for constructing stripe sparsity constraints based on the RBS regularization method specifically include:

[0173] In some embodiments, the construction of stripe sparsity constraints based on the RBS regularization method includes:

[0174] The striped image S corresponding to the remote sensing image to be processed is divided into multiple image blocks along the cross-striped noise direction; the height of each image block is the same as the height of the striped image, the width of each image block is the same, and the sum of the widths of all image blocks is the same as the width of the striped image.

[0175] To improve the group sparsity of each stripe block, a weight matrix is ​​introduced to adaptively control the penalty level. The block weight matrix for each image block is determined, and the stripe sparsity constraint is constructed using RBS regularization, applying the formula:

[0176] ;

[0177] ;

[0178] ;

[0179] in, This represents the z-th image patch. s is the total number of image patches. For image blocks width, , This represents the e-th row of the z-th image patch. , Let be the block weight matrix corresponding to the z-th image block, specifically a non-negative weighting matrix. This represents the e-th row in the block weight matrix corresponding to the z-th image block; It is a non-zero parameter, for example, set to 10. -16 To avoid the denominator being 0; This is the sparse constraint for the stripes. For parameters.

[0180] It should be noted that, express Norm (Weighted) Norm is a hybrid norm often used in structured sparse learning (such as group sparse and multi-task learning), which combines... Norm (within-group smoothing) and Norm (between groups sparsity), and introduce weight matrix To adjust the importance of different groups.

[0181] Further, the numerical fidelity term corresponding to the remote sensing image to be processed is obtained, and S105 is executed. An optimization model for removing stripe noise is constructed by combining the smoothing constraint term, the statistical constraint term, and the stripe sparsity constraint, including:

[0182]

[0183] in, The numerical fidelity term corresponding to the remote sensing image to be processed; , , These are all regularization parameters used to balance statistical feature constraints, smoothness constraints, and stripe sparsity constraints. Indicates the calculation of internal elements " The L1 norm of ".

[0184] In the embodiments of this application, the regularization parameter , , Used to balance smoothness constraints, statistical characteristic constraints, and fringe sparsity constraints. Parameters Depending on the severity of the image banding, images with severe banding will be preferred over those with smaller banding. ;parameter Used to control the contribution of statistical constraint terms to the overall optimization model, for example, setting... (n is the number of image columns); parameters The value used to control the contribution of the fringe sparsity constraint term to the model can be selected according to the actual situation. .

[0185] Further, step S106 is executed to calculate the optimized model to remove stripe noise from the remote sensing image to be processed, and output a clean image, including:

[0186] Define the problem as minimizing a clean image:

[0187]

[0188] By introducing auxiliary variables H and V, the problem of minimizing the clean image is transformed into a constrained minimization problem, and the formula is applied:

[0189] ;

[0190] H= ;

[0191] ;

[0192] Determine the corresponding augmented Lagrangian function and apply the formula:

[0193] ;

[0194] In some embodiments, the ADMN algorithm can be applied to solve the problem, decomposing the augmented Lagrangian function into three subproblems: H, V, and X. Based on the soft-threshold shrinkage operator, the following formula can be obtained:

[0195] ;

[0196] ;

[0197] Among them, the soft threshold shrinkage operator , , For placeholders in the soft threshold shrinkage operator, For example, when applying the soft threshold shrinkage operator, let , ;

[0198] Furthermore, iterative calculations based on the Fast Fourier Transform (FFT) are performed to obtain the two-dimensional matrix form of the clean image, applying the formula:

[0199] ;

[0200] It should be noted that the stripe image S is calculated simultaneously with the clean image, including:

[0201] Define the problem of minimizing the fringe pattern:

[0202] ;

[0203] By introducing auxiliary variables The corresponding augmented Lagrange function is obtained, and the formula is applied:

[0204] ;

[0205] ;

[0206] Solving this problem using the ADMM algorithm can decompose it into two subproblems, Q and S, where Q is an auxiliary variable used for computation.

[0207] For subproblem Q, the corresponding solution can be obtained based on the soft threshold shrinkage operator. :

[0208] ;

[0209] in, , ;

[0210] in, , Indicates the first Lagrange multipliers corresponding to each image patch The OK.

[0211] For subproblem S, iterative calculation based on the fast two-dimensional Fourier transform can be performed to obtain the two-dimensional matrix form of the stripe image, applying the formula:

[0212] ;

[0213] In one iteration, after calculating the two-dimensional matrix form of the striped image S and the two-dimensional matrix form of the clean image X, the following steps are also included:

[0214] For Lagrange multipliers , , Update:

[0215] ;

[0216] ;

[0217] ;

[0218] If the iteration termination condition is met, output the clean image and the corresponding stripe image obtained in the last iteration.

[0219] Among them, H, V, and Q are auxiliary variables. , , For Lagrange multipliers; , , The penalty parameter can be set. = = The two-dimensional matrix forms of the clean image and the striped image with the same iteration number are calculated simultaneously in the same iteration process.

[0220] In some embodiments, the iteration termination condition can be:

[0221] If the difference between two consecutive clean images is less than a threshold, it can be set to 10. -5 Alternatively, if the maximum number of iterations is reached, it can be set to 30.

[0222] If either of the two conditions is met, the iteration can be terminated, the calculation process can be ended, and the final clean image X and the corresponding striped image S can be obtained.

[0223] For example, if the clean image obtained in the k-th iteration The clean image obtained from the (k-1)th iteration If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, output the clean image and the corresponding stripe image obtained in the last iteration.

[0224] This method can efficiently and accurately remove stripe noise from remote sensing images.

[0225] The following are apparatus embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of this application.

[0226] Please see below. Figure 4 This is a schematic diagram of a remote sensing image stripe noise removal device provided in an exemplary embodiment of this application. This device can be implemented as all or part of a terminal through software, hardware, or a combination of both, or it can be integrated as an independent module on a server. The remote sensing image stripe noise removal device in this embodiment can be applied to a terminal or the cloud. The device 40 includes a data acquisition module 401, a constraint construction module 402, and a calculation module 403, wherein:

[0227] The data acquisition module 401 is used to acquire the remote sensing image to be processed;

[0228] The constraint construction module 402 is used to determine the edge weight factor of each pixel in the remote sensing image to be processed, and to construct a smoothing constraint term based on the edge weight factor;

[0229] The constraint construction module 402 is further configured to determine the position of stripe noise in the remote sensing image to be processed based on the gradient information of the remote sensing image to be processed, calculate the average cross trajectory profile of the remote sensing image to be processed in combination with the position of the stripe noise, and construct statistical constraint terms based on the average cross trajectory profile.

[0230] The constraint construction module 402 is also used to construct stripe sparse constraints based on the RBS regularization method;

[0231] The calculation module 403 is used to combine the smoothing constraint term, the statistical constraint term, and the fringe sparsity constraint to construct an optimization model for removing fringe noise;

[0232] The calculation module 403 is also used to calculate the optimization model based on the alternating direction multiplier method to remove stripe noise in the remote sensing image to be processed and output a clean image.

[0233] It should be noted that the apparatus 40 provided in the above embodiments is only illustrated by the division of the above functional modules when performing the remote sensing image stripe noise removal method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus provided in the above embodiments and the remote sensing image stripe noise removal method embodiments belong to the same concept, and its implementation process can be found in the method embodiments, which will not be repeated here.

[0234] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0235] Please see Figure 5 This is a structural block diagram of an electronic device provided in an embodiment of this application.

[0236] like Figure 5 As shown, the electronic device 500 includes a processor 501 and a memory 502.

[0237] In this embodiment, the processor 501 is the control center of the computer system, and can be a processor of a physical machine or a processor of a virtual machine. The processor 501 may include one or more processing cores, such as a 4-core processor or an 8-core processor. The processor 501 can be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array).

[0238] Processor 501 may also include a main processor and a coprocessor. The main processor is a processor used to process data in the wake-up state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor used to process data in the standby state.

[0239] Memory 502 may include one or more computer-readable storage media, which may be non-transitory. Memory 502 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this application, the non-transitory computer-readable storage media in memory 502 is used to store at least one instruction, which is executed by processor 501 to implement the method in the embodiments of this application.

[0240] In some embodiments, the electronic device 500 further includes a peripheral device interface 503 and at least one peripheral device 504. The processor 501, memory 502, and peripheral device interface 503 can be connected via a bus or signal line. Each peripheral device 504 can be connected to the peripheral device interface 503 via a bus, signal line, or circuit board. Specifically, the peripheral device 504 includes: a display screen, a camera, and audio circuitry. The peripheral device interface 503 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 501 and memory 502.

[0241] In some embodiments of this application, the processor 501, memory 502, and peripheral device interface 503 are integrated on the same chip or circuit board; in other embodiments of this application, any one or two of the processor 501, memory 502, and peripheral device interface 503 can be implemented on separate chips or circuit boards. This application does not specifically limit the implementation in this regard.

[0242] The block diagram of the electronic device shown in the embodiments of this application does not constitute a limitation on the electronic device 500. The electronic device 500 may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0243] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the methods in any of the foregoing embodiments. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0244] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0245] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for removing stripe noise from a remote sensing image, characterized in that, The method comprises the following steps: acquiring a to-be-processed remote sensing image; determining an edge weight factor of each pixel in the to-be-processed remote sensing image, and constructing a smoothing constraint term based on the edge weight factor; determining a position of stripe noise in the to-be-processed remote sensing image based on gradient information of the to-be-processed remote sensing image, calculating an average cross-track profile of the to-be-processed remote sensing image according to the position of the stripe noise, and constructing a statistical constraint term based on the average cross-track profile; constructing a stripe sparse constraint based on an RBS regularization method; combining the smoothing constraint term, the statistical constraint term and the stripe sparse constraint to construct an optimization model for removing the stripe noise; calculating the optimization model to remove the stripe noise in the to-be-processed remote sensing image, and outputting a clean image without the stripe noise.

2. The method of claim 1, wherein, The method for determining the edge weight factor of each pixel in the to-be-processed remote sensing image comprises the following steps: performing smoothing processing on the to-be-processed remote sensing image through guided filtering to obtain a smoothing component and a high-frequency component; calculating a pixel DN value standard deviation corresponding to the smoothing component at each pixel position and a pixel DN value standard deviation corresponding to the high-frequency component, calculating an initial weight factor at each pixel position based on the pixel DN value standard deviations, and applying the formula: ； performing normalization processing on the initial weight factor, and determining the edge weight factor at each pixel position according to a comparison result of the initial weight factor after the normalization processing and a preset normalization threshold, which is expressed by the formula: ； The method for constructing the smoothing constraint term based on the edge weight factor comprises the following steps: ； wherein, is a position coordinate of a pixel, subscript denotes the smooth component, subscript denotes the high-frequency component, is a standard deviation of pixel DN values at coordinates in the smooth component, is a standard deviation of pixel DN values at coordinates in the high-frequency component, is an initial weight factor at coordinates in the to-be-processed remote sensing image, is the initial weight factor after normalization processing, is a preset normalization threshold, is a parameter, W is an edge weight matrix, is an edge weight factor at coordinates in the to-be-processed remote sensing image; is the smooth constraint term, is the clean image, is the to-be-processed remote sensing image, is a partial differential operator along a direction of stripe noise, is a partial differential operator across the direction of stripe noise, is a parameter, denotes calculation of an L1 norm of internal elements .

3. The method of claim 2, wherein, The method for determining the position of the stripe noise in the to-be-processed remote sensing image based on the gradient information of the to-be-processed remote sensing image comprises the following steps: calculating a gradient difference value between a gradient along a stripe noise direction and a gradient across the stripe noise direction of the to-be-processed remote sensing image to generate a binary image, and determining a type of each pixel on the to-be-processed remote sensing image according to the binary image, which is expressed by the formula: ； wherein, m is the number of lines of the remote sensing image to be processed, n is the number of columns of the remote sensing image to be processed, is the cross direction gradient of the pixel of the i-th line and j-th column of the remote sensing image to be processed, is the along direction gradient of the pixel of the i-th line and j-th column of the remote sensing image to be processed, is the binary map, equal to 1 indicates that the pixel of the i-th line and j-th column is a pixel with the presence of the stripe noise, equal to 0 indicates that the pixel of the i-th line and j-th column is a pixel without the presence of the stripe noise, denotes the absolute value of the inner element . determining a number of pixels with the stripe noise on all columns of an i-th row of the to-be-processed remote sensing image according to the binary image, and determining that the i-th row has the stripe noise if the number is greater than a preset pixel number threshold, thereby obtaining the position of the stripe noise, which is expressed by the formula: ； wherein, is a preset pixel number threshold, is a parameter, is used to indicate the position of the stripe noise, represents that the i-th row has the stripe noise.

4. The method of claim 3, wherein, The method for combining the position of the stripe noise to calculate the average cross-track profile of the to-be-processed remote sensing image and constructing the statistical constraint term based on the average cross-track profile comprises the following steps: constructing a one-dimensional variational model for calculating an average cross-track profile curve, which comprises the following steps: ； solving the one-dimensional variational model through an iterative weighted least squares method, which is expressed by the formula: ； updating a fidelity weight matrix for the k-th iteration, which is expressed by the formula: ； ； performing the k-th iteration calculation based on the updated fidelity weight matrix to obtain the average cross-track profile curve, which is expressed by the formula: ； If the average cross trajectory profile curve obtained in the k-th iteration is... The average cross trajectory profile curve obtained from the (k-1)th iteration If the relative Frobenius norm error is less than a preset threshold, or if k+1 equals a preset number of iterations, output the average cross trajectory profile curve of the kth iteration. constructing the statistical constraint term based on the output average cross-track profile curve, which is expressed by the formula: ； wherein, is an estimated average cross-track profile curve, is a column vector with all elements , , D is a Gaussian blur kernel , is a parameter for adjusting the smoothness of the average cross-track profile curve, is a parameter, k represents the iteration number, is a fidelity weight matrix of the kth iteration, is the statistical constraint term, is an average cross-track profile curve calculated based on the remote sensing image to be processed, is a parameter, P is a column vector representing the position of the stripe noise, T represents the transpose of the matrix, is the Hadamard product of the two matrices, represents the L2 norm of the internal element , represents the L1 norm of the internal element , .

5. The method of claim 4, wherein, The method for constructing the stripe sparse constraint based on the RBS regularization method comprises the following steps: Divide the fringe image corresponding to the to-be-processed remote sensing image into a plurality of image blocks along a cross-fringe noise direction; the height of each image block is the same as the height of the fringe image, the width of each image block is the same, and the sum of the widths of all the image blocks is the same as the width of the fringe image; Determine a block weight matrix of each image block, and the fringe sparsity constraint is constructed by RBS regularization, and the formula is: ； ； ； in, Represents a striped image. This represents the z-th image patch. s is the total number of image patches. For image blocks width, , This represents the e-th row of the z-th image patch. , Let z be the block weight matrix corresponding to the z-th image block. This represents the e-th row in the block weight matrix corresponding to the z-th image block. For a non-zero parameter, This is the sparse constraint for the stripes. For parameters, Weighted for block weight matrix Mixed norm.

6. The method of claim 5, wherein, Obtain a numerical fidelity term corresponding to the to-be-processed remote sensing image, and combine the smooth constraint term, the statistical constraint term and the fringe sparsity constraint to construct an optimization model for removing fringe noise, including: wherein, is a numerical fidelity term corresponding to the remote sensing image to be processed.

7. The method of claim 6, wherein, The calculation module is further used to calculate the optimization model based on the alternating direction multiplier method to remove the fringe noise in the to-be-processed remote sensing image, and output a clean image. The processor executes the program to implement the steps of the method in any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. ； H= ； ； The processor executes the program to implement the steps of the method in any one of claims 1 to 7. ； The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. ； The processor executes the program to implement the steps of the method in any one of claims 1 to 7. ； The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. ； ； ； The processor executes the program to implement the steps of the method in any one of claims 1 to 7. ； The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7. where H, V, Q are auxiliary variables, 、 、 is a Lagrange multiplier, 、 、 is a penalty parameter, the two-dimensional matrix form of the clean image and the two-dimensional matrix form of the stripe image of the same iteration sequence number are calculated simultaneously in the same iteration process; denotes the auxiliary variable corresponding to the zth image block; denotes the auxiliary variable corresponding to the zth image block in the zth row of the zth column. in the zth row of the zth column.

8. A device for removing stripe noise from a remote sensing image, characterized in that In the case of meeting the iteration termination condition, output the clean image and the corresponding fringe image obtained by the last iteration; Including: A data acquisition module is configured to acquire a to-be-processed remote sensing image; A constraint construction module is configured to determine an edge weight factor of each pixel in the to-be-processed remote sensing image, and construct a smooth constraint term based on the edge weight factor; The constraint construction module is further configured to determine the position of fringe noise in the to-be-processed remote sensing image based on gradient information of the to-be-processed remote sensing image, calculate an average cross-track profile of the to-be-processed remote sensing image based on the position of the fringe noise, and construct a statistical constraint term based on the average cross-track profile; The constraint construction module is further configured to construct a fringe sparsity constraint based on an RBS regularization method; A calculation module is configured to combine the smooth constraint term, the statistical constraint term and the fringe sparsity constraint to construct an optimization model for removing fringe noise.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The calculation module is further configured to calculate the optimization model based on the alternating direction multiplier method to remove the fringe noise in the to-be-processed remote sensing image, and output a clean image. 10.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The processor executes the program to implement the steps of the method in any one of claims 1 to 7. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 7.

Images