A residual regulation unrolled non-negative auto-encoder training and image reconstruction method

Abstract

The application discloses a residual regulation unrolled non-negative auto-encoder training and image reconstruction method, and belongs to the technical field of image processing. The method forms a non-negative image block matrix from an image containing noise, shielding or abnormal pixels, generates a base matrix, a non-negative coefficient matrix and an initial state of sparse interference by an encoder, and takes the non-negative coefficient matrix as a latent representation. A plurality of alternating direction multiplier method unrolling units update the base matrix original variable, the coefficient matrix original variable, the base matrix auxiliary variable, the coefficient matrix auxiliary variable, the sparse interference matrix and the scaling dual variable in turn; a residual control module adjusts the penalty coefficient, the non-negative soft threshold and the execution state of the subsequent unit according to the normalized original residual and the dual residual, and terminates in advance when the stop condition is continuously satisfied. The decoder generates a reconstructed image block according to the product of the two non-negative auxiliary variables. In the training stage, the parameters are updated through a joint loss and back propagation, and the method can be used for degraded image reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for training and reconstructing an unfolded non-negative autoencoder with residual control. Background Technology

[0002] Images may be affected by sensor noise, local occlusion, impulse interference, and anomalous pixels during acquisition, transmission, or storage. Image reconstruction requires estimating the main components from the disturbed observation data and suppressing the interference for subsequent display, identification, or measurement.

[0003] Nonnegative matrix factorization (NMF) uses nonnegative basis features and their combination coefficients to represent image data. When solving constrained matrix factorization using the alternating direction multiplier method, it is usually necessary to preset penalty coefficients, regularization weights, stopping thresholds, and the maximum number of iterations. Fixed parameters are difficult to reflect the constraint states of different images or different iteration stages, which may lead to continued calculations even after the residuals have met the conditions, or the maintenance of an inappropriate update scale when the two types of residuals are imbalanced.

[0004] Autoencoders extract latent representations through an encoder, which then generates the output through a decoder. These latent representations typically do not directly correspond to the basic features and combination coefficients in nonnegative matrix factorization. Deep unrolling methods can map optimization iterations to sequential network units, but it is still necessary to explicitly distinguish between the basis matrix and the coefficient matrix, and between original variables and auxiliary variables. When changing the penalty coefficients, scaling the dual variables must also be handled simultaneously to maintain the continuity of the dual states of adjacent units.

[0005] Therefore, an unfolded nonnegative autoencoder for image reconstruction is needed to make the data relationships between the encoder, nonnegative latent representation, unfolding unit and decoder explicit, and to adjust the penalty coefficient, projection threshold and execution depth according to the dimension-consistent original residual and dual residual. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a residual-controlled unfolded nonnegative autoencoder training and image reconstruction method. Under the condition that the basic features and combination coefficients satisfy the nonnegativity constraint, this invention establishes a dimensionally consistent correspondence between the internal state of the autoencoder and the matrix factorization variables; adjusts the penalty coefficient and execution depth according to the consistency constraint state of each unfolded unit; reconstructs the main image from the degraded image and estimates sparse interference; and solves the problems of generating initial states from a shared basic dictionary and sample corrections, maintaining the continuity of dual states when changing the penalty coefficient, and reducing unnecessary computation.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for training an unfolded non-negative autoencoder with residual control, comprising:

[0008] Obtain training image pairs consisting of degraded input images and corresponding clean target images, and construct a non-negative image block matrix Y∈R from the degraded input images. + m×n ;

[0009] An expansion-type non-negative autoencoder is constructed, comprising an encoder, multiple sequentially connected alternating direction multiplier expansion units, a residual control module, and a decoder. The encoder is used to map the non-negative image block matrix to the initial state of the basis matrix, the initial state of the non-negative coefficient matrix, and the initial state of the sparse interference matrix, wherein the non-negative coefficient matrix serves as a latent representation.

[0010] Execute the alternating direction multiplier expansion units according to the network connection order, for the original variable W∈R of the basis matrix. m×r The original variables of the coefficient matrix H∈R r×n Auxiliary variable Z of the basis matrix W ∈R m×r Coefficient matrix auxiliary variable Z H ∈R r×n , Sparse interference matrix S∈R m×n Update the corresponding scaling dual variables, and make the updated base matrix auxiliary variables non-negative and the L2 norm of each column equal to 1, and the updated coefficient matrix auxiliary variables non-negative;

[0011] The residual control module generates the original residual based on the consistency constraint deviation of each alternating direction multiplier expansion unit, generates the dual residual based on the change of auxiliary variables of adjacent alternating direction multiplier expansion units, normalizes the original residual and the dual residual, and adjusts the penalty coefficient, non-negative soft threshold and execution state used by the next alternating direction multiplier expansion unit according to their relative magnitude.

[0012] The decoder calculates the product X̂=Z based on the auxiliary variables of the current base matrix and the auxiliary variables of the current coefficient matrix. W Z H Generate non-negative reconstructed image patches;

[0013] The joint loss is constructed based on the reconstruction loss between the non-negative reconstructed image patch and the target image patch corresponding to the target clean image, the sparse constraint loss of the latent representation and sparse interference, and the consistency loss between the original variable and the auxiliary variable. The joint loss is then backpropagated through the executed alternating direction multiplier expansion unit to update the trainable parameters of the unfolded non-negative autoencoder.

[0014] Furthermore, the encoder includes:

[0015] A coefficient encoding branch is used to perform feature mapping and non-negative activation on the non-negative image block matrix to generate the initial state of the non-negative coefficient matrix.

[0016] The basis matrix adaptive branch is used to combine the basic dictionary shared across training samples with the correction amount of the current sample, and perform non-negative column normalized projection to generate the initial state of the basis matrix.

[0017] The sparse interference branch is used to generate the initial state of the sparse interference matrix based on the difference between the product of the input matrix and the initial state of the base matrix and the initial state of the coefficient matrix.

[0018] Furthermore, each alternating direction multiplier expansion unit performs the following update process:

[0019] Subtract the sparse interference estimate of the previous expansion unit from the non-negative image patch matrix, and update the original variables of the basis matrix and the original variables of the coefficient matrix;

[0020] Perform a nonnegative column normalized projection on the sum of the original variables of the updated basis matrix and the corresponding scaled dual variables to obtain the auxiliary variables of the updated basis matrix;

[0021] Perform non-negative soft thresholding on the sum of the original variables and the corresponding scaled dual variables of the updated coefficient matrix to obtain the auxiliary variables of the updated coefficient matrix;

[0022] Update the sparse disturbance matrix and the two sets of scaling dual variables;

[0023] The product of the two updated auxiliary variables is used as the non-negative reconstructed image patch of the expanded unit.

[0024] Furthermore, the original residual and the dual residual are calculated in the following manner:

[0025] The original residual of the k-th alternating direction multiplier expansion unit and dual residual They are respectively:

[0026]

[0027]

[0028] Here, vec indicates vectorization in a fixed order, and the semicolon indicates vertically connecting two vectors. The penalty coefficient for the k-th expansion unit is given by the superscript k, which indicates the corresponding variable after the k-th expansion unit is updated.

[0029] The residual control module normalizes the original residual and the dual residual according to the number of elements in the basis matrix and the coefficient matrix, and updates the penalty coefficient of the next expansion unit according to the following rules based on the relative magnitude of the normalized original residual and the dual residual:

[0030] When the normalized original residual is greater than the normalized dual residual by a preset multiple, the penalty coefficient is increased within the preset upper limit.

[0031] When the normalized dual residual is greater than the normalized original residual by the preset multiple, the penalty coefficient is reduced within the preset lower limit.

[0032] In all other cases, the penalty coefficient remains unchanged;

[0033] Wherein, the preset multiple is greater than 1.

[0034] Furthermore, when the penalty coefficients are updated, the two sets of scaling dual variables are recalibrated according to the ratio of the old and new penalty coefficients; and the non-negative soft threshold of the coefficient matrix auxiliary variable of the next expansion unit is determined according to the ratio of the sparse regular weight to the updated penalty coefficient.

[0035] Furthermore, the residual control module is also used to control the stop execution state of the unfolding unit:

[0036] When the normalized original residual and normalized dual residual of q consecutive expansion units are not greater than their respective stopping thresholds, the execution of subsequent expansion units is stopped, and the product of the current two auxiliary variables is passed to the decoder.

[0037] Otherwise, continue executing subsequent expansion units until the maximum number of expansion units K is reached;

[0038] Where q and K are positive integers, and q is not greater than K.

[0039] The present invention also provides a method for image reconstruction using an unfolded non-negative autoencoder trained by the above-described training method, comprising:

[0040] The image to be processed is obtained, and it is non-negatively normalized, divided into image blocks, and arranged by column matrix to obtain a non-negative image block matrix.

[0041] The non-negative image patch matrix is ​​input into the trained unfolded non-negative autoencoder, which generates the initial state of the basis matrix, the initial state of the non-negative coefficient matrix, and the initial state of the sparse interference matrix.

[0042] The alternating direction multiplier expansion unit is executed sequentially to update the original variables of the basis matrix, the original variables of the coefficient matrix, the auxiliary variables of the basis matrix, the auxiliary variables of the coefficient matrix, the sparse interference matrix, and the scaling dual variables. The residual control module adjusts the penalty coefficient, the non-negative soft threshold, and the execution status of subsequent alternating direction multiplier expansion units based on the normalized original residual and the normalized dual residual.

[0043] When the residual stopping condition is met or the maximum number of expanded units K is reached, the decoder calculates the product X̂=Z of the current non-negative basis matrix auxiliary variable and the current non-negative coefficient matrix auxiliary variable. W Z H Generate non-negative reconstructed image patches;

[0044] A reconstructed image is formed based on the non-negative reconstructed image blocks.

[0045] Further, forming a reconstructed image based on the non-negative reconstructed image blocks includes:

[0046] Based on the spatial position and column arrangement order recorded when dividing the image blocks, the non-negative reconstructed image blocks are inversely vectorized and inversely arranged.

[0047] For overlapping non-negative reconstructed image blocks, non-negative fusion weights are determined based on the distance from pixels in each image block to the image block boundary. Pixel values ​​at the same spatial location are weighted and summed and then divided by the sum of the corresponding fusion weights.

[0048] The fused pixel values ​​are restored to the numerical range of the image to be processed based on the numerical transformation parameters recorded during non-negative normalization, and pixel values ​​that exceed the numerical range are truncated to obtain the reconstructed image.

[0049] The present invention also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the above-described training method and / or the above-described image reconstruction method.

[0050] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the above-described training method and / or the above-described image reconstruction method.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] First, set an auxiliary variable Z with the same dimension as the original variables W of the basis matrix and H of the coefficient matrix. W Z H This achieves variable splitting through W=Z. W H=Z H By splitting variables, data fidelity updates and non-negative projections can be performed separately, avoiding direct subtraction of matrices of different sizes, and ensuring a dimensionally consistent correspondence between the network's internal state and the matrix decomposition variables.

[0053] Second, the encoder provides a reasonable initialization state. The shared basic dictionary and sample correction values ​​generate the initial values ​​of the basis matrix, the coefficient encoding branch generates the main latent representation, and the sparse interference branch generates the initial values ​​of the interference, thereby establishing the data relationship between the input image, the internal state, and the decoded output.

[0054] Third, the residual control mechanism enables adaptive adjustment of the penalty coefficient. The normalized original residual reflects the consistency constraint deviation, while the normalized dual residual reflects changes in the auxiliary variable. Both types of residuals jointly adjust the bounded penalty coefficient. When the penalty coefficient changes, the scaling of the dual variable is synchronously recalibrated, and the non-negative soft threshold is updated in conjunction, thereby maintaining the continuity of the dual state of adjacent units.

[0055] Fourth, adaptive expansion depth reduces computational load. Once the residuals continuously meet the stopping condition, subsequent expansion units are stopped, thus avoiding unnecessary computations from continuing to execute expansion units after the residuals have met the conditions, and improving inference efficiency.

[0056] Fifth, the unfolding unit preserves the unity of interpretability and trainability. The unfolding unit retains the computational relationships of data fidelity, non-negative projection, and sparse shrinkage, and supports the backpropagation of joint loss along the executed units, combining the interpretability of traditional optimization algorithms with the modeling capabilities of deep learning. Attached Figure Description

[0057] Figure 1 The flowchart illustrates the implementation of the residual-controlled unfolded non-negative autoencoder training method provided in this embodiment of the invention. Detailed Implementation

[0058] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0059] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0060] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0061] like Figure 1 As shown, the residual-controlled unfolded non-negative autoencoder training method provided in this embodiment includes:

[0062] S1. Obtain training image pairs consisting of degraded input images and corresponding clean target images, and construct a non-negative image block matrix Y∈R from the degraded input images. + m×n ;

[0063] S2. Construct an expansion non-negative autoencoder including an encoder, multiple sequentially connected alternating direction multiplier expansion units, a residual control module and a decoder. The encoder is used to map the non-negative image block matrix to the initial state of the basis matrix, the initial state of the non-negative coefficient matrix and the initial state of the sparse interference matrix. The non-negative coefficient matrix is ​​used as a latent representation.

[0064] S3. Execute the alternating direction multiplier expansion units according to the network connection order, for the original variables W∈R of the basis matrix. m×r The original variables of the coefficient matrix H∈R r×n Auxiliary variable Z of the basis matrix W ∈R m×r Coefficient matrix auxiliary variable Z H ∈R r×n , Sparse interference matrix S∈R m×n Update the corresponding scaling dual variables, and make the updated base matrix auxiliary variables non-negative and the L2 norm of each column equal to 1, and the updated coefficient matrix auxiliary variables non-negative;

[0065] S4. The residual control module generates the original residual based on the consistency constraint deviation of each alternating direction multiplier expansion unit, generates the dual residual based on the change of auxiliary variables of adjacent alternating direction multiplier expansion units, normalizes the original residual and the dual residual, and adjusts the penalty coefficient, non-negative soft threshold and execution state used by the next alternating direction multiplier expansion unit according to their relative magnitude.

[0066] S5. The decoder calculates the product X̂=Z of the current base matrix auxiliary variable and the current coefficient matrix auxiliary variable. W Z H Generate non-negative reconstructed image patches;

[0067] S6. The joint loss is constructed based on the reconstruction loss between the non-negative reconstructed image patch and the target image patch corresponding to the target clean image, the sparse constraint loss of the latent representation and sparse interference, and the consistency loss between the original variable and the auxiliary variable. The joint loss is then backpropagated through the executed alternating direction multiplier expansion unit to update the trainable parameters of the unfolded non-negative autoencoder.

[0068] Based on this, this embodiment also provides a method for image reconstruction using an unfolded non-negative autoencoder trained by the above training method, including:

[0069] 1) Obtain the image to be processed, and perform nonnegative normalization, image block division, and column matrix arrangement to obtain a nonnegative image block matrix;

[0070] 2) Input the non-negative image patch matrix into the trained unfolded non-negative autoencoder, and the encoder generates the initial state of the basis matrix, the initial state of the non-negative coefficient matrix, and the initial state of the sparse interference matrix;

[0071] 3) The alternating direction multiplier expansion unit is executed sequentially to update the original variables of the basis matrix, the original variables of the coefficient matrix, the auxiliary variables of the basis matrix, the auxiliary variables of the coefficient matrix, the sparse interference matrix, and the scaling dual variables. The residual control module adjusts the penalty coefficient, the non-negative soft threshold, and the execution status of the subsequent alternating direction multiplier expansion unit according to the normalized original residual and the normalized dual residual.

[0072] 4) When the residual stopping condition is met or the maximum number of expanded units K is reached, the decoder calculates the product X̂=Z of the current non-negative basis matrix auxiliary variable and the current non-negative coefficient matrix auxiliary variable. W Z H Generate non-negative reconstructed image patches;

[0073] 5) Form a reconstructed image based on the non-negative reconstructed image blocks.

[0074] The specific implementation of the relevant technical content involved in this invention will be further explained below.

[0075] Without altering the technical relationship defined in the claims, those skilled in the art can set the image block size, potential dimension, maximum number of unfolded units, loss weight, and numerical implementation method according to application requirements.

[0076] I. Symbols, Inputs, and Outputs

[0077] Suppose that the degraded input image is formed after nonnegative normalization, image block division, and column arrangement:

[0078]

[0079] Where each y j ∈ℝ + m For a given image patch, r represents the dimension of the latent features. The dimensions of each variable are as follows:

[0080] W, Z W U W ∈ℝ m×r ;

[0081] H, Z H U H ∈ℝ r×n ;

[0082] S, Y, X̂∈ℝ m×n ;

[0083] B∈ℝ + m×r Δ W (Y)∈ℝ m×r .

[0084] Where W and H are the original variables of the data fidelity term; Z W Z H For the corresponding constraint auxiliary variables; U W U H For scaling the dual variable; Z H Z represents the primary latent representation; S represents sparse interference. The decoding output of each unfolded layer is Z. W Z H Do not directly subtract or splice matrices of different sizes.

[0085] II. Construction of Training Image Pairs and Image Patch Matrix

[0086] The training image pair includes the registered degraded input image and the target clean image. The degraded input image can be acquired directly or obtained by applying a degradation operator with recorded parameters to a clean image. When using manual degradation, the noise or occlusion type, generation parameters, abnormal pixel locations, random seed, and program version should be recorded for verification.

[0087] Image blocks are extracted by sliding according to predetermined block height, block width, and step size. Each image block is vectorized in the same order and arranged into a matrix column according to its spatial position. The clean target image is used to construct the target image block matrix X according to the same position and order.

[0088] Normalization uses the lower bound 'a' and upper bound 'b' determined by the training set:

[0089]

[0090] Where b > a, and a and b remain unchanged during verification and inference. If the input device has already output non-negative calibration data, the range parameters of the device can be used for equivalent non-negative normalization.

[0091] III. The three initialization branches of the encoder

[0092] 1. Coefficient encoding branch

[0093] Feature mapping is performed on each column of Y using shared parameters. One executable form is:

[0094]

[0095] Where A1∈ℝ p×m A2∈ℝr×p b1∈ℝ p b2∈ℝ r σ is the element-wise activation function, 1 n Let n be a vector of all ones. Softplus guarantees that the output is non-negative. Alternatively, an element-wise non-negative affine mapping can be used, as long as the output dimension and non-negativity remain unchanged.

[0096] 2. Adaptive branches of the basis matrix

[0097] First, calculate the cross-image patch statistics:

[0098]

[0099] Then, the sample correction amount is generated and rearranged using the feature map with output dimension mr:

[0100]

[0101] Add it to the shared base dictionary B, and project it column by column:

[0102]

[0103] For a column v of the matrix to be projected j Let a j =[v j ] + =max(v j ,0). When‖a j When ‖2>ε, let Π C_W (v j )=a j / ‖a j ‖2; when ‖a j When 2 ≤ ε, the non-negative normalized result of the corresponding column in the basic dictionary is used. If the corresponding column in the basic dictionary is also less than the threshold, a pre-fixed non-negative unit vector is used. This avoids undefined values ​​caused by zero-column normalization.

[0104] 3. Sparse interference branches

[0105] First calculate the initial residual R 0 =YW 0 H 0 Then, shrinking is performed according to the predetermined groups. One form of grouping by image block column is as follows:

[0106]

[0107] Among them, κ S ≥0. If the interference to be processed is defined as pixel-wise sparse, element-wise soft thresholding can also be used for shrinkage. Let U W 0 =0、UH 0 =0, and send all initial states into the first expansion unit.

[0108] IV. Single Alternating Direction Multiplier Expansion Unit

[0109] Let the k-th expansion unit receive W k-1 H k-1 S k-1 Z W k-1 Z H k-1 U W k-1 U H k-1 and ρ k Let A k-1 =YS k-1 A complete and dimensionally consistent update method is as follows.

[0110] 1. Update of original variables in the basis matrix:

[0111]

[0112] 2. Update the original variables of the coefficient matrix:

[0113]

[0114] Due to ρ k >0, the above contains I r The coefficient matrix is ​​a positive definite matrix. In implementation, Cholesky decomposition or other linear equation solvers can be used, eliminating the need to explicitly calculate the matrix inverse.

[0115] 3. Projection of auxiliary variables from the basis matrix:

[0116]

[0117] 4. Update of non-negative soft thresholds for auxiliary variables in the coefficient matrix:

[0118]

[0119] 5. Sparse Disturbance Update: Let R k =YW k H k Use ℓ by column 2,1 shrink

[0120]

[0121] 6. Scaling dual variable updates:

[0122]

[0123]

[0124] The above steps provide a closed update method.

[0125] V. Primitive-Dual Residual Regulation

[0126] The k-th expansion unit uses the following two types of residuals:

[0127]

[0128]

[0129] To avoid dimensionality errors, the basis matrix residuals and coefficient matrix residuals are first vectorized separately, and then the two vectors are joined vertically. The normalized residuals used for control are:

[0130]

[0131]

[0132] The penalty coefficient is updated according to the following bounded rule:

[0133]

[0134] Where μ is the residual comparison factor greater than 1, τinc and τdec are the increase and decrease factors greater than 1, respectively, and ρmin and ρmax are the lower and upper limits of the penalty coefficient. When the penalty coefficient changes, it is executed before entering the next expansion unit:

[0135]

[0136] This recalibration is used to maintain the continuity of the unscaled dual variable before and after the penalty coefficient switch. The non-negative soft threshold of the next expanded unit is updated synchronously with the new penalty coefficient.

[0137] To control the unfolding depth, a binary decision variable is defined:

[0138]

[0139] And consecutive satisfaction count:

[0140]

[0141] When t k When ≥q, denote the current unrolling index as κ=k, and stop executing unrolling units from k+1 to K; if the condition is not continuously satisfied up to K, let κ=K. The final non-negative reconstructed image block is:

[0142]

[0143] During training, the stop decision does not participate in gradient calculation; the joint loss is only backpropagated along the actually executed unfolded units.

[0144] VI. Decoder and Joint Loss

[0145] The decoder receives the Z of the kth executed expansion unit. W k and Z H k , using matrix multiplication Z W k Z H k Generate non-negative reconstructed image patches for this layer. The decoder does not use W. k H k As output.

[0146] The final joint loss for the training samples can be:

[0147]

[0148] Where α>0, γ, ξ, δ, η≥0, A is the set of indices of the expanded units, and ω k η ≥ 0 represents intermediate supervision weights. When no intermediate supervision is used, η can be set to 0. Each loss weight is determined according to predetermined rules on the training and validation sets.

[0149] Trainable parameters include a shared base dictionary B, coefficient encoding branch parameters, sample correction generation parameters, and sparse interference initialization branch parameters; they may also include positive parameterized λ. H , λ S Alternatively, the unit step size can be expanded. During training, the average of the batch joint loss is used as the objective, gradients are calculated via backpropagation, and parameters are updated by a pre-defined optimizer. After each update of the basic dictionary, nonnegative projection or nonnegative parameterization is performed to preserve B∈ℝ. + m×r .

[0150] VII. Training Process

[0151] The training process is as follows:

[0152] 1) Determine the training set, validation set and their mutually exclusive partitions, construct training image pairs that correspond one-to-one with the degraded input image and the target clean image, and save the degradation process and data partition records.

[0153] 2) Convert each training image pair into Y and X according to the fixed block size, step size, vectorization order and normalization parameters.

[0154] 3) Initialize the shared base dictionary, three encoding branch parameters, regular expression parameters, and ρ. 1 ρmin ρ max μ, τ inc τ dec ε pri ε dual q and K. These parameters are written to the configuration record before a training run begins.

[0155] 4) Input Y into the three initialization branches to generate W. 0 =Z W 0 H 0 =Z H 0 S 0 U W 0 =0 and U H 0 =0.

[0156] 5) Start the complete expanded unit update from k=1 and calculate r̄ k and d̄ k Update ρ k+1 Recalibrate U if necessary W k and U H k And update the consecutive satisfaction count; when the consecutive stopping condition is met, set κ=k, otherwise let k←k+1 until K.

[0157] 6) With Z W κ Z H κ The final non-negative reconstructed image patch is generated, the joint loss is calculated, and backpropagation is performed along the actual network path to update the trainable parameters.

[0158] 7) Repeat the initialization, expansion, update, and decoding training for each training batch, and end the training according to the pre-recorded number of training rounds or the validation set stopping rule, and save the model parameters, normalization parameters, image patch parameters, residual control parameters, and software environment information.

[0159] 8) Record the reconstruction error, structure preservation index, average number of execution expansion units and runtime resource index on the independent validation set, and select the model to be deployed based on the index.

[0160] VIII. Image Reconstruction Process

[0161] The image reconstruction process is as follows:

[0162] 1) Read the image to be processed and its bit depth, size and effective numerical range, and use the normalization parameters saved during training to obtain a non-negative image.

[0163] 2) Generate a non-negative image block matrix Y using the same block size, stride, and vectorization order as during training, and record the spatial position of each column.

[0164] 3) Load the trained shared basic dictionary, encoding branch parameters, expansion unit parameters, and residual control parameters, and execute the three initialization branches.

[0165] 4) Perform a maximum of K expansion units according to the above expansion update method. Exit early when q consecutive expansion units meet the residual threshold, and exit the layer or the Z value of the Kth layer. W Z H Generate non-negative reconstructed image patches.

[0166] 5) Inverse vectorize each column according to its recorded spatial location and place it back into the image plane. For non-overlapping blocks, directly merge adjacent image blocks; for overlapping blocks, set a non-negative window weight w for each block. j (u,v), and press

[0167]

[0168] The reconstructed values ​​covering the same pixel are merged, where Ω(u,v) is the set of image patches covering pixel (u,v). The window weights can be determined based on the distance from the pixel to the image patch boundary to reduce abrupt changes in patch boundaries.

[0169] 6) Use the values ​​a and b recorded during training to perform inverse normalization, truncate the pixel values ​​to the effective numerical range of the image to be processed, and perform numerical conversion according to the bit depth of the input image to obtain the reconstructed image.

[0170] This embodiment also provides an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the processor executes the computer program, it implements the training method described above, and / or the image reconstruction method described above.

[0171] This embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to implement the above-described training method and / or the above-described image reconstruction method.

[0172] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0173] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0174] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0175] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0176] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for training a residual-controlled unfolded non-negative autoencoder, characterized in that, include: Obtain training image pairs consisting of degraded input images and corresponding clean target images, and construct a non-negative image block matrix Y from the degraded input images; An expansion-type non-negative autoencoder is constructed, comprising an encoder, multiple sequentially connected alternating direction multiplier expansion units, a residual control module, and a decoder. The encoder is used to map the non-negative image block matrix to the initial state of the basis matrix, the initial state of the non-negative coefficient matrix, and the initial state of the sparse interference matrix, wherein the non-negative coefficient matrix serves as a latent representation. Execute the alternating direction multiplier expansion units according to the network connection order, and perform the original variables W of the basis matrix, H of the original variables of the coefficient matrix, and Z of the auxiliary variables of the basis matrix. W Coefficient matrix auxiliary variable Z H Update the sparse disturbance matrix S and the corresponding scaling dual variable, and make the auxiliary variable of the updated basis matrix non-negative and the L2 norm of each column 1, and the auxiliary variable of the updated coefficient matrix non-negative; The residual control module generates the original residual based on the consistency constraint deviation of each alternating direction multiplier expansion unit, generates the dual residual based on the change of auxiliary variables of adjacent alternating direction multiplier expansion units, normalizes the original residual and the dual residual, and adjusts the penalty coefficient, non-negative soft threshold and execution state used by the next alternating direction multiplier expansion unit according to their relative magnitude. The decoder generates non-negative reconstructed image patches based on the product of the auxiliary variables of the current basis matrix and the auxiliary variables of the current coefficient matrix; The joint loss is constructed based on the reconstruction loss between the non-negative reconstructed image patch and the target image patch corresponding to the target clean image, the sparse constraint loss of the latent representation and sparse interference, and the consistency loss between the original variable and the auxiliary variable. The joint loss is then backpropagated through the executed alternating direction multiplier expansion unit to update the trainable parameters of the unfolded non-negative autoencoder.

2. The residual-controlled expansion non-negative autoencoder training method according to claim 1, characterized in that, The encoder includes: A coefficient encoding branch is used to perform feature mapping and non-negative activation on the non-negative image block matrix to generate the initial state of the non-negative coefficient matrix. The basis matrix adaptive branch is used to combine the basic dictionary shared across training samples with the correction amount of the current sample, and perform non-negative column normalized projection to generate the initial state of the basis matrix. The sparse interference branch is used to generate the initial state of the sparse interference matrix based on the difference between the product of the input matrix and the initial state of the base matrix and the initial state of the coefficient matrix.

3. The residual-controlled expansion non-negative autoencoder training method according to claim 1, characterized in that, Each alternating direction multiplier expansion unit performs the following update process: Subtract the sparse interference estimate of the previous expansion unit from the non-negative image patch matrix, and update the original variables of the basis matrix and the original variables of the coefficient matrix; Perform a nonnegative column normalized projection on the sum of the original variables of the updated basis matrix and the corresponding scaled dual variables to obtain the auxiliary variables of the updated basis matrix; Perform non-negative soft thresholding on the sum of the original variables and the corresponding scaled dual variables of the updated coefficient matrix to obtain the auxiliary variables of the updated coefficient matrix; Update the sparse disturbance matrix and the two sets of scaling dual variables; The product of the two updated auxiliary variables is used as the non-negative reconstructed image patch of the expanded unit.

4. The residual-controlled expansion non-negative autoencoder training method according to claim 1, characterized in that, The original residual and the dual residual are calculated in the following manner: The original residual of the k-th alternating direction multiplier expansion unit and dual residual They are respectively: Here, vec indicates vectorization in a fixed order, and the semicolon indicates vertically connecting two vectors. The penalty coefficient for the k-th expansion unit is given by the superscript k, which indicates the corresponding variable after the k-th expansion unit is updated. The residual control module normalizes the original residual and the dual residual according to the number of elements in the basis matrix and the coefficient matrix, and updates the penalty coefficient of the next expansion unit according to the following rules based on the relative magnitude of the normalized original residual and the dual residual: When the normalized original residual is greater than the normalized dual residual by a preset multiple, the penalty coefficient is increased within the preset upper limit. When the normalized dual residual is greater than the normalized original residual by the preset multiple, the penalty coefficient is reduced within the preset lower limit. In all other cases, the penalty coefficient remains unchanged; Wherein, the preset multiple is greater than 1.

5. The residual-controlled expansion non-negative autoencoder training method according to claim 4, characterized in that, When the penalty coefficients are updated, the two sets of scaling dual variables are recalibrated according to the ratio of the old and new penalty coefficients; and the non-negative soft threshold of the auxiliary variable of the coefficient matrix of the next expansion unit is determined according to the ratio of the sparse regular weight to the updated penalty coefficient.

6. The residual-controlled expansion non-negative autoencoder training method according to claim 4, characterized in that, The residual control module is also used to control the stop execution state of the unfolding unit: When the normalized original residual and normalized dual residual of q consecutive expansion units are not greater than their respective stopping thresholds, the execution of subsequent expansion units is stopped, and the product of the current two auxiliary variables is passed to the decoder. Otherwise, continue executing subsequent expansion units until the maximum number of expansion units K is reached; Where q and K are positive integers, and q is not greater than K.

7. A method for image reconstruction using an unfolded non-negative autoencoder trained according to any one of claims 1-6, characterized in that, include: The image to be processed is obtained, and it is non-negatively normalized, divided into image blocks, and arranged by column matrix to obtain a non-negative image block matrix. The non-negative image patch matrix is ​​input into the trained unfolded non-negative autoencoder, which generates the initial state of the basis matrix, the initial state of the non-negative coefficient matrix, and the initial state of the sparse interference matrix. The alternating direction multiplier expansion unit is executed sequentially to update the original variables of the basis matrix, the original variables of the coefficient matrix, the auxiliary variables of the basis matrix, the auxiliary variables of the coefficient matrix, the sparse interference matrix, and the scaling dual variables. The residual control module adjusts the penalty coefficient, the non-negative soft threshold, and the execution status of subsequent alternating direction multiplier expansion units based on the normalized original residual and the normalized dual residual. When the residual stopping condition is met or the maximum number of unfolded units K is reached, the decoder generates a non-negative reconstructed image block based on the product of the current non-negative basis matrix auxiliary variable and the current non-negative coefficient matrix auxiliary variable. A reconstructed image is formed based on the non-negative reconstructed image blocks.

8. The image reconstruction method according to claim 7, characterized in that, Forming a reconstructed image based on the non-negative reconstructed image blocks includes: Based on the spatial position and column arrangement order recorded when dividing the image blocks, the non-negative reconstructed image blocks are inversely vectorized and inversely arranged. For overlapping non-negative reconstructed image blocks, non-negative fusion weights are determined based on the distance from pixels in each image block to the image block boundary. Pixel values ​​at the same spatial location are weighted and summed and then divided by the sum of the corresponding fusion weights. The fused pixel values ​​are restored to the numerical range of the image to be processed based on the numerical transformation parameters recorded during non-negative normalization, and pixel values ​​that exceed the numerical range are truncated to obtain the reconstructed image.

9. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the training method as described in any one of claims 1-7, and / or, the image reconstruction method as described in claim 8 or 9.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to implement the training method as described in any one of claims 1-7, and / or the image reconstruction method as described in claim 8 or 9.

Images