An image processing method based on singular value loss

Abstract

The present disclosure relates to a singular value loss-based image processing method, and relates to the technical field of image processing, obtaining an output sample image and an output image output by an image processing model, extracting N feature parameters from the output image, and screening n feature parameters from the N feature parameters, calculating a loss function according to the n feature parameters, the output image and the output sample image, and training the image processing model through the loss function, to improve the performance of the model, the image processing model can be trained for m times, in each iteration, new n feature parameters are reacquired to calculate a new loss function, and after m iterations, a trained image processing model is obtained. In each iteration process, the feature parameters used for model training are different, avoiding the situation that the trained model is in false convergence, so that the finally obtained model has good performance, and the obtained model is used for image processing, improving the image processing effect.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more specifically, to an image processing method based on singular value loss. Background Technology

[0002] In recent years, with the rapid development of machine learning, it has been widely applied in various technical fields. For example, trained machine learning models can be applied to image processing. In machine learning, the loss function is a mathematical function that describes the difference between the model's predicted value and the expected value. Minimizing the loss function can better guide the predicted value to fit the expected value during model training. Therefore, during model training, minimizing the loss function can be used as the training objective. Specifically, this involves finding the point where the error gradient is zero; this point is called the minimum. In reality, there are usually multiple minimums, and most of them are local minima. Therefore, during model training, the loss function is very prone to getting trapped in local minima. Local minima can lead to spurious convergence, resulting in poor predictive ability of the trained model, with the predicted value deviating significantly from the expected value, leading to unsatisfactory actual processing results. Summary of the Invention

[0003] The purpose of this disclosure is to provide an image processing method based on singular value loss, which aims to solve the above-mentioned problems.

[0004] To achieve the above objectives, a first aspect of this disclosure provides an image processing method based on singular value loss. The method includes: acquiring an output image of an image processing model based on an input sample image, and acquiring an output sample image corresponding to the input sample image; extracting N feature parameters from the output image, and selecting n feature parameters from the N feature parameters, where N and n are both positive integers, and N is greater than or equal to n; obtaining a loss function based on the n feature parameters, the output image, and the output sample image; iteratively training the image processing model using the loss function, obtaining a trained image processing model after m iterations, wherein the step of selecting n feature parameters from the N feature parameters needs to be re-executed in each iteration of the m iterations of training the image processing model, where m is a positive integer greater than 2; inputting the image to be processed into the trained image generation model, and obtaining the processed image output by the trained image generation model.

[0005] Optionally, extracting N feature parameters from the output image includes: obtaining the image size X*Y of the output image, where X and Y are both positive integers; and extracting the N feature parameters based on the image size X*Y, where the smaller of X and Y is equal to N.

[0006] Optionally, obtaining the loss function based on the n feature parameters, the output image, and the output sample image includes: performing singular value decomposition on the output image to obtain a first orthogonal matrix, a second orthogonal matrix, and a first diagonal matrix; performing singular value decomposition on the output sample image to obtain a third orthogonal matrix, a fourth orthogonal matrix, and a second diagonal matrix; obtaining a filtering tensor based on the n feature parameters; obtaining a first eigenvalue tensor based on the filtering tensor and the first diagonal matrix; obtaining a second eigenvalue tensor based on the filtering tensor and the second diagonal matrix; obtaining the restored output image based on the first orthogonal matrix, the second orthogonal matrix, and the first eigenvalue tensor; and obtaining the restored output sample image based on the restored output image and the restored output sample image.

[0007] Optionally, obtaining the first eigenvalue tensor based on the filtering tensor and the first diagonal matrix includes: obtaining a first intermediate tensor based on the product of the filtering tensor and the first diagonal matrix; and expanding the first intermediate tensor to obtain the first eigenvalue tensor. Obtaining the second eigenvalue tensor based on the filtering tensor and the second diagonal matrix includes: obtaining a second intermediate tensor based on the product of the filtering tensor and the second diagonal matrix; and expanding the second intermediate tensor to obtain the second eigenvalue tensor.

[0008] Optionally, the first intermediate tensor is expanded to obtain a first diagonal matrix; the first diagonal matrix is ​​expanded to obtain the first eigenvalue tensor; the expansion process of the second intermediate tensor to obtain the second eigenvalue tensor includes: expanding the second intermediate tensor to obtain a second diagonal matrix; expanding the second diagonal matrix to obtain the second eigenvalue tensor.

[0009] Optionally, the filtering tensor is a matrix with 1 row and N columns, where the elements at the positions corresponding to the n feature parameters in the filtering tensor are 1, and the elements at the other positions are 0.

[0010] Optionally, the n feature parameters are any n of the N feature parameters, or the n feature parameters with the largest feature values ​​among the N feature parameters.

[0011] A second aspect of this disclosure provides an image processing apparatus, comprising: an image acquisition module for acquiring an output image output by an image processing model based on an input sample image, and acquiring an output sample image corresponding to the input sample image; a filtering module for extracting N feature parameters from the output image and filtering out n feature parameters from the N feature parameters, wherein N and n are both positive integers, and N is greater than or equal to n; a loss calculation module for obtaining a loss function based on the n feature parameters, the output image, and the output sample image; an iterative training module for iteratively training the image processing model using the loss function, obtaining a trained image processing model after m iterations, wherein the step of filtering out n feature parameters from the N feature parameters needs to be re-executed in each iteration of the m iterations of training the image processing model, wherein m is a positive integer greater than 2; and an image generation module for inputting the image to be processed into the trained image generation model to obtain a processed image output by the trained image generation model.

[0012] A third aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in the first aspect.

[0013] A fourth aspect of this disclosure provides an electronic device, the electronic device comprising: a memory having a computer program stored thereon;

[0014] A processor for executing the computer program in the memory to implement the steps of the method described in the first aspect.

[0015] This disclosure provides an image processing method based on singular value loss. The method involves inputting an input sample image into an image processing model to obtain the output image. It also involves obtaining the desired output sample image corresponding to the input sample image, extracting N feature parameters from the output image, and selecting a subset of these N feature parameters (i.e., n feature parameters). A loss function is calculated based on these n feature parameters, the output image, and the output sample image. This loss function is then used to train the image processing model. To improve model performance, the model can be trained through m iterations. In each iteration, new n feature parameters are obtained, and a new loss function is calculated. After m iterations, a trained image processing model is obtained. In each iteration, the feature parameters used for model training are different. Even if the n feature parameters used in some iterations lead to local minima, the n feature parameters are reselected in other iterations, allowing the model to escape from local minima and avoid spurious convergence. This results in a well-performing final model, which, when used for image processing, improves image processing efficiency.

[0016] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings:

[0018] Figure 1 This is a graph of the MAE loss function;

[0019] Figure 2 This is a graph of the MSE loss function;

[0020] Figure 3 This is a flowchart illustrating an image processing method based on singular value loss according to an exemplary embodiment;

[0021] Figure 4 yes Figure 3 A flowchart of the sub-steps of step S130 of an image processing method based on singular value loss is shown.

[0022] Figure 5 The image to be processed;

[0023] Figure 6 Images obtained by existing image processing methods;

[0024] Figure 7The image obtained by the image processing method based on singular value loss provided in this disclosure;

[0025] Figure 8 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment;

[0026] Figure 9 This is a block diagram illustrating an electronic device for an image processing method based on singular value loss, according to an exemplary embodiment. Detailed Implementation

[0027] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0028] In recent years, with the rapid development of machine learning, it has been widely applied in various technical fields. For example, trained machine learning models can be applied to image processing. In machine learning, the loss function is a mathematical function that describes the difference between the model's predicted value and the expected value, where the predicted value refers to the model's output and the expected value refers to the true value. The loss function can be used to measure the quality of a model, and minimizing the loss function can better guide the predicted value to fit the expected value during model training. Therefore, during model training, minimizing the loss function can be used as the training objective. Specifically, this involves finding the point where the error gradient is zero; the point where the error gradient is zero is called the minimum.

[0029] Loss functions come in various forms, with common ones including MAE (Mean Absolute Error) loss function and MSE (Mean Square Error) loss function.

[0030] In the calculation of the loss function for Model-Analogous Error (MAE), MAE directly calculates the correct error between the model's predicted values ​​and the actual values ​​to obtain the MAE loss function. The MAE loss function is as follows: Figure 1 As shown, MAE cannot be differentiated near the approximate solution, making it inconvenient to solve. Through... Figure 1 It is evident that MAE possesses a stable gradient value, which leads to continuous oscillations in the model.

[0031] In the MSE loss function calculation method, the Euclidean distance between the predicted value and the expected value in space is predicted, and the resulting MSE loss function is as follows: Figure 2 As shown, when the predicted value differs significantly from the expected value, the gradient value is large, which may lead to gradient explosion. Near the approximate solution, the gradient value is small, which may lead to gradient vanishing, slowing down model training.

[0032] During model training, the loss function is calculated by comparing all predicted values ​​with all expected values. In this process, the global loss distribution of the model remains stable and essentially unchanged. However, training the model with only these values ​​can lead to local minima. Since the input values ​​remain unchanged during iterative training, the loss function gets trapped in a local minimum, also known as a local optimum. In reality, there are usually multiple minimums, and most of them are local minima. Therefore, the loss function is highly susceptible to getting trapped in local minima. Local minima can cause the model to spuriously converge, resulting in poor predictive ability. The predicted values ​​output by the model deviate significantly from the expected values, leading to unsatisfactory actual processing results. For example, when performing image restoration, the restored image may have low clarity or even fail to achieve the desired restoration effect.

[0033] To address the aforementioned problems, this disclosure provides an image processing method based on singular value loss, which can be applied to... Figure 8 The image processing device 100 shown and Figure 9 The electronic device 700 shown in this embodiment is exemplified by its application in an electronic device. This electronic device can be a server, or it can be a terminal device such as a smartphone, tablet, or television. For details, please refer to [link to relevant documentation]. Figure 3 The image processing method based on singular value loss may include the following steps:

[0034] Step S110: Obtain the output image of the image processing model based on the input sample image, and obtain the output sample image corresponding to the input sample image.

[0035] Sample images are pre-collected and used to train the image processing model. Training samples include input sample images and their corresponding output sample images; in essence, input and output sample images exist in pairs. The sample images are selected based on the application scenario of the image processing model. For example, when applying the model to old photo restoration, the input sample image might be a low-quality black and white photo, and the output sample image might be a restored high-quality photo. Similarly, when applying the model to image restoration, the input sample image might be an image covered by oil stains, mosaics, or other obfuscation, and the output sample image might be the image after removing the obfuscation. Furthermore, when applying the model to blur an image, the input sample image might be a sharp image, and the output sample image might be a blurred image.

[0036] It can be understood that the input sample image is an image to be processed by an image processing model, and the output sample image is the image expected to be output when the input sample image is used as the input of the image processing model. The output image is the image actually output by the image processing model when the input sample image is used as the input.

[0037] Optionally, in order to obtain a model with better performance, the image processing model needs to be iteratively trained multiple times. For example, iterate m times, where m is a positive integer greater than 2, and m can be 1000 times. The image processing model in this step can be the model at any one of the m iterations (which can be called the i-th iteration). For example, the image processing model in the i-th iteration can be the model in the initial training, or the model in the 100th iteration process.

[0038] Step S120: Extract N feature parameters from the output image, and screen out n feature parameters from the N feature parameters, where both N and n are positive integers, and N is greater than or equal to n.

[0039] Through a feature extraction algorithm, N feature parameters are extracted from the input image. Optionally, the feature parameters can be in the form of a matrix or an array. In order to avoid falling into local minima in this embodiment of the present disclosure, when performing one round of model training, not all the feature parameters (i.e., N feature parameters) can be used, but some feature parameters are screened out from the N feature parameters for the model training in this round.

[0040] In one implementation, the feature parameters can be extracted from the input image in the following manner: Obtain the image size X*Y of the output image, where both X and Y are positive integers. According to the image size X*Y, extract the N feature parameters, where the smaller of the values of X and Y is equal to N. It can be understood that when X>Y, then N = Y, and Y feature parameters are extracted from the output image. When X<Y, then N = X, and X feature parameters are extracted from the output image.

[0041] In one implementation, the feature parameters can be screened in the following manner: According to a preset screening strategy, screen out n feature parameters from the N feature parameters of the input image. As one way, the preset screening strategy stipulates that n are randomly selected from the N feature parameters. According to this preset screening strategy, n feature parameters are randomly selected from the N feature parameters.

[0042] As another way, the preset screening strategy stipulates that the n with the largest eigenvalues are selected from the N feature parameters. According to this preset screening strategy, screen out the n feature parameters with the largest eigenvalues from the N feature parameters.

[0043] As another approach, multiple different screening strategies can be pre-defined, and different screening strategies can be used in different rounds to select n feature parameters for model training.

[0044] Optionally, the preset screening strategy stipulates that n changes as the number of screening rounds increases during the screening process. For example, n increases as the number of screening rounds increases. As one approach, for each increase of a first preset quantity 'a' in the screening rounds, n increases by a second preset quantity 'b' based on the previous round. For example, the first preset quantity 'a' can be, but is not limited to, 1, 7, 200, 250, etc., and the second preset quantity 'b' can be, but is not limited to, 6, 8, 11, etc.

[0045] For example, let's take an initial value of n = 1, a first preset quantity a = 1, and a second preset quantity b = 6 as an example. During initial training, n = 1, and one feature parameter is selected from N feature parameters. In the second round of iterative training, as training progresses, the second round of iterative training begins. Using the preset selection strategy, n = 1 + 6 = 7, and 7 feature parameters are selected from N feature parameters. This process continues until m iterations have been completed.

[0046] As another approach, the second preset quantity is an increment based on the previous round. Different preset selection strategies exist for different iteration ranges. For example, with an initial value of 1 for n, between iterations i (0-1000), n increases by 6 every 250 training rounds. For instance, when i=250, n=1+6=7; when i=500, n=7+6=13. Between iterations i (1000-4000), n increases by 5 every 300 training rounds. When the number of iterations exceeds 4000, n increases by 10 every 100 training rounds.

[0047] Step S130: Obtain the loss function based on the n feature parameters, the output image, and the output sample image.

[0048] The loss function is obtained based on n feature parameters, the output image, and the output sample image. The obtained loss function only uses some of the feature parameters in the output image.

[0049] Step S140: Iteratively train the image processing model using the loss function, and obtain the trained image processing model after m iterations. In each iteration of the m iterations of training the image processing model, the step of selecting n feature parameters from the N feature parameters needs to be re-executed, where m is a positive integer greater than 2.

[0050] In each iteration of training, it is necessary to reselect n feature parameters from the N feature parameters, calculate the loss function using the newly selected feature parameters, and iterate the model using the new loss function until the m iterations are completed.

[0051] Step S150: Input the image to be processed into the trained image generation model to obtain the processed image output by the trained image generation model.

[0052] After m iterations of training, a trained image generation model is obtained. The trained image generation model is then used to process images, such as repairing or restoring the input image.

[0053] This embodiment provides an image processing method based on singular value loss. An input sample image is input into an image processing model to obtain the output image. The desired output sample image corresponding to the input sample image is also obtained. N feature parameters are extracted from the output image, and a subset of these N feature parameters, i.e., n feature parameters, are selected. A loss function is calculated based on the n feature parameters, the output image, and the output sample image. The image processing model is then trained using this loss function. To improve model performance, the image processing model can be trained through m iterations. In each iteration, new n feature parameters are obtained, and a new loss function is calculated. After m iterations, the trained image processing model is obtained. In each iteration, the feature parameters used for model training are different. Even if the n feature parameters used in some rounds lead to local minima, the n feature parameters are reselected in other iterations, so that the model training can get rid of local minima and avoid getting stuck in local minima. This avoids the trained model being in a false convergence situation, resulting in a model with good performance. The obtained model can then be used for image processing to improve the image processing effect, obtain images with richer details, and achieve higher image quality.

[0054] In one implementation, such as Figure 4 As shown, step S130 includes the following sub-steps:

[0055] Sub-step S131: Perform singular value decomposition on the output image to obtain a first orthogonal matrix, a second orthogonal matrix, and a first diagonal matrix; and perform singular value decomposition on the output sample image to obtain a third orthogonal matrix, a fourth orthogonal matrix, and a second diagonal matrix.

[0056] One approach is to use the Singular Value Decomposition (SVD) algorithm for decomposition. The SVD algorithm performs singular value decomposition on the output image to obtain the first orthogonal matrix. First diagonal matrix Second orthogonal matrix For example, feature extraction is performed on the input image to obtain the first feature matrix. Then, the SVD algorithm is used to analyze the first feature matrix. Perform eigenvalue decomposition, and convert the first feature matrix... Decomposed into the first orthogonal matrix First diagonal matrix Second orthogonal matrix The form of multiplication, that is If the first characteristic matrix Let N be an X-row, Y-column matrix, i.e., X*Y. Taking X less than Y as an example, and referring to the example in the previous embodiment, when X is less than Y, N=X, then the first orthogonal matrix is ​​obtained. Let X be a square matrix, and let the first diagonal matrix be... An array of size 1*N, for example, the array is [ , , … ], the second orthogonal matrix It is a Y*Y square matrix.

[0057] Similarly, the Singular Value Decomposition (SVD) algorithm is used to perform singular value decomposition on the output sample image to obtain the third orthogonal matrix. The second diagonal matrix and the fourth orthogonal matrix For example, feature extraction is performed on the output sample image to obtain the second feature matrix A. Then, the SVD algorithm is used to perform feature analysis on the second feature matrix A, decomposing the second feature matrix into a third orthogonal matrix. The second diagonal matrix and the fourth orthogonal matrix The form of multiplication, that is The output image and the output sample image are the same size. Continuing with the example above, the second feature matrix A is an X-row, Y-column matrix, i.e., X*Y. The resulting third orthogonal matrix... Let X be a square matrix, and let the second diagonal matrix be... An array of size 1*X, for example, the array is [ , , ... ], the fourth orthogonal matrix It is a Y*Y square matrix.

[0058] Sub-step S132: Obtain the screening tensor based on the n feature parameters.

[0059] Based on the n feature parameters, a filtering tensor T is constructed, where the filtering tensor T is a 1-row, N-column matrix. In the filtering tensor T, the elements corresponding to the n feature parameters are 1, and the elements in the remaining positions are 0. It can be understood that, in the N feature parameters, each position corresponding to a feature parameter is 1, and multiplying any number by 1 leaves the result unchanged, thus retaining the n feature parameters after processing. In the N feature parameters, all positions except the n feature parameters are set to 0, and multiplying any number by 0 results in 0, thus deleting the feature parameters in the remaining positions. For example, the filtering tensor T can be [1,0,0,1,0…1].

[0060] Sub-step S133: Obtain a first eigenvalue tensor based on the filtering tensor and the first diagonal matrix, and obtain a second eigenvalue tensor based on the filtering tensor and the second diagonal matrix.

[0061] The first eigenvalue tensor can be obtained from the filtering tensor and the first diagonal matrix as follows: First, based on the filtering tensor T and the first diagonal matrix... The product between them yields the first intermediate tensor. ,Right now = Then, the first intermediate tensor is expanded to obtain the first eigenvalue tensor. As one method, the first intermediate tensor... Expanding the array yields the first diagonal square. That is, to expand it into an N x N diagonal matrix. Then, consider the first diagonal matrix. Expand the model to obtain the first eigenvalue tensor. Wherein, the magnitude of the first eigenvalue tensor is equal to the magnitude of the first feature matrix. The sizes are all the same, X*Y. It can be understood that, in this embodiment, the first diagonal square matrix of N rows and N columns is... Expanded into an X-row, Y-column first eigenvalue tensor The image will then need to be reconstructed using two orthogonal matrices obtained from the output image and the first diagonal matrix. The rows of the first orthogonal matrix are the same as the columns of the expanded first diagonal matrix, which facilitates the subsequent reconstruction calculation.

[0062] Similarly, the second eigenvalue tensor can be obtained from the filtering tensor and the second diagonal matrix as follows: first, based on the filtering tensor T and the second diagonal matrix... The product between them yields the second intermediate tensor. The second intermediate tensor is then expanded to obtain the second eigenvalue tensor. As one method, the second intermediate tensor... Expanding the matrix yields a second diagonal square. That is, to expand it into an N x N diagonal matrix. Then consider the second diagonal matrix. The expansion is performed to obtain the second eigenvalue tensor. The second eigenvalue tensor has the same size as the second eigenma matrix A, meaning their rows and columns are identical and can both be represented as X*Y. This can be understood as representing the second diagonal matrix of N rows and N columns. Expanded into a second eigenvalue tensor with X rows and Y columns The image will then need to be reconstructed using two orthogonal matrices and a second diagonal matrix obtained from the output sample image. The rows of the third orthogonal matrix are the same as the columns of the expanded second diagonal matrix, which facilitates the subsequent reconstruction calculation.

[0063] Sub-step S134: Obtain the restored output image based on the first orthogonal matrix, the second orthogonal matrix, and the first eigenvalue tensor; and obtain the restored output sample image based on the third orthogonal matrix, the fourth orthogonal matrix, and the second eigenvalue tensor.

[0064] Through the first orthogonal matrix Second orthogonal matrix and the first eigenvalue tensor Obtain the output image after the response. .

[0065] Through the third orthogonal matrix Fourth orthogonal matrix Second eigenvalue tensor Obtain the restored output sample image .

[0066] Sub-step S135: Obtain the SVD loss function based on the recovered output image and the recovered output sample image.

[0067] Calculate the SVD loss function: .in, Let i be the SVD loss function, and i be the actual number of iterations.

[0068] In this embodiment, the output image is decomposed into two orthogonal matrices and a first diagonal matrix. Similarly, the output sample image is decomposed into two orthogonal matrices and a second diagonal matrix. A filtering tensor is obtained by selecting n feature parameters from N feature parameters. This filtering tensor is used to filter out some data from the first diagonal matrix, resulting in a first eigenvalue tensor. The first eigenvalue tensors of the two orthogonal matrices decomposed from the output image are then used to reconstruct the output image, resulting in a reconstructed output image with fewer features than the original. Likewise, a filtering tensor is used to filter out a portion of the second diagonal matrix, resulting in a second eigenvalue tensor. The two orthogonal matrices and the second eigenvalue tensor of the output sample image are then used to reconstruct the output sample image, resulting in a restored output sample image with fewer features than the original. Based on the restored output images and the restored sample images, an SVD loss function is obtained. Using the SVD loss function obtained from partial features avoids getting trapped in local minima. Training the model using the obtained SVD loss function can prevent spurious convergence and improve the performance of the final model.

[0069] When training a model using a loss function, the problem of getting trapped in local minima can be avoided as long as the features used each time are different. This means that in m iterations of training, it is acceptable as long as the features used are not identical each time. When calculating the loss function, feature reduction can be performed on either the output image or the output sample image. In another implementation, feature reduction can be performed only on the output image. For example, singular value decomposition (SVD) is performed on the output image to obtain a first orthogonal matrix, a second orthogonal matrix, and a first diagonal matrix. A selection tensor is obtained based on the n feature parameters selected from the N feature parameters. Then, the restored output image is obtained based on the first orthogonal matrix, the second orthogonal matrix, and the first feature value tensor. Finally, the SVD loss function is obtained based on the restored output image and the output sample image.

[0070] In another implementation, feature reduction can be performed only on the output sample image. For example, singular value decomposition (SVD) is performed on the output sample image to obtain a third orthogonal matrix, a fourth orthogonal matrix, and a second diagonal matrix. A filtering tensor is obtained based on the n feature parameters. Then, a second eigenvalue tensor is obtained based on the filtering tensor and the second diagonal matrix. The restored output sample image is then obtained based on the third orthogonal matrix, the fourth orthogonal matrix, and the second eigenvalue tensor. Finally, the SVD loss function is obtained based on the output image and the restored output sample image.

[0071] The image processing method based on singular value loss provided in the above embodiments can be applied to image inpainting scenarios. To verify the effectiveness of the image processing method based on singular value loss provided in this disclosure, both this disclosure and the prior art use 11 pairs of sample images for 11,000 iterations of training. During the training process, data statistics were performed. The average PSNR (Peak Signal-to-Noise Ratio) of the image processing model trained in this disclosure is 33.50, while the average PSNR of the model obtained by the prior art is 33.08. The average PSNR of 33.50 corresponding to this disclosure is greater than the existing average PSNR of 33.08, indicating that this disclosure can obtain output images of higher quality. Furthermore, it employs methods such as... Figure 5 The image to be processed is shown for verification. In existing methods, the trained model processes the image to be processed and obtains... Figure 6 The image shown. After processing the image using the singular value loss-based image processing method provided in this disclosure, the result is... Figure 7 The image shown. After comparison... Figure 6 and Figure 7 Details such as windows, roof, and walls are visible. Figure 7 Compare Figure 6 It possesses richer details. Based on the average PSNR of both methods and their processing results, it is evident that the image processed by this disclosure has richer details, surpassing images obtained using existing technologies. When the display screen showing the processing results is large, the superiority of the image obtained by this disclosure becomes even more apparent when the processed image is magnified.

[0072] To implement the above-described method embodiments, this embodiment provides an image processing apparatus. Figure 8 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment. Please refer to... Figure 8 The image processing device 100 includes: an image acquisition module 110, a filtering module 120, a loss calculation module 130, an iterative training module 140, and an image generation module 150.

[0073] The image acquisition module 110 is used to acquire the output image output by the image processing model based on the input sample image, and to acquire the output sample image corresponding to the input sample image;

[0074] The filtering module 120 is used to extract N feature parameters from the output image and filter out n feature parameters from the N feature parameters, where N and n are both positive integers, and N is greater than or equal to n;

[0075] Loss calculation module 130 is used to obtain a loss function based on the n feature parameters, the output image, and the output sample image;

[0076] The iterative training module 140 is used to iteratively train the image processing model using the loss function, and obtain the trained image processing model after m iterations. In each iteration of the m iterations of training the image processing model, the step of selecting n feature parameters from the N feature parameters needs to be re-executed, where m is a positive integer greater than 2.

[0077] The image generation module 150 is used to input the image to be processed into the trained image generation model to obtain the processed image output by the trained image generation model.

[0078] Optionally, the filtering module 120 includes a size acquisition module and a feature extraction module.

[0079] The size acquisition module is used to acquire the image size X*Y of the output image, where X and Y are both positive integers;

[0080] The feature extraction module is used to extract the N feature parameters based on the image size X*Y, wherein the smaller of X and Y values ​​is equal to N.

[0081] Optionally, the loss calculation module 130 includes: a decomposition module, a tensor filtering module, a feature tensor acquisition module, an image restoration module, and an SVD loss function calculation module.

[0082] The decomposition module is used to perform singular value decomposition on the output image to obtain a first orthogonal matrix, a second orthogonal matrix, and a first diagonal matrix, and to perform singular value decomposition on the output sample image to obtain a third orthogonal matrix, a fourth orthogonal matrix, and a second diagonal matrix.

[0083] The tensor filtering module is used to obtain the filtering tensor based on the n feature parameters;

[0084] The eigenvalue tensor acquisition module is used to obtain a first eigenvalue tensor based on the filtering tensor and the first diagonal matrix, and to obtain a second eigenvalue tensor based on the filtering tensor and the second diagonal matrix;

[0085] The image restoration module is used to obtain a restored output image based on the first orthogonal matrix, the second orthogonal matrix, and the first eigenvalue tensor, and to obtain a restored output sample image based on the third orthogonal matrix, the fourth orthogonal matrix, and the second eigenvalue tensor.

[0086] The SVD loss function calculation module is used to obtain the SVD loss function based on the restored output image and the restored output sample image.

[0087] Optionally, the feature value tensor acquisition module includes: a first intermediate tensor acquisition module, a first expansion module, a second intermediate tensor acquisition module, and a second expansion module.

[0088] The first intermediate tensor acquisition module is used to obtain the first intermediate tensor based on the product between the filtered tensor and the first diagonal matrix;

[0089] The first expansion module is used to expand the first intermediate tensor to obtain the first feature value tensor;

[0090] The second intermediate tensor acquisition module is used to obtain the second intermediate tensor based on the product between the filtered tensor and the second diagonal matrix;

[0091] The second expansion module is used to expand the second intermediate tensor to obtain the second feature value tensor.

[0092] Optionally, the first expansion module includes: a first expansion submodule and a second expansion submodule.

[0093] The first expansion submodule is used to expand the first intermediate tensor to obtain a first diagonal matrix;

[0094] The second expansion submodule is used to expand the first diagonal matrix to obtain the first eigenvalue tensor.

[0095] The second expansion module includes: a third expansion submodule and a fourth expansion submodule.

[0096] The third expansion submodule is used to expand the second intermediate tensor to obtain the second diagonal matrix;

[0097] The fourth expansion submodule is used to expand the second diagonal matrix to obtain the second eigenvalue tensor.

[0098] Optionally, the filtering tensor is a matrix with 1 row and N columns, where the elements at the positions corresponding to the n feature parameters in the filtering tensor are 1, and the elements at the other positions are 0.

[0099] Optionally, the n feature parameters are any n of the N feature parameters, or the n feature parameters with the largest feature values ​​among the N feature parameters.

[0100] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0101] Figure 9 This is a block diagram illustrating an electronic device for an image processing method based on singular value loss, according to an exemplary embodiment, such as... Figure 9As shown, the electronic device 700 may include a processor 701 and a memory 702. The electronic device 700 may also include one or more of a multimedia component 703, an input / output (I / O) interface 704, and a communication component 705.

[0102] The processor 701 controls the overall operation of the electronic device 700 to complete all or part of the steps in the image processing method based on singular value loss described above. The memory 702 stores various types of data to support the operation of the electronic device 700. This data may include, for example, instructions for any application or method operating on the electronic device 700, and application-related data such as contact data, sent and received messages, pictures, audio, video, etc. The memory 702 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. Multimedia component 703 may include a screen and an audio component. The screen may be, for example, a touchscreen, and the audio component is used to output and / or input audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signals may be further stored in memory 702 or transmitted via communication component 705. The audio component also includes at least one speaker for outputting audio signals. I / O interface 704 provides an interface between processor 701 and other interface modules, such as a keyboard, mouse, buttons, etc. These buttons may be virtual or physical buttons. Communication component 705 is used for wired or wireless communication between the electronic device 700 and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IoT, eMTC, or other 5G technologies, or combinations thereof, is not limited here. Therefore, the corresponding communication component 705 may include: a Wi-Fi module, a Bluetooth module, an NFC module, etc.

[0103] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described image processing method based on singular value loss.

[0104] In another exemplary embodiment, a computer-readable storage medium including program instructions is also provided, which, when executed by a processor, implement the steps of the singular value loss-based image processing method described above. For example, the computer-readable storage medium may be the memory 702 including the program instructions described above, which may be executed by the processor 701 of the electronic device 700 to complete the singular value loss-based image processing method described above.

[0105] In another exemplary embodiment, a computer program product is also provided, comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described singular value loss-based image processing method when executed by the programmable device.

[0106] In summary, the image processing method based on singular value loss provided in this disclosure involves inputting an input sample image into an image processing model to obtain the output image of the model. Furthermore, it obtains the expected output sample image corresponding to the input sample image, extracts N feature parameters from the output image, and selects a subset of these N feature parameters (i.e., n feature parameters). A loss function is calculated based on these n feature parameters, the output image, and the output sample image. This loss function is then used to train the image processing model. To improve model performance, the model can be trained through m iterations. In each iteration, new n feature parameters are obtained, and a new loss function is calculated. After m iterations, a trained image processing model is obtained. In each iteration, the feature parameters used for model training are different. Even if the n feature parameters used in some iterations lead to local minima, the n feature parameters are reselected in other iterations, allowing the model training to escape from local minima and avoid falling into them. This prevents the trained model from being in a state of false convergence, resulting in a well-performing final model. Applying this model to image processing can improve image processing results.

[0107] The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0108] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0109] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A method for image processing based on singular value loss, characterized in that, The method comprises: obtaining an output image output by an image processing model based on an input sample image, and obtaining an output sample image corresponding to the input sample image; extracting N feature parameters from the output image, and screening n feature parameters from the N feature parameters, wherein N and n are positive integers, and N is greater than or equal to n; obtaining a loss function according to the n feature parameters, the output image and the output sample image; iteratively training the image processing model through the loss function, and obtaining a trained image processing model after m iterations, wherein the step of screening n feature parameters from the N feature parameters needs to be re-executed in each iteration of the m iterations of iteratively training the image processing model, and m is a positive integer greater than 2; inputting a to-be-processed image into the trained image processing model to obtain a processed image output by the trained image processing model.

2. The method of claim 1, wherein, The step of extracting N feature parameters from the output image comprises: obtaining an image size X*Y of the output image, wherein X and Y are positive integers; extracting the N feature parameters according to the image size X*Y, wherein the value of the smaller one of X and Y is equal to N.

3. The method of claim 1, wherein, The step of obtaining a loss function according to the n feature parameters, the output image and the output sample image comprises: performing singular value decomposition on the output image to obtain a first orthogonal matrix, a second orthogonal matrix and a first diagonal matrix, and performing singular value decomposition on the output sample image to obtain a third orthogonal matrix, a fourth orthogonal matrix and a second diagonal matrix; obtaining a screening tensor according to the n feature parameters; obtaining a first eigenvalue tensor according to the screening tensor and the first diagonal matrix, and obtaining a second eigenvalue tensor according to the screening tensor and the second diagonal matrix; obtaining a restored output image according to the first orthogonal matrix, the second orthogonal matrix and the first eigenvalue tensor, and obtaining a restored output sample image according to the third orthogonal matrix, the fourth orthogonal matrix and the second eigenvalue tensor; obtaining an SVD loss function according to the restored output image and the restored output sample image.

4. The method of claim 3, wherein, The step of obtaining a first eigenvalue tensor according to the screening tensor and the first diagonal matrix comprises: obtaining a first intermediate tensor according to the product between the screening tensor and the first diagonal matrix; performing extension processing on the first intermediate tensor to obtain the first eigenvalue tensor; The step of obtaining a second eigenvalue tensor according to the screening tensor and the second diagonal matrix comprises: obtaining a second intermediate tensor according to the product between the screening tensor and the second diagonal matrix; performing extension processing on the second intermediate tensor to obtain the second eigenvalue tensor.

5. The method of claim 4, wherein, The step of performing extension processing on the first intermediate tensor to obtain the first eigenvalue tensor comprises: performing extension on the first intermediate tensor to obtain a first diagonal square matrix; performing extension on the first diagonal square matrix to obtain the first eigenvalue tensor; The expansion processing is performed on the second intermediate tensor to obtain the second eigenvalue tensor, including: The second intermediate tensor is expanded to obtain a second diagonal matrix; The second diagonal matrix is expanded to obtain the second eigenvalue tensor.

6. The method of claim 3, wherein, The screening tensor is a matrix with 1 row and N columns, and elements at positions corresponding to the n characteristic parameters in the screening tensor are 1, and elements at other positions are 0.

7. The method of claim 1, wherein, The n characteristic parameters are any n of the N characteristic parameters, or n characteristic parameters with the largest eigenvalues in the N characteristic parameters.

8. An image processing apparatus characterized by comprising: The device comprises: An image acquisition module configured to acquire an output image output by an image processing model based on an input sample image, and acquire an output sample image corresponding to the input sample image; A screening module configured to extract N characteristic parameters from the output image, and screen n characteristic parameters from the N characteristic parameters, wherein N and n are positive integers, and N is greater than or equal to n; A loss calculation module configured to obtain a loss function according to the n characteristic parameters, the output image, and the output sample image; An iterative training module configured to iteratively train the image processing model through the loss function, and obtain a trained image processing model after m iterations, wherein the step of screening n characteristic parameters from the N characteristic parameters needs to be re-executed in each iteration of the m times of iterative training of the image processing model, and m is a positive integer greater than 2; An image generation module configured to input a to-be-processed image into the trained image processing model, and obtain a processed image output by the trained image processing model.

9. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method in any one of claims 1-7.

10. An electronic device, comprising: Comprise: A memory having a computer program stored thereon; A processor configured to execute the computer program in the memory to implement the steps of the method in any one of claims 1-7.

Images