Image repair method and device

Abstract

The embodiment of the present invention provides a method and device for repairing an image, including: obtaining a sample image set, wherein the sample image set includes N sample images, and the sample image is an image obtained by an image acquisition device shooting a target object; performing S outer loop processing on the sample image set, each outer loop processing includes t inner loop processing, and each inner loop processing obtains an estimated original image of a randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing, wherein S and t are integers, S is less than N, and i is an integer less than S; and determining the estimated original image obtained after the S-th outer loop processing as the target image after the sample image is repaired. Through the present invention, the problem of low efficiency of image repair in the related art is solved, thereby achieving the effect of improving the efficiency of image repair.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing, and in particular, to a method and device for repairing an image. Background Art

[0002] Image is a very common information carrier, which can convey information and ideas visually. In the process of image acquisition, transmission and storage, the image may be affected by blur and noise due to various reasons, which reduces the image quality and clarity, and affects the image visualization and recognition accuracy. For example, in the fields of medicine and astronomy, the images collected by imaging equipment are often easily affected by blur and noise due to hardware equipment limitations, which reduces the image quality and clarity, resulting in inaccurate diagnosis results or unclear image observation data.

[0003] Image restoration is to restore the texture and details of a degraded image affected by blur and noise as much as possible to obtain a clear, high-quality image.

[0004] The image observation model affected by blur and noise (such as Gaussian noise) can usually be expressed as:

[0005]

[0006] in, is the original image, is the observed image affected by blur and noise, is the fuzzy operator, is noise. Due to the existence of noise, the problem is ill-posed. Directly solving the inverse problem is not feasible. For this inverse problem, the regularization method is one of the effective methods to deal with this problem, but the regularization method in the prior art requires gradient calculation for each sample. When the total number of samples is large, the calculation of the gradient in each iteration consumes more resources and is inefficient.

[0007] There is currently no effective solution to the above problems. Summary of the invention

[0008] The embodiments of the present invention provide a method and device for repairing an image, so as to at least solve the problem of low efficiency of image repair in the related art.

[0009] According to an embodiment of the present invention, a method for repairing an image is provided, comprising: obtaining a sample image set, wherein the sample image set includes N sample images, and the N sample images are images obtained by an image acquisition device shooting a target object; performing an outer loop process S times on the sample image set, each outer loop process includes t inner loop processes, each inner loop process in the i-th outer loop process randomly selects a sample image from the sample image set, and each inner loop process obtains the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop process. The estimated original image of the sample image, t times of the inner loop processing in the i-th outer loop processing obtain t estimated original images, the i-th estimated original image obtained by the i-th outer loop processing is obtained according to the t estimated original images, and in the case of the existence of the i+1-th outer loop processing, the i+1-th outer loop processing obtains the i+1-th full gradient according to the i-th estimated original image, wherein S and t are integers, S is less than N, and i is an integer less than S; the estimated original image obtained after the S-th outer loop processing is determined as the target image after the sample image is repaired.

[0010] In an exemplary embodiment, each inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set, and each inner loop processing obtains an estimated original image of the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing, including: the k-th inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set as the k-th selected sample image, wherein k is an integer less than t; according to the k-th selected sample image, the i-th full gradient obtained by the i-th outer loop processing, the i-1-th estimated original image obtained by the i-1-th outer loop processing, and the k-1-th estimated original image obtained by the k-1-th inner loop processing in the i-th outer loop processing, the k-th variance reduction gradient of the k-th inner loop processing in the i-th outer loop processing is obtained; and the estimated original image of the k-th selected sample image is obtained according to the k-th variance reduction gradient and the i-th full gradient.

[0011] In an exemplary embodiment, obtaining a kth variance reduction gradient according to the kth selected sample image, the ith full gradient obtained by the ith outer loop processing, the i-1th estimated original image obtained by the i-1th outer loop processing, and the k-1th estimated original image obtained by the k-1th inner loop processing in the i-1th outer loop processing includes:

[0012] The k-th variance reduction gradient is obtained by the following formula:

[0013]

[0014] in, is the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the estimated original image obtained by the outer loop processing for the i-1th time, , is the preset initial value, is the full gradient obtained by the outer loop processing for the i-th time, is the gradient operation, is the loss function operation, is the kth variance reducing gradient.

[0015] In an exemplary embodiment, obtaining the estimated original image of the sample image selected for the kth time according to the kth variance reduction gradient and the ith full gradient includes: obtaining the kth function value obtained by the kth inner loop processing in the i-th outer loop processing; obtaining the estimated original image of the sample image selected for the kth time by the following formula:

[0016]

[0017] in, is the estimated original image of the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the sample image selected for the kth time, is the default value. is the transposed matrix of the preset matrix, It is the kth function value obtained by the kth inner loop processing in the ith outer loop processing.

[0018] In an exemplary embodiment, the kth inner loop processing in the i-th outer loop processing further includes: obtaining the kth function value obtained by the kth inner loop processing in the i-th outer loop processing by the following formula:

[0019]

[0020]

[0021] in, is the kth function value obtained by the kth inner loop processing in the i-th outer loop processing, is the function to find the maximum value, , Y is the preset value range, is the default value. is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the two-norm operation, is the loss function operation, is a sample image randomly selected from the sample image set by the k-1th inner loop in the i-th outer loop processing, is the inner product operation, is the preset matrix.

[0022] In an exemplary embodiment, obtaining the i-th estimated original image obtained by the i-th outer loop processing based on t estimated original images includes: determining the mean of the t estimated original images as the i-th estimated original image obtained by the i-th outer loop processing.

[0023] In an exemplary embodiment, in the case where there is an i+1th outer loop processing, the i+1th outer loop processing obtains the i+1th full gradient according to the i-th estimated original image, including:

[0024] The i+1th full gradient is obtained by the following formula:

[0025]

[0026] in, is the i+1th full gradient, is the gradient operation, is the loss function operation, , is the c+1th sample image among the N sample images, is the estimated original image obtained by the i-th outer loop processing.

[0027] According to another embodiment of the present invention, there is provided an apparatus for repairing an image, comprising: an acquisition module, for acquiring a sample image set, wherein the sample image set comprises N sample images, and the sample images are images obtained by an image acquisition device photographing a target object; a processing module, for performing S times of outer loop processing on the sample image set, each of the outer loop processing comprises t times of inner loop processing, each of the inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set, and each of the inner loop processing obtains a randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing. The estimated original image of the sample image is taken, t inner loop processes in the i-th outer loop process obtain t estimated original images, the i-th estimated original image obtained by the i-th outer loop process is obtained according to the t estimated original images, and in the case of the existence of the i+1-th outer loop process, the i+1-th outer loop process obtains the i+1-th full gradient according to the i-th estimated original image, wherein S and t are integers, S is less than N, and i is an integer less than S; a determination module is used to determine the estimated original image obtained after the S-th outer loop process is completed as the target image after the sample image is repaired.

[0028] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, wherein the computer program implements the steps of any of the above methods when executed by a processor.

[0029] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0030] According to yet another embodiment of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0031] Through the present invention, the sample image is processed by two layers of inner and outer loops to obtain an estimated original image, thereby determining the repaired image. Since the number of times S of the outer loop is less than the number N of sample images, the calculation cost of the full gradient is reduced, and the complexity of the loop processing is reduced, so that the real image can be repaired faster. In addition, in the inner loop, the noise variance between the random gradient and the full gradient is reduced by reducing the gradient through variance, making it possible to use a relatively large constant step size to accelerate convergence, thereby reducing the iteration complexity. Therefore, the problem of low efficiency of image repair in the related art can be solved, and the effect of improving the efficiency of image repair can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 is a hardware structure block diagram of a mobile terminal according to a method for repairing an image according to an embodiment of the present invention;

[0033] Figure 2 is a flowchart of a method for repairing an image according to an embodiment of the present invention;

[0034] Figure 3 is a cyclic processing flow of image restoration according to an embodiment of the present invention;

[0035] Figure 4 is a cyclic processing flow of random noise gradient according to an embodiment of the present invention;

[0036] Figure 5 4 is a structural block diagram of an apparatus for repairing an image according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0038] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0039] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for repairing an image according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0040] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the method for repairing an image in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0041] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0042] In this embodiment, a method running on the above mobile terminal or network architecture is provided. Figure 2 is a flowchart of a method for repairing an image according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0043] Step S202, obtaining a sample image set, wherein the sample image set includes N sample images, and the sample images are images obtained by photographing the target object with an image acquisition device;

[0044] The above-mentioned sample images may be images obtained by photographing the target object in different environments. Affected by the shooting environment, there are noise and blur in the sample images. The target object may be any photographed object, for example, images of planets taken during astronomical observations; or images of underwater creatures taken during scientific research, etc. The above-mentioned N sample images may be images of the target object taken at different times, or images of the target object taken at different angles. Affected by the shooting environment, the collected sample images are noisy and blurry, and the clarity of the sample images is low (the above-mentioned degraded images may be images with low clarity). The purpose of this application is to repair sample images affected by the environment to obtain target images with higher clarity.

[0045] Step S204, performing S times of outer loop processing on the sample image set, each of the outer loop processing includes t times of inner loop processing, each inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set, each inner loop processing obtains an estimated original image of the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing, t times of inner loop processing in the i-th outer loop processing obtain t estimated original images, and the i-th estimated original image obtained by the i-th outer loop processing is obtained according to the t estimated original images, and in the case of the existence of the i+1-th outer loop processing, the i+1-th outer loop processing obtains the i+1-th full gradient according to the i-th estimated original image, wherein S and t are integers, S is less than t, S is less than N, and i is less than or equal to S;

[0046] The S times of outer loop processing can be any number less than N, such as 2, 3, 4, etc.; the t times of inner loop processing can be any number. By adding inner loop processing to calculate random gradient noise, the number of outer loop processing is greatly reduced, so that only a few outer loop processing times need to calculate the full gradient, which reduces the amount of calculation during image restoration and improves the efficiency of image restoration.

[0047] Taking the i-th outer loop processing as an example (i is an integer less than S), the i-th full gradient is calculated by the following formula:

[0048]

[0049] Where N is the number of sample images, is the c+1th image among N sample images, It is a gradient operation, that is, calculating the partial derivatives of the loss function value in the direction of each parameter. It is the loss function operation, such as LogisticRegression (logistic regression), Lasso loss, cross entropy loss, etc. It is the estimated original image obtained by the outer loop processing for the i-1th time.

[0050] Optionally, each inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set, and each inner loop processing obtains an estimated original image of the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing, including: the k-th inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set as the k-th selected sample image, wherein k is an integer and k is less than or equal to t, j is an integer and j is less than or equal to N; according to the k-th selected sample image, the i-th full gradient obtained by the i-th outer loop processing, the i-1-th estimated original image obtained by the i-1-th outer loop processing, and the k-1-th estimated original image obtained by the k-1-th inner loop processing in the i-th outer loop processing, a k-th variance reduction gradient is obtained; according to the k-th variance reduction gradient and the i-th full gradient, the estimated original image of the k-th selected sample image is obtained.

[0051] Each outer loop processing includes t inner loop processings, and the i-th outer loop processing includes t inner loop processings:

[0052] Taking the kth inner loop processing as an example, a sample image is randomly selected from the sample image set. As the sample image selected for the kth time, the kth variance reduction gradient of the kth inner loop process in the i-th outer loop process is obtained by the following formula:

[0053]

[0054] in, is the sample image selected for the kth time, is the k-1th estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the i-1th estimated original image obtained by the i-1th outer loop processing, is the i-th full gradient obtained by the i-th outer loop processing, is the gradient operation, is the loss function operation, is the kth variance reducing gradient.

[0055] The random noise gradient is constrained by calculating the variance reduction gradient, so that the noise gradient approaches the full gradient, reducing the noise variance between the random gradient and the full gradient, ensuring the learning rate of the model, and ensuring that the convergence speed of the model output to the optimal solution of the model will not decrease due to the reduction of the learning rate.

[0056] Optionally, obtaining the estimated original image of the sample image selected for the kth time according to the kth variance reduction gradient and the i-th full gradient includes: obtaining the kth function value obtained by the kth inner loop processing in the i-th outer loop processing; and obtaining the estimated original image of the sample image selected for the kth time by the following formula:

[0057]

[0058] in, is the estimated original image of the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the sample image selected for the kth time, is the default value. is the transposed matrix of the preset matrix, It is the kth function value obtained by the kth inner loop processing in the ith outer loop processing.

[0059] The kth inner loop processing in the i-th outer loop processing further includes:

[0060] The kth function value obtained by the kth inner loop processing in the i-th outer loop processing is obtained by the following formula:

[0061]

[0062]

[0063] in, is the kth function value obtained by the kth inner loop processing in the i-th outer loop processing, is the function to find the maximum value, , Y is the preset value range, is the default value. is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the two-norm operation, is the loss function operation, is a sample image randomly selected from the sample image set by the k-1th inner loop in the i-th outer loop processing, is the inner product operation, is the preset matrix, It is the preset initial value.

[0064] After the t inner loops of the ith outer loop are completed, t estimated original images are obtained. The average of the t estimated original images is determined as the ith estimated original image obtained by the ith outer loop.

[0065] Specifically, the i-th estimated original image can be obtained by the following formula:

[0066]

[0067] And update:

[0068] in, is the function value of the tth inner loop processing in the i-th outer loop processing, , It is the preset initial value.

[0069] Step S206: determine the estimated original image obtained after the S-th outer loop processing is completed as the repaired image obtained by repairing the damaged original image.

[0070] Optionally, the executor of the above steps may be a background processor, or other devices with similar processing capabilities, or a machine that integrates at least an image acquisition device and a data processing device, wherein the image acquisition device may include a graphics acquisition module such as a camera, and the data processing device may include a computer, a mobile phone or other terminal, but is not limited thereto.

[0071] Through the above steps, the variance reduction technology is introduced to calculate the noise variance between the random noise gradient and the full gradient, and the random noise gradient is approximated to the full gradient, thereby reducing the noise variance between the random gradient and the full gradient, so that a relatively large constant step size can be used to accelerate convergence, reduce the complexity of the loop processing, and reach or approach the optimal solution through fewer loops, so as to repair the real image faster. The problem of low efficiency of image repair in related technologies is solved and the efficiency of image repair is improved.

[0072] As an optional implementation, in the case where there is an i+1th outer loop processing, the i+1th outer loop processing obtains the i+1th full gradient according to the i-th estimated original image, including: obtaining the i+1th full gradient by the following formula:

[0073]

[0074] in, is the i+1th full gradient, is the gradient operation, is the loss function operation, , is the c+1th sample image among the N sample images, is the estimated original image obtained by the i-th outer loop processing.

[0075] As an optional implementation, Figure 3 is a cyclic processing flow of image restoration according to an embodiment of the present invention, such as Figure 3 As shown, the entire loop process can be divided into S outer loop processing and t inner loop processing. The specific process is as follows: First, initialize the variable , It is a preset initial value, which can be set according to actual conditions, such as 0, 1, etc.

[0076] In each outer loop processing (the total number of outer loop processing S is very small), first calculate the reference point (estimate the original image ) at the full gradient (Since S is very small, the full gradient is calculated in the outer loop processing The cost is not high), and initialize As the initial value of the inner loop processing.

[0077] Then in the inner loop, a random gradient with reduced variance is calculated , and then use To update , and calculate the reference point after the inner loop processing (estimate the original image ) is used to assign initial values ​​to the inner loop iterations in the next outer loop processing.

[0078] Optionally, for fields such as medical imaging, photographic restoration, security monitoring and lossless compression, medical images generally have noise and artifacts, such as CT and MRI images. Image restoration can be used to remove noise and artifacts and improve image quality, thereby assisting doctors in making more accurate diagnoses; old photos generally may have problems such as damage, stains, creases, etc. Image restoration can be used to repair these problems and make old photos look clearer and more realistic; images taken by security surveillance cameras are often affected by factors such as light and weather, resulting in a decrease in image quality. Image restoration can improve image quality and improve monitoring effects; in terms of lossless compression, image restoration allows lossless compression of images, reducing the size of image files without losing quality. Among them, the image observation model affected by blur and noise (such as Gaussian noise) can usually be expressed as:

[0079]

[0080] In the formula, is the original image, To observe the image, is the fuzzy operator, For noise.

[0081] Based on the prior information of the image, the image can be repaired to obtain the following image:

[0082]

[0083] In the formula, , The diameter is The convex compact set of is a convex regularization function, is a penalty matrix (not necessarily diagonal) that specifies the image The desired structured sparsity pattern in Indicates that when in the sample data set Predict the original image The smooth convex loss function when , the corresponding expectation is expressed as .

[0084] In the above formula, the regularization function can be expressed as:

[0085]

[0086] In the formula, Is the diameter Convex compact set, assuming the regularized norm is The norm or nuclear norm can be expressed as:

[0087]

[0088] The repaired image can be expressed as:

[0089]

[0090] And the solution is obtained by looping through the following two formulas:

[0091]

[0092]

[0093] At the same time, in order to reduce the amount of calculation, an inner loop is added in each loop processing to convert part of the full gradient The calculation of is converted into the calculation of random noise gradient, where the calculation process of random noise gradient can be as follows Figure 4 As shown, the specific process is:

[0094] First, for the dual variable Find the maximum value, and then follow Direction in steps Perform a stochastic gradient descent cycle. is the weighting coefficient, and the output of the loop processing is the value of each iteration The non-uniform average.

[0095] At the same time, in order to avoid the problem that the step size decays with the number of cycles, resulting in slower convergence and reduced image restoration efficiency, the variance reduction technology is used to reduce the noise variance between the random gradient and the full gradient, thereby accelerating convergence and improving the efficiency of image restoration. The specific process is as described above. Figure 3 shown.

[0096] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0097] In this embodiment, a device for repairing an image is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0098] Figure 5 is a structural block diagram of an apparatus for repairing an image according to an embodiment of the present invention, such as Figure 5As shown, the device includes: an acquisition module 502, used to acquire a sample image set, wherein the sample image set includes N sample images, and the sample image is an image obtained by an image acquisition device shooting a target object; a processing module 504, used to perform S times of outer loop processing on the sample image set, each of the outer loop processing includes t times of inner loop processing, each of the inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set, and each of the inner loop processing obtains the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing an estimated original image, t times of the inner loop processing in the i-th outer loop processing obtain t estimated original images, the i-th estimated original image obtained by the i-th outer loop processing is obtained according to the t estimated original images, and in the case of the existence of the i+1-th outer loop processing, the i+1-th outer loop processing obtains the i+1-th full gradient according to the i-th estimated original image, wherein S and t are integers, S is less than N, and i is an integer less than S; a determination module 506 is used to determine the estimated original image obtained after the S-th outer loop processing is completed as the target image after the sample image is repaired.

[0099] In an exemplary embodiment, the device is also used for randomly selecting a sample image from the sample image set as the kth selected sample image in the kth inner loop processing in the i-th outer loop processing, wherein k is an integer less than t; obtaining the kth variance reduction gradient of the kth inner loop processing in the i-th outer loop processing according to the kth selected sample image, the i-th full gradient obtained by the i-th outer loop processing, the i-1th estimated original image obtained by the i-1th outer loop processing, and the k-1th estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing; and obtaining the estimated original image of the kth selected sample image according to the kth variance reduction gradient and the i-th full gradient.

[0100] In an exemplary embodiment, the apparatus is further used to obtain the kth variance reduction gradient by the following formula:

[0101]

[0102] in, is the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the estimated original image obtained by the outer loop processing for the i-1th time, , is the preset initial value, is the full gradient obtained by the outer loop processing for the i-th time, is the gradient operation, is the loss function operation, is the kth variance reduction gradient

[0103] In an exemplary embodiment, the apparatus is further configured to obtain a k-th function value obtained by the k-th inner loop processing in the i-th outer loop processing;

[0104] The estimated original image of the sample image selected for the kth time is obtained by the following formula:

[0105]

[0106] in, is the estimated original image of the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the sample image selected for the kth time, is the default value. is the transposed matrix of the preset matrix, It is the kth function value obtained by the kth inner loop processing in the ith outer loop processing.

[0107] In an exemplary embodiment, the apparatus is further configured to obtain the kth function value obtained by the kth inner loop processing in the i-th outer loop processing by the following formula:

[0108]

[0109]

[0110] in, is the kth function value obtained by the kth inner loop processing in the i-th outer loop processing, is the function to find the maximum value, , Y is the preset value range, is the default value. is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the two-norm operation, is the loss function operation, is a sample image randomly selected from the sample image set by the k-1th inner loop process in the i-th outer loop process, is the inner product operation, is the preset matrix.

[0111] In an exemplary embodiment, the apparatus is further configured to determine a mean value of t estimated original images as the i-th estimated original image obtained by the i-th outer loop processing.

[0112] In an exemplary embodiment, the apparatus is further configured to obtain the (i+1)th full gradient by the following formula:

[0113]

[0114] in, is the i+1th full gradient, is the gradient operation, is the loss function operation, , is the c+1th sample image among the N sample images, is the estimated original image obtained by the i-th outer loop processing.

[0115] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0116] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above methods are implemented.

[0117] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0118] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0119] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0120] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method described in each embodiment of the present application when executed by a processor.

[0121] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0122] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0123] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for repairing an image, characterized in that: include: Acquire a sample image set, wherein the sample image set includes N sample images, and the sample images are images obtained by photographing a target object with an image acquisition device; Performing S times of outer loop processing on the sample image set, each outer loop processing includes t times of inner loop processing; Wherein, each inner loop process in the i-th outer loop process randomly selects a sample image from the sample image set; Each time the inner loop process obtains an estimated original image of the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop process; The t inner loop processes in the ith outer loop process obtain t estimated original images, and the ith estimated original image obtained by the ith outer loop process is obtained according to the t estimated original images; In the case where there is an i+1th outer loop processing, the i+1th outer loop processing obtains an i+1th full gradient according to the i-th estimated original image, wherein S and t are integers, S is less than N, and i is an integer less than S; Determine the estimated original image obtained after the S-th outer loop processing as the target image after the sample image is repaired; Wherein, each inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set, and each inner loop processing obtains an estimated original image of the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop processing, including: The kth inner loop processing in the i-th outer loop processing randomly selects a sample image from the sample image set as the kth selected sample image, where k is an integer less than t; Obtaining a k-th variance reduction gradient of the k-th inner loop processing in the i-th outer loop processing according to the k-th selected sample image, the i-th full gradient obtained by the i-th outer loop processing, the i-1-th estimated original image obtained by the i-1-th outer loop processing, and the k-1-th estimated original image obtained by the k-1-th inner loop processing in the i-th outer loop processing; Obtaining an estimated original image of the sample image selected for the kth time according to the kth variance reduction gradient and the ith full gradient; Among them, the k-th variance reduction gradient is obtained by the following formula: ； in, is the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the estimated original image obtained by the outer loop processing for the i-1th time, , is the preset initial value, is the full gradient obtained by the outer loop processing for the i-th time, is the gradient operation, is the loss function operation, is the kth variance reducing gradient.

2. The method according to claim 1, characterized in that Obtaining an estimated original image of the k-th selected sample image according to the k-th variance reduction gradient and the i-th full gradient, comprising: Obtaining a k-th function value obtained by the k-th inner loop processing in the i-th outer loop processing; The estimated original image of the sample image selected for the kth time is obtained by the following formula: ； in, is the estimated original image of the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the sample image selected for the kth time, is the default value. is the transposed matrix of the preset matrix, It is the kth function value obtained by the kth inner loop processing in the ith outer loop processing.

3. The method according to claim 2, characterized in that The kth inner loop processing in the i-th outer loop processing further includes: The kth function value obtained by the kth inner loop processing in the i-th outer loop processing is obtained by the following formula: ； ； in, is the kth function value obtained by the kth inner loop processing in the i-th outer loop processing, is the function to find the maximum value, , Y is the preset value range, is the default value. is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the two-norm operation, is the loss function operation, is a sample image randomly selected from the sample image set by the k-1th inner loop process in the i-th outer loop process, is the inner product operation, is the preset matrix.

4. The method according to claim 1, characterized in that: Obtaining the i-th estimated original image obtained by the i-th outer loop processing according to the t estimated original images, comprising: The average value of the t estimated original images is determined as the i-th estimated original image obtained by the i-th outer loop processing.

5. The method according to claim 1, characterized in that In the case where there is an i+1th outer loop processing, the i+1th outer loop processing obtains an i+1th full gradient according to the i-th estimated original image, including: The i+1th full gradient is obtained by the following formula: ； in, is the i+1th full gradient, is the gradient operation, is the loss function operation, , is the c+1th sample image among the N sample images, is the estimated original image obtained by the i-th outer loop processing.

6. A device for repairing an image, characterized in that: include: An acquisition module, used for acquiring a sample image set, wherein the sample image set includes N sample images, and the N sample images are damaged images in which the original images are damaged to different degrees; A processing module, used for performing S times of outer loop processing on the sample image set, each outer loop processing includes t times of inner loop processing; Wherein, each inner loop process in the i-th outer loop process randomly selects a sample image from the sample image set; Each time the inner loop process obtains an estimated original image of the randomly selected sample image according to the i-th full gradient obtained by the i-th outer loop process; The t inner loop processes in the ith outer loop process obtain t estimated original images, and the ith estimated original image obtained by the ith outer loop process is obtained according to the t estimated original images; In the case where there is an i+1th outer loop processing, the i+1th outer loop processing obtains an i+1th full gradient according to the i-th estimated original image, wherein S and t are integers, S is less than N, and i is an integer less than S; A determination module, used for determining the estimated original image obtained after the S-th outer loop processing is completed as a repaired image obtained by repairing the damaged original image; The device is also used for randomly selecting a sample image from the sample image set as the kth selected sample image in the kth inner loop processing in the i-th outer loop processing, where k is an integer less than t; Obtaining a k-th variance reduction gradient of the k-th inner loop processing in the i-th outer loop processing according to the k-th selected sample image, the i-th full gradient obtained by the i-th outer loop processing, the i-1-th estimated original image obtained by the i-1-th outer loop processing, and the k-1-th estimated original image obtained by the k-1-th inner loop processing in the i-th outer loop processing; Obtaining an estimated original image of the sample image selected for the kth time according to the kth variance reduction gradient and the ith full gradient; Among them, the k-th variance reduction gradient is obtained by the following formula: ； in, is the sample image selected for the kth time, is the estimated original image obtained by the k-1th inner loop processing in the i-th outer loop processing, is the estimated original image obtained by the outer loop processing for the i-1th time, , is the preset initial value, is the full gradient obtained by the outer loop processing for the i-th time, is the gradient operation, is the loss function operation, is the kth variance reducing gradient.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 5 when executed by a processor.

8. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 5.

Images