Image processing method and device, electronic equipment and storage medium

By predicting the target quality standards in advance in the image reconstruction model and monitoring the output of the intermediate layer, the problem of large and time-consuming calculation resources of the image processing model in the prior art is solved, and a faster image reconstruction processing is achieved.

CN118945483AActive Publication Date: 2024-11-12BEIJING XIAOMI MOBILE SOFTWARE CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202310520182.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-09
Publication Date
2024-11-12
Estimated Expiration
2043-05-09

AI Technical Summary

Technical Problem

When used in real-life image reconstruction models, the computing resources are consumed and the inference is time-consuming, making it difficult to quickly and accurately perform image processing.

Method used

By predicting the target quality standards of image reconstruction in advance, and monitoring the real-time intermediate layer output during image reconstruction model processing, the processing is completed when the output meets the target quality standards, reducing computing resource consumption and improving processing speed.

Benefits of technology

While ensuring image quality, the computing resource consumption of the image processing model is reduced, the processing speed of the image processing model is improved, and faster image reconstruction processing is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945483A_ABST
    Figure CN118945483A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method and device, electronic equipment and a storage medium, and relates to the field of image processing, and the method mainly comprises the steps: obtaining a to-be-processed initial image; performing reasoning training on the initial image to obtain an image reconstruction stopping condition corresponding to the initial image; and performing image reconstruction processing on the initial image based on a neural network model, and when a middle layer reconstruction image of the neural network model meets the image reconstruction stopping condition, stopping performing image reconstruction processing on the initial image to obtain a target reconstruction image corresponding to the initial image. According to the method, a target quality standard of image reconstruction is predicted in advance, real-time middle layer output monitoring is carried out on a reconstruction model, and when the target quality standard is met, reconstruction is ended, and the target quality standard is taken as final output. Compared with the conventional image reconstruction with a complete process, the method has the advantages that under the condition of ensuring the quality, the calculation resource consumption of the model is reduced, and the processing speed of the model is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to an image processing method, device, electronic device and storage medium. Background Art

[0002] In the field of image processing, deep learning has been widely used in image processing tasks such as image reconstruction, and has shown excellent performance in terms of accuracy. However, since the existing image reconstruction models are usually complex in structure, when the models are applied to real-world deployments, they inevitably face the problems of high consumption of computing resources and long inference time. Therefore, how to better enable the existing image processing models to process images quickly and accurately is a problem that needs to be solved at present. Summary of the invention

[0003] The present disclosure provides an image processing method, device, electronic device and storage medium to solve the problems in the related art and improve the training calculation speed of the image reconstruction model.

[0004] The first aspect of the present disclosure proposes an image processing method, which includes: acquiring an initial image to be processed; performing inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image; performing image reconstruction processing on the initial image based on a neural network model, and when the intermediate layer reconstructed image of the neural network model meets the stop image reconstruction condition, stopping the image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image.

[0005] In some embodiments of the present disclosure, performing inference training on the initial image to obtain a stopping image reconstruction condition for reconstructing the initial image includes: performing inference training on the initial image according to an image quality control discrimination neural network model to obtain a stopping image reconstruction condition corresponding to the initial image.

[0006] In some embodiments of the present disclosure, image reconstruction is performed on the initial image based on a neural network model. When the reconstructed image of the intermediate layer of the neural network model meets the condition for stopping image reconstruction, the image reconstruction processing of the initial image is stopped to obtain a target reconstructed image corresponding to the initial image, which includes: reconstructing the initial image using the image reconstruction processing neural network model in units of layers of the neural network model to obtain the reconstructed image of the intermediate layer of the neural network; judging whether the reconstructed image of the intermediate layer of the neural network meets the condition for stopping image reconstruction; if the reconstructed image of the intermediate layer of the neural network meets the condition for stopping image reconstruction, stopping the image reconstruction processing of other layers of the neural network model to perform image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image; if the reconstructed image of the intermediate layer of the neural network does not meet the condition for stopping image reconstruction, continuing to perform image reconstruction processing on the reconstructed image of the intermediate layer of the neural network until the reconstructed image of the intermediate layer of the neural network meets the condition for stopping image reconstruction.

[0007] In some embodiments of the present disclosure, the condition for stopping image reconstruction is the number of image reconstruction times, and the judging whether the image reconstructed by the neural network intermediate layer satisfies the condition for stopping image reconstruction includes: determining whether the number of reconstruction processes performed on the initial image has reached the number of image reconstruction times; if the number of image reconstruction times has been reached, determining that the image reconstructed by the neural network intermediate layer satisfies the condition for stopping image reconstruction; if the number of image reconstruction times has not been reached, determining that the image reconstructed by the neural network intermediate layer does not satisfy the condition for stopping image reconstruction.

[0008] In some embodiments of the present disclosure, the condition for stopping image reconstruction is a target reconstructed image quality standard, and the judging whether the image reconstructed by the neural network intermediate layer satisfies the condition for stopping image reconstruction comprises: comparing the quality of the image reconstructed by the neural network intermediate layer with the target reconstructed image quality standard; if the quality of the image reconstructed by the neural network intermediate layer satisfies the target reconstructed image quality standard, determining that the image reconstructed by the neural network intermediate layer satisfies the condition for stopping image reconstruction; if the quality of the image reconstructed by the neural network intermediate layer does not satisfy the target reconstructed image quality standard, determining that the image reconstructed by the neural network intermediate layer does not satisfy the condition for stopping image reconstruction.

[0009] In some embodiments of the present disclosure, it also includes: constructing the image quality control discrimination neural network model based on the image reconstruction processing neural network model.

[0010] The second aspect of the present disclosure proposes an image processing device, which includes: an acquisition unit, which acquires an initial image to be processed; a processing unit, which is used to perform inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image; and a judgment unit, which is used to perform image reconstruction processing on the initial image based on a neural network model, and when the intermediate layer reconstructed image of the neural network model meets the stop image reconstruction condition, stop the image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image.

[0011] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect embodiment of the present disclosure.

[0012] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in the first aspect embodiment of the present disclosure.

[0013] A fifth aspect embodiment of the present disclosure provides a computer program product, including a computer program, which executes the method described in the first aspect embodiment of the present disclosure when a processor is used to execute the computer program.

[0014] The image processing method, device, electronic device and storage medium provided by the present disclosure include: obtaining an initial image to be processed; performing inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image; performing image reconstruction processing on the initial image based on a neural network model, and when the intermediate layer reconstructed image of the neural network model meets the stop image reconstruction condition, stopping the image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image. When the method performs image reconstruction processing, the target quality standard of image reconstruction is predicted in advance, and when the image reconstruction model is processed, the real-time intermediate layer output of the model is monitored, and when the real-time intermediate layer output meets the target quality standard, the reconstruction processing is terminated and the output is used as the final output of the image. Compared with the previous image reconstruction processing of a complete process, this method reduces the computing resource consumption of the image processing model and improves the processing speed of the image processing model while ensuring quality.

[0015] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations of the present disclosure.

[0017] Figure 1 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0018] Figure 2 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0019] Figure 3 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0020] Figure 4 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0021] Figure 5 A flowchart of an image processing method provided by an embodiment of the present disclosure;

[0022] Figure 6 A flowchart of an image reconstruction method provided by an embodiment of the present disclosure;

[0023] Figure 7 A schematic diagram of the structure of an image discrimination model provided by an embodiment of the present disclosure;

[0024] Figure 8 A schematic diagram of the structure of an image reconstruction neural network model provided by an embodiment of the present disclosure;

[0025] Fig. 9 A block diagram of an image processing device provided by an embodiment of the present disclosure;

[0026] Fig.10 A block diagram of an image processing device provided by an embodiment of the present disclosure;

[0027] Fig.11 A block diagram of an image processing device provided by an embodiment of the present disclosure;

[0028] Fig.12 A block diagram of a composition of an image processing device provided in a disclosed embodiment;

[0029] Fig.13 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0031] In the process of image formation and transmission, images are inevitably disturbed by noise. The noise in the image is often intertwined with the signal, which will make the details of the image itself, such as boundary contours and lines, blurred. At the same time, due to hardware reasons such as equipment or sensors, the images we get are often low-resolution images, but in a large number of electronic image application fields, people often expect to get high-resolution images. Therefore, the research on image reconstruction denoising and image super-resolution is of great significance. In the field of image processing, deep learning has been widely used in image processing tasks such as image reconstruction, and has shown excellent performance in accuracy. However, since the existing image reconstruction models are usually complex in structure, when the models are applied to real deployment, they inevitably face the problems of high computing resource consumption and long reasoning time. Therefore, how to better enable the existing image processing models to quickly and accurately process images is a problem that needs to be solved at present.

[0032] In order to solve the problems existing in the related art, this method predicts the target quality standard of image reconstruction in advance when processing image reconstruction, and monitors the real-time intermediate layer output of the model when processing the image reconstruction model. When the real-time intermediate layer output meets the target quality standard, the reconstruction process is terminated and the output is used as the final output of the image. Compared with the previous image reconstruction process of the complete process, this method reduces the computing resource consumption of the image processing model and improves the processing speed of the image processing model while ensuring the quality.

[0033] Figure 1 Flow chart of an image processing method provided by an embodiment of the present disclosure. Figure 1 As shown, steps 101-103 are included.

[0034] Step 101: Obtain an initial image to be processed.

[0035] It should be noted here in the embodiments of the present disclosure that the initial image to be processed, that is, the image data of the image reconstruction task to be processed, may be low-noise image data, may be low-resolution image data, or may be other image data that requires image reconstruction. The specific image data type is not limited in the embodiments of the present disclosure.

[0036] Among them, image reconstruction mainly includes super-resolution (Image SuperResolution) and image denoising (Image Denoising) processing of the initial image to obtain the target image. Image super-resolution refers to the technology of using image processing methods to restore low-resolution (LR) images to high-resolution (HR) images through computers. High-resolution images mean that the images have high pixel density and can provide more image details. Image denoising refers to using image processing methods to reduce the noise of images with high noise through computers to obtain low-noise images.

[0037] Step 102: Perform inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image.

[0038] It should be noted here in the embodiments of the present disclosure that the inference training can be to train the input image data through a neural network model to obtain the image reconstruction conditions corresponding to the image data. The embodiments of the present disclosure do not limit the specific processing model or the method of obtaining the image reconstruction conditions.

[0039] The condition for stopping image reconstruction, that is, the quality standard of image reconstruction corresponding to the image data, may include the number of image reconstruction times, or may include commonly used evaluation indicators of image reconstruction as quality standards, such as structural similarity SSIM (Structural SIMilarity), or peak signal-to-noise ratio PSNR (Peak Signal-to-Noise Ratio), or other commonly used evaluation indicators of image reconstruction. The specific condition for stopping image reconstruction is not limited by the present disclosure.

[0040] Step 103, performing image reconstruction processing on the initial image based on the neural network model, when the intermediate layer reconstructed image of the neural network model meets the stop image reconstruction condition, stop performing image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image.

[0041] It should be noted that the neural network model refers to a trained image reconstruction model for image processing of the initial image. The model type can be a convolutional neural network, a multi-layer perception, a graph neural network, other neural network models, or an integrated combination of one or more neural network models. The specific model type and the specific model network structure are not limited by the embodiments of the present disclosure. The middle layer of the neural network model refers to the model layer after the input layer of the neural network model, that is, the hidden layer of the neural network model. It can be a convolutional layer or a layer of other neural networks. The specific type of layer is not limited by the embodiments of the present disclosure.

[0042] In summary, the image processing method provided by the present disclosure includes: obtaining an initial image to be processed; performing inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image; performing image reconstruction processing on the initial image based on a neural network model, and when the intermediate layer reconstructed image of the neural network model meets the stop image reconstruction condition, stopping the image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image. When the method performs image reconstruction processing, the target quality standard of image reconstruction is predicted in advance, and when the image reconstruction model is processed, the real-time intermediate layer output of the model is monitored, and when the real-time intermediate layer output meets the target quality standard, the reconstruction processing is terminated and the output is used as the final output of the image. Compared with the previous image reconstruction processing of a complete process, this method reduces the computing resource consumption of the image processing model and improves the processing speed of the image processing model while ensuring quality.

[0043] Further, in some embodiments of the present disclosure, when performing the inference training on the initial image in step 102 to obtain the stop image reconstruction condition for image reconstruction of the initial image, it can be achieved by but not limited to the following method, specifically: Figure 2 As shown, the following steps are included:

[0044] Step 201 , performing inference training on the initial image according to the image quality control discrimination neural network model to obtain a stop image reconstruction condition corresponding to the initial image.

[0045] It should be noted here in the embodiments of the present disclosure that the image quality control discrimination neural network model is used to obtain the predicted stopping image reconstruction conditions. It can be a convolutional neural network, or a deep neural network with multiple convolutional neural layers, multi-layer perceptron layers and pooling layers, or other types of neural network models. The specific model type and model structure are not limited in the embodiments of the present disclosure.

[0046] In a possible implementation, the image quality control discrimination neural network model that judges the input information mainly includes a deep neural network of neural network modules such as convolutional neural network and multi-layer perceptron, and its final output is to determine whether the output of the intermediate layer for image reconstruction meets the target quality or the number of inferences for inference. Figure 7 As shown, it is a structural diagram of the image discrimination model of the present method. The structure of the judgment neural network M_J of the present embodiment is a deep neural network model composed of multiple neural network modules. The image quality control discrimination neural network model for judging the input information mainly includes a deep neural network of neural network modules such as a convolutional neural network, a pooling layer, and a multi-layer perceptron. Its final output is to determine whether the output of the image reconstruction processing neural network model meets the output quality or the number of inferences of the image reconstruction processing neural network model.

[0047] Among them, Convolutional Neural Networks (CNN) is a deep neural network with a convolutional structure. Convolutional neural networks contain a feature extractor consisting of a convolution layer and a subsampling layer. The feature extractor can be regarded as a filter, and the convolution process can be regarded as using a trainable filter to convolve an input image or a convolution feature plane (feature map). Convolutional neural networks can be composed of multiple convolutional neural units, and convolutional neurons can refer to x-th order neurons. s and intercept 1 as input operation unit, the form of the operation unit can be defined as:

[0048]

[0049] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s For X S , b is the bias term of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the convolutional neural network to convert the input signal in the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolution layer. The activation function can be a sigmoid, relu or tanh function. A neural network is a network formed by connecting many of the above-mentioned single neural units together, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the characteristics of the local receptive field. The local receptive field can be an area composed of several neural units.

[0050] A multilayer perceptron is a neural network consisting of fully connected layers with at least one hidden layer, and the output of each hidden layer is transformed by an activation function. The specific form of this operation unit can be defined as:

[0051]

[0052] Where φ represents the activation function, h represents the hidden variable, and O represents the output. In a deep neural network, more hidden layers allow the network to better depict complex situations in the real world. Theoretically, the more parameters a model has, the higher its complexity and the greater its "capacity", which means it can complete more complex learning tasks. Therefore, a multi-layer perceptron can have a performance capability that exceeds that of a single-layer perceptron.

[0053] In one possible implementation, the input x of the image quality control discriminant neural network model that determines the input information is an image, which is a matrix with channel dimensions, which mainly contains information such as pixel values ​​and image features. The pixel value of an image can be a red, green, and blue (RGB) color value, and the pixel value can be a long integer representing the color. For example, the pixel value is 256*Red+100*Green+76Blue, where Blue represents the blue component, Green represents the green component, and Red represents the red component. In each color component, the smaller the value, the lower the brightness, and the larger the value, the higher the brightness. For grayscale images, the pixel value is the grayscale value, which does not contain multiple channels such as RGB.

[0054] Further, in some embodiments of the present disclosure, when performing step 103 to reconstruct the initial image based on the neural network model, when the intermediate layer reconstructed image of the neural network model satisfies the stop image reconstruction condition, the image reconstruction processing of the initial image is stopped, and the target reconstructed image corresponding to the initial image is obtained, which can be achieved by but not limited to the following method, specifically as follows: Figure 3 As shown, the following steps are included:

[0055] Step 301, taking the layer of the neural network model as a unit, using the image reconstruction processing neural network model to reconstruct the initial image, and obtain the reconstructed image of the intermediate layer of the neural network.

[0056] It should be noted that the embodiments of the present disclosure refer to the basic units of layers when constructing a neural network. Other basic units of neural networks may also be used, and the present disclosure does not limit this. In addition, the intermediate layer may be the first layer or any of the multiple intermediate layers. When obtaining the output of the intermediate layer of the neural network, the output features of the intermediate layer may be extracted and distinguished layer by layer, or the output features of the intermediate layer may be extracted and distinguished by skipping layers. The specific method of obtaining the intermediate layer is not limited by the embodiments of the present disclosure.

[0057] Regarding the image reconstruction processing neural network model, in a possible implementation, the image reconstruction processing neural network model is a deep neural network with multiple convolutional neural layers, multi-layer perceptron layers, and pooling layers, the output of which is an image, and its main purpose is to reconstruct the input image x. Figure 8 As shown in the figure, it is a structural diagram of the image reconstruction neural network model of the present method. The image reconstruction processing neural network model is a deep neural network with multiple convolutional neural layers, multi-layer perceptron layers, and pooling layers, and its main purpose is to reconstruct the input image x.

[0058] Among them, image reconstruction mainly includes image super resolution and image denoising. Image super resolution refers to the technology of using image processing methods to restore low resolution (LR) images to high resolution (HR) images through computers. High resolution images mean that the images have high pixel density and can provide more image details. Image denoising refers to the technology of using image processing methods to reduce the noise of high-noise images through computers to obtain low-noise images.

[0059] Specifically, the image super-resolution method based on deep learning, that is, the image reconstruction from LR to HR is realized by using the deep neural network learning method. Among them, the image super-resolution method based on reconstruction can statistically estimate the high-resolution image with the maximum posterior probability through the frequency domain algorithm or the spatial domain algorithm. The image super-resolution method based on learning can include two stages: the training stage and the testing stage.

[0060] In the image reconstruction process of the image super-resolution task, in the training stage, the initial super-resolution model and training set are first established. The training set may include multiple low-resolution images and the high-resolution image corresponding to each low-resolution image. Through the low-resolution images in the training set and the high-resolution images corresponding to them, the correspondence between the high-resolution image and the low-resolution image is learned, and then the values ​​of the parameters in the initial super-resolution model are corrected to converge the error between the high-resolution image and the reconstructed image, and finally the newly-created super-resolution model obtained after training is determined. In the testing stage, the newly-created super-resolution model can be used to guide the super-resolution reconstruction of the image.

[0061] Among them, the method for obtaining image pairs in the training set (a low-resolution image and a high-resolution image corresponding to the low-resolution image form an image pair) can be to first obtain a high-resolution image, and then process the high-resolution image through a fuzzy function to obtain a corresponding low-resolution image, wherein the fuzzy function is irreversible.

[0062] Specifically, the image denoising method based on deep learning, that is, the image reconstruction from high-noise image to low-noise image using the deep neural network learning method. Among them, the image denoising method based on reconstruction can statistically estimate the low-noise image with the maximum posterior probability through the frequency domain algorithm or the spatial domain algorithm. The image denoising method based on learning can include two stages: the training stage and the testing stage.

[0063] In the image reconstruction process of image denoising, in the training stage, the initial denoising model and training set are first established. The training set may include multiple low-noise images and high-noise images corresponding to each low-noise image. Through the low-noise images in the training set and the high-noise images corresponding to them, the correspondence between the high-noise image and the low-noise image is learned, and then the values ​​of the parameters in the initial super-noise model are corrected to converge the error between the high-noise image and the reconstructed image, and finally the newly-created super-noise model obtained after training is determined. In the testing stage, the super-noise reconstruction of the image can be guided according to the newly-created super-noise model.

[0064] Among them, the method for obtaining an image pair in the training set (a low-noise image and a high-noise image corresponding to the low-noise image form an image pair) can be to first obtain a low-noise image, and then perform noise processing on the low-noise image to obtain a corresponding high-noise image, wherein the noise processing is irreversible.

[0065] In addition, it should be noted that, in a possible implementation, the training of the image quality control discrimination neural network model for judging the input information depends on the image reconstruction processing neural network model:

[0066] First, after inputting the initial image, the intermediate state variable h is obtained by reconstructing the i-th Block of the neural network model i , compare its PSNR and SSIM with the original image x;

[0067] Secondly, given a clean image I and a noisy image K of size m×n, the peak signal-to-noise ratio (PSNR) is defined as:

[0068]

[0069] in is the maximum possible pixel value of the image, and MSE is the mean square error of the two images, which is specifically defined as:

[0070]

[0071] SSIM (Structural SIMilarity) is used to compare the structural similarity of two images. It contains three comparison measures: luminance, contrast, and structure. Its specific definition is:

[0072]

[0073] where μ x is the mean of x, μ y is the mean of y, is the variance of x, is the variance of y, σ x σ y is the covariance of x and y, c1 and c2 are constants to avoid incomprehensible calculations. SSIM is:

[0074] SSIM(x,y)=[l(x,y) α c(x, y) β s(x, y) γ ] (Formula 6)

[0075] Setting α, β, and γ to 1 gives a simplified formula for SSIM:

[0076]

[0077] Finally, by calculating the intermediate variable h i The training data is obtained by the difference between the PSNR and SSIM of the original image x. We define the difference as g i , h of each layer of intermediate variables i The difference between the index and x has a corresponding gi By accumulating g at each layer i To obtain the training data of the discriminative neural network.

[0078] The image quality control discrimination neural network model that judges the input information obtains the PSNR and SSIM of the original image x and the image target quality, and uses the required difference g during inference r If it is determined that the i-th layer is required to achieve the target image quality, the image reconstruction processing neural network model outputs the image when reasoning to the i-th Block.

[0079] Step 302, determining whether the image reconstructed by the neural network intermediate layer satisfies the condition for stopping image reconstruction. If the image reconstructed by the neural network intermediate layer satisfies the condition for stopping image reconstruction, executing step 303; if the image reconstructed by the neural network intermediate layer does not satisfy the condition for stopping image reconstruction, executing step 304.

[0080] It should be noted here in the embodiments of the present disclosure that the judgment, when executed, can utilize the image quality control discrimination neural network model constructed in the present disclosure, or can utilize other external judgment methods, such as an external judgment code program for judgment. The specific place where the judgment is executed is not limited by the embodiments of the present disclosure.

[0081] Step 303: Stop other layers of the image reconstruction neural network model to perform image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image.

[0082] The embodiments of the present disclosure that need to be explained have been explained in the previous text and will not be repeated here. The embodiments of the present disclosure do not limit the specific stopping method.

[0083] Step 304, continue to perform image reconstruction processing on the neural network intermediate layer reconstructed image until the neural network intermediate layer reconstructed image meets the image reconstruction stopping condition.

[0084] The embodiments of the present disclosure that need to be explained have been explained in the previous text and will not be repeated here. The embodiments of the present disclosure do not limit the specific implementation method.

[0085] Further, in some embodiments of the present disclosure, when executing step 302 to determine whether the reconstructed image of the neural network intermediate layer meets the condition for stopping image reconstruction, when the condition for stopping image reconstruction is the number of image reconstructions, the determination of whether the reconstructed image of the neural network intermediate layer meets the condition for stopping image reconstruction can be achieved by but not limited to the following methods, specifically: Figure 4 As shown, the following steps are included:

[0086] Step 401, determining whether the number of reconstructions of the initial image has reached the number of image reconstructions. If the number of image reconstructions has reached the number of image reconstructions, executing step 402; if the number of image reconstructions has not reached the number of image reconstructions, executing step 403.

[0087] It should be noted here in the embodiments of the present disclosure that the number of image reconstructions, that is, the number of times required to achieve the target quality of image reconstruction, can be obtained by the image quality control discrimination neural network model of the present disclosure, and can also be obtained by other methods. The specific acquisition method is not limited by the embodiments of the present disclosure.

[0088] Step 402, determining whether the image reconstructed by the intermediate layer of the neural network satisfies the condition for stopping image reconstruction.

[0089] It should be noted here in the embodiments of the present disclosure that the determination of whether the image reconstruction conditions are met can be judged by the image quality control discrimination neural network model of the present disclosure during execution, or by an external execution code. The specific judgment method and judgment place are not limited by the embodiments of the present disclosure.

[0090] Step 403, determining that the image reconstructed by the intermediate layer of the neural network does not satisfy the condition for stopping image reconstruction.

[0091] It should be noted here in the embodiments of the present disclosure that the determination of whether the image reconstruction conditions are met can be judged by the image quality control discrimination neural network model of the present disclosure during execution, or by an external execution code. The specific judgment method and judgment place are not limited by the embodiments of the present disclosure.

[0092] Further, in some embodiments of the present disclosure, when executing step 302 to determine whether the image reconstructed by the neural network intermediate layer meets the condition for stopping image reconstruction, when the condition for stopping image reconstruction is a target reconstructed image quality standard, the determination of whether the image reconstructed by the neural network intermediate layer meets the condition for stopping image reconstruction can be achieved by but not limited to the following methods, specifically: Figure 5 As shown, the following steps are included:

[0093] Step 501, compare the quality of the image reconstructed by the intermediate layer of the neural network with the target reconstructed image quality standard. If the quality of the image reconstructed by the intermediate layer of the neural network meets the target reconstructed image quality standard, execute step 502; if the quality of the image reconstructed by the intermediate layer of the neural network does not meet the target reconstructed image quality standard, execute step 503.

[0094] It should be noted here in the embodiments of the present disclosure that the target reconstructed image quality standard may be an indicator standard of an image super-resolution task in an image reconstruction task, or the embodiments of the present disclosure do not limit this.

[0095] Step 502, determining whether the image reconstructed by the intermediate layer of the neural network satisfies the condition for stopping image reconstruction.

[0096] It should be noted here in the embodiments of the present disclosure that the determination of whether the image reconstruction conditions are met can be judged by the image quality control discrimination neural network model of the present disclosure during execution, or by an external execution code. The specific judgment method and judgment place are not limited by the embodiments of the present disclosure.

[0097] Step 503, determining that the image reconstructed by the intermediate layer of the neural network does not satisfy the condition for stopping image reconstruction.

[0098] It should be noted here in the embodiments of the present disclosure that the determination of whether the image reconstruction conditions are met can be judged by the image quality control discrimination neural network model of the present disclosure during execution, or by an external execution code. The specific judgment method and judgment place are not limited by the embodiments of the present disclosure.

[0099] Based on the above method, when performing the image reconstruction processing of the embodiment of the present invention, it is necessary to train and construct an image reconstruction processing neural network model and an image quality control discrimination neural network model. Specifically, based on the image reconstruction processing neural network model, the image quality control discrimination neural network model is constructed.

[0100] Among them, the image reconstruction processing neural network model of the embodiment of the present invention is used to reconstruct the image, and the model is trained by the currently popular image reconstruction processing data. The specific type and structure of the model, and the specific image reconstruction processing data are not limited by this disclosure. The image quality control discrimination neural network model of the embodiment of the present invention is used to obtain the quality standard of the image reconstruction. The model can be trained by, but is not limited to, the image quality standard data constructed by the trained image reconstruction processing model. The specific type and structure of the model, and the specific model training data are not limited by this disclosure.

[0101] In summary, this method predicts the target quality standard of image reconstruction in advance when processing image reconstruction, and monitors the real-time intermediate layer output of the model when the image reconstruction model is processing. When the real-time intermediate layer output meets the target quality standard, the reconstruction process is ended and the output is used as the final output of the image. Compared with the previous image reconstruction process of the complete process, this method reduces the computing resource consumption of the image processing model and improves the processing speed of the image processing model while ensuring the quality.

[0102] Based on the above method, the intermediate layer output of the image reconstruction processing model can be judged according to the predicted template quality condition, so as to terminate it in advance to save computing resources and time. Figure 6 The figure is a flowchart of the image reconstruction method, which is based on the neural network adaptive inference technology of neural network discrimination, and the main steps include:

[0103] S1. A neural network model for image quality control discrimination and input information judgment and a neural network model for image reconstruction processing are constructed.

[0104] S2. Use the image quality control discrimination neural network model to infer the existing input x, and obtain the stopping inference condition (i.e., stopping image reconstruction condition) required by the image reconstruction processing neural network model.

[0105] S3, use the image reconstruction processing neural network model to infer the input x, start inference from the first block, and obtain the intermediate state variable h 1 .

[0106] S4. Use the image quality control discrimination neural network model to determine the existing intermediate variable h 1 Whether it meets the set quality standards, if not, the next Block of the image reconstruction processing neural network model is used for reasoning to obtain the next intermediate state variable h 2 .

[0107] S5, in h i If the set quality standard is not reached, the next block of the image reconstruction processing neural network model is used for reasoning to obtain h (i+1) , use the image quality control discrimination neural network model to determine the existing intermediate state variable h (i+1) Whether it meets the set quality standard, if not, loop S5.

[0108] S6, if h i If the set quality standards are met, the output will be carried out.

[0109] In summary, this solution monitors the real-time intermediate layer output during image reconstruction processing according to the image reconstruction quality conditions predicted in advance. When the intermediate layer meets the output, it is used as the final output and image reconstruction is stopped. Since the image reconstruction process is stopped based on the predicted image quality standard control, on the one hand, it avoids the computational loss of the previous process that needs to go through the entire process, and on the other hand, it also reduces the time consumed by the image reconstruction task.

[0110] Fig. 9 FIG. 9 is a schematic diagram of the structure of an image processing device 900 provided in an embodiment of the present disclosure. Fig. 9As shown, the image processing device comprises:

[0111] An acquisition unit 901 acquires an initial image to be processed;

[0112] The processing unit 902 performs inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image;

[0113] The judgment unit 903 performs image reconstruction processing on the initial image based on the neural network model. When the intermediate layer reconstructed image of the neural network model meets the stopping image reconstruction condition, the image reconstruction processing on the initial image is stopped to obtain the target reconstructed image corresponding to the initial image.

[0114] In some embodiments of the present disclosure, Fig.10 As shown, the processing unit 902 includes:

[0115] The inference module 9021 performs inference training on the initial image according to the image quality control discrimination neural network model to obtain the stop image reconstruction condition corresponding to the initial image.

[0116] In some embodiments of the present disclosure, Fig.11 As shown, the judging unit 903 includes:

[0117] The first processing module 9031 reconstructs the initial image using the image reconstruction processing neural network model in units of layers of the neural network model to obtain a reconstructed image of the neural network intermediate layer;

[0118] A judgment module 9032 is used to judge whether the image reconstructed by the intermediate layer of the neural network meets the condition for stopping image reconstruction;

[0119] The second processing module 9033 stops the image reconstruction processing of the other layers of the neural network model to perform image reconstruction processing on the initial image if the reconstructed image of the intermediate layer of the neural network meets the stopping image reconstruction condition, so as to obtain a target reconstructed image corresponding to the initial image;

[0120] The third processing module 9034 continues to perform image reconstruction processing on the reconstructed image of the intermediate layer of the neural network if the reconstructed image of the intermediate layer of the neural network does not meet the condition for stopping image reconstruction until the reconstructed image of the intermediate layer of the neural network meets the condition for stopping image reconstruction.

[0121] In some embodiments of the present disclosure, when the image reconstruction stopping condition is the number of image reconstruction times, the second processing module 9033 is further configured to:

[0122] Determining whether the number of times the initial image is reconstructed reaches the number of times of image reconstruction;

[0123] If the number of image reconstructions has been reached, determining that the image reconstructed by the intermediate layer of the neural network meets the condition for stopping image reconstruction;

[0124] If the number of image reconstruction times is not reached, it is determined that the image reconstructed by the intermediate layer of the neural network does not meet the condition for stopping image reconstruction.

[0125] In some embodiments of the present disclosure, when the image reconstruction stopping condition is the target reconstruction image quality standard, the second processing module 9033 is further configured to:

[0126] Comparing the quality of the image reconstructed by the intermediate layer of the neural network with the target reconstructed image quality standard;

[0127] If the quality of the image reconstructed by the neural network intermediate layer meets the target reconstructed image quality standard, determining that the image reconstructed by the neural network intermediate layer meets the image reconstruction stopping condition;

[0128] If the quality of the image reconstructed by the intermediate layer of the neural network does not meet the target reconstructed image quality standard, it is determined that the image reconstructed by the intermediate layer of the neural network does not meet the condition for stopping image reconstruction.

[0129] In some embodiments of the present disclosure, Fig.12 As shown, the image processing device also includes:

[0130] The training construction unit 904 constructs the image quality control discrimination neural network model based on the image reconstruction processing neural network model.

[0131] In the embodiment of the present disclosure, when correcting an image, the initial image to be processed is first obtained; the initial image is subjected to inference training to obtain the stop image reconstruction condition corresponding to the initial image; the initial image is subjected to image reconstruction processing based on a neural network model, and when the intermediate layer reconstructed image of the neural network model meets the stop image reconstruction condition, the image reconstruction processing of the initial image is stopped to obtain the target reconstructed image corresponding to the initial image. When the image is reconstructed, the method predicts the target quality standard of image reconstruction in advance, and when the image reconstruction model is processed, the real-time intermediate layer output of the model is monitored, and when the real-time intermediate layer output meets the target quality standard, the reconstruction processing is terminated and the output is used as the final output of the image. Compared with the previous image reconstruction processing with a complete process, this method reduces the computing resource consumption of the image processing model and improves the processing speed of the image processing model while ensuring the quality.

[0132] In the embodiments provided in the present application, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the functions in the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. A function of the functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0133] Fig.13 1 is a block diagram of an electronic device 1300 for implementing the above-mentioned image processing method according to an exemplary embodiment. For example, the electronic device 1300 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0134] Reference Fig.13 , the electronic device 1300 may include one or more of the following components: a processing component 1302 , a memory 1304 , a power component 1306 , a multimedia component 1308 , an audio component 1310 , an input / output (I / O) interface 1312 , a sensor component 1314 , and a communication component 1316 .

[0135] The processing component 1302 generally controls the overall operation of the electronic device 1300, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1302 may include one or more processors 1320 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 1302 may include one or more modules to facilitate the interaction between the processing component 1302 and other components. For example, the processing component 1302 may include a multimedia module to facilitate the interaction between the multimedia component 1308 and the processing component 1302.

[0136] The memory 1304 is configured to store various types of data to support operations on the electronic device 1300. Examples of such data include instructions for any application or method operating on the electronic device 1300, contact data, phone book data, messages, pictures, videos, etc. The memory 1304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0137] The power supply component 1306 provides power to the various components of the electronic device 1300. The power supply component 1306 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 1300.

[0138] The multimedia component 1308 includes a screen that provides an output interface between the electronic device 1300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1308 includes a front camera and / or a rear camera. When the electronic device 1300 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0139] The audio component 1310 is configured to output and / or input audio signals. For example, the audio component 1310 includes a microphone (MIC), and when the electronic device 1300 is in an operation mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1304 or sent via the communication component 1316. In some embodiments, the audio component 1310 also includes a speaker for outputting audio signals.

[0140] I / O interface 1312 provides an interface between processing component 1302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0141] The sensor assembly 1314 includes one or more sensors for providing various aspects of status assessment for the electronic device 1300. For example, the sensor assembly 1314 can detect the open / closed state of the electronic device 1300, the relative positioning of components, such as the display and keypad of the electronic device 1300, and the sensor assembly 1314 can also detect the position change of the electronic device 1300 or a component of the electronic device 1300, the presence or absence of user contact with the electronic device 1300, the orientation or acceleration / deceleration of the electronic device 1300, and the temperature change of the electronic device 1300. The sensor assembly 1314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 1314 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1314 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0142] The communication component 1316 is configured to facilitate wired or wireless communication between the electronic device 1300 and other devices. The electronic device 1300 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 1316 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1316 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0143] In an exemplary embodiment, the electronic device 1300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above methods.

[0144] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1304 including instructions, and the instructions can be executed by the processor 320 of the electronic device 1300 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0145] An embodiment of the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the image processing method described in the above embodiment of the present disclosure.

[0146] An embodiment of the present disclosure further provides a computer program product, including a computer program, which executes the image processing method described in the above embodiment of the present disclosure when a processor is used.

[0147] An embodiment of the present disclosure also proposes a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive signals from a memory of an electronic device and send signals to the processor, the signals include computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the image processing method described in the above embodiments of the present disclosure.

[0148] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0149] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0150] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0151] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processing module, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (control method), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

[0152] It should be understood that the various parts of the embodiments of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0153] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0154] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0155] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. An image processing method, characterized in that: include: Obtaining an initial image to be processed; Performing inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image; The initial image is reconstructed based on a neural network model. When the intermediate layer reconstructed image of the neural network model satisfies the image reconstruction stopping condition, the image reconstruction processing of the initial image is stopped to obtain a target reconstructed image corresponding to the initial image.

2. The method according to claim 1, characterized in that The performing of inference training on the initial image to obtain a stopping image reconstruction condition for image reconstruction of the initial image comprises: The initial image is inferred and trained according to the image quality control discrimination neural network model to obtain a stop image reconstruction condition corresponding to the initial image.

3. The method according to claim 2, characterized in that The image reconstruction processing is performed on the initial image based on the neural network model, and when the intermediate layer reconstructed image of the neural network model meets the stopping image reconstruction condition, the image reconstruction processing on the initial image is stopped, and the target reconstructed image corresponding to the initial image is obtained, which includes: Taking the layers of the neural network model as units, reconstructing the initial image using the image reconstruction processing neural network model to obtain a reconstructed image of the intermediate layer of the neural network; Determining whether the image reconstructed by the intermediate layer of the neural network meets the condition for stopping image reconstruction; If the reconstructed image of the intermediate layer of the neural network meets the stopping image reconstruction condition, stop the image reconstruction processing neural network model other layers to perform image reconstruction processing on the initial image to obtain a target reconstructed image corresponding to the initial image; If the reconstructed image of the neural network intermediate layer does not satisfy the condition for stopping image reconstruction, continue to perform image reconstruction processing on the reconstructed image of the neural network intermediate layer until the reconstructed image of the neural network intermediate layer satisfies the condition for stopping image reconstruction.

4. The method according to claim 3, characterized in that: The condition for stopping image reconstruction is the number of times of image reconstruction, and judging whether the image reconstructed by the intermediate layer of the neural network meets the condition for stopping image reconstruction includes: Determining whether the number of times the initial image is reconstructed reaches the number of times of image reconstruction; If the number of image reconstruction times has been reached, determining that the image reconstructed by the intermediate layer of the neural network meets the condition for stopping image reconstruction; If the number of image reconstruction times is not reached, it is determined that the image reconstructed by the intermediate layer of the neural network does not meet the condition for stopping image reconstruction.

5. The method according to claim 3, characterized in that The condition for stopping image reconstruction is a target reconstructed image quality standard, and judging whether the image reconstructed by the intermediate layer of the neural network meets the condition for stopping image reconstruction includes: Comparing the quality of the image reconstructed by the intermediate layer of the neural network with the target reconstructed image quality standard; If the quality of the image reconstructed by the neural network intermediate layer meets the target reconstructed image quality standard, determining that the image reconstructed by the neural network intermediate layer meets the image reconstruction stopping condition; If the quality of the image reconstructed by the intermediate layer of the neural network does not meet the target reconstructed image quality standard, it is determined that the image reconstructed by the intermediate layer of the neural network does not meet the condition for stopping image reconstruction.

6. The method according to any one of claims 1 to 5, characterized in that Also includes: Based on the image reconstruction processing neural network model, the image quality control discrimination neural network model is constructed.

7. An image processing device, characterized in that: include: An acquisition unit, used for acquiring an initial image to be processed; A processing unit, used for performing inference training on the initial image to obtain a stop image reconstruction condition corresponding to the initial image; A judgment unit is used to perform image reconstruction processing on the initial image based on a neural network model. When the intermediate layer reconstructed image of the neural network model meets the image reconstruction stopping condition, the image reconstruction processing on the initial image is stopped to obtain a target reconstructed image corresponding to the initial image.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Image reconstruction method and system

    CN110807821A

  • Mixed pixel bilinear deep unmixing method and system, application and storage medium

    CN112529865A

  • Image detection method and device, electronic equipment and storage medium

    CN114299366A

  • Image reconstruction model training method and device, electronic equipment and storage medium

    CN114494006A

  • Image reconstruction by modeling image formation as one or more neural networks

    CN114503118A