Image processing method, device, terminal device and storage medium

Abstract

The embodiment of the present invention proposes an image processing method, a device terminal device and a storage medium, which relate to the field of image processing technology; and can control the adjustment of image details in the process of converting image resolution according to user needs. The method includes: obtaining a user input image and a target magnification of the difference between the resolution of the user input image and the target resolution preset by the user; obtaining an image to be processed according to the user input image; inputting the image to be processed and the target magnification into a preset multi-layer neural network model for image detail enhancement, and using the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on the image that meets the target resolution, and obtain and output a complete detail image; superimposing the complete detail image and the user input image magnified by the target magnification to obtain the target image.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of image processing technology, and in particular to an image processing method, a device terminal equipment and a storage medium. Background Art

[0002] With the development of information technology, people use more and more images in their work and life. In addition to improving hardware such as optical devices and photosensitive sensors, image quality can also be improved by processing the collected images. For example, the public security and traffic departments need to use images to identify suspects or vehicles involved in accidents, but due to the limitations of the shooting environment, the image resolution is low and the target is blurred. In view of the above situation, the image can be processed to obtain a clear image with a higher resolution.

[0003] Super-resolution imaging (SR) refers to an image processing technology that enhances image resolution. However, conventional super-resolution imaging can only adjust the resolution of the converted image and ignore the details of the image during image processing. Summary of the invention

[0004] The embodiments of the present invention provide an image processing method, an apparatus, a terminal device and a storage medium, which can control the adjustment of image details in the process of converting image resolution according to user requirements.

[0005] In a first aspect, an embodiment of the present invention provides an image processing method, which is applied to an electronic device terminal, and the method includes: obtaining a user input image and a target magnification of the difference between the resolution of the user input image and a target resolution preset by the user; obtaining an image to be processed based on the user input image; inputting the image to be processed and the target magnification into a preset multi-layer neural network model for image detail enhancement, and using the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on an image that meets the target resolution, to obtain and output a complete detail image; wherein the image that meets the target resolution is preset or obtained based on the image to be processed; and superimposing the complete detail image and the user input image magnified by the target magnification to obtain a target image.

[0006] In the above-mentioned image processing method, the image to be processed obtained based on the user input image is input into the multi-layer neural network model for image detail enhancement. The multi-layer neural network model extracts detail features that can be fused with the image without low-frequency information that meets the target resolution according to the target magnification required by the image to be processed to complete the resolution conversion. The multi-layer neural network model obtains a preset image without low-frequency information that meets the target resolution according to the type of the image to be processed, or calculates the image to be processed to obtain an image without low-frequency information that meets the target resolution, and enhances the detail features on the image without low-frequency information that meets the target resolution to obtain a complete detail image that meets the target resolution. The user input image is magnified by the target magnification to obtain the user input image that meets the target resolution, and the complete detail image that meets the target resolution is superimposed with the user input image that meets the target resolution to obtain a target image with successful resolution conversion, and the image details of the target image match the target magnification of the image. In the process of converting the image resolution, the user input image is controlled to lose image details after the resolution conversion, so as to ensure that the adjustment of the image details is controlled in the process of converting the image resolution according to user requirements.

[0007] In one possible implementation, the multi-layer neural network for image detail enhancement is preset by the following method:

[0008] Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images:

[0009] Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images;

[0010] According to the target magnification, performing a first image interpolation method on the low-resolution image to obtain a high-resolution image;

[0011] Acquire a difference image between the original image and the high-resolution image;

[0012] The multi-layer neural network model is trained by using the low-resolution image as an input sample and the difference image as a control sample.

[0013] In one possible implementation, the multi-layer neural network for image detail enhancement is preset by the following method:

[0014] Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images:

[0015] Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images;

[0016] According to the target magnification, performing the first image interpolation method on the low-resolution image to obtain a high-resolution image;

[0017] Acquire a difference image between the original image and the high-resolution image;

[0018] Obtaining a first residual image of the low-resolution image;

[0019] The multi-layer neural network model is trained by using the first residual image as an input sample and the difference image as a control sample.

[0020] In one possible implementation manner, obtaining the image to be processed according to the user input image includes:

[0021] Performing a first image interpolation method on the user input image to obtain a first image magnified by a target magnification;

[0022] Performing a second image interpolation method on the user input image to obtain a second image magnified by a target magnification;

[0023] Obtaining a second residual image of the first image at the target magnification and the second image at the target magnification;

[0024] The second residual image is used as the image to be processed.

[0025] In one possible implementation, superimposing the complete detail image and the user input image magnified by the target magnification includes:

[0026] Performing the first image interpolation method processing on the user input image to obtain a third image magnified by a target magnification;

[0027] The full detail image and the third image are superimposed.

[0028] In one possible implementation, the method further includes:

[0029] Acquire a detail adjustment weight selected by a user, and assign the detail adjustment weight to the complete detail image;

[0030] The step of superimposing the complete detail image and the user input image magnified by the target magnification includes:

[0031] The complete detail image assigned with the detail adjustment weight and the user input image enlarged with the target resolution are superimposed to obtain a target image.

[0032] In one possible implementation, the first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation;

[0033] The second image interpolation method includes: nearest neighbor interpolation.

[0034] In a second aspect, an embodiment of the present invention provides an image processing device, which is provided in an electronic device terminal, and the device includes:

[0035] A resolution acquisition module, which acquires a user input image and a target magnification of a difference between the resolution of the user input image and a target resolution preset by the user;

[0036] An image acquisition module, used for obtaining an image to be processed according to the image input by the user;

[0037] An input module, used for inputting the image to be processed and the target resolution into a preset multi-layer neural network model for image detail enhancement, and using the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on the image meeting the target resolution, to obtain and output a complete detail image; wherein the image meeting the target resolution is preset or obtained according to the image to be processed;

[0038] The input image superposition module is used to superimpose the complete detail image and the user input image magnified by the target magnification to obtain the target image.

[0039] In one possible implementation, the device further includes a first neural network training module, and the first neural network training module is specifically used to:

[0040] Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images:

[0041] Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images;

[0042] According to the target magnification, performing a first image interpolation method on the low-resolution image to obtain a high-resolution image;

[0043] Acquire a difference image between the original image and the high-resolution image;

[0044] The multi-layer neural network model is trained by using the low-resolution image as an input sample and the difference image as a control sample.

[0045] In one possible implementation, the device further includes a second neural network training module, and the second neural network training module is specifically used to:

[0046] Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images:

[0047] Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images;

[0048] According to the target magnification, performing the first image interpolation method on the low-resolution image to obtain a high-resolution image;

[0049] Acquire a difference image between the original image and the high-resolution image;

[0050] Obtaining a first residual image of the low-resolution image;

[0051] The multi-layer neural network model is trained by using the first residual image as an input sample and the difference image as a control sample.

[0052] In one possible implementation, the image acquisition module includes:

[0053] A first interpolation submodule, configured to perform a first image interpolation method on the user input image to obtain a first image magnified by a target magnification;

[0054] A second interpolation submodule, configured to perform a second image interpolation method on the user input image to obtain a second image magnified by a target magnification;

[0055] A residual image acquisition submodule, used to obtain a second residual image of the first image with the target magnification and the second image with the target magnification;

[0056] The image determination submodule is used to use the second residual image as the image to be processed.

[0057] In one possible implementation, the input image overlay module includes:

[0058] A third interpolation submodule, configured to perform the first image interpolation method on the user input image to obtain a third image magnified by a target magnification;

[0059] The enlarged image superposition submodule is used to superimpose the complete detail image and the third image.

[0060] In one possible implementation, the device further includes:

[0061] A weight adjustment module, used for obtaining a detail adjustment weight selected by a user, and assigning the detail adjustment weight to the complete detail image;

[0062] The input image superposition module includes an adjusted image superposition submodule, which is used to superimpose the complete detail image assigned with the detail adjustment weight and the user input image enlarged to the target resolution to obtain a target image.

[0063] In one possible implementation, the first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation;

[0064] The second image interpolation method includes: nearest neighbor interpolation.

[0065] In a third aspect, an embodiment of the present invention provides a terminal device, comprising: at least one processor; and at least one memory communicatively connected to the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the method provided in the first aspect.

[0066] In a fourth aspect, an embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute the method provided in the first aspect.

[0067] It should be understood that the second to fourth aspects of the embodiments of the present invention are consistent with the technical solutions of the first aspect of the embodiments of the present invention, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0069] Figure 1 is a flow chart of training a multi-layer neural network model for image detail enhancement according to an embodiment of the present invention;

[0070] Figure 2 is a flow chart of training a multi-layer neural network model for image detail enhancement according to another embodiment of the present invention;

[0071] Figure 3 is a flowchart of the steps of the image processing method proposed in an embodiment of the present invention;

[0072] Figure 4It is a schematic diagram of a processing flow of an example of the present invention for a user input image;

[0073] Figure 5 is a functional module diagram of an image processing device proposed in an embodiment of the present invention;

[0074] Figure 6 A schematic diagram of the structure of an electronic terminal device provided by an embodiment of the present invention;

[0075] Figure 7 A schematic diagram of the structure of a terminal device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0076] In order to better understand the technical solution of this specification, the embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0077] It should be clear that the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0078] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit this specification. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0079] In the application fields of medical imaging, monitoring and security, there is a high demand for image resolution, which determines the accuracy of the information obtained. However, it is not easy to directly acquire high-resolution images. The reasons include: limited by the performance of optical sensors and related devices; affected by the environment or transmission equipment, the acquired images will produce noise, affecting the resolution image and details of the image. Therefore, image processing technology is used to convert the original image with lower resolution into an image with higher resolution, or to restore the low-resolution image obtained by compressing the original image to the original image. When performing resolution conversion, the super-resolution imaging of the existing technology ignores the details of the image; or ignores the influence of image resolution on image details.

[0080] In view of the above problems, an embodiment of the present invention proposes an image processing method, in which an image to be processed obtained based on a user input image and a target ratio of resolution conversion are used in a multi-layer neural network model for image detail enhancement. The multi-layer neural network model for image detail enhancement extracts detail features of the image to be processed, and enhances the detail features according to the target ratio of resolution conversion to obtain a complete detail image that meets the target resolution; the complete detail image is superimposed with the user input image converted to the target resolution, and the obtained image not only has its resolution converted to the target resolution, but also has its image details enhanced according to the detail transformation required for the resolution conversion to the target resolution.

[0081] In order to more intelligently implement the image processing method proposed in the embodiment of the present invention, the embodiment of the present invention first collects training samples and control samples to construct a sample set, and finally trains the pre-constructed multi-layer neural network model based on the sample set to obtain a multi-layer neural network model for image detail enhancement. Among them, the multi-layer neural network model can be a model commonly used in related technologies, and can include convolutional neural networks, deep neural networks, recursive neural networks, etc. The multi-layer neural network model for image detail enhancement can be used to execute some or all of the steps in the above-mentioned image processing method proposed in the embodiment of the present invention.

[0082] refer to Figure 1 , Figure 1 is a flowchart of a multi-layer neural network model for training image detail enhancement according to an embodiment of the present invention; the electronic terminal device can execute Figure 1 The proposed steps are to collect a sample set, the sample set includes training samples and control samples, train a multi-layer neural network to obtain a multi-layer neural network model for image detail enhancement, and set the model locally to use the obtained multi-layer neural network model for image detail enhancement to execute part of the steps or all of the steps of the image processing method.

[0083] The steps for setting up a multi-layer neural network for image detail enhancement include:

[0084] Step S111: Acquire multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images.

[0085] The original image may be an image with high resolution and clear details, which is acquired by using a high-performance optical camera device with proper exposure.

[0086] The resolution of the original image is the target resolution preset by the user. For example, the user presets that the electronic terminal device can adjust the image resolution to 4096×4096 when executing the image processing method proposed in the embodiment of the present invention, then 4096×4096 is the target resolution preset by the user. The user can preset multiple target resolutions.

[0087] Step S112: reducing the resolution of the target original image by the target magnification to obtain a low-resolution image, wherein the target original image is any image among the multiple original images.

[0088] The target magnification is any of the pre-set magnifications, and the magnification of the target original image to reduce the resolution is obtained according to the target resolution and the user's intention to build a multi-layer neural network model for image detail enhancement. For example, according to the target resolution of 4096×4096, the target magnification is set to 2 times, then the trained multi-layer neural network model for image detail enhancement can convert an image with a resolution of 2048×2048 to an image with a resolution of 4096×4096.

[0089] After setting the target resolution, you can reduce the resolution of the target original image at different magnifications. Reducing the resolution of the target original image means reducing the resolution and image clarity of the target original image by using different downsampling algorithms, blurring some pixels of the image, or adding noise according to the preset target magnification.

[0090] A low-resolution image is an image whose resolution is reduced by a target factor relative to the original image.

[0091] For example, after obtaining an original image with a resolution of 4096×4096, 2 times and 4 times are used as target magnifications in turn to reduce the resolution of the original image, thereby obtaining a low-resolution image of 2048×2048 and a low-resolution image of 1024×1024.

[0092] Step S113: performing a first image interpolation method on the low-resolution image according to the target magnification to obtain a high-resolution image.

[0093] The first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation. The high-resolution image obtained by interpolating the low-resolution image cannot completely restore the original image. The resolution of the high-resolution image is the same as that of the original image, but compared with the original image, it lacks image detail information, has lower clarity, and has more noise.

[0094] Step S114: obtaining a difference image between the original image and the high-resolution image.

[0095] The difference image between the original image and the high-resolution image has image details of the original image compared to the high-resolution image.

[0096] Step S115: using the low-resolution image as an input sample and the difference image as a control sample to train the multi-layer neural network model.

[0097] The low-resolution image is input into the multi-layer neural network model, and the multi-layer neural network model extracts the detail features of the low-resolution image; the multi-layer neural network model performs filtering and other processing on the low-resolution image, and adjusts the resolution of the low-resolution image to the resolution of the original image, so as to obtain an image without basic image information and with the same resolution as the original image; the detail features of the low-resolution image are enhanced on the image without basic image information and with the same resolution as the original image, so as to obtain the output image of the multi-layer neural network model.

[0098] The loss function of the output image and the difference image is calculated, and the parameters of the multi-layer neural network model are adjusted according to the loss function until the output image obtained by the multi-layer neural network model is infinitely close to the difference image and the value of the loss function is less than a preset threshold.

[0099] The input samples collected by the above-mentioned training multi-layer neural network model are low-resolution images obtained after the resolution of the original image is reduced by a target magnification; the target magnification of the original image resolution reduction can be preset to one or more. Then the resolution of the low-resolution image is adjusted to the resolution of the original image by interpolation calculation to obtain a high-resolution image. The difference image between the high-resolution image and the original image is the real image details lost after the original image is processed by downsampling, image blurring, adding noise, etc. The low-resolution image is input into the multi-layer neural network model as an input sample. After the multi-layer neural network model is calculated, the image details of the predicted low-resolution image and the original image are obtained. The difference image between the high-resolution image and the original image is used as a control sample to test whether the image details predicted by the multi-layer neural network model are the same as the real image details. The parameters of the multi-layer neural network model are adjusted according to the test results. When the output image of the multi-layer neural network model is infinitely close to the difference image between the high-resolution image and the original image, the multi-layer neural network model obtains the function of extracting the detail features of the low-resolution image and restoring the image details of the original image in the process of converting the resolution of the low-resolution image according to the target magnification.

[0100] Figure 2 is a flowchart of another embodiment of the present invention for training a multi-layer neural network model for image detail enhancement; the electronic terminal device can execute Figure 2 The proposed steps are to collect a sample set, the sample set includes training samples and control samples, train a multi-layer neural network to obtain a multi-layer neural network model for image detail enhancement, and set the model locally to use the obtained multi-layer neural network model for image detail enhancement to execute part of the steps or all of the steps of the image processing method.

[0101] refer to Figure 2 , the steps for setting up a multi-layer neural network for image detail enhancement include:

[0102] The image processing proposed in the embodiment of the present invention can input the user input image after preprocessing into the multi-layer neural network model for image detail enhancement, or can input the residual image of the user input image into the multi-layer neural network model for image detail enhancement. The residual image has a small value, and the electronic terminal device occupies less hardware resources when calculating the residual image.

[0103] Step S121: Acquire multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images.

[0104] Step S122: reducing the resolution of the target original image by the target magnification to obtain a low-resolution image, wherein the target original image is any image among the multiple original images.

[0105] Step S123: performing the first image interpolation method on the low-resolution image according to the target magnification to obtain a high-resolution image.

[0106] Step S124: obtaining a difference image between the original image and the high-resolution image.

[0107] Step S125: Obtain a first residual image of the low-resolution image.

[0108] The method of obtaining the first residual image of a low-resolution image may include: obtaining the first residual image by calculating the base layer and detail layer of the low-resolution image; converting the resolution of the low-resolution image by using the first image interpolation method and the second image interpolation method respectively, and subtracting the images after the resolution conversion by the above two image interpolation methods to obtain the first residual image; the pixel points inserted by the second image interpolation method are the pixel points closest to the insertion position, and there is no information about other pixel points; the pixel points inserted by the second image interpolation method are obtained after calculating the pixel points at different positions, the inserted pixel points carry the information of other pixel points, and the subtraction of the two only retains the image details of the low-resolution image.

[0109] Step S126: using the first residual image as an input sample and the difference image as a control sample to train the multi-layer neural network model.

[0110] The first residual image is input into the multi-layer neural network model, and the multi-layer neural network model extracts detail features of the first residual image; the detail features are supplemented at some pixel points of the first residual image to obtain an output image of the multi-layer neural network model.

[0111] The loss function of the output image and the difference image is calculated, and the parameters of the multi-layer neural network model are adjusted according to the loss function until the output image obtained by the multi-layer neural network model is infinitely close to the difference image and the value of the loss function is less than a preset threshold.

[0112] The input sample collected by the above-mentioned training multi-layer neural network model is the residual image of the low-resolution image, and the residual image only retains the image detail information of the low-resolution image. The difference image between the high-resolution image and the original image is the image detail that is actually lost after the original image is processed by downsampling, image blurring, adding noise, etc. The residual image is input as an input sample into the multi-layer neural network model, and after calculation, the multi-layer neural network model predicts the complete image detail information after the image detail information retained by the residual image is supplemented on the residual image. The difference image between the high-resolution image and the original image is used as a control sample to test whether the complete image detail information predicted by the multi-layer neural network model is the same as the real image detail, and the parameters of the multi-layer neural network model are adjusted according to the test results. When the output image of the multi-layer neural network model is infinitely close to the difference image between the high-resolution image and the original image, the multi-layer neural network model obtains the function of extracting the detail features of the residual image of the low-resolution image and restoring the image details of the original image in the process of converting the resolution of the low-resolution image according to the target magnification through learning.

[0113] The above-mentioned embodiment of the present invention proposes two methods of training a multi-layer neural network model for image detail enhancement: using a low-resolution image as an input sample and a difference image as a control sample to train the multi-layer neural network model; using the first residual image as an input sample and a difference image as a control sample to train the multi-layer neural network model.

[0114] The method of setting the obtained multi-layer neural network model for image detail enhancement in an electronic terminal device includes: the multi-layer neural network model for image detail enhancement obtained by training the multi-layer neural network model with a low-resolution image as an input sample and a difference image as a control sample, and the multi-layer neural network model for image detail enhancement obtained by training the multi-layer neural network model with a first residual image as an input sample and a difference image as a control sample are simultaneously set in the electronic terminal device.

[0115] First, the low-resolution image is used as the input sample and the difference image is used as the control sample to train the multi-layer neural network model. On this basis, the first residual image is used as the input sample and the difference image is used as the control sample to further train it to obtain a multi-layer neural network model for image detail enhancement, and it is set in the electronic terminal device.

[0116] First, the multi-layer neural network model is trained by taking the first residual image as the input sample and the difference image as the control sample. Then, on this basis, the low-resolution image is taken as the input sample and the difference image is taken as the control sample for further training to obtain a multi-layer neural network model for image detail enhancement, which is then set in an electronic terminal device.

[0117] An electronic terminal device is provided with at least one multi-layer neural network model for image detail enhancement obtained based on the above two training methods, which can respond to different mode selection instructions, select a model matching the image to be processed from at least one multi-layer neural network model for image detail enhancement obtained based on the above two training methods, receive the input image to be processed, and execute corresponding calculation steps.

[0118] For example, when the mode selection instruction corresponds to directly extracting the detail image of the user input image, the method of receiving the multi-layer neural network model for image detail enhancement that extracts the detail features of the image to be processed includes: the multi-layer neural network model for image detail enhancement performs filtering and other processing on the image to be processed, and adjusts the resolution of the image to be processed to the target resolution to obtain an image without basic image information and with a resolution of the target resolution; the detail features of the image to be processed are enhanced on the image without basic image information and with a resolution of the target resolution, and a complete detail image is output.

[0119] When the mode selection instruction corresponds to extracting detail images from the residual image, the multi-layer neural network model for image detail enhancement that receives the image to be processed extracts detail features of the image to be processed, including: enhancing the detail features on the residual image and outputting a complete detail image.

[0120] refer to Figure 3 , Figure 3 is a flowchart of the steps of the image processing method proposed in an embodiment of the present invention, which is applied to an electronic terminal device, and the steps include:

[0121] The electronic terminal device may be a mobile phone, a computer, a wearable device, a camera with an integrated computing module, a display with an integrated computing module, etc.

[0122] Step S211: obtaining a user input image and a target magnification of a difference between a resolution of the user input image and a target resolution preset by a user.

[0123] The target resolution pre-set by the user is one of the resolutions that can be achieved by the multi-layer neural network model for image detail enhancement. For example, the resolutions that can be achieved by the multi-layer neural network model for image detail enhancement include 4096×4096, 2048×2048, 1024×1024, etc. In response to the selection operation triggered by the user, the target resolution pre-set by the user is determined in the above resolutions.

[0124] Step S212: obtaining an image to be processed according to the user input image.

[0125] Obtaining the image to be processed according to the user input image may include preprocessing the user input image to obtain the image to be processed, such as image denoising, image color correction, image bad pixel removal, and invalid frame removal, etc., which may be processed accordingly according to some conditions of the imaging system itself. Alternatively, obtaining the image to be processed according to the user input image may include calculating a residual image of the user input image as the image to be processed.

[0126] In one embodiment of the present invention, obtaining an image to be processed according to an image input by a user comprises the following steps:

[0127] Step S212-1: Perform a first image interpolation method on the user input image to obtain a first image magnified by a target magnification. The first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation.

[0128] Step S212-2: Perform a second image interpolation method on the user input image to obtain a second image magnified by a target magnification. The second image interpolation method includes: nearest neighbor interpolation.

[0129] Step S212-3: obtaining a second residual image of the first image at the target magnification and the second image at the target magnification;

[0130] Step S212-4: taking the second residual image as the image to be processed.

[0131] The pixel points inserted by the second image interpolation method are the pixel points closest to the insertion position and do not have information about other pixel points; the pixel points inserted by the second image interpolation method are obtained after relevant calculations are performed on pixel points at different positions. The inserted pixel points carry information about other pixel points. The resolution of the second residual image obtained by subtracting the two is the target resolution, and the second residual image only retains the image details of the user input image.

[0132] Step S213: inputting the image to be processed and the target magnification into a preset multi-layer neural network model for image detail enhancement, and utilizing the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on an image that meets the target resolution, to obtain and output a complete detail image; wherein, the image that meets the target resolution is pre-set or obtained based on the image to be processed.

[0133] The multi-layer neural network model for image detail enhancement obtained by training the multi-layer neural network model using the image processing method of the embodiment of the present invention has the function of enhancing the detail features of the image to be processed on the image meeting the target resolution, and obtaining and outputting a complete detail image.

[0134] When the input image to be processed is a preprocessed user input image, the image that meets the target resolution may be a pixel matrix without image information pre-set in the multi-layer neural network model for image detail enhancement. Alternatively, the image that meets the target resolution may be an image obtained by converting the resolution of the image to be processed to the target resolution by the multi-layer neural network model for image detail enhancement, and then filtering the image to be processed after the resolution conversion to remove the image base layer.

[0135] When the input image to be processed is a residual image of the user input image, the image that meets the target resolution can be a second residual image input to the multi-layer neural network model for image detail enhancement, and the second residual image is obtained by calculating the memory residual of the user input image.

[0136] Step S214: superimposing the complete detail image and the user input image magnified by the target magnification to obtain a target image.

[0137] In an embodiment of the present invention, an image to be processed obtained based on an image input by a user is input into a multi-layer neural network model for image detail enhancement. The multi-layer neural network model extracts detail features from the image to be processed that can be fused with an image without low-frequency information that meets the target resolution according to a target magnification of the image to be processed to complete the resolution conversion requirement. The multi-layer neural network model obtains a preset image without low-frequency information that meets the target resolution according to the type of the image to be processed, or calculates the image to be processed to obtain an image without low-frequency information that meets the target resolution, and enhances detail features on the image without low-frequency information that meets the target resolution to obtain a complete detail image that meets the target resolution. The user input image is magnified by a target magnification to obtain a user input image that meets the target resolution, and the complete detail image that meets the target resolution is superimposed on the user input image that meets the target resolution to obtain a target image with successful resolution conversion, and the image details of the target image match the target magnification of the image. In the process of converting the image resolution, the user input image is controlled to lose image details after the resolution conversion, so as to ensure that the adjustment of the image details is controlled in the process of converting the image resolution according to user requirements.

[0138] In another embodiment of the present invention, the step of superimposing the complete detail image and the user input image with the magnified target magnification comprises:

[0139] Step S214 - 1 : performing the first image interpolation method processing on the user input image to obtain a third image magnified by a target magnification.

[0140] Step S214-2: superimposing the complete detail image and the third image.

[0141] The user input image is interpolated using the first image interpolation method, and while converting the resolution of the user input image, a basis is provided for obtaining a residual image of the user input image.

[0142] In another embodiment of the present invention, the step of superimposing the complete detail image and the user input image with the magnified target magnification comprises:

[0143] Step S214a: Obtain the detail adjustment weight selected by the user, and assign the detail adjustment weight to the complete detail image.

[0144] Assigning detail adjustment weights to the full detail image may refer to weighting each pixel in the full detail image using the detail adjustment weights.

[0145] Step S214b: superimposing the complete detail image assigned with the detail adjustment weight and the user input image enlarged to the target resolution to obtain a target image.

[0146] By assigning user-selected detail adjustment weights to the complete detail image, the presentation intensity of the image details in the final output image can be controlled, and the presentation degree of the image details during the image resolution conversion process can be flexibly adjusted.

[0147] refer to Figure 4 , Figure 4 It is a schematic diagram of the processing flow of an example of the present invention for a user input image. The user input image can be input into a multi-layer neural network model for image detail enhancement through image binary processing.

[0148] The user input image can also be processed by the first image interpolation method to obtain a first image of the target magnification; and processed by the second image interpolation method to obtain a second image of the target magnification; on the one hand, the first image and the second image are processed by image subtraction to obtain a second residual image, and the second residual image is input into the multi-layer neural network model for image detail enhancement through image two-choice processing. On the other hand, the first image is superimposed with the complete detail image to which detail adjustment weights are assigned. The user input image is processed by the first image interpolation method to obtain the user input image of the target magnification superimposed with the complete detail image, and can also be subtracted from the second image to obtain the second residual image that can be input into the multi-layer neural network model for image detail enhancement.

[0149] The second residual image or the user input image is input into the multi-layer neural network model for image detail enhancement. The multi-layer neural network model for image detail enhancement extracts detail features from the second residual image or the user input image, and enhances the image that meets the target resolution, and outputs a complete detail image with a resolution of the target magnification.

[0150] The complete detail image is processed by assigning detail adjustment weights to obtain a superposition coefficient on the user input image of the target magnification. By adjusting the detail adjustment weights, the detail presentation degree of the target image can be adjusted.

[0151] The electronic terminal device receives the target resolution input by the user, interpolates the user input image according to the target resolution, obtains the first image of the target resolution, and the second image of the target resolution, obtains the residual image of the target resolution, and the image that can be superimposed with the complete detail image. The multi-layer neural network model for image detail enhancement outputs the complete detail image of the target resolution according to the received target resolution, and can flexibly convert the image resolution according to the user's needs, and flexibly adjust the intensity of the image details according to the user's needs for the image details.

[0152] Figure 5 is a functional module diagram of an image processing device proposed in an embodiment of the present invention, wherein the image processing device is arranged in a terminal device, such as Figure 5 As shown, the device comprises:

[0153] A resolution acquisition module 51 is used to acquire a user input image and a target magnification of a difference between the resolution of the user input image and a target resolution preset by the user;

[0154] An image acquisition module 52, used to obtain an image to be processed according to the user input image;

[0155] An input module 53 is used to input the image to be processed and the target resolution into a preset multi-layer neural network model for image detail enhancement, and use the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on the image meeting the target resolution, so as to obtain and output a complete detail image; wherein the image meeting the target resolution is preset or obtained according to the image to be processed;

[0156] The input image superposition module 54 is used to superimpose the complete detail image and the user input image magnified by the target magnification to obtain a target image.

[0157] Figure 5 The image processing device provided in the embodiment shown can be used to execute the Figures 1 to 4 The technical solution of the method embodiment shown, its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.

[0158] Optionally, the device further includes a first neural network training module, wherein the first neural network training module is specifically configured to:

[0159] Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images:

[0160] Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images;

[0161] According to the target magnification, performing a first image interpolation method on the low-resolution image to obtain a high-resolution image;

[0162] Acquire a difference image between the original image and the high-resolution image;

[0163] The multi-layer neural network model is trained by using the low-resolution image as an input sample and the difference image as a control sample.

[0164] Optionally, the device further includes a second neural network training module, wherein the second neural network training module is specifically configured to:

[0165] Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images:

[0166] Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images;

[0167] According to the target magnification, performing the first image interpolation method on the low-resolution image to obtain a high-resolution image;

[0168] Acquire a difference image between the original image and the high-resolution image;

[0169] Obtaining a first residual image of the low-resolution image;

[0170] The multi-layer neural network model is trained by using the first residual image as an input sample and the difference image as a control sample.

[0171] Optionally, the image acquisition module includes:

[0172] A first interpolation submodule, configured to perform a first image interpolation method on the user input image to obtain a first image magnified by a target magnification;

[0173] A second interpolation submodule, configured to perform a second image interpolation method on the user input image to obtain a second image magnified by a target magnification;

[0174] A residual image acquisition submodule, used to obtain a second residual image of the first image with the target magnification and the second image with the target magnification;

[0175] The image determination submodule is used to use the second residual image as the image to be processed.

[0176] Optionally, the input image overlay module includes:

[0177] A third interpolation submodule, configured to perform the first image interpolation method on the user input image to obtain a third image magnified by a target magnification;

[0178] The enlarged image superposition submodule is used to superimpose the complete detail image and the third image.

[0179] Optionally, the device further comprises:

[0180] A weight adjustment module, used for obtaining a detail adjustment weight selected by a user, and assigning the detail adjustment weight to the complete detail image;

[0181] The input image superposition module includes an adjusted image superposition submodule, which is used to superimpose the complete detail image assigned with the detail adjustment weight and the user input image enlarged to the target resolution to obtain a target image.

[0182] Optionally, the first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation;

[0183] The second image interpolation method includes: nearest neighbor interpolation.

[0184] The device provided in the above-mentioned embodiment is used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effects can be further referred to the relevant description in the method embodiment, which will not be repeated here.

[0185] The device provided in the above embodiment can be, for example, a chip or a chip module. The device provided in the above embodiment is used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect can be further referred to the relevant description in the method embodiment, which will not be repeated here.

[0186] Regarding the various modules / units included in the various devices described in the above embodiments, they can be software modules / units, or hardware modules / units, or they can be partially software modules / units and partially hardware modules / units. For example, for various devices applied to or integrated in a chip, the various modules / units included therein can all be implemented in the form of hardware such as circuits, or at least some of the modules / units can be implemented in the form of software programs, which run on a processor integrated inside the chip, and the remaining modules / units can be implemented in the form of hardware such as circuits; for various devices applied to or integrated in a chip module, the various modules / units included therein can all be implemented in the form of hardware such as circuits, and different modules / units can be located in the same component (such as a chip, circuit module, etc.) or different components of the chip module, or at least some of the modules / units can be implemented in the form of software programs. The software program may be implemented in the form of a program, which runs on a processor integrated in the chip module, and the remaining modules / units may be implemented in the form of hardware such as circuits; for each device applied to or integrated in the electronic terminal device, each module / unit contained therein may be implemented in the form of hardware such as circuits, and different modules / units may be located in the same component (for example, a chip, a circuit module, etc.) or in different components in the electronic terminal device, or at least some modules / units may be implemented in the form of a software program, which runs on a processor integrated in the electronic terminal device, and the remaining (if any) modules / units may be implemented in the form of hardware such as circuits.

[0187] Figure 6 A structural schematic diagram of an electronic terminal device provided in an embodiment of the present invention, the electronic terminal device 600 includes a processor 610, a memory 611, and a computer program stored in the memory 611 and executable on the processor 610. The processor 610 implements the steps in the aforementioned method embodiment when executing the program. The electronic terminal device provided in the embodiment can be used to execute the technical solution of the method embodiment shown above. The implementation principle and technical effects thereof can be further referred to the relevant description in the method embodiment, which will not be repeated here.

[0188] Figure 7 A schematic diagram of the structure of a terminal device provided in one embodiment of this specification, such as Figure 7 As shown, the terminal device may include at least one processor; and at least one memory in communication with the processor, wherein: the memory stores program instructions executable by the processor, and the processor calls the program instructions to execute the instructions of this specification. Figure 1~Figure 4 The illustrated embodiment provides an image processing method.

[0189] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the terminal device 100. In other embodiments of the present invention, the terminal device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0190] like Figure 7 As shown, the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a mobile communication module 150, a wireless communication module 160, an indicator 192, a camera 193, a display screen 194, and the like.

[0191] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0192] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0193] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0194] The processor 110 executes various functional applications and data processing by running the programs stored in the internal memory 121, such as implementing the present invention. Figure 1~Figure 4 The illustrated embodiment provides an image processing method.

[0195] The wireless communication function of the terminal device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0196] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0197] The terminal device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0198] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0199] The terminal device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

[0200] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.

[0201] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0202] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the terminal device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0203] Video codecs are used to compress or decompress digital videos. The terminal device 100 may support one or more video codecs. Thus, the terminal device 100 may play or record videos in various coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0204] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the terminal device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the terminal device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0205] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute the instructions of the present invention. Figure 1~Figure 4The illustrated embodiment provides an image processing method.

[0206] The above-mentioned non-transitory computer-readable storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, - but not limited to - an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0207] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0208] The program code embodied on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wire, optical cable, radio frequency (RF), etc., or any suitable combination of the foregoing.

[0209] Computer program code for performing the operations of the present specification may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0210] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0211] In the description of the embodiments of the present invention, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0212] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this specification, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0213] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of this specification includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of this specification belong.

[0214] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0215] It should be noted that the terminals involved in the embodiments of the present invention may include but are not limited to personal computers (PCs), personal digital assistants (PDAs), wireless handheld devices, tablet computers, mobile phones, MP3 players, MP4 players, etc.

[0216] In the several embodiments provided in this specification, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0217] In addition, each functional unit in each embodiment of this specification may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0218] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the method described in each embodiment of this specification. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program codes.

[0219] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

Claims

1. An image processing method, characterized in that: Applied to electronic terminal equipment, the method comprises: Acquire a user input image and a target magnification of a difference between a resolution of the user input image and a target resolution preset by a user; Obtaining an image to be processed according to the user input image; Inputting the image to be processed and the target magnification into a preset multi-layer neural network model for image detail enhancement, using the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on an image that meets the target resolution, and obtaining and outputting a complete detail image; wherein the image that meets the target resolution is preset or obtained according to the image to be processed; Superimposing the complete detail image and the user input image magnified by the target magnification to obtain a target image; The multi-layer neural network for image detail enhancement is preset by the following method: Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images: Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images; According to the target magnification, performing a first image interpolation method on the low-resolution image to obtain a high-resolution image; Acquire a difference image between the original image and the high-resolution image; The multi-layer neural network model is trained based on the low-resolution image and the difference image.

2. The method according to claim 1, characterized in that The training of the multi-layer neural network model based on the low-resolution image and the difference image comprises: The multi-layer neural network model is trained by using the low-resolution image as an input sample and the difference image as a control sample.

3. The method according to claim 1, characterized in that: The training of the multi-layer neural network model based on the low-resolution image and the difference image comprises: obtaining a first residual image of the low-resolution image; The multi-layer neural network model is trained by using the first residual image as an input sample and the difference image as a control sample.

4. The method according to any one of claims 1 to 3, characterized in that: The step of obtaining the image to be processed according to the user input image comprises: Performing a first image interpolation method on the user input image to obtain a first image magnified by a target magnification; Performing a second image interpolation method on the user input image to obtain a second image magnified by a target magnification; Obtaining a second residual image of the first image at the target magnification and the second image at the target magnification; The second residual image is used as the image to be processed.

5. The method according to any one of claims 1 to 3, characterized in that: The step of superimposing the complete detail image and the user input image magnified by the target magnification includes: Performing a first image interpolation process on the user input image to obtain a third image magnified by a target magnification; The full detail image and the third image are superimposed.

6. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: Acquire a detail adjustment weight selected by a user, and assign the detail adjustment weight to the complete detail image; The step of superimposing the complete detail image and the user input image magnified by the target magnification includes: The complete detail image assigned with the detail adjustment weight and the user input image enlarged with the target resolution are superimposed to obtain a target image.

7. The method according to claim 4, characterized in that The first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation; The second image interpolation method includes: nearest neighbor interpolation.

8. An image processing device, characterized in that: Set in an electronic terminal device, the device includes: A resolution acquisition module, which acquires a user input image and a target magnification of a difference between the resolution of the user input image and a target resolution preset by the user; An image acquisition module, used for obtaining an image to be processed according to the image input by the user; An input module, used for inputting the image to be processed and the target resolution into a preset multi-layer neural network model for image detail enhancement, and using the multi-layer neural network model for image detail enhancement to enhance the detail features of the image to be processed on the image meeting the target resolution, to obtain and output a complete detail image; wherein the image meeting the target resolution is preset or obtained according to the image to be processed; An input image superposition module, used for superimposing the complete detail image and the user input image magnified by the target magnification to obtain a target image; The multi-layer neural network for image detail enhancement is preset by the following method: Obtain multiple original images, and train the multi-layer neural network model according to the following steps for each of the original images: Reducing the target magnification of the resolution of the target original image to obtain a low-resolution image, wherein the target original image is any one of the multiple original images; According to the target magnification, performing a first image interpolation method on the low-resolution image to obtain a high-resolution image; Acquire a difference image between the original image and the high-resolution image; The multi-layer neural network model is trained based on the low-resolution image and the difference image.

9. The device according to claim 8, characterized in that The device further includes a first neural network training module, wherein the first neural network training module is specifically configured to: The multi-layer neural network model is trained by using the low-resolution image as an input sample and the difference image as a control sample.

10. The device according to claim 8, characterized in that The device further includes a second neural network training module, wherein the second neural network training module is specifically configured to: Obtaining a first residual image of the low-resolution image; The multi-layer neural network model is trained by using the first residual image as an input sample and the difference image as a control sample.

11. The device according to any one of claims 8 to 10, characterized in that: The image acquisition module comprises: A first interpolation submodule, configured to perform a first image interpolation method on the user input image to obtain a first image magnified by a target magnification; A second interpolation submodule, configured to perform a second image interpolation method on the user input image to obtain a second image magnified by a target magnification; A residual image acquisition submodule, used to obtain a second residual image of the first image with the target magnification and the second image with the target magnification; The image determination submodule is used to use the second residual image as the image to be processed.

12. The device according to any one of claims 8 to 10, characterized in that: The input image overlay module comprises: A third interpolation submodule, configured to perform a first image interpolation process on the user input image to obtain a third image magnified by a target magnification; The enlarged image superposition submodule is used to superimpose the complete detail image and the third image.

13. The device according to any one of claims 8 to 10, characterized in that: The device also includes: A weight adjustment module, used for obtaining a detail adjustment weight selected by a user, and assigning the detail adjustment weight to the complete detail image; The input image superposition module includes an adjusted image superposition submodule, which is used to superimpose the complete detail image assigned with the detail adjustment weight and the user input image enlarged to the target resolution to obtain a target image.

14. The device according to claim 11, characterized in that The first image interpolation method includes: any one of bilinear interpolation, bicubic interpolation, Lanczos interpolation, and spline interpolation; The second image interpolation method includes: nearest neighbor interpolation.

15. A terminal device, comprising: at least one processor; as well as At least one memory in communication with the processor, characterized in that: The memory stores program instructions executable by the processor, and the processor can execute the method according to any one of claims 1 to 7 by calling the program instructions.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions enable the computer to execute the method according to any one of claims 1 to 7.

Images