Image processing method, device and equipment

Abstract

The present application provides an image processing method, device, and equipment, etc. The method generates a special image filter and performs super-resolution on the image according to the image filter, thereby improving the image super-resolution effect. The image filter has filter parameters corresponding to each pixel in the image to be super-resolution, and pixels with different texture features have different filter parameters. The image super-resolution method, device, and equipment, etc. can be applied to various scenes such as video, games, and photography to improve the image effects in these scenes, thereby enhancing the user experience.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and equipment for image processing. Background Art

[0002] Image super-resolution refers to the reconstruction of a low-resolution image to obtain a high-resolution image.

[0003] At present, low-resolution images can be super-resolved by neural networks to obtain high-resolution images. For example, super-resolution convolutional neural networks (SRCNN), enhanced super-resolution generative adversarial networks (ESRGAN), and very deep networks for super-resolution (VDSR) can be used to super-resolve low-resolution images to obtain high-resolution images. However, the above neural networks can only perform good super-resolution on images of some texture types, for example, images of building types can be well super-resolved, and the above neural networks cannot perform good super-resolution on face images. That is, the reliability of existing image super-resolution methods for image processing is low. Summary of the invention

[0004] The present application provides an image processing method, apparatus and device, which improve the reliability of image processing.

[0005] In a first aspect, an embodiment of the present application provides an image processing method. When it is necessary to obtain a super-resolution image of a first image, an image filter corresponding to the first image can be determined, and the first image can be super-resolution processed according to the image filter of the first image to obtain a super-resolution image of the first image. The image filter has filter parameters corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters. In other words, the determined image filter has different filter parameters corresponding to pixels with different texture features.

[0006] In the above process, since the determined image filter includes filter parameters corresponding to each pixel in the first image, and the filter parameters corresponding to pixels with different texture features in the first image are different, when the first image is super-resolution processed by the image filter, different super-resolution processing can be performed on pixels with different texture features in the first image, so that the super-resolution processing performed on each pixel in the image is related to the texture characteristics of the pixel itself, so that the effect of super-resolution processing on the image is better and the reliability of image processing is improved.

[0007] In a possible implementation, the image filter corresponding to the first image can be determined by the following feasible implementation method: obtaining a texture image of the first image; determining C local texture images and a weight value of each local texture image in the texture image according to the texture features of each pixel in the texture image, and determining the image filter according to the C local texture images and the weight value of each local texture image. Wherein, the texture features of the textures in the local texture images are the same, the number of channels of the image filter is C, and C=f 2 ×t, f is a preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1. In other possible implementations, the preset magnification may also be changed to other factors, which may be specifically determined according to the requirements of super-resolution.

[0008] In one possible implementation, the (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image; wherein, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are integers greater than 1 respectively.

[0009] In the above process, a texture image of the first image is first obtained, multiple local images and weight values ​​of each local image are determined in the texture image, and an image filter is determined according to the local texture image and the features of the local texture image. Since the texture features of the textures in the local texture images are the same, in the image filter obtained by the above method, the first image has filter parameters corresponding to each pixel in the first image, and the filter parameters corresponding to pixels with different texture features are different. When the first image is super-resolution processed by the image filter, different super-resolution processes can be performed on different pixels in the first image according to the image filter, thereby improving the reliability of image processing.

[0010] In a possible implementation, the texture image of the first image may be obtained by the following feasible implementation: compressing the first image according to a preset size, the size of the compressed first image being the preset size; and determining the texture image based on the compressed first image.

[0011] In the above process, after the first image is compressed, the number of pixels included in the compressed first image is smaller, so that the number of pixels in the determined texture image is also smaller. In this way, in the subsequent process of processing the texture image, the amount of data processing can be reduced, thereby improving the image processing efficiency.

[0012] In a possible implementation, the image filter corresponding to the first image can be determined by the following feasible implementation method: the first image is processed by a recognition model to obtain the image filter of the first image, the recognition model is learned from multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content in the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

[0013] The recognition model can be learned by the following feasible implementation method: multiple groups of sample images can be input into the recognition model, and the recognition model can learn the multiple groups of sample images. Since the multiple groups of sample images include sample images with multiple texture characteristics, the recognition model can learn: in the process of super-resolution of sample low-resolution images (second sample images) with various texture characteristics to sample high-resolution images (first sample images), the sample image filter (or filter parameter) corresponding to the sample low-resolution image. After the recognition model learns the multiple groups of sample images, the recognition model has the function of determining the image filter of the image. Among them, only when the filter parameters corresponding to the pixels with different texture characteristics in the image filter are different, the high-resolution image obtained by super-resolution of the image filter is a high-resolution image with a better super-resolution effect. Since the sample high-resolution image in each group of sample images is a high-resolution image with a better super-resolution effect, after the recognition model is learned according to the above multiple groups of sample images, the recognition model has the function of outputting the image filter of the image, and the output image filter has the following characteristics: the image filter has a filter parameter corresponding to each pixel in the image, and the filter parameters corresponding to the pixels with different texture characteristics are different.

[0014] In the above process, the recognition model is learned in advance, and the image filter of the first image can be obtained through the recognition model. By training the recognition model, the recognition model can learn the filter parameters corresponding to images with various texture features from low-resolution images to high-resolution images. Therefore, in the image filter output by the recognition model, the filter parameters of the image filter corresponding to pixels with different texture features in the first image are different, and different super-resolution processing can be performed on different pixels in the first image according to the image filter, thereby improving the reliability of image processing.

[0015] In a possible implementation, the first image can be processed by the recognition model in the following feasible implementation manner: compressing the first image according to a preset size, and the size of the compressed first image is the preset size; processing the compressed first image by the recognition model.

[0016] In the above process, after the first image is compressed, the number of pixels included in the compressed first image is smaller, so that the recognition model can process fewer pixels to obtain the image filter of the first image, thereby improving the efficiency of the recognition model in determining the image filter of the first image.

[0017] In a possible implementation, the first image can be super-resolution processed by the following feasible implementation method: obtain a gradient image of the first image, the gradient image has the same size as the first image, the size of the gradient image is M*N, and M and N are integers greater than 1 respectively; process the gradient image through an image filter to obtain a second image, and the size of the second image is (f*N)*(f*N); enlarge the first image by f times to obtain a third image, and the size of the third image is (f*N)*(f*N); obtain a super-resolution image based on the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

[0018] In the above process, the gradient image of the first image is first processed by an image filter to obtain a second image, and then a super-resolution image is obtained based on the second image and a third image that is amplified from the first image. The data volume of the gradient image is smaller than that of the first image, so the gradient image can be quickly processed by the image filter, and then the super-resolution image can be quickly determined.

[0019] In a possible implementation, the gradient image is processed by an image filter to obtain a second image, including: processing the gradient image by an image filter to obtain f 2 sub-images, each sub-image has a size of (f*N)*(f*N); according to f 2 The sub-images determine the second image.

[0020] In the above process, different sub-images can represent fine texture characteristics of different regions in the first image. Therefore, the determined second image can represent the fine texture characteristics of the first image.

[0021] In a possible implementation, the gradient image is processed by an image filter of the first image to obtain f 2 The method comprises: processing the gradient image by using the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

[0022] In the above process, since each sub-image is obtained based on the parameters in different channels of the image filter, and the parameters in different channels correspond to the texture characteristics of different regions in the first image, each sub-image can represent the fine texture features of different regions in the first image.

[0023] In one possible implementation, according to f 2 The second image is determined by performing the following operations: 2 The pixels at the same pixel position in the first sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f; according to the pixel position of the pixels in each image block in the sub-image, the M*N image blocks are spliced ​​to obtain the second image.

[0024] In the above process, since each sub-image can represent the fine texture features of different areas in the first image, the second image can be obtained by splicing through the above method, which can not only increase the size of the second image, but also enable the second image to represent the fine texture features of the first image.

[0025] In a second aspect, an embodiment of the present application provides an image processing device, including: a determination module and a super-resolution module, wherein:

[0026] The determination module is used to determine an image filter corresponding to the first image, wherein the image filter has filter parameters corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters corresponding to them;

[0027] The super-resolution module is used to perform super-resolution processing on the first image according to the image filter of the first image to obtain a super-resolution image of the first image.

[0028] In a possible implementation manner, the determining module is specifically configured to:

[0029] Acquire a texture image of the first image;

[0030] According to the texture features of each pixel in the texture image, C local texture images and a weight value of each local texture image are determined in the texture image, and the texture features of the textures in the local texture images are the same; C=f 2 ×t, f is the preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1;

[0031] The image filter is determined according to the C local texture images and the weight value of each local texture image, and the number of channels of the image filter is C.

[0032] In a possible implementation, the (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image;

[0033] Among them, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are respectively integers greater than 1.

[0034] In a possible implementation manner, the determining module is specifically configured to:

[0035] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0036] The texture image is determined according to the first image after compression.

[0037] In a possible implementation manner, the determining module is specifically configured to:

[0038] The first image is processed by a recognition model to obtain an image filter of the first image, wherein the recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content of the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

[0039] In a possible implementation manner, the determining module is specifically configured to:

[0040] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0041] The first compressed image is processed by using a recognition model.

[0042] In a possible implementation manner, the super-resolution module is specifically used for:

[0043] Acquire a gradient image of the first image, wherein the gradient image has the same size as the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively;

[0044] Processing the gradient image by the image filter to obtain a second image, wherein the size of the second image is (f*N)*(f*N), wherein f is a preset magnification;

[0045] Enlarging the first image by f times to obtain a third image, wherein the size of the third image is (f*N)*(f*N);

[0046] The super-resolution image is obtained according to the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

[0047] In a possible implementation manner, the super-resolution module is specifically used for:

[0048] The gradient image is processed by the image filter to obtain f 2 sub-images, each sub-image has a size of (f*N)*(f*N);

[0049] According to the f 2 The second image is determined by using a plurality of sub-images.

[0050] In a possible implementation manner, the super-resolution module is specifically used for:

[0051] The gradient image is processed by the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

[0052] In a possible implementation manner, the super-resolution module is specifically used for:

[0053] For the f 2 The pixels at the same pixel position in the sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f;

[0054] The M*N image blocks are spliced ​​according to the pixel positions of the pixels in each image block in the sub-image to obtain the second image.

[0055] In a third aspect, an embodiment of the present application provides a computer system, including: a memory, a processor, and a computer program, wherein the computer program is stored in the memory, and the processor runs the computer program and performs the following steps:

[0056] Determine an image filter corresponding to the first image, wherein the image filter has a filter parameter corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters;

[0057] The first image is super-resolved according to the image filter of the first image to obtain a super-resolved image of the first image.

[0058] In a possible implementation manner, the processor is specifically configured to:

[0059] Acquire a texture image of the first image;

[0060] According to the texture features of each pixel in the texture image, C local texture images and a weight value of each local texture image are determined in the texture image, and the texture features of the textures in the local texture images are the same; C=f 2 ×t, f is the preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1;

[0061] The image filter is determined according to the C local texture images and the weight value of each local texture image, and the number of channels of the image filter is C.

[0062] In a possible implementation, the (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image;

[0063] Among them, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are respectively integers greater than 1.

[0064] In a possible implementation manner, the processor is specifically configured to:

[0065] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0066] The texture image is determined according to the first image after compression.

[0067] In a possible implementation manner, the processor is specifically configured to:

[0068] The first image is processed by a recognition model to obtain an image filter of the first image, wherein the recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content of the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

[0069] In a possible implementation manner, the processor is specifically configured to:

[0070] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0071] The first compressed image is processed by using a recognition model.

[0072] In a possible implementation manner, the processor is specifically configured to:

[0073] Acquire a gradient image of the first image, wherein the gradient image has the same size as the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively;

[0074] Processing the gradient image by the image filter to obtain a second image, wherein the size of the second image is (f*N)*(f*N), wherein f is a preset magnification;

[0075] Enlarging the first image by f times to obtain a third image, wherein the size of the third image is (f*N)*(f*N);

[0076] The super-resolution image is obtained according to the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

[0077] In a possible implementation manner, the processor is specifically configured to:

[0078] The gradient image is processed by the image filter to obtain f 2 sub-images, each sub-image has a size of (f*N)*(f*N);

[0079] According to the f 2 The second image is determined by using a plurality of sub-images.

[0080] In a possible implementation manner, the processor is specifically configured to:

[0081] The gradient image is processed by the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

[0082] In a possible implementation manner, the processor is specifically configured to:

[0083] For the f 2 The pixels at the same pixel position in the sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f;

[0084] The M*N image blocks are spliced ​​according to the pixel positions of the pixels in each image block in the sub-image to obtain the second image.

[0085] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a computer program, and the computer program is used to implement the image processing method as described in any one of the first aspects.

[0086] In a fifth aspect, an embodiment of the present application further provides a chip or integrated circuit, comprising: an interface circuit and a processor. The processor is used to call program instructions through the interface circuit to implement the image processing method as described in any one of the first aspects.

[0087] In a sixth aspect, an embodiment of the present application further provides a computer program or a computer program product, wherein the computer program or the computer program product comprises computer-readable instructions, and wherein the computer-readable instructions, when read by one or more processors, implement the image processing method described in any one of the first aspects.

[0088] The image processing method, device and equipment provided by the embodiments of the present application can first determine the image filter of the first image when it is necessary to obtain a super-resolution image of the first image, and then perform super-resolution processing on the first image through the image filter to obtain the super-resolution image of the first image. Among them, the image filter has filter parameters corresponding to each pixel in the first image, and the filter parameters corresponding to pixels with different texture features are different. In this way, when the first image is super-resolution processed by the image filter, different super-resolution processing can be performed on the pixels with different texture features in the first image, so that the super-resolution processing performed on each pixel in the image is related to the texture characteristics of the pixel itself, so that the effect of super-resolution processing on the image is better, and the reliability of image processing is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] Figure 1 A schematic diagram of a super-resolution image provided in an embodiment of the present application;

[0090] Figure 2 A schematic diagram of a texture image provided in an embodiment of the present application;

[0091] Figure 3 A schematic diagram of a gradient image provided in an embodiment of the present application;

[0092] Figure 4 A schematic diagram of an image filter provided in an embodiment of the present application;

[0093] Figure 5 A schematic diagram of a flow chart of a method for determining an image filter provided in an embodiment of the present application;

[0094] Figure 6 A schematic diagram of a local texture image provided in an embodiment of the present application;

[0095] Figure 7 A schematic diagram of another image filter provided in an embodiment of the present application;

[0096] Figure 8 A schematic diagram of an image provided in an embodiment of the present application;

[0097] Fig. 9 A schematic diagram of a flow chart of another method for determining an image filter provided in an embodiment of the present application;

[0098] Fig.10 A collection of sample images provided for the embodiments of the present application;

[0099] Fig.11 A schematic diagram of a flow chart of a method for obtaining a sample image set provided in an embodiment of the present application;

[0100] Fig.12 A schematic flow chart of a method for super-resolution processing of a first image by an image filter provided in an embodiment of the present application;

[0101] Fig.13 A schematic diagram of processing pixels by using filter parameters provided in an embodiment of the present application;

[0102] Fig.14 A schematic diagram of another method of processing pixels by using filter parameters provided in an embodiment of the present application;

[0103] Fig.15 A schematic diagram of a process for determining an image block provided in an embodiment of the present application;

[0104] Fig.16 A schematic diagram of the process of stitching image blocks provided in an embodiment of the present application;

[0105] Fig.17 A schematic diagram of the process of an image processing method provided in an embodiment of the present application;

[0106] Fig.18 A schematic diagram of the image processing process provided by an embodiment of the present application;

[0107] Fig.19 A schematic diagram of an application scenario provided for an embodiment of the present application;

[0108] Fig. 20 A schematic diagram of the generation and use process of the processing model provided in the embodiment of the present application;

[0109] Fig.21 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0110] Fig. 22 A schematic diagram of the hardware structure of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0111] To facilitate understanding, first, the concepts involved in this application are explained.

[0112] Image super-resolution: It can also be called image super-resolution, which refers to the process of reconstructing a low-resolution image to obtain a high-resolution image. Figure 1 , image super-resolution is explained. Figure 1 This is a schematic diagram of a super-resolution image provided in an embodiment of the present application. Figure 1 , including the original image and the super-resolution image, wherein the original image is super-resolution processed to obtain the super-resolution image, and the image content in the original image and the super-resolution image is the same. For example, assuming that the resolution of the original image is a*b and the magnification is 3, the resolution of the super-resolution image is 3a*3b. In the resolution a*b of the image, a refers to the number of pixels included in the image in the horizontal direction, and b refers to the number of pixels included in the image in the vertical direction. The resolution of an image can also be called the size of the image.

[0113] Texture: It can also be called lines, which refers to the patterns or lines on the surface of an object.

[0114] Texture image: refers to an image that includes the texture of the original image. Figure 2 , describe the texture image. Figure 2 A schematic diagram of a texture image provided in an embodiment of the present application. Figure 2 , including the original image and the texture image, the texture image includes the texture in the original image. It should be noted that Figure 2It is just a schematic illustration of a type of texture image in the original image. For example, the type of texture image may include a local binary patterns (LBP) type, a Gaussian Markov random field (GMRF) type, a gray-level co-occurrence matrix (GLCM) type, etc.

[0115] Texture features: refers to the characteristics of texture. Different objects in an image have different texture features. For example, see Figure 2 , the texture features of the sky, the texture features of the building, the texture features of the hair, the texture features of the face, the texture features of the clothes, the texture features of the grass, etc. are all different.

[0116] Gradient image: If the original image is regarded as a two-dimensional discrete function, then the gradient image is the image obtained by taking the derivative of the two-dimensional discrete function. Figure 3 , the gradient image is explained. Figure 3 This is a schematic diagram of a gradient image provided in an embodiment of the present application. Figure 3 , including the original image and the gradient image. The original image is usually three-channel, and the gradient image is usually one-channel. Therefore, processing the gradient image of the original image can reduce the amount of calculation. For example, the amount of calculation for the gradient image is one third of the amount of calculation for the original image.

[0117] Image filter: can process images to improve the resolution of the image. The image filter involved in this application is a three-dimensional filter, and the three dimensions can be denoted as H, W and C respectively. For ease of understanding, below, combined with Figure 4 The image filter will be described. Figure 4 This is a schematic diagram of an image filter provided in an embodiment of the present application. Figure 4 , the image filter is a three-dimensional filter, the horizontal size of the image filter is W, the vertical size is H, and the number of channels is C. Among them, Figure 1 Each square in represents a filter parameter, and the image filter includes H*W*C filter parameters. The essence of processing an image through an image filter is to process the image through the filter parameters in the image filter.

[0118] Downsampling: refers to the compression process. For example, downsampling an image refers to compressing the image.

[0119] Upsampling: refers to the process of enlarging. For example, upsampling an image means enlarging the image.

[0120] In the present application, when it is necessary to obtain a super-resolution image of the first image, the image filter of the first image can be determined first, and then the first image can be super-resolution processed through the image filter to obtain the super-resolution image of the first image. Among them, the image filter includes filter parameters corresponding to each pixel in the first image, and the filter parameters corresponding to pixels with different texture features in the first image are different. In this way, when the first image is super-resolution processed through the image filter, different super-resolution processing can be performed on pixels with different texture features in the first image, so that the super-resolution processing performed on each pixel in the image is related to the texture characteristics of the pixel itself, so that the effect of super-resolution processing on the image is better, and the reliability of image processing is improved.

[0121] The technical solution shown in the present application is described in detail below through specific embodiments. It should be noted that the following embodiments can exist independently or in combination with each other, and the same or similar contents will not be described repeatedly in different embodiments.

[0122] For ease of understanding, two methods of determining the image filter corresponding to the first image are first introduced. Figure 5-Figure 8 The embodiment shown is a method of determining an image filter. Figure 9-11 The illustrated embodiment is another way to determine the image filter.

[0123] Figure 5 A flowchart of a method for determining an image filter provided in an embodiment of the present application. Figure 5 , the method may include:

[0124] S501: Acquire a texture image of a first image.

[0125] The first image is an image to be super-resolution processed, and the first image is usually an image with a lower resolution.

[0126] Optionally, the texture image of the first image may be acquired through a convolutional neural network. For example, the texture image of the first image may be acquired through any one of an LBP model, a GMRF model, or a GLCM model.

[0127] In order to improve the efficiency of obtaining the texture image of the first image, the first image may be first compressed to a preset size (the first image may be downsampled to obtain an image of the preset size), and then the texture image of the compressed first image may be obtained. Optionally, when the size of the first image is smaller than the preset size, the first image may not be compressed. For example, the preset size may be 256*256, 512*512, etc. In actual application, the preset size may be set according to actual needs. It should be noted that the size of the image involved in this application refers to the resolution of the image. For example, the size of the first image is M*N, which means that the first image includes M pixels in the horizontal direction and N pixels in the vertical direction.

[0128] The size of the texture image of the compressed first image is the same as the size of the compressed first image. For example, assuming that the size of the first image is M*N and the size of the compressed first image is M1*N1, the size of the texture image of the compressed first image is M1*N1. Since the number of pixels included in the compressed first image is reduced, the texture image of the compressed first image can be quickly acquired. Furthermore, the number of pixels included in the texture image of the compressed first image is also less, thereby reducing the amount of data for subsequent processing of the texture image.

[0129] S502: Determine C local texture images and a weight value of each local texture image in the texture image according to texture features of each pixel in the texture image.

[0130] Where C = f 2 ×t, f is the preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1.

[0131] Optionally, the size of t can be preset, t = a 2 , a is usually an odd number greater than or equal to 1, for example, a can be 1, 3, 5, 7, etc.

[0132] For example, suppose the first image needs to be magnified f=3 times, and suppose the number of filter parameters corresponding to each pixel is t=3 2 =9, then C=81.

[0133] Optionally, each local texture image includes a portion of the texture in the texture image, and the texture features of the textures in a local texture image are the same. For example, a local texture image only includes face textures, and the texture features of the face textures are the same, or a local texture image only includes sky textures, and the texture features of the sky textures are the same.

[0134] The size of the local texture image may be equal to the size of the texture image. If the first image is compressed to a preset size in S502, the size of the texture image and the size of the local texture image are both the preset size, and if the first image is not compressed in S502, the size of the texture image and the size of the local texture image are both the same as the size of the first image.

[0135] Next, combine Figure 6 , the process of determining the local texture image is explained.

[0136] Figure 6 This is a schematic diagram of a local texture image provided by an embodiment of the present application. Figure 6 , assuming that C is 5, then 5 local texture images can be determined in the texture image, which are recorded as: local texture image 1, local texture image 2, local texture image 3, local texture image 4 and local texture image 5.

[0137] See also Figure 6 , local texture image 1 is a local texture image corresponding to a building, and local texture image 1 includes a texture corresponding to the building. Local texture image 2 is a local texture image corresponding to the sky, and local texture image 2 includes a texture corresponding to the sky. Local texture image 3 is a local texture image corresponding to the grass, and local texture image 3 includes a texture corresponding to the grass. Local texture image 4 is a local texture image corresponding to a car, and local texture image 4 includes a texture corresponding to a car. Local texture image 5 is a local texture image corresponding to clouds, and local texture image 5 includes a texture corresponding to clouds.

[0138] It should be noted that Figure 6 The partial local texture images determined in the texture image are merely illustrated in the form of examples. Of course, other local texture images may also be determined in the texture image, and the embodiments of the present application do not specifically limit this.

[0139] A local texture image may correspond to a weight value. For example, the weight value of the local texture image may be a number between 0 and 1.

[0140] Optionally, the weight value of the local texture image can be determined according to the object type included in the local texture image. For example, the correspondence between the object type and the weight value can be preset, and accordingly, the weight value of the local texture image can be determined according to the object type in the local texture image and the correspondence. For example, the object type of the object in the local texture image can be identified, and the weight value of the local texture image can be determined according to the object type and the object relationship.

[0141] Optionally, the weight value of the local texture image can be determined based on the texture features of the texture in the local texture image. For example, the correspondence between the texture features and the weight value can be preset, and accordingly, the weight value of the local texture image can be determined based on the texture features of the texture in the local texture image and the correspondence.

[0142] Optionally, the weight value of each local texture image can also be determined by a preset model, and the preset model is obtained by learning multiple groups of sample images. Each group of sample images may include a sample low-resolution image and a corresponding sample high-resolution image. Multiple groups of sample images can be input into the preset model, and the preset model can be learned from the multiple groups of sample images to learn the weight value of the sample local texture image corresponding to each sample low-resolution image when super-resolving from the sample low-resolution image to the sample high-resolution image. After the preset model learns the multiple groups of sample images, the preset model may have the function of determining the weight value of each local texture image. Therefore, the local texture image can be input into the preset model so that the preset model outputs the weight value of the local texture image.

[0143] S503 . Determine an image filter according to the C local texture images and the weight value of each local texture image. The image filter is a three-dimensional image filter, and the number of channels of the image filter is C.

[0144] Optionally, each local texture image can be multiplied by the corresponding weight value to obtain an updated local texture image. Each updated local texture image corresponds to a channel of the image filter, and the pixel value in an updated local texture image is a filter parameter in a channel of the image filter. The lateral size (W) of the image filter is the same as the lateral size of the local texture image (the number of pixels included in each row of the local texture image), and the longitudinal size (H) of the image filter is the same as the longitudinal size of the local texture image (the number of pixels included in each column of the local texture image).

[0145] The (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image. Wherein, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are integers greater than 1 respectively.

[0146] Next, combine Figure 7 , the image filter is described.

[0147] Figure 7 This is a schematic diagram of another image filter provided in an embodiment of the present application. Figure 7, assuming C = 5, the five local texture images are recorded as: local texture image 1, local texture image 2, local texture image 3, local texture image 4 and local texture image 5. After multiplying the pixel values ​​of the five local texture images by the corresponding weight values, the updated local texture images can be obtained according to Figure 7 The pixel values ​​of the five updated local texture images are arranged in the manner shown, and the pixel value in each updated local texture image is a filter parameter of a channel in the image filter.

[0148] It should be noted that Figure 7 It only illustrates the relationship between the local texture image and the image filter, and does not represent the actual form of the image filter.

[0149] In the above process, by setting the weight value for each local texture image, the filter parameters of the determined image filter are related to the object type in the image. In this way, when the image is processed by the image filter, a smooth transition can be made between different areas in the processed image, avoiding the appearance of obvious dividing lines between different areas in the processed image.

[0150] Next, combine Figure 8 , the image processing effect after adopting the weight value of the local texture image is explained.

[0151] Figure 8 This is a schematic diagram of an image provided in an embodiment of the present application. Figure 8 , the original image is super-resolved by image filter 1 to obtain super-resolved image 1, and the original image is super-resolved by filter 2 to obtain super-resolved image 2. In the process of determining image filter 1, the weight value of the local texture image is not used, that is, the image filter 1 is directly determined based on the local texture image. In the process of determining image filter 2, the weight value of the local texture image is used, that is, the image filter 2 is determined based on the local texture image and the weight value of the local texture image.

[0152] See also Figure 8 In the super-resolution image 1, there is a relatively obvious dividing line between the sky in the upper left corner and the rest of the image, that is, the transition between the sky in the upper left corner and the rest of the image is not smooth.

[0153] See also Figure 8 In super-resolution image 2, there is no obvious dividing line between the sky in the upper left corner and the rest of the image, that is, the transition between the sky in the upper left corner and the rest of the image is smooth.

[0154] exist Figure 5In the illustrated embodiment, a texture image of the first image is first obtained, a plurality of local images and a weight value of each local image are determined in the texture image, and an image filter is determined based on the local texture image and the weight value of the local texture image. Since the texture features of the textures in a local texture image are the same, the image filter obtained by the above method has filter parameters corresponding to each pixel in the first image, and the filter parameters corresponding to pixels with different texture features are different. When the first image is super-resolution processed by the image filter, different super-resolution processes can be performed on different pixels in the first image according to the image filter, thereby improving the reliability of image processing.

[0155] Fig. 9 A flowchart of another method for determining an image filter provided in an embodiment of the present application. Fig. 9 , the method may include:

[0156] S901. Compress a first image according to a preset size, and the size of the compressed first image is the preset size.

[0157] Optionally, when the size of the first image is larger than a preset size, the first image may be compressed. When the size of the first image is smaller than the preset size, the first image may not be compressed.

[0158] For example, the preset size may be 256*256 / 512*512, etc. In actual application, the preset size may be set according to actual needs.

[0159] S902: Process the first image by using a recognition model to obtain an image filter of the first image.

[0160] It should be noted that S901 may be an optional step. When S901 is executed, the first image after compression is processed by the recognition model in S902, so that the data processing amount of the recognition model can be reduced, thereby improving the efficiency of the image filter output by the recognition model. When S901 is not executed, the original first image is processed in S902.

[0161] The recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content in the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

[0162] Optionally, the first sample image may be a high-resolution image, and the second sample image may be a low-resolution image. For example, the resolution of the first sample image is greater than or equal to a first threshold, and the resolution of the second sample image is less than or equal to a second threshold.

[0163] In execution Fig. 9 Before the embodiment shown, the recognition model can be learned in advance. The process of learning the recognition model is described below.

[0164] First, obtain sample images with different texture features to obtain a sample image set. For example, the sample image set can be as follows: Fig.10 shown. Fig.10 A collection of sample images provided for the embodiments of this application. Fig.10 ,The sample image set includes multiple animal sample images, multiple face sample images and multiple sky sample images.

[0165] Next, combine Fig.11 , a method for obtaining a set of sample images is introduced. Fig.11 This is a flow chart of the method for obtaining a sample image set provided in an embodiment of the present application. Fig.11 , you can first obtain a super-resolution dataset, a dataset captured by a search engine, or a dataset collected by yourself. Each dataset includes multiple initial sample images. The initial sample images in each dataset are input to a dataset processor, which processes the initial sample images in each dataset to obtain a sample image set.

[0166] Optionally, the initial sample images in each data set can be segmented (patch extraction) to extract the desired sample images from the initial sample images. For example, an initial sample image including a face, sky, and building is segmented to obtain a sample image including only a face, a sample image including only a sky, and a sample image including only a building. The sample images obtained by image segmentation are then rotated, stretched, scaled, and processed by the data augmentation module to obtain a sample image set. Through the above processing, the sample image set can include a rich set of sample images.

[0167] Optional, sample image set Dataset = {Texture info |Texture∈AllData}.

[0168] Among them, Texture is the texture feature, and Data is the texture feature set. info is an image with texture features Texture, for example, Texture info ={sky image|face image|animal image...}.

[0169] After obtaining the sample image set, multiple groups of sample images can be determined in the sample image set. For example, for a sample high-resolution image in the sample image set, the sample high-resolution image can be compressed to obtain a corresponding sample low-resolution image; or, for a sample low-resolution image in the sample image set, the sample low-resolution image can be processed to obtain a corresponding sample high-resolution image.

[0170] After determining to obtain multiple groups of sample images, the multiple groups of sample images can be input into the recognition model, and the recognition model can learn the multiple groups of sample images. Since the multiple groups of sample images include sample images with multiple texture characteristics, the recognition model can learn to obtain: in the process of super-resolution of sample low-resolution images (second sample images) with various texture characteristics to sample high-resolution images (first sample images), the sample image filters (or filter parameters) corresponding to the sample low-resolution images. After the recognition model learns the multiple groups of sample images, the recognition model has the function of determining the image filter of the image. Among them, only when the filter parameters corresponding to the pixels with different texture characteristics in the image filter are different, the high-resolution image obtained by super-resolution of the image filter is a high-resolution image with a better super-resolution effect. Since the sample high-resolution images in each group of sample images are high-resolution images with a better super-resolution effect, after the recognition model learns according to the above multiple groups of sample images, the recognition model has the function of outputting the image filter of the image, and the output image filter has the following characteristics: the image filter has filter parameters corresponding to each pixel in the image, and the filter parameters corresponding to the pixels with different texture characteristics are different.

[0171] Optionally, data representing the first image, a preset magnification factor, and the number t of filter parameters corresponding to each pixel may be input into the recognition model, and the recognition model outputs an image filter of the first image based on the received data. The data representing the first image may be the first image itself, or a grayscale image of the first image, etc. For example, the recognition model may determine the number of channels C of the image filter based on the preset magnification factor f and the number t of filter parameters corresponding to each pixel, and determine the image filter based on the number of channels C and the data representing the first image. The number of channels of the image filter output by the recognition model is C.

[0172] Optional, Fig. 9 The process shown in the embodiment can be described as: DTF (H,W,C) =Conv(resize(Input) (H,W) ), where Input is the original image input, resize(Input) (H,W) It means that the input original image is scaled to size H*W, Conv represents the recognition model (convolutional neural network), DTF(H,W,C) is the image filter output by the recognition model, H is the longitudinal size of the image filter, W is the lateral size of the image filter, and C is the number of channels of the image filter.

[0173] exist Fig. 9 In the embodiment shown, a recognition model is learned in advance, and an image filter of the first image can be obtained through the recognition model. By training the recognition model, the recognition model can learn filter parameters corresponding to images with various texture features from low-resolution images to high-resolution images. Therefore, in the image filter output by the recognition model, the filter parameters of the image filter corresponding to pixels with different texture features in the first image are different, and different super-resolution processing can be performed on different pixels in the first image according to the image filter, thereby improving the reliability of image processing.

[0174] After the image filter of the first image is obtained by any of the above methods, the first image may be super-resolution processed by the image filter to obtain a super-resolution image of the first image.

[0175] Next, through Figure 12-16 The illustrated embodiment introduces a process of super-resolution processing of a first image by using an image filter.

[0176] Fig.12 This is a flow chart of a method for super-resolution processing of a first image by an image filter provided in an embodiment of the present application. Fig.12 ,include:

[0177] S1201. Obtain a gradient image of a first image.

[0178] The size of the gradient image is the same as that of the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively.

[0179] Optionally, the gradient image of the first image may be obtained through a convolutional neural network. For example, data representing the first image may be input into a convolutional neural network, and the output of the convolutional neural network is the gradient image of the first image.

[0180] Optionally, this step can be described as: GGF (H,W,1) =Conv(Input(H,W,3)), where Input (H,W,3) Refers to the first input image, which has 3 channels and is usually an RGB image or a YUV image. GGF (H,W,1) Refers to the output guided gradient image.

[0181] S1202, process the gradient image through an image filter to obtain f 2 Zhangzi image.

[0182] The size of each sub-image is (f*N)*(f*N).

[0183] Optionally, it is possible to first determine whether H*W (W is the number of filter parameters included in a channel in the horizontal direction, and H is the number of filter parameters included in a channel in the vertical direction) of the image filter is consistent with the size of the gradient image (determine whether W is the same as the number of pixels included in the gradient image in the horizontal direction, and whether H is the same as the number of pixels included in the gradient image in the vertical direction). If H*W of the image filter is inconsistent with the size of the gradient image, first adjust H*W of the image filter to the size of the gradient image, and the adjusted W of the image filter is the same as the number of pixels included in the gradient image in the horizontal direction, and the adjusted H of the image filter is the same as the number of pixels included in the gradient image in the vertical direction.

[0184] The gradient image is processed by the parameters in the k*t+1 to (k+1)*t channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

[0185] When acquiring each sub-image, first determine the corresponding channel in the image filter, and process the gradient image according to the filter parameters in the corresponding channel in the image filter (process each pixel in the gradient image) to obtain a sub-image. For example, the gradient image is processed by the filter parameters in the 1st to tth channels in the image filter to obtain the 0th sub-image. The gradient image is processed by the filter parameters in the t+1th to 2tth channels in the image filter to obtain the 1st sub-image. The gradient image is processed by the filter parameters in the 2t+1th to 3tth channels in the image filter to obtain the 2nd sub-image. And so on, until f is obtained. 2 Zhangzi image.

[0186] For any sub-image, in the process of obtaining the sub-image, the filter parameters corresponding to each pixel in the gradient image are first determined, and the pixel is processed according to the corresponding filter parameters to obtain the pixel value of the corresponding pixel in the sub-image. The filter parameters corresponding to the pixels in the gradient image can be determined by the following two feasible implementation methods:

[0187] A feasible implementation method is: if the H*W size of the image filter is the same as the size of the gradient image.

[0188] In this feasible implementation, the coordinates of the pixels in the gradient image correspond to the (h, w) coordinates of the corresponding filter parameters, where h is a positive integer between 0 and H-1, and w is a positive integer between 0 and W-1.

[0189] For example, the filter parameters h and w corresponding to the pixel (0,0) in the gradient image are 0 and 0, respectively. The filter parameters h and w corresponding to the pixel (1,2) in the gradient image are 1 and 2, respectively.

[0190] Another feasible implementation is that the H*W size of the image filter is different from the size of the gradient image.

[0191] In this feasible implementation, the correspondence between the coordinates of the pixels in the gradient image and the filter parameters (h, w) can be determined according to the ratio of H*W of the image filter to the size of the gradient image.

[0192] For example, if W is half of the horizontal size of the gradient image and H is half of the vertical coordinate of the gradient image, then the filter parameters h and w corresponding to the pixels (0,0), (0,1), (1,0), and (1,1) in the gradient image are 0 and 0 respectively.

[0193] Among them, each pixel in the gradient image corresponds to f filter parameters. There are many ways to process a pixel through f filter parameters. Figure 13-14 , introduces a process of processing pixels in a gradient image through filter parameters.

[0194] Fig.13 A schematic diagram of processing pixels through filter parameters provided in an embodiment of the present application. Fig.14 Another schematic diagram of processing pixels through filter parameters provided in an embodiment of the present application.

[0195] See also Figure 13-14 , assuming that the magnification factor f = 3, the number of filter parameters corresponding to one pixel is 9, then the number of channels C of the image filter is 81 ( Fig.13 Only some channels are shown in the figure). When acquiring the 0th sub-image, the gradient image is processed according to the filter parameters in the 1-9 channels in the image filter.

[0196] See also Fig.13 , for the pixel (0,0) in the gradient image, the filter parameters corresponding to the pixel (0,0) are the 9 filter parameters of the image filter with h=0, w=0, C=1 to 9. Assuming that the pixel value of the pixel (0,0) in the gradient image is 0.5, the matrix formed by the 9 filter parameters corresponding to the pixel (0,0) is Since there are no pixels around the pixel (0,0), we can fill the pixel (0,0) with pixels 0, place the center (0.2) of the matrix opposite the pixel (0,0), multiply the elements at the corresponding positions, add them together, and take the average value to get the pixel value at the pixel (0,0).

[0197] For the pixels at the edge of the gradient image, Fig.13 The pixels are processed in the manner shown.

[0198] See also Fig.14 , for the pixel (1,1) in the gradient image, the filter parameters corresponding to the pixel (1,1) are the 9 filter parameters of the image filter with h=1, w=1, C=1 to 9. Assuming that the pixel value of the pixel (1,1) in the gradient image is 0.5, the matrix formed by the 9 filter parameters corresponding to the pixel (1,1) is The center (0.1) of the matrix is ​​aligned with the pixel (1,1), the elements at the corresponding positions are multiplied and added, and then the average is taken to obtain the pixel value at the pixel (1,1).

[0199] For non-edge pixels in the gradient image, Fig.14 The pixels are processed in the manner shown.

[0200] It should be noted that Figure 13-14 The process of processing pixels by using filter parameters is merely illustrated in a schematic form, and is not intended to limit the processing of pixels by using filter parameters.

[0201] S1203, respectively 2 The pixels at the same pixel position in each sub-image are spliced ​​together to obtain M*N image blocks.

[0202] The size of each image block is f*f.

[0203] Optional, for f 2 The coordinate f of the sub-image is (0,0) 2 Pixels are stitched together to obtain an image block corresponding to the coordinate (0,0). 2 The coordinate f of the sub-image is (0,1) 2 Pixels are stitched together to obtain an image block corresponding to the coordinate (0,1). And so on, M*N image blocks are obtained.

[0204] Next, combine Fig.15 , the process of determining the image block is explained.

[0205] Fig.15 A schematic diagram of the process of determining an image block provided in an embodiment of the present application. Fig.15, assuming f = 3, we can determine that 9 sub-images are obtained, which are respectively recorded as sub-image 1, sub-image 2, ..., sub-image 9. The 9 pixels at the coordinate (0,0) of the 9 sub-images are spliced ​​to obtain the image block 1 corresponding to the coordinate (0,0), the 9 pixels at the coordinate (0,1) of the 9 sub-images are spliced ​​to obtain the image block 2 corresponding to the coordinate (0,1), ..., the 9 pixels at the coordinate (2,2) of the 9 sub-images are spliced ​​to obtain the image block 9 corresponding to the coordinate (2,2).

[0206] It should be noted that Fig.15 The numbers in the sub-images and image blocks shown are pixel numbers. For example, "11" in sub-image 1 is the number of pixel (0,0) in sub-image 1, and "12" in sub-image 1 is the number of pixel (0,1) in sub-image 1. "11" in image block 1 is the number of pixel (0,0) in image block 1, and "21" in image block 11 is the number of pixel (0,1) in image block 1.

[0207] S1204 . Perform splicing processing on the M*N image blocks according to the pixel positions of the pixels in each image block in the sub-image to obtain a second image.

[0208] The size of the second image is (f*N)*(f*N).

[0209] Optionally, the placement position of each image block can be determined according to the pixel position of the pixel in each image block in the sub-image, and the M*N image blocks can be spliced ​​according to the placement position of each image block. The placement position of each image block corresponds to the pixel position of the pixel in the image block in the sub-image.

[0210] For example, if the pixel position of the pixel in the image block 1 in the sub-image is (0, 0), and the pixel position of the pixel in the image block 2 in the sub-image is (0, 1), then the placement position of the image block 2 is located after the image block 1.

[0211] Next, combine Fig.16 , the process of stitching image blocks is explained.

[0212] Fig.16 A schematic diagram of the process of stitching image blocks provided in the embodiment of the present application. Fig.16 , assuming that 9 image blocks are obtained, which are respectively recorded as image block 1, image block 2, ..., image block 9. Assume that the pixel position of each pixel in the 9 image blocks is as follows Fig.15 As shown, according to the pixel position of each pixel in the image block in the sub-image, the 9 image blocks are spliced ​​to obtain Fig.16 The second image is shown.

[0213] It should be noted that Fig.16 The numbers in the image blocks and the second image shown are pixel numbers, for example, "11" in image block 1 is the number of pixel (0,0) in image block 1, and "21" in image block 11 is the number of pixel (0,1) in image block 1. "11" in the second image is the number of pixel (0,0) in the second image, and "21" in the second image is the number of pixel (0,1) in the second image.

[0214] Optionally, S1202-S1204 can be described as: DTFS (H,W,C) =Conv(GGF (H,W,1) ,DTF (H,W,C)) ), where GGF (H,W,1) Represents gradient image, DTF (H,W,C)) Represents image filter; DTFS (H,W,C) Feature map representing the texture fine-tuning after fusing the gradient image with the image filter.

[0215] Through the steps shown in S1202-S1204, the second image can be determined based on the image filter and the gradient image. Of course, the second image can also be determined by other feasible implementation methods, which are not specifically limited in the embodiment of the present application.

[0216] S1205 , magnify the first image by f times to obtain a third image.

[0217] The size of the third image is (f*N)*(f*N).

[0218] Optionally, the first image may be magnified by f times by bicubic difference magnification to obtain the third image. Of course, the first image may also be magnified by other feasible implementation methods, which are not specifically limited in the embodiment of the present application.

[0219] S1206: Obtain a super-resolution image based on the first image and the second image.

[0220] The size of the super-resolution image is (f*N)*(f*N).

[0221] Optionally, the pixel values ​​in the third image and the pixel values ​​in the second image may be added correspondingly to obtain a super-resolution image.

[0222] For example, the pixel value at the pixel position (0,0) in the third image and the pixel value at the pixel position (0,0) in the second image may be added to obtain the pixel value at the pixel position (0,0) in the super-resolution image.

[0223] exist Fig.12In the embodiment shown, after the image filter is determined, the gradient image of the first image is first processed by the image filter to obtain the second image, and then the super-resolution image is obtained based on the second image and the third image obtained by magnifying the first image. The data amount of the gradient image is smaller than the data amount of the first image, so the gradient image can be quickly processed by the image filter, and then the super-resolution image can be quickly determined.

[0224] Next, combine Fig.17 , an image processing method is introduced.

[0225] Fig.17 This is a flowchart of the image processing method provided in the embodiment of the present application. Fig.17 , the method may include:

[0226] S1701. Determine an image filter corresponding to a first image.

[0227] The image filters corresponding to pixels with different texture features in the first image have different filter parameters.

[0228] It should be noted that, Figure 5-Figure 8 The method shown in the embodiment determines the image filter, or, by Figure 9-11 The method shown determines the image filter and will not be described in detail here.

[0229] S1702. Perform super-resolution processing on the first image according to the image filter of the first image to obtain a super-resolution image of the first image.

[0230] It should be noted that, Figure 12-16 The method shown in the embodiment performs super-resolution processing on the first image to obtain a super-resolution image of the first image, which will not be described in detail here.

[0231] Optionally, this step can be expressed as: HR (H,W) =Conv(DTFS (H,W,C) ,LR (H / f,W / f) ), among which HR (H,W) For super-resolution images, DTFS (H,W,C) is the second image, LR (H / f,W / f) is the first image, H is the horizontal size of the first image, W is the vertical size of the first image, and f is the magnification.

[0232] The image processing method provided in the embodiment of the present application can first determine the image filter of the first image when it is necessary to obtain a super-resolution image of the first image, and then perform super-resolution processing on the first image through the image filter to obtain the super-resolution image of the first image. Among them, the filter parameters in the image filter have a corresponding relationship with the pixels in the first image, and the filter parameters of the image filter corresponding to the pixels with different texture features in the first image are different. In this way, when the first image is super-resolution processed by the image filter, different super-resolution processing can be performed on the pixels with different texture features in the first image, so that the super-resolution processing performed on each pixel in the image is related to the texture characteristics of the pixel itself, so that the effect of super-resolution processing on the image is better, and the reliability of image processing is improved.

[0233] Next, combine Fig.18 , the image processing process is explained.

[0234] Fig.18 This is a schematic diagram of the image processing process provided by the embodiment of the present application. Fig.18 When the original image (first image) needs to be super-resolved, the first image is first downsampled and Figure 5-Figure 8 The method shown in the embodiment determines the image filter, or, by Figure 9-11 The method shown in the embodiment determines the image filter. After the image filter is determined, the image filter is upsampled so that the H*W size of the upsampled image filter is the same as the size of the first image.

[0235] A gradient image of the first image is also obtained by Figure 12-16 The method shown in the embodiment processes the image filter and the gradient image to obtain a second image. The first image is also upsampled to obtain an enlarged image. Figure 12-16 The method shown in the embodiment processes the second image and the enlarged image to obtain a super-resolution image.

[0236] exist Fig.18 In the illustrated embodiment, the first image includes objects such as the sky, clouds, cars, buildings, and grass. In the process of super-resolution processing of the first image, different objects correspond to different filter coefficients, so that different super-resolution processing can be performed on different objects in the first image, thereby avoiding problems such as jagged edges, white edges, and blurring of some objects caused by performing the same super-resolution processing on all objects in the image, making the super-resolution image more realistic and natural, and improving the reliability of image processing.

[0237] The image processing method shown in the above embodiment can be applied to a variety of application scenarios. For example, the image processing method shown in the above embodiment can be applied to the display of images by a terminal device, that is, before the terminal device displays an image, the image is super-processed by the image processing method shown in the above embodiment, so that the image displayed by the terminal device is clearer and more natural. For example, the image processing method shown in the above embodiment can also be applied to the video call scenario of the terminal device, that is, during the process of the terminal device and other terminal devices communicating through video, the terminal device can receive video frames from other terminal devices, and the terminal device can super-process each frame of the video frame by the method shown in the above embodiment, so that the video picture displayed by the terminal device is clearer and more natural. For example, the image processing method shown in the above embodiment can also be applied to the scenario of the terminal device playing the video, that is, during the process of the terminal device playing the video, the terminal device can super-process each frame of the video frame by the method shown in the above embodiment, so that the video picture displayed by the terminal device is clearer and more natural. For example, the image processing method shown in the above embodiment can also be applied to the game scenario, that is, before the terminal device displays the game picture, the game picture can be super-processed by the method shown in the above embodiment, so that the game picture displayed by the terminal device is clearer and more natural.

[0238] Optionally, the terminal device involved in the embodiments of the present application may be a mobile phone, a computer, a television, a vehicle-mounted terminal (or an unmanned driving system), an augmented reality (AR) device, a virtual reality (VR) device, a mixed reality device, a wearable device, a smart home device, a drone terminal device, etc.

[0239] Of course, the image processing method shown in the above embodiment can also be applied to other application scenarios, and the embodiments of the present application do not specifically limit this.

[0240] Next, combine Fig.19 , introduces an application scenario of an image processing method.

[0241] Fig.19 This is a schematic diagram of an application scenario provided by the embodiment of this application. Fig.19 , an application is installed in the terminal device, for example, the application may be an image display application, a video playback application, a video call application, etc. The application may obtain a media stream (for example, a video stream), a decoder in the terminal device decodes the media stream to obtain an original image, the terminal device performs super-resolution processing on the original image through the image processing method shown in the above embodiment to obtain a super-resolution image, and the terminal device displays the super-resolution image.

[0242] Optionally, when the media stream is a video stream, when the video frames are processed by the image processing method shown in the embodiment of the present application, since the content (or scene) in the video usually does not change in a short period of time, in order to improve processing efficiency, the image filter can be determined once every T frames, that is, the image filter determined based on a frame of image can be applied to the T-1 frame image after the frame, for example, T can be any number between 5 and 10.

[0243] Optionally, the image processing method shown in the above embodiment can be implemented by a processing model. For example, sample data can be obtained, and the sample data can be trained to obtain a processing model, and the processing model can implement the above image processing method. Optionally, in actual application, the above processing model can be trained on a personal computer (PC) according to the sample data, and the above processing model can be converted into an offline model on the computer, and the offline model can be moved to any other terminal device (for example, a mobile device such as a mobile phone or a tablet computer) so that the terminal device can perform image processing through the processing model.

[0244] It should be noted that the training process of the processing model can refer to the training process of the recognition model in the above embodiment, which will not be repeated here.

[0245] Next, combine Fig. 20 The process of generating and using the processing model is explained.

[0246] Fig. 20 This is a schematic diagram of the generation and use of the processing model provided in the embodiment of the present application. Fig. 20 , a processing model can be trained on a PC according to sample data, and the processing model can be converted into an offline model on the computer. The offline model is installed on the mobile terminal. For example, after the application in the mobile terminal obtains the media stream (for example, a video stream), the decoder in the terminal device decodes the media stream to obtain the original image, and the terminal device performs super-resolution processing on the original image through the offline processing model to obtain a super-resolution image, and the terminal device displays the super-resolution image.

[0247] Fig.21 This is a schematic diagram of the structure of the image processing device provided in the embodiment of the present application. Fig.21 The image processing device 10 may include: a determination module 11 and a super-resolution module 12, wherein:

[0248] The determination module 11 is used to determine an image filter corresponding to the first image, wherein the image filter has filter parameters corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters.

[0249] The super-resolution module 12 is used to perform super-resolution processing on the first image according to the image filter of the first image to obtain a super-resolution image of the first image.

[0250] Optionally, the determination module 11 may execute the above Figure 5 S501-S503 in the embodiment, Fig. 9 S901-S902 in the embodiment, and Fig.17 S1701 in the embodiment.

[0251] Optionally, the super-resolution module can perform the above Fig.12 S1201-S1206 in the embodiment, and Fig.17 S1702 in the embodiment.

[0252] It should be noted that the image processing device shown in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0253] In a possible implementation manner, the determining module 11 is specifically configured to:

[0254] Acquire a texture image of the first image;

[0255] According to the texture features of each pixel in the texture image, C local texture images and a weight value of each local texture image are determined in the texture image, and the texture features of the textures in the local texture images are the same; C=f 2 ×t, f is the preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1;

[0256] The image filter is determined according to the C local texture images and the weight value of each local texture image, and the number of channels of the image filter is C.

[0257] In a possible implementation manner, the determining module 11 is specifically configured to:

[0258] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0259] The texture image is determined according to the first image after compression.

[0260] In a possible implementation, the (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image;

[0261] Among them, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are respectively integers greater than 1.

[0262] In a possible implementation manner, the determining module 11 is specifically configured to:

[0263] The first image is processed by a recognition model to obtain an image filter of the first image, wherein the recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content of the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

[0264] In a possible implementation manner, the determining module 11 is specifically configured to:

[0265] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0266] The first compressed image is processed by using a recognition model.

[0267] In a possible implementation manner, the super-resolution module 12 is specifically used for:

[0268] Acquire a gradient image of the first image, wherein the gradient image has the same size as the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively;

[0269] Processing the gradient image by the image filter to obtain a second image, wherein the size of the second image is (f*N)*(f*N), wherein f is a preset magnification;

[0270] Enlarging the first image by f times to obtain a third image, wherein the size of the third image is (f*N)*(f*N);

[0271] The super-resolution image is obtained according to the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

[0272] In a possible implementation manner, the super-resolution module 12 is specifically used for:

[0273] The gradient image is processed by the image filter to obtain f 2sub-images, each sub-image has a size of (f*N)*(f*N);

[0274] According to the f 2 The second image is determined by using a plurality of sub-images.

[0275] In a possible implementation manner, the super-resolution module 12 is specifically used for:

[0276] The gradient image is processed by the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

[0277] In a possible implementation manner, the super-resolution module 12 is specifically used for:

[0278] For the f 2 The pixels at the same pixel position in the sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f;

[0279] The M*N image blocks are spliced ​​according to the pixel positions of the pixels in each image block in the sub-image to obtain the second image.

[0280] It should be noted that the image processing device shown in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0281] Fig. 22 This is a schematic diagram of the hardware structure of the computer system provided in the embodiment of the present application. Fig. 22 The computer system 20 includes: a memory 21 and a processor 22, wherein the memory 21 and the processor 22 communicate with each other; illustratively, the memory 21 and the processor 22 can communicate with each other via a communication bus 23, the memory 21 is used to store a computer program, and the processor 22 executes the computer program to perform the following steps:

[0282] Determine an image filter corresponding to the first image, wherein the image filter has a filter parameter corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters;

[0283] The first image is super-resolved according to the image filter of the first image to obtain a super-resolved image of the first image.

[0284] Optionally, the processor 21 may implement Fig.21 The functions of the determination module 11 and the super-resolution module 12 shown in the embodiment.

[0285] It should be noted that the image processing device shown in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0286] In a possible implementation manner, the processor 22 is specifically configured to:

[0287] Acquire a texture image of the first image;

[0288] According to the texture features of each pixel in the texture image, C local texture images and a weight value of each local texture image are determined in the texture image, and the texture features of the textures in the local texture images are the same; C=f 2 ×t, f is the preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, and t is an integer greater than or equal to 1;

[0289] The image filter is determined according to the C local texture images and the weight value of each local texture image, and the number of channels of the image filter is C.

[0290] In a possible implementation manner, the processor 22 is specifically configured to:

[0291] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0292] The texture image is determined according to the first image after compression.

[0293] In a possible implementation, the (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image;

[0294] Among them, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are respectively integers greater than 1.

[0295] In a possible implementation manner, the processor 22 is specifically configured to:

[0296] The first image is processed by a recognition model to obtain an image filter of the first image, wherein the recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content of the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

[0297] In a possible implementation manner, the processor 22 is specifically configured to:

[0298] Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size;

[0299] The first compressed image is processed by using a recognition model.

[0300] In a possible implementation manner, the processor 22 is specifically configured to:

[0301] Acquire a gradient image of the first image, wherein the gradient image has the same size as the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively;

[0302] Processing the gradient image by the image filter to obtain a second image, wherein the size of the second image is (f*N)*(f*N), wherein f is a preset magnification;

[0303] Enlarging the first image by f times to obtain a third image, wherein the size of the third image is (f*N)*(f*N);

[0304] The super-resolution image is obtained according to the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

[0305] In a possible implementation manner, the processor 22 is specifically configured to:

[0306] The gradient image is processed by the image filter to obtain f 2 sub-images, each sub-image has a size of (f*N)*(f*N);

[0307] According to the f 2 The second image is determined by using a plurality of sub-images.

[0308] In a possible implementation manner, the processor 22 is specifically configured to:

[0309] The gradient image is processed by the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f2 -1.

[0310] In a possible implementation manner, the processor 22 is specifically configured to:

[0311] For the f 2 The pixels at the same pixel position in the sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f;

[0312] The M*N image blocks are spliced ​​according to the pixel positions of the pixels in each image block in the sub-image to obtain the second image.

[0313] It should be noted that the image processing device shown in the embodiment of the present application can execute the technical solution shown in the above method embodiment, and its implementation principle and beneficial effects are similar, which will not be repeated here.

[0314] Optionally, the processor may be a CPU, or other general-purpose processors, DSP, ASIC, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc. The steps in the authentication method embodiment disclosed in the present application may be directly implemented as being executed by a hardware processor, or may be implemented as being executed by a combination of hardware and software modules in the processor.

[0315] The present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to implement the image processing method described in the above embodiment.

[0316] The embodiment of the present application also provides a chip or integrated circuit, including: a memory and a processor;

[0317] The memory is used to store program instructions and sometimes also to store intermediate data;

[0318] The processor is used to call the program instructions stored in the memory to implement the image processing method as described above.

[0319] Optionally, the memory may be independent or integrated with the processor. In some implementations, the memory may be located outside the chip or integrated circuit.

[0320] An embodiment of the present application further provides a program product, which includes a computer program stored in a storage medium, and the computer program is used to implement the above-mentioned image processing method.

[0321] All or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned memory (storage medium) includes: read-only memory (English: read-only memory, abbreviated: ROM), RAM, flash memory, hard disk, solid state drive, magnetic tape (English: magnetic tape), floppy disk (English: floppy disk), optical disc (English: optical disc) and any combination thereof.

[0322] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0323] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0324] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0325] Obviously, those skilled in the art can make various changes and modifications to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0326] In the present application, the term "include" and its variations may refer to non-restrictive inclusion; the term "or" and its variations may refer to "and / or". The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. In the present application, "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previously associated objects are in an "or" relationship.

Claims

1. An image processing method, characterized in that: include: Acquire a texture image of the first image; In the texture image, a plurality of local texture images and a weight value of each local texture image are determined according to the texture features of each pixel in the texture image, wherein the texture features of the textures in the local texture images are the same; Determine an image filter corresponding to the first image according to the multiple local texture images and a weight value of each local texture image, wherein the image filter has a filter parameter corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters; The first image is super-resolved according to the image filter of the first image to obtain a super-resolved image of the first image.

2. The method according to claim 1, characterized in that Also includes: Determine C local texture images and a weight value of each local texture image in the texture image, wherein the texture features of the textures in the local texture images are the same; C=f 2 ×t, f is a preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, t is an integer greater than or equal to 1, and the number of channels of the image filter is C.

3. The method according to claim 2, characterized in that The (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image; Among them, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are respectively integers greater than 1.

4. The method according to claim 1, characterized in that: The step of acquiring the texture image of the first image comprises: Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size; The texture image is determined according to the first image after compression.

5. The method according to claim 1, characterized in that The determining of the image filter corresponding to the first image further includes: The first image is processed by a recognition model to obtain an image filter of the first image, wherein the recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content of the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

6. The method according to claim 5, characterized in that The processing of the first image by using a recognition model comprises: Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size; The first compressed image is processed by using a recognition model.

7. The method according to any one of claims 1 to 6, characterized in that: The step of performing super-resolution processing on the first image according to the image filter of the first image to obtain a super-resolution image of the first image includes: Acquire a gradient image of the first image, wherein the gradient image has the same size as the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively; Processing the gradient image by the image filter to obtain a second image, wherein the size of the second image is (f*N)*(f*N), wherein f is a preset magnification; Enlarging the first image by f times to obtain a third image, wherein the size of the third image is (f*N)*(f*N); The super-resolution image is obtained according to the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

8. The method according to claim 7, characterized in that The step of processing the gradient image by using the image filter to obtain a second image comprises: The gradient image is processed by the image filter to obtain f 2 sub-images, each sub-image has a size of (f*N)*(f*N); According to the f 2 The second image is determined by using a plurality of sub-images.

9. The method according to claim 8, characterized in that The gradient image is processed by the image filter to obtain f 2 A sub-image, including: The gradient image is processed by the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

10. The method according to claim 8 or 9, characterized in that: According to the f 2 The method of determining the second image by using a sub-image comprises: For the f 2 The pixels at the same pixel position in the sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f; The M*N image blocks are spliced ​​according to the pixel positions of the pixels in each image block in the sub-image to obtain the second image.

11. An image processing device, characterized in that: include: Determine the module and the super-resolution module, where The determination module is used to obtain a texture image of a first image; in the texture image, determine a plurality of local texture images and a weight value of each local texture image according to the texture features of each pixel in the texture image, wherein the texture features of the textures in the local texture images are the same; determine an image filter corresponding to the first image according to the plurality of local texture images and the weight value of each local texture image, wherein the image filter has a filter parameter corresponding to each pixel in the first image, and pixels with different texture features have different filter parameters; The super-resolution module is used to perform super-resolution processing on the first image according to the image filter of the first image to obtain a super-resolution image of the first image.

12. The device according to claim 11, characterized in that The determination module is specifically used for: Determine C local texture images and a weight value of each local texture image in the texture image, wherein the texture features of the textures in the local texture images are the same; C=f 2 ×t, f is a preset magnification, t is the number of filter parameters corresponding to each pixel in the first image, f is greater than 1, t is an integer greater than or equal to 1, and the number of channels of the image filter is C.

13. The device according to claim 12, characterized in that The (x, y)th filter parameter in the ith channel of the image filter is: the product of the pixel value of the (x, y)th pixel in the ith local texture image and the weight value of the ith local texture image; Among them, i is a positive integer less than or equal to C, x is a positive integer less than or equal to M, y is a positive integer less than or equal to N, M is the number of pixels included in the first image in the horizontal direction, N is the number of pixels included in the first image in the vertical direction, and M and N are respectively integers greater than 1.

14. The device according to claim 11, characterized in that The determination module is specifically used for: Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size; The texture image is determined according to the first image after compression.

15. The device according to claim 11, characterized in that The determining module is also used for: The first image is processed by a recognition model to obtain an image filter of the first image, wherein the recognition model is obtained by learning multiple groups of samples, each group of samples includes a first sample image and a second sample image, the image content of the first sample image and the second sample image is the same, and the resolution of the first sample image is greater than the resolution of the second sample image.

16. The device according to claim 15, characterized in that The determination module is specifically used for: Compressing the first image according to a preset size, so that the size of the first image after compression is the preset size; The first compressed image is processed by using a recognition model.

17. The device according to any one of claims 11 to 16, characterized in that: The super-resolution module is specifically used for: Acquire a gradient image of the first image, wherein the gradient image has the same size as the first image, and the size of the gradient image is M*N, where M and N are integers greater than 1 respectively; Processing the gradient image by the image filter to obtain a second image, wherein the size of the second image is (f*N)*(f*N), wherein f is a preset magnification; Enlarging the first image by f times to obtain a third image, wherein the size of the third image is (f*N)*(f*N); The super-resolution image is obtained according to the second image and the third image, and the size of the super-resolution image is (f*N)*(f*N).

18. The device according to claim 17, characterized in that The super-resolution module is specifically used for: The gradient image is processed by the image filter to obtain f 2 sub-images, each sub-image has a size of (f*N)*(f*N); According to the f 2 The second image is determined by using a plurality of sub-images.

19. The device according to claim 18, characterized in that The super-resolution module is specifically used for: The gradient image is processed by the parameters in the k*t+1th to (k+1)*tth channels in the image filter to obtain the kth sub-image, where k is 0, 1, ..., f 2 -1.

20. The device according to claim 18 or 19, characterized in that The super-resolution module is specifically used for: For the f 2 The pixels at the same pixel position in the sub-image are spliced ​​to obtain M*N image blocks, and the size of each image block is f*f; The M*N image blocks are spliced ​​according to the pixel positions of the pixels in each image block in the sub-image to obtain the second image.

21. A computer system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program, and the processor runs the computer program to execute the image processing method according to any one of claims 1 to 10.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed by one or more processors, the computer program implements the image processing method according to any one of claims 1 to 10.

Images