Image processing method and device, computer equipment and storage medium

By utilizing image processing models for feature extraction and fusion on mobile devices, combined with training image sets and compression models, the problem of limited computing resources in existing technologies is solved, enabling fast super-resolution reconstruction of animations and resolution enhancement of old anime videos.

CN118918010APending Publication Date: 2024-11-08WUHAN TCL CORP RES CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310499874.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-04-27
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

Existing deep learning models such as Real ESRGAN and Real CUGAN are computationally intensive and slow when performing animation super-resolution reconstruction on devices with limited computing resources, such as mobile devices.

Method used

By acquiring image data of the image to be processed in a preset channel, feature extraction and fusion are performed using an image processing model to generate a super-resolution image. By combining a training image set and a compression model, the amount of data processing is reduced and the reconstruction speed is improved.

Benefits of technology

It enables fast super-resolution reconstruction on mobile devices, improves the super-resolution effect of animations, and is suitable for resolution enhancement of older anime videos with lower resolution.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918010A_ABST
    Figure CN118918010A_ABST
Patent Text Reader

Abstract

The invention provides an image processing method and apparatus, a computer device and a storage medium. The method comprises the steps of obtaining first image data of a to-be-processed image in a preset first channel; inputting the first image data into an image processing model for image processing to obtain second image data of the to-be-processed image in the first channel; and generating a target image corresponding to the to-be-processed image based on the second image data. According to the embodiment of the invention, based on the first image data of the to-be-processed image in the preset first channel, super-resolution reconstruction is carried out through the image processing model to obtain the second image data of the to-be-processed image in the first channel, and then the super-resolution image is generated, so that super-resolution reconstruction of the image data based on a single channel is realized; and the data processing amount during super-resolution reconstruction is reduced, so that the super-resolution reconstruction speed is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of super-resolution reconstruction, and in particular to an image processing method, apparatus, computer equipment and storage medium. Background Art

[0002] Super-resolution reconstruction is an image processing technology that reconstructs low-resolution (LR) images into high-resolution (HR) images. Super-resolution reconstruction of animations is generally achieved based on deep learning, such as the Real ESRGAN model and Real CUGAN model in deep learning.

[0003] However, the computational complexity of the Real ESRGAN model, Real CUGAN model, etc. is usually large. When they are applied to devices such as mobile terminals, the computing resources in the devices are often insufficient to meet the computing requirements of these models, resulting in slow animation super-resolution reconstruction. Summary of the invention

[0004] The present application provides an image processing method, apparatus, computer device and storage medium, aiming to solve the technical problem of how to improve the speed of animation super-resolution reconstruction.

[0005] In one aspect, the present application provides an image processing method, the image processing method comprising:

[0006] Acquire first image data of the image to be processed in a preset first channel;

[0007] Inputting the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel;

[0008] Based on the second image data, a target image corresponding to the image to be processed is generated.

[0009] In a possible implementation of the present application, generating a target image corresponding to the image to be processed based on the second image data includes:

[0010] Acquire image data of the image to be processed in a preset second channel and image data of the image to be processed in a preset third channel;

[0011] A target image corresponding to the image to be processed is generated based on the second image data, the image data of the image to be processed in a preset second channel, and the image data of the image to be processed in a preset third channel.

[0012] In a possible implementation of the present application, inputting the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel includes:

[0013] Using the image processing model, extracting first image features from the first image data to obtain a first feature map;

[0014] Performing second image feature extraction on the first feature map to obtain a second feature map;

[0015] fusing the second feature map and the first feature map to obtain a fused feature map;

[0016] The fused feature map is upsampled to obtain second image data of the image to be processed in the first channel.

[0017] In a possible implementation of the present application, extracting a second image feature from the first feature map to obtain a second feature map includes:

[0018] Performing convolution processing on the first feature map to obtain a convolution feature map;

[0019] Perform activation processing on the convolution feature map to obtain the second feature map.

[0020] In a possible implementation of the present application, before the first image data is input into an image processing model for image processing to obtain the second image data of the first channel of the image to be processed, the step further includes:

[0021] Acquire a training image set, the training image set comprising a plurality of training image groups, each of the training image groups comprising a first image and a second image corresponding to the first image, and a resolution of the first image is smaller than a resolution of the second image;

[0022] Acquire third image data of the first image in the preset first channel and fourth image data of the second image in the preset first channel;

[0023] Training a preset network model based on the third image data and the fourth image data to obtain a basic model;

[0024] Compressing the basic model to obtain a compressed model;

[0025] The compression model is quantized to obtain the image processing model.

[0026] In a possible implementation of the present application, the step of obtaining a training image set includes:

[0027] acquiring a plurality of third images;

[0028] Determine a first image corresponding to each third image in the plurality of third images; wherein a resolution of the first image is smaller than a resolution of the third image;

[0029] Determine a second image corresponding to each third image in the plurality of third images; wherein a resolution of the second image is greater than a resolution of the third image;

[0030] The training image set is constructed based on the first image and the second image.

[0031] In a possible implementation of the present application, determining the first image corresponding to each of the plurality of third images includes:

[0032] Downsampling the plurality of third images respectively to obtain a downsampled image corresponding to each third image;

[0033] Perform image compression processing on the downsampled image to obtain a first image corresponding to each of the third images.

[0034] In a possible implementation of the present application, determining the second image corresponding to each of the plurality of third images includes:

[0035] Performing bilateral filtering on the plurality of third images respectively to obtain a first filtered image corresponding to each of the third images;

[0036] Performing bilateral filtering on the first filtered image to obtain a second filtered image corresponding to each of the third images;

[0037] Based on the third image, the first filtered image, and the second filtered image, a first image corresponding to each of the third images is determined.

[0038] In another aspect, the present application provides an image processing device, the device comprising:

[0039] An acquisition unit, used for acquiring first image data of an image to be processed in a preset first channel;

[0040] A determination unit, configured to input the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel;

[0041] A generating unit is used to generate a target image corresponding to the image to be processed based on the second image data.

[0042] On the other hand, the present application also provides a computer device, the computer device comprising:

[0043] one or more processors;

[0044] Memory; and

[0045] One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement any one of the above-mentioned image processing methods.

[0046] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps in any one of the above-mentioned image processing methods.

[0047] The image processing method, device, computer equipment and storage medium provided in the embodiment of the present application include: obtaining the first image data of the image to be processed in the preset first channel; inputting the first image data into the image processing model for image processing to obtain the second image data of the image to be processed in the first channel; and generating the target image corresponding to the image to be processed based on the second image data. Compared with the traditional method, the embodiment of the present application performs super-resolution reconstruction based on the first image data of the image to be processed in the preset first channel through the image processing model to obtain the second image data of the image to be processed in the first channel, and then generates a super-resolution image, so as to realize the super-resolution reconstruction based on the image data of a single channel, reduce the amount of data processing during the super-resolution reconstruction, thereby improving the speed of super-resolution reconstruction, and when the image to be processed is a single-frame animation, it is different from the natural video data that needs to restore the reality through the optical flow information, and the animation does not need the optical flow information to achieve the sense of reality in the real scene, so compared with the natural video data, the super-resolution of the single-frame animation based on the single-channel image data has a better super-resolution effect, thereby improving the speed of super-resolution reconstruction and ensuring a better super-resolution effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 is a schematic flow chart of an embodiment of the image processing method provided in the embodiments of the present application;

[0050] Figure 2 is a schematic flow chart of another embodiment of the image processing method provided in the embodiments of the present application;

[0051] Figure 3It is a schematic diagram of an equivalent structure of the basic model and the compression model provided in the embodiments of the present application;

[0052] Figure 4 is a schematic diagram of a network structure of a compression model provided in an embodiment of the present application;

[0053] Figure 5 is a schematic diagram of the structure of an embodiment of an image processing device provided in an embodiment of the present application;

[0054] Figure 6 It is a schematic diagram of the structure of an embodiment of a computer device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0056] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0057] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0058] The embodiments of the present application provide an image processing method, apparatus, computer device, and storage medium, which are described in detail below.

[0059] See also Figure 1 , Figure 1The following is a flow chart of an embodiment of an image processing method provided in an embodiment of the present application. The image processing method can be applied to devices with less computing resources such as mobile terminals and televisions. The image processing method includes:

[0060] 101. Acquire first image data of an image to be processed in a preset first channel;

[0061] In an embodiment of the present application, the image to be processed is generally a frame of animation. For example, the image to be processed may be a frame of an animation video that needs to be displayed on a mobile terminal when a user plays the animation video through the mobile terminal.

[0062] In an embodiment of the present application, the preset first channel may be a brightness channel in a preset color space. The preset color space may be a color space with a brightness channel such as a YUV color space (Luminance, Chrominance, Chroma, brightness, chroma, concentration), an HSV color space (Hue, Saturation, Value, hue, saturation, brightness). Taking the preset color space as the YUV color space as an example, the first image data of the image to be processed in the preset first channel may be the brightness channel data of the image to be processed in the YUV color space, and the brightness channel data includes the brightness value of each pixel in the image to be processed in the YUV color space. It should be noted that if the current color space of the image to be processed is not the preset color space, the image to be processed needs to be converted to the preset color space first, and the format of the image to be processed after conversion to the preset color space is generally YUV444.

[0063] 102. Input the first image data into the image processing model for image processing to obtain second image data of the image to be processed in the first channel;

[0064] In the embodiment of the present application, super-resolution reconstruction is performed based on the first image data through an image processing model to obtain second image data. The image processing model may be a neural network model. The generation process of the image processing model is described in detail in Figure 2 The embodiment shown.

[0065] In some embodiments of the present application, the specific process of the image processing model obtaining the second image data based on the input first image data can be referred to Figure 4 , and described as follows: extracting the first image feature from the input first image data (which can be Figure 4 The first convolution process in is implemented) to obtain the first feature map; the second image feature extraction is performed on the first feature map (which can be Figure 4The second convolution processing in the above method is implemented to obtain a second feature map; the second feature map and the first feature map are fused to obtain a fused feature map; the fused feature map is upsampled (which can be implemented by transposed convolution) to obtain second image data of the image to be processed in the first channel, and the second image data is output.

[0066] Further, extracting the second image feature from the first feature map to obtain the second feature map may include: performing convolution processing on the first feature map (for example Figure 4 The second convolution process in ) is performed to obtain a convolution feature map; the convolution feature map is activated (for example Figure 4 The second feature map is obtained by performing the activation process in the convolution operation. The second feature extraction process may also include multiple consecutive convolution operations and activation operations, for example Figure 4 6 consecutive convolution and activation processes are shown in FIG.

[0067] It can be understood that, since the second image data is a model prediction result based on super-resolution reconstruction of the first image data, a target image with a higher resolution than the image to be processed can be generated based on the second image data.

[0068] 103. Generate a target image corresponding to the image to be processed based on the second image data.

[0069] In an embodiment of the present application, based on the second image data, generating a target image corresponding to the image to be processed may include: obtaining image data of the image to be processed in a preset second channel and image data of the image to be processed in a preset third channel; based on the second image data, the image data of the image to be processed in the preset second channel and the image data of the image to be processed in the preset third channel, generating a target image corresponding to the image to be processed. Taking the preset color space as the YUV color space (Luminance, Chrominance, Chroma, brightness, chroma, concentration) as an example, the preset second channel may be a chroma channel, and the preset third channel may be a concentration channel. The image data of the image to be processed in the preset second channel may be the chroma channel data of the image to be processed in the YUV color space, and the chroma channel data includes the chroma value of each pixel in the image to be processed in the YUV color space. The image data of the image to be processed in the preset third channel may be the concentration channel data of the image to be processed in the YUV color space, and the concentration channel data includes the concentration value of each pixel in the image to be processed in the YUV color space.

[0070] It should be noted that the second image data is image data in a target image with a higher resolution than the image to be processed. Due to the difference between the resolution represented by the second image data and the resolution of the image to be processed, if the resolution represented by the second image data is much greater than the resolution of the image to be processed, the format of the reconstructed target image will be incorrect. Therefore, when the ratio of the resolution represented by the second image data to the resolution of the image to be processed is greater than 2, it is necessary to upsample the image to be processed before obtaining the image data of the image to be processed in the preset second channel and the image data of the image to be processed in the preset third channel, so as to improve the resolution of the image to be processed, so that the ratio of the resolution represented by the second image data to the resolution of the image to be processed is less than or equal to 2. When the ratio of the resolution represented by the second image data to the resolution of the image to be processed is less than or equal to 2 times, the target image in the format of YUV444 or YUV420 can be directly reconstructed.

[0071] The image processing method provided in the embodiment of the present application performs super-resolution reconstruction based on the first image data of the image to be processed in the preset first channel through the image processing model, obtains the second image data of the image to be processed in the first channel, and then generates a super-resolution image, realizing super-resolution reconstruction based on single-channel image data, reducing the amount of data processing during super-resolution reconstruction, thereby improving the speed of super-resolution reconstruction, and when the image to be processed is a single-frame animation, unlike natural video data that needs to restore reality through optical flow information, animation does not need optical flow information to achieve the sense of reality in the real scene, so compared with natural video data, the super-resolution of single-frame animation based on single-channel image data has better super-resolution effect, thereby improving the speed of super-resolution reconstruction while ensuring a better super-resolution effect. In addition, the image processing method provided in the embodiment of the present application can also be used for resolution enhancement of old animation videos with lower resolution to improve the viewing experience of old animation videos.

[0072] See also Figure 2 , Figure 2 This is a flow chart of another embodiment of the image processing method provided in the embodiment of the present application. Figure 1 In the embodiment shown, before the first image data is input into the image processing model for image processing to obtain the second image data of the first channel of the image to be processed, the method may further include:

[0073] 201. Obtain a training image set, where the training image set includes a plurality of training image groups, each training image group includes a first image and a second image corresponding to the first image, and a resolution of the first image is smaller than a resolution of the second image;

[0074] In an embodiment of the present application, obtaining a training image set may include: obtaining multiple third images; determining a first image corresponding to each of the multiple third images; wherein the resolution of the first image is less than the resolution of the third image; determining a second image corresponding to each of the multiple third images; wherein the resolution of the second image is greater than the resolution of the third image; and constructing a training image set based on the first image and the second image. It can be seen that the first image and the second image corresponding to the third image are different images of the third image at different resolutions.

[0075] In the embodiment of the present application, the third image is generally a frame of animation in an animation video, and can be extracted from the animation video. It should be noted that the resolution of the third image is the original resolution of the frame of animation, that is, the third image is the original image of the frame of animation. The first image corresponding to the third image is generally blurrier than the third image, and the second image corresponding to the third image is generally clearer than the third image.

[0076] In a further embodiment, determining the first image corresponding to each of the plurality of third images may include: downsampling the plurality of third images respectively to obtain a downsampled image corresponding to each third image, wherein the downsampling may be implemented based on cubic interpolation technology, and the specific principle is not described in detail here; performing image compression processing on the downsampled images to obtain the first image corresponding to each third image for the target video frame, wherein the image compression processing may be implemented by a multimedia video processing tool, and the multimedia video processing tool may be FFmpeg (Fast Forward Mpeg) or the like. It is understandable that through downsampling and image compression, the resolution of the image will be reduced to reach the first resolution. An example of the process of downsampling and image compression is as follows:

[0077] I LR =FFmpeg(I HR ↓)

[0078] Among them, I LR Refers to the first image, I HR refers to the third image, FFmpeg (*) refers to image compression processing, and ↓ indicates downsampling.

[0079] In a further embodiment, the second image corresponding to the third image can be obtained by an image detail enhancement algorithm based on a multi-scale to improve the efficiency of image detail enhancement. Specifically, determining the second image corresponding to each of the plurality of third images may include: performing bilateral filtering on the plurality of third images to obtain a first filtered image corresponding to each third image; performing bilateral filtering on the first filtered image to obtain a second filtered image corresponding to each third image; and determining the first image corresponding to each third image based on the third image, the first filtered image, and the second filtered image. It should be noted that, in the images of multiple scales, the first filtered image is an image of the first scale, and the second filtered image is an image of the second scale. Of course, in the step of determining the second image corresponding to each of the plurality of third images, after obtaining the second filtered image, the second filtered image may also be subjected to bilateral filtering to obtain a third filtered image, and then based on the third image, the first filtered image, the second filtered image, and the third filtered image, the second image corresponding to the third image is determined. Among them, the third filtered image is an image of the third scale. Bilateral Filters is a nonlinear filtering method that can effectively remove noise in smooth areas while improving the edge clarity of animated images, thereby enhancing the protection and enhancement effect of subtitles in animated images.

[0080] In a further embodiment, a plurality of third images are respectively subjected to bilateral filtering processing by a first convolution kernel, and a first filtered image is subjected to bilateral filtering processing by a second convolution kernel. The size of the first convolution kernel is greater than or equal to a preset value, and / or the size of the second convolution kernel is greater than or equal to a preset value, for example, the preset value may be 9, that is, the convolution kernel is set to a large kernel, thereby integrating information of more pixels in the image during filtering to improve the filtering effect. Furthermore, since the large kernel convolution is applied in the generation process of the second image, rather than in the subsequent model training process of the preset network model, the amount of data calculation during model training can be reduced, and the image processing model obtained by the subsequent model training will also have the above-mentioned advantages brought by the large kernel convolution.

[0081] In order to more clearly illustrate the acquisition process of the second image, the acquisition process of the second image can be exemplified as follows:

[0082] B1=Bilateral Filters(I HR , K, δ1, δ2)

[0083] B2=Bilateral Filters(B1, K, δ1, δ2)

[0084] B3=Bilateral Filters(B2, K, δ1, δ2)

[0085] D1=I HR -B1

[0086] D2=B1-B2

[0087] D3=B2-B3

[0088] r=(1-W1*sign(D1))*D1+W2*D2+W3*D3+I HR

[0089] Among them, B1, B2, B3 are the first filtered image, the second filtered image, and the third filtered image respectively, BilateralFilters(*) is bilateral filtering processing, K refers to the convolution kernel size, and the value of K can be greater than or equal to 9, δ1 and δ2 are the spatial domain filter variance and the range filter variance respectively, and the values ​​of δ1 and δ2 can both be 200. D1, D2, and D3 refer to the differences between images, r is the second image, W1, W2, and W3 are all weight values, and the values ​​of W1 and W2 can both be 0.5, and the value of W3 can be 0.25. sgn(*) refers to the sign function. It can be seen from the above formula that based on the third image, the first filtered image, the second filtered image, and the third filtered image, the second image corresponding to the third image is determined, specifically based on the differences between the images (such as D1, D2, and D3), and then these differences are fused into the third image in a weighted combination (such as W1, W2, and W3) to obtain the second image.

[0090] 202. Acquire third image data of the first image in a preset first channel and fourth image data of the second image in the preset first channel;

[0091] In the embodiment of the present application, the first channel is preset Figure 1 The preset first channel in the illustrated embodiment is the same. Taking the preset color space as the YUV color space as an example, the third image data of the first image in the preset first channel may be the brightness channel data of the first image in the YUV color space, and the brightness channel data includes the brightness value of each pixel in the first image in the YUV color space. Similarly, the fourth image data of the second image in the preset first channel may be the brightness channel data of the second image in the YUV color space, and the brightness channel data includes the brightness value of each pixel in the second image in the YUV color space. It should be noted that if the current color space of the first image and the second image is not the preset color space, the first image and the second image need to be converted to the preset color space first, and the format of the first image after conversion to the preset color space is generally YUV444.

[0092] 203. Training a preset network model based on the third image data and the fourth image data to obtain a basic model;

[0093] In an embodiment of the present application, the preset network model may be a model such as a convolutional neural network (CNN). Training the preset network model based on the third image data and the fourth image data may include: using the fourth image data as the model prediction result of the third image data to train the preset network model. During model training, the third image data is specifically input into the preset network model, and based on the difference between the content output by the preset network model and the fourth image data, the model is optimized to obtain a basic model.

[0094] 204. Compress the basic model to obtain a compressed model;

[0095] In some embodiments of the present application, the preset network model is a re-parameterized neural network model. Generally speaking, the model generation process includes a model training phase, a model reasoning phase, etc. When the re-parameterization technology is applied, the model training phase and the model reasoning phase are decoupled, that is, in the model training phase, the model includes multiple convolution branches, and in the model reasoning phase, these multiple convolution branches are converted into a single convolution branch to obtain a compressed model, for example Figure 3 As shown, Figure 3 The basic model in the model training phase includes multiple convolution branches to enhance the model's fitting ability. Figure 3 In the model inference phase, these multiple convolution branches are converted into a single convolution branch to obtain a compressed model to reduce the computational complexity and improve the performance of the model. Figure 3 The specific network structure of the compression model is as follows Figure 4 As shown in Figure 2, the specific principle of applying the reparameterization technique to transform multiple convolution branches into a single convolution branch will not be described in detail here.

[0096] 205. Perform quantization processing on the compression model to obtain an image processing model.

[0097] In an embodiment of the present application, the compression model is applied to a specific terminal device, such as a mobile terminal that needs to play anime videos, by quantizing the compression model for model reasoning; after the model reasoning is completed, the image processing model is obtained.

[0098] In a further embodiment, when quantizing the compression model, half-precision floating point quantization, int16 quantization (16-bit integer) and other quantization methods may be used to lightweight the model and improve the computational efficiency of the model.

[0099] The image processing method provided in the embodiment of the present application generates an image processing model based on image data of images with different resolutions in a preset first channel, thereby achieving lightweight models and greatly reducing the amount of data calculation in the model, thereby improving the speed of super-resolution reconstruction. Furthermore, since animation frames are different from natural video data, their optical flow information is basically invalid. Therefore, the image processing model generated based on the image data of the image in the preset first channel can also ensure better super-resolution effects while improving the super-resolution reconstruction speed of single-frame animations.

[0100] In order to better implement the image processing method in the embodiment of the present application, on the basis of the image processing method, an image processing device is also provided in the embodiment of the present application, such as Figure 5 As shown, the image processing device 500 includes:

[0101] An acquisition unit 501 is used to acquire first image data of an image to be processed in a preset first channel;

[0102] A determination unit 502 is used to input the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel;

[0103] The generating unit 503 is configured to generate a target image corresponding to the image to be processed based on the second image data.

[0104] Compared with the traditional method, the embodiment of the present application performs super-resolution reconstruction through an image processing model based on the first image data of the image to be processed in a preset first channel, obtains the second image data of the image to be processed in the first channel, and then generates a super-resolution image, thereby realizing super-resolution reconstruction based on image data of a single channel, reducing the amount of data processing during super-resolution reconstruction, and thus improving the speed of super-resolution reconstruction.

[0105] In some embodiments of the present application, the generating unit 503 is specifically configured to:

[0106] Acquire image data of the image to be processed in the preset second channel and image data of the image to be processed in the preset third channel;

[0107] A target image corresponding to the image to be processed is generated based on the second image data, the image data of the image to be processed in the preset second channel, and the image data of the image to be processed in the preset third channel.

[0108] In some embodiments of the present application, the determining unit 502 is specifically configured to:

[0109] Using the image processing model, extracting first image features from the first image data to obtain a first feature map;

[0110] Performing second image feature extraction on the first feature map to obtain a second feature map;

[0111] Fusing the second feature map with the first feature map to obtain a fused feature map;

[0112] The fused feature map is upsampled to obtain second image data of the image to be processed in the first channel.

[0113] In some embodiments of the present application, the determining unit 502 is specifically configured to:

[0114] Performing convolution processing on the first feature map to obtain a convolution feature map;

[0115] The convolution feature map is activated to obtain a second feature map.

[0116] In some embodiments of the present application, the determining unit 502 is further configured to:

[0117] Acquire a training image set, the training image set includes a plurality of training image groups, each training image group includes a first image and a second image corresponding to the first image, and a resolution of the first image is smaller than a resolution of the second image;

[0118] Acquire third image data of the first image in the preset first channel and fourth image data of the second image in the preset first channel;

[0119] Training a preset network model based on the third image data and the fourth image data to obtain a basic model;

[0120] Compress the basic model to obtain a compressed model;

[0121] The compression model is quantized to obtain an image processing model.

[0122] In some embodiments of the present application, the determining unit 502 is further configured to:

[0123] acquiring a plurality of third images;

[0124] Determine a first image corresponding to each third image in a plurality of third images; wherein a resolution of the first image is smaller than a resolution of the third image;

[0125] Determining a second image corresponding to each third image in a plurality of third images; wherein a resolution of the second image is greater than a resolution of the third image;

[0126] Based on the first image and the second image, a training image set is constructed.

[0127] In some embodiments of the present application, the determining unit 502 is further configured to:

[0128] Downsampling the plurality of third images respectively to obtain a downsampled image corresponding to each third image;

[0129] The downsampled image is subjected to image compression processing to obtain a first image corresponding to each third image.

[0130] In some embodiments of the present application, the determining unit 502 is further configured to:

[0131] Performing bilateral filtering on the plurality of third images respectively to obtain a first filtered image corresponding to each third image;

[0132] Performing bilateral filtering on the first filtered image to obtain a second filtered image corresponding to each third image;

[0133] Based on the third image, the first filtered image, and the second filtered image, a first image corresponding to each third image is determined.

[0134] In addition to the above-described image processing method and device, the embodiment of the present application further provides a computer device, which integrates any image processing device provided in the embodiment of the present application, and the computer device includes:

[0135] one or more processors;

[0136] Memory; and

[0137] One or more applications, wherein the one or more applications are stored in the memory and are configured to execute the steps of any one of the embodiments of the above-mentioned image processing method by the processor.

[0138] The present application also provides a computer device that integrates any image processing device provided in the present application. Figure 6 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0139] The computer device may include one or more processing core processors 601, one or more computer readable storage medium storage units 602, a power supply 603, an input unit 604 and other components. Those skilled in the art will understand that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0140] The processor 601 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the storage unit 602 and calling data stored in the storage unit 602, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.

[0141] The storage unit 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the storage unit 602. The storage unit 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the storage unit 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the storage unit 602 may also include a memory controller to provide the processor 601 with access to the storage unit 602.

[0142] The computer device also includes a power supply 603 for supplying power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 603 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0143] The computer device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0144] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in the embodiment of the present application, the processor 601 in the computer device will load the executable file corresponding to the process of one or more application programs into the storage unit 602 according to the following instructions, and the processor 601 will run the application program stored in the storage unit 602, thereby realizing various functions, as follows:

[0145] The first image data of the image to be processed in a preset first channel is obtained; the first image data is input into the image processing model for image processing to obtain the second image data of the image to be processed in the first channel; based on the second image data, a target image corresponding to the image to be processed is generated.

[0146] The embodiment of the present application performs super-resolution reconstruction through an image processing model based on first image data of an image to be processed in a preset first channel, obtains second image data of the image to be processed in the first channel, and then generates a super-resolution image, thereby realizing super-resolution reconstruction based on image data of a single channel, reducing the amount of data processing during super-resolution reconstruction, and thus improving the speed of super-resolution reconstruction.

[0147] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. The computer-readable storage medium stores a plurality of instructions, which can be loaded by a processor to execute the steps of any image processing method provided in the embodiment of the present application. For example, the instruction can execute the following steps:

[0148] The first image data of the image to be processed in a preset first channel is obtained; the first image data is input into the image processing model for image processing to obtain the second image data of the image to be processed in the first channel; based on the second image data, a target image corresponding to the image to be processed is generated.

[0149] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0150] The above is a detailed introduction to an image processing method, device, computer equipment and storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. An image processing method, characterized in that: The image processing method comprises: Acquire first image data of the image to be processed in a preset first channel; Inputting the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel; Based on the second image data, a target image corresponding to the image to be processed is generated.

2. The image processing method according to claim 1, characterized in that: The step of generating a target image corresponding to the image to be processed based on the second image data includes: Acquire image data of the image to be processed in a preset second channel and image data of the image to be processed in a preset third channel; A target image corresponding to the image to be processed is generated based on the second image data, the image data of the image to be processed in a preset second channel, and the image data of the image to be processed in a preset third channel.

3. The image processing method according to claim 1, characterized in that: The step of inputting the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel includes: Using the image processing model, extracting first image features from the first image data to obtain a first feature map; Performing second image feature extraction on the first feature map to obtain a second feature map; fusing the second feature map and the first feature map to obtain a fused feature map; The fused feature map is upsampled to obtain second image data of the image to be processed in the first channel.

4. The image processing method according to claim 3, characterized in that: The extracting a second image feature from the first feature map to obtain a second feature map includes: Performing convolution processing on the first feature map to obtain a convolution feature map; Perform activation processing on the convolution feature map to obtain the second feature map.

5. The image processing method according to claim 1, wherein: Before the first image data is input into the image processing model for image processing to obtain the image to be processed, the method further includes: Acquire a training image set, the training image set comprising a plurality of training image groups, each of the training image groups comprising a first image and a second image corresponding to the first image, and a resolution of the first image is smaller than a resolution of the second image; Acquire third image data of the first image in the preset first channel and fourth image data of the second image in the preset first channel; Training a preset network model based on the third image data and the fourth image data to obtain a basic model; Compressing the basic model to obtain a compressed model; The compression model is quantized to obtain the image processing model.

6. The image processing method according to claim 5, characterized in that: The step of acquiring a training image set comprises: acquiring a plurality of third images; Determine a first image corresponding to each third image in the plurality of third images; wherein a resolution of the first image is smaller than a resolution of the third image; Determine a second image corresponding to each third image in the plurality of third images; wherein a resolution of the second image is greater than a resolution of the third image; The training image set is constructed based on the first image and the second image.

7. The image processing method according to claim 6, characterized in that: The determining the first image corresponding to each third image in the plurality of third images comprises: Downsampling the plurality of third images respectively to obtain a downsampled image corresponding to each third image; Perform image compression processing on the downsampled image to obtain a first image corresponding to each of the third images.

8. The image processing method according to claim 6, characterized in that: The determining the second image corresponding to each third image in the plurality of third images includes: Performing bilateral filtering on the plurality of third images respectively to obtain a first filtered image corresponding to each of the third images; Performing bilateral filtering on the first filtered image to obtain a second filtered image corresponding to each of the third images; Based on the third image, the first filtered image, and the second filtered image, a first image corresponding to each of the third images is determined.

9. An image processing device, characterized in that: The device comprises: An acquisition unit, used for acquiring first image data of an image to be processed in a preset first channel; A determination unit, configured to input the first image data into an image processing model for image processing to obtain second image data of the image to be processed in the first channel; A generating unit is used to generate a target image corresponding to the image to be processed based on the second image data.

10. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the image processing method according to any one of claims 1 to 8.