An image processing method, apparatus, device, storage medium, and product

Abstract

The embodiment of the application discloses an image processing method, device, equipment, storage medium and product. The method comprises the following steps: obtaining an original image I and an enhanced image E corresponding to the original image I; performing n times of downsampling processing on the original image I to obtain a thumbnail original image I n ’ at a scale P n ’; performing n times of downsampling processing on the enhanced image E to obtain a thumbnail enhanced image set; extracting feature information of each thumbnail enhanced image in the thumbnail enhanced image set at a corresponding scale; and performing fusion processing on an up-sampling image of the thumbnail original image I n ’ and the feature information of each thumbnail enhanced image in the thumbnail enhanced image set at the corresponding scale to obtain a fusion image corresponding to the original image I. The method can be applied to an artificial intelligence model. It can be seen that the thumbnail original image I n ’ is enhanced by fusing the feature information of the thumbnail enhanced image at different scales, thereby improving the image enhancement effect and making the image after the enhancement processing more real.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to an image processing method, apparatus, device, storage medium, and product. Background Technology

[0002] With advancements in scientific research, image restoration technology has been widely applied in the restoration of old images. The core of image restoration technology is enhancing the original image; for example, performing grayscale transformation, spatial domain enhancement, or histogram transformation. However, practice has shown that thumbnail-enhanced images obtained after enhancing the original image often exhibit distortion and poor enhancement results. Summary of the Invention

[0003] This invention provides an image processing method, apparatus, device, storage medium, and product that can improve the effect of image enhancement, making the enhanced image more realistic.

[0004] On one hand, embodiments of this application provide an image processing method, including:

[0005] Obtain the original image I and the corresponding enhanced image E;

[0006] The original image I is downsampled n times to obtain the scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images, which contains thumbnail enhanced images at n scales, where n is a positive integer.

[0007] Extract feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale;

[0008] For the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I.

[0009] On one hand, embodiments of this application provide an image processing apparatus, which includes:

[0010] The acquisition unit is used to acquire the original image I and the enhanced image E corresponding to the original image I;

[0011] The processing unit is used to perform n downsampling operations on the original image I to obtain the scale P. n The original image below is a thumbnail. nThe enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images, which contains thumbnail enhanced images at n scales, where n is a positive integer.

[0012] And features for extracting the features of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale;

[0013] And for the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I.

[0014] In one implementation, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are represented as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i , where i is a positive integer less than or equal to n; the number of fusion processes is n; when j = 1, the process of the j-th fusion process includes:

[0015] For the thumbnail original image I n ', and thumbnail-enhanced image E n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1.

[0016] In one implementation, when j>1 and less than or equal to n-1, the j-th fusion process includes:

[0017] Obtain the intermediate image X obtained from the (j-1)th fusion process. j-1 ;

[0018] For the intermediate image X j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The first image to be fused below;

[0019] For the first image to be fused, and the thumbnail-enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j .

[0020] In one embodiment, the processing unit is further configured to:

[0021] Based on the original thumbnail image I n The upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1.

[0022] In one implementation, the processing unit is used to, based on the thumbnail original image I nThe upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1, which is specifically used for:

[0023] Based on the positional relationship of each pixel in the image, determine the relationship between each pixel in the intermediate image X1 and the original thumbnail image I. n The difference information of corresponding pixels in the upsampled image;

[0024] The first target pixel in the intermediate image X1 is determined based on the difference information;

[0025] Replace the first target pixel with a thumbnail of the original image I. n The corrected image Y1 is obtained by taking the pixels corresponding to the first target pixel in the upsampled image.

[0026] In one implementation, the difference information for each pixel in the intermediate image X1 includes: the difference between that pixel and the original thumbnail image I. n The difference between the color channel values ​​of the corresponding pixel in the upsampled image of ' in the three color channels;

[0027] The processing unit is used to determine the first target pixel in the intermediate image X1 based on the difference information, specifically for:

[0028] If the k-th pixel in the intermediate image X1 is similar to the original image I... n If the difference between the color channel values ​​of the corresponding pixel in the upsampled image of ' is greater than the first difference threshold, then the k-th pixel is determined as the first target pixel.

[0029] The intermediate image X1 includes at least one pixel, and the k-th pixel is any one of the at least one pixels.

[0030] In one implementation, the processing unit is used to, based on the thumbnail original image I n The upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1, which is specifically used for:

[0031] The second target pixel in the intermediate image X1 is determined based on the color channel values ​​of each pixel in the three color channels.

[0032] Replace the second target pixel with a thumbnail of the original image I. n The corrected image Y1 is obtained by finding the pixels in the upsampled image that correspond to the second target pixel.

[0033] In one implementation, the processing unit is configured to determine a second target pixel in the intermediate image X1 based on the color channel values ​​of each pixel in the three color channels, specifically configured to:

[0034] If the difference between the maximum and minimum color channel values ​​of the qth pixel in the intermediate image X1 is greater than the second difference threshold, then the qth pixel is determined as the second target pixel.

[0035] The intermediate image X1 includes at least one pixel, and the q-th pixel is any one of the at least one pixels.

[0036] In one implementation, when j>1 and less than or equal to n-1, the j-th fusion process includes:

[0037] Obtain the corrected image Y j-1 Correcting image Y j-1 The intermediate image X obtained from the (j-1)th fusion process j-1 The result was obtained after correction;

[0038] For the corrected image Y j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The second image to be fused below;

[0039] For the second image to be fused, and the thumbnail-enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j ;

[0040] For the intermediate image X j The correction is performed to obtain the corrected image Y. j .

[0041] In one implementation, when j = n-1, the processing unit is used to process the intermediate image X. j The correction is performed to obtain the corrected image Y. j Specifically used for:

[0042] Based on the original image I, the intermediate image X n-1 The correction is performed to obtain the corrected image Y. n-1 .

[0043] In one implementation, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are represented as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i , where i is a positive integer less than or equal to n; the processing unit is used to extract feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale, specifically for:

[0044] For thumbnail-enhanced image E i'Perform upsampling to obtain a scale of P' i-1 The sampled image;

[0045] Calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the original image is used to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 The following feature information.

[0046] Accordingly, this application provides a smart device, which includes:

[0047] A processor is used to load and execute computer programs;

[0048] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described image processing method.

[0049] Accordingly, this application provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and executed by the above-described image processing method.

[0050] Accordingly, this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned image processing method.

[0051] In this embodiment, the original image I and the enhanced image E corresponding to the original image I are obtained; the original image I is downsampled n times to obtain the scale P. n The original image below is a thumbnail. n '; and perform n downsampling operations on the enhanced image E to obtain a set of thumbnail enhanced images; extract the feature information of each thumbnail enhanced image in the set at the corresponding scale; and perform thumbnail original image I n The upsampled image of the original image I and the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at their respective scales are fused to obtain the fused image corresponding to the original image I. It can be seen that by fusing the feature information of the thumbnail-enhanced images at different scales, the original thumbnail image I can be fused. n The enhancement process improves the image enhancement effect, making the enhanced image more realistic. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 An architecture diagram of an image processing system provided in this application embodiment;

[0054] Figure 2 An image processing method provided in this application embodiment;

[0055] Figure 3 Another image processing method provided in the embodiments of this application;

[0056] Figure 4a This is a schematic diagram illustrating the extraction of feature information provided in an embodiment of this application;

[0057] Figure 4b This is a schematic diagram of an image processing flow provided in an embodiment of this application;

[0058] Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application;

[0059] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0061] This application relates to artificial intelligence. The following is a brief introduction to the relevant terms and concepts of artificial intelligence:

[0062] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0063] AI technology is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, large application processing technologies, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0064] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of AI and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning / deep learning typically includes techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0065] Deep learning: The concept of deep learning originates from the research of artificial neural networks. A multilayer perceptron with multiple hidden layers is a type of deep learning structure. Deep learning discovers distributed feature representations of data by combining low-level features to form more abstract high-level representations of attribute categories or features. This application's embodiments mainly involve training an initial model using a reference dataset to obtain an image enhancement model; the image enhancement model can perform image processing on the acquired original image I and its corresponding enhanced image E, and output a fused image corresponding to the original image I.

[0066] The image processing scheme proposed in the embodiments of this application is briefly introduced below, which can improve the image enhancement effect and make the enhanced image more realistic. Please refer to... Figure 1 , Figure 1 This is an architecture diagram of an image processing system provided in an embodiment of this application. Figure 1As shown, the image processing system may include: a terminal device 101 and a server 102. The image processing method provided in this application embodiment can be executed by the server 102 or by a terminal device equipped with an image processing device. The terminal device 101 may include, but is not limited to: smartphones (such as Android phones, iOS phones, etc.), tablet computers, portable personal computers, mobile internet devices (MIDs), vehicle terminals, and other intelligent devices. This application embodiment does not limit this. The server 102 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This application embodiment does not limit this.

[0067] It should be noted that, Figure 1 The terminal device 101 and the server 102 can be directly or indirectly connected via wired or wireless communication, and this application does not impose any limitations on this. The number of terminal devices and servers is only for illustrative purposes and does not constitute an actual limitation of this application; for example, the image processing system may also include a terminal device 103 or a server 104.

[0068] The general principles of image processing schemes are as follows:

[0069] (1) Server 102 acquires the original image I (such as an old photograph to be restored) and the corresponding enhanced image E; for example, the original image I and the enhanced image E can be sent to the server by the terminal device 101. The content of the original image I and the enhanced image E is the same, and the original image I and the enhanced image E can be grayscale images, color images, infrared images, visible light images, etc. After acquiring the original image I and the enhanced image E, server 102 performs n downsampling operations on the original image I to obtain scale P. n The original image below is a thumbnail. n In one implementation, the length and width of the original image I are respectively the length and width of the scaled-down original image I. n ' ' length and width 2 nThe server 102 performs n downsampling operations on the enhanced image E to obtain a set of thumbnail enhanced images. This set contains thumbnail enhanced images at n scales, where n is a positive integer. For example, if the scales of the original image I and the enhanced image E are 1000*1000, and n is 2, then the server 102 performs two downsampling operations on the original image I to obtain a thumbnailed original image I2' at scale P2 (250*250); and performs two downsampling operations on the enhanced image E to obtain a set of thumbnail enhanced images, including thumbnail enhanced images E1' and E2'. The scale corresponding to thumbnail enhanced image E1' is P1 (500*500), and the scale corresponding to thumbnail enhanced image E2' is P2 (250*250).

[0070] (2) Server 102 extracts feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale. In one embodiment, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set can be represented as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i Where i is a positive integer less than or equal to n. Server 102 extracts the thumbnail-enhanced image E. i-1 'At scale P i-1 The specific method for obtaining the feature information is as follows: Server 102 analyzes the thumbnail-enhanced image E in the thumbnail-enhanced image set. i 'Perform upsampling to obtain a scale of P' i-1 The sampled image; calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the image (e.g., the difference between the thumbnail-enhanced image E) i-1 The RGB values ​​of each pixel in the image are subtracted from the RGB values ​​of the corresponding pixels in the sampled image to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 Feature information below (such as thumbnail enhanced image E) i-1 (The content details contained in ').

[0071] (3) For the thumbnail original image I n The upsampled image of the original image I and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at their respective scales are fused to obtain the fused image corresponding to the original image I. The fused image can be understood as another enhanced image different from the enhanced image E, relative to the original image I. In one implementation, the server 102 performs a fusion process on the thumbnailed original image I... n The process of fusing the feature information of the upsampled image and each thumbnail-enhanced image in the set at the corresponding scale is as follows:

[0072] S11: Thumb up the original image I n In the set of upsampled and thumbnail-enhanced images E n-1 'At scale P n-1 The feature information below is fused (e.g., the original image is thumbnailed). n The RGB values ​​of each pixel in the upsampled image are added to the RGB values ​​of the corresponding pixels in the feature information to obtain the intermediate image X1.

[0073] S12: Upsample the intermediate image X1 to obtain the image at scale P. n-2 The first image to be merged.

[0074] S13: Combine the first image to be fused with the set of thumbnail-enhanced images E n-2 'At scale P n-2 The feature information below is fused to obtain the intermediate image X2.

[0075] Repeat steps S12 and S13 until the fused image (i.e., intermediate image X) corresponding to the original image I is obtained. n-1 The scale of the fused image is P1.

[0076] In another implementation, after each intermediate image is obtained, server 102 can also correct the intermediate image; server 102 performs correction on the thumbnail original image I. n The process of fusing the feature information of the upsampled image and each thumbnail-enhanced image in the set at the corresponding scale is as follows:

[0077] S21: Thumb up the original image I n In the set of upsampled and thumbnail-enhanced images E n-1 'At scale P n-1 The feature information below is fused (e.g., the original image is thumbnailed). n The intermediate image X1 is obtained by adding the RGB values ​​of each pixel in the upsampled image to the corresponding RGB values ​​in the feature information. Based on the thumbnail original image I... n The upsampled image is used to correct the intermediate image X1 to obtain the corrected image Y1.

[0078] S22: Upsample the corrected image Y1 to obtain the image at scale P. n-2 The second image to be fused.

[0079] S23: Combine the second image to be fused with the set of thumbnail-enhanced images E n-2 'At scale P n-2 The feature information below is fused to obtain the intermediate image X2. Based on the thumbnail original image I, the feature information is fused to obtain the intermediate image X2. n-1The upsampled image of ' is used to correct the intermediate image X2 to obtain the corrected image Y2.

[0080] Repeat steps S22 and S23 until an intermediate image X is obtained. n-1 Based on the original image I, the intermediate image X n-1 After correction, the fused image corresponding to image I (i.e., the corrected image Y) is obtained. n-1 The scale of the fused image is P1.

[0081] In this embodiment, the original image I and the enhanced image E corresponding to the original image I are obtained; the original image I is downsampled n times to obtain the scale P. n The original image below is a thumbnail. n '; and perform n downsampling operations on the enhanced image E to obtain a set of thumbnail enhanced images; extract the feature information of each thumbnail enhanced image in the set at the corresponding scale; and perform thumbnail original image I n The upsampled image of the original image I and the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at their respective scales are fused to obtain the fused image corresponding to the original image I. It can be seen that by fusing the feature information of the thumbnail-enhanced images at different scales, the original thumbnail image I can be fused. n The enhancement process improves the image enhancement effect, making the enhanced image more realistic.

[0082] Based on the above image processing scheme, this application proposes a more detailed image processing method. The data transmission method proposed in this application will be described in detail below with reference to the accompanying drawings.

[0083] Please see Figure 2 , Figure 2 An image processing method provided in this application embodiment can be executed by a computer device, specifically, the computer device may be... Figure 1 The server 102 shown can also be a terminal device equipped with an image processing device or an image processing model. For example... Figure 2 As shown, the image processing method may include the following steps S201-S204:

[0084] S201. Obtain the original image I and the enhanced image E corresponding to the original image I.

[0085] The original image I and the enhanced image E contain the same image content. The original image I and the enhanced image E can be grayscale images, color images, infrared images, visible light images, etc. In one embodiment, the original image I and the enhanced image E have the same image type; for example, when the original image I is a color image, the enhanced image E is also a color image; similarly, when the original image I is a grayscale image, the enhanced image E is also a grayscale image.

[0086] S202. Perform n downsampling operations on the original image I to obtain scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images.

[0087] Downsampling methods can include, but are not limited to, Nearest Neighbor Interpolation, Bilinear Interpolation, and Bicubic Interpolation. The set of thumbnail-enhanced images contains thumbnail-enhanced images at n scales, where n is a positive integer.

[0088] In one implementation, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set can be represented as E1'…E n ', E i 'This is the thumbnail enhanced image obtained after downsampling the enhanced image E i times.' i The corresponding scale is P i , where i is a positive integer less than or equal to n. For example, suppose the scale of the original image I and the enhanced image E is 1000*1000, and n is 3; then the server 102 performs downsampling processing on the original image I 3 times to obtain a thumbnailed original image I3' at scale P3 (125*125); and performs downsampling processing on the enhanced image E 3 times to obtain a set of thumbnailed enhanced images, which includes a thumbnailed enhanced image E1' after one downsampling processing of the enhanced image E, a thumbnailed enhanced image E2' after two downsampling processing, and a thumbnailed enhanced image E3' after three downsampling processing; where the scale corresponding to the thumbnailed enhanced image E1' is P1 (500*500), the scale corresponding to the thumbnailed enhanced image E2' is P2 (250*250), and the scale corresponding to the thumbnailed enhanced image E3' is P3 (125*125).

[0089] S203. Extract the feature information of each thumbnail-enhanced image in the set of thumbnail-enhanced images at the corresponding scale.

[0090] In one implementation, the feature information can specifically be an image; for example, a thumbnail-enhanced image E. i'At scale P i The feature information below is a map with a scale of P. i The image.

[0091] The computer device extracts feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale in the same way. Taking the extraction of feature information of thumbnail-enhanced image E from the thumbnail-enhanced image set by the computer device as an example... i-1 At the corresponding scale (i.e., P) i-1 Taking the feature information of ) as an example: the computer analyzes the thumbnail-enhanced image E in the set of thumbnail-enhanced images. i 'Perform upsampling to obtain a scale of P' i-1 The sampled image; calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the original image is used to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 The following feature information.

[0092] For example, suppose the enhanced image E is a color image (i.e., a thumbnail-enhanced image E). i-1 'and thumbnail enhanced image E i (All are color images), then the computer device calculates the thumbnail-enhanced image E. i-1 The method for finding the difference between the sampled image and the thumbnail-enhanced image E is as follows: i-1 The color channel value (i.e., RGB value) of each pixel in the sample image is subtracted from the RGB value of the corresponding pixel in the sampled image.

[0093] S204, For the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I.

[0094] The fused image, in relation to the original image I, can be understood as another enhanced image, different from the enhanced image E. In one implementation, a computer device performs a thumbnail-processing of the original image I. n The process of fusing the feature information of the upsampled image and each thumbnail-enhanced image in the set at the corresponding scale is as follows:

[0095] S11: Thumb up the original image I n In the set of upsampled and thumbnail-enhanced images E n-1 'At scale P n-1 The feature information is fused to obtain the intermediate image X1. Taking the original image I and the enhanced image E as color images as an example, the original image I is scaled down. n The RGB values ​​of each pixel in the upsampled image and the E values ​​in the thumbnail-enhanced image set.n-1 'At scale P n-1 The RGB values ​​of the corresponding pixels in the feature information below are added together to obtain the intermediate image X1.

[0096] S12: Upsample the intermediate image X1 to obtain the image at scale P. n-2 The first image to be merged.

[0097] S13: Combine the first image to be fused with the set of thumbnail-enhanced images E n-2 'At scale P n-2 The feature information below is fused to obtain the intermediate image X2.

[0098] Repeat steps S12 and S13 until the fused image (i.e., intermediate image X) corresponding to the original image I is obtained. n-1 The scale of the fused image is P1.

[0099] In another implementation, after each intermediate image is obtained, server 102 can also correct the intermediate image; server 102 performs correction on the thumbnail original image I. n The process of fusing the feature information of the upsampled image and each thumbnail-enhanced image in the set at the corresponding scale is as follows:

[0100] S21: Thumb up the original image I n 'The set of upsampled images and thumbnail-enhanced images E' n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1. Based on the thumbnailed original image I, the feature information is fused to obtain the intermediate image X1. n The upsampled image is used to correct the intermediate image X1 to obtain the corrected image Y1.

[0101] S22: Upsample the corrected image Y1 to obtain the image at scale P. n-2 The second image to be fused.

[0102] S23: Combine the second image to be fused with the set of thumbnail-enhanced images E n-2 'At scale P n-2 The feature information below is fused to obtain the intermediate image X2. Based on the thumbnail original image I, the feature information is fused to obtain the intermediate image X2. n-1 The upsampled image is used to correct the intermediate image X2 to obtain the corrected image Y2.

[0103] Repeat steps S22 and S23 until an intermediate image X is obtained. n-1 Based on the original image I, the intermediate image X n-1 After correction, the fused image corresponding to the original image I (i.e., the corrected image Y) is obtained.n-1 The scale of the fused image is P1.

[0104] In this embodiment, the original image I and the enhanced image E corresponding to the original image I are obtained; the original image I is downsampled n times to obtain the scale P. n The original image below is a thumbnail. n '; and perform n downsampling operations on the enhanced image E to obtain a set of thumbnail enhanced images; extract the feature information of each thumbnail enhanced image in the set at the corresponding scale; and perform thumbnail original image I n The upsampled image of the original image I and the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at their respective scales are fused to obtain the fused image corresponding to the original image I. It can be seen that by fusing the feature information of the thumbnail-enhanced images at different scales, the original thumbnail image I can be fused. n The enhancement process improves the image enhancement effect, making the enhanced image more realistic.

[0105] Please see Figure 3 , Figure 3 Another image processing method provided in this application embodiment can be executed by a computer device, specifically, the computer device may be... Figure 1 The server 102 shown can also be a terminal device equipped with an image processing device or an image processing model. For example... Figure 3 As shown, the image processing method may include the following steps S301-S306:

[0106] S301. Obtain the original image I and the enhanced image E corresponding to the original image I.

[0107] S302. Perform n downsampling operations on the original image I to obtain the scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images.

[0108] For detailed implementation methods of steps S301 and S302, please refer to Figure 2 The implementation methods of steps S201 and S202 will not be described in detail here.

[0109] S303. Extract the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale.

[0110] The thumbnail-enhanced images at n scales in the thumbnail-enhanced image set can be represented as E1'…E n ', E i 'This is the thumbnail enhanced image obtained after downsampling the enhanced image E i times.'i The corresponding scale is P i Where i is a positive integer less than or equal to n. Computer equipment extracts thumbnail-enhanced image E. i-1 'At scale P i-1 The following feature information can be represented as:

[0111] LapE i-1 =E i-1 '-upsample(E i ')

[0112] Among them, LapE i-1 Enhance image E for thumbnail i-1 'At scale P i-1 Feature information below, upsample(E) i ') indicates the thumbnail enhancement of image E i Perform upsampling.

[0113] Figure 4a This is a schematic diagram illustrating feature information extraction provided in an embodiment of this application. For example... Figure 4a As shown, the thumbnail-enhanced image E i The corresponding scale is P i Thumbnail Enhanced Image E i-1 The corresponding scale is P i-1 Computer equipment enhances thumbnail images E i 'Perform upsampling to obtain a scale of P' i-1 The sampled image; calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the image (e.g., the difference between the thumbnail-enhanced image E) i-1 The RGB values ​​of each pixel in the image are subtracted from the RGB values ​​of the corresponding pixels in the sampled image to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 The feature information below, that is, the thumbnail enhanced image E i-1 'At scale P i-1 The feature information below is based on the thumbnail enhanced image E i-1 'and thumbnail enhanced image E i The above method allows computer equipment to extract feature information of each thumbnail-enhanced image at its corresponding scale.

[0114] S304, For the thumbnail original image I n The upsampled image of ' and the thumbnail-enhanced image E n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1.

[0115] Thumbnail Original Image I n' is the thumbnail-enhanced image E obtained by downsampling the original image I n times. n-1 ' is obtained by downsampling the enhanced image E corresponding to the original image I (n-1 times). (Original image I is shown in the image below.) n The scale of ' is P n Thumbnail Enhanced Image E n-1 The scale of ' is P n-1 As can be seen from step S303, the thumbnail-enhanced image E n-1 'At scale P n-1 The feature information below can be based on the thumbnail enhanced image E n-1 'and thumbnail enhanced image E n 'get.

[0116] In one implementation, the computer device displays a thumbnail of the original image I. n The upsampled image of ' and the thumbnail-enhanced image E n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1, which can be represented as:

[0117] X1 = LapE n-1 +upsample(I n ')

[0118] Where X1 is the intermediate image (the image after the first fusion process), LapE n-1 Enhance image E for thumbnail n-1 'At scale P n-1 Feature information below, upsample(I n ') is a thumbnail of the original image I n The upsampled image.

[0119] Fusion processing can be understood as shrinking the original image I n The basic information of each pixel in the upsampled image is compared with the thumbnail-enhanced image E. n-1 'At scale P n-1 The feature information is superimposed; for example, for color images, fusion processing refers to combining the features of the original image I into a single image. n The RGB values ​​of each pixel in the upsampled image are added to the RGB values ​​of the corresponding pixels in the feature information to obtain the intermediate image X1. Specifically, assuming the original image I is scaled down... n The RGB values ​​of the t-th pixel in the upsampled image are: R = 155, G = 86, B = 133; the RGB values ​​of the corresponding pixel in the feature information are: R = 15, G = 32, B = 3; then the original image I after fusion processing is downsized. nThe RGB values ​​of the t-th pixel in the upsampled image are: R = 170, G = 118, B = 136. For example, for grayscale images, fusion processing refers to shrinking the original image I... n The grayscale values ​​of each pixel in the upsampled image and the thumbnail-enhanced image E n-1 'At scale P n-1 The gray values ​​of the corresponding pixels in the feature information below are added together to obtain the intermediate image X1.

[0120] Furthermore, when j>2, the process of the j-th fusion process is as follows: obtain the intermediate image X obtained from the (j-1)-th fusion process. j-1 ; For the intermediate image X j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The first image to be fused; the first image to be fused, and the thumbnail enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j For example, after obtaining an intermediate image X1, the computer device upsamples the intermediate image X1 to obtain an image at scale P. n-2 The first image to be fused is then combined with the image from the set of thumbnail-enhanced images E. n-2 'At scale P n-2 The feature information below is fused to obtain the intermediate image X2. By repeating the above method, the fused image (i.e., the intermediate image X) corresponding to the original image I can be obtained. n-1 The scale of the merged image is P1. It should be noted that the fusion process can reduce the original image I to a smaller scale. n The low-frequency information in the upsampled image is used as the image content and color of the fused image, and the thumbnail-enhanced image E is then used. n-1 'At scale P n-1 The feature information (i.e., high-frequency information) is used as supplementary details of the fused image, and the original image I is enhanced through iterative iteration.

[0121] In another implementation, after obtaining the intermediate image X1, the computer device continues to execute step S305.

[0122] S305, Based on the original thumbnail image I n The intermediate image X1 is corrected to obtain the corrected image Y1.

[0123] In one implementation, based on the thumbnail original image I n Correcting the intermediate image X1 means: based on the thumbnail original image I nThe upsampled image is used to filter out the first target pixel (i.e., anomaly) in the intermediate image X1, and the original image I is then scaled down. n The upsampled image X1 optimizes the first target pixel in the intermediate image X1 to obtain the corrected image Y1. Specifically, the computer device determines the relationship between each pixel in the intermediate image X1 and the original image I based on the positional relationship of each pixel in the image. n The difference information of corresponding pixels in the upsampled image is obtained; and the first target pixel in the intermediate image X1 is determined based on the difference information; the first target pixel is replaced with the thumbnail original image I. n The corrected image Y1 is obtained by taking the pixels corresponding to the first target pixel in the upsampled image.

[0124] In one embodiment, the difference information for each pixel in the intermediate image X1 includes: the difference between the pixel and the original thumbnail image I. n The difference between the color channel values ​​of the corresponding pixel in the upsampled image X1 across the three color channels. The specific implementation method for the computer device to determine the first target pixel in the intermediate image X1 based on the difference information is as follows: Determine the k-th pixel in the intermediate image X1 and the pixel in the thumbnail original image I. n In the upsampled image of ', for each corresponding pixel, does there exist at least one color channel value difference greater than a first difference threshold? If the k-th pixel in the intermediate image X1 differs from the original image I... n If the difference between the color channel values ​​of corresponding pixels in at least one color channel of the upsampled image is greater than a first difference threshold, then the k-th pixel is determined as the first target pixel; wherein, the intermediate image X1 includes at least one pixel, and the k-th pixel is any one of the at least one pixels. When the first difference threshold is 50, the corrected image Y1 can be represented as:

[0125] Y1 = upsample(I n ')(|X1-upsample(I n ')|>50)

[0126] Y1 = X1(|X1-upsample(I n ')|≤50)

[0127] Where Y1 is the corrected image, upsample(I n ') is a thumbnail of the original image I n The upsampled image of ', where X1 is the intermediate image, when |X1-upsample(I n When ')|>50, Y1=upsample(I n '). This indicates that if the pixels in the intermediate image X1 are similar to those in the thumbnail original image I...n If, in the upsampled image, the pixel has a color channel value difference greater than 50 across at least one of the three color channels, then that pixel is replaced with the original thumbnail image I. n The corresponding pixel in the upsampled image; when |X1-upsample(I n When | ≤ 50, Y1 = X1. This means that if the pixels in the intermediate image X1 are similar to those in the original image X1, then X1 = X1. n If the difference between the color channel values ​​of the corresponding pixel in the upsampled image is less than or equal to 50 in all three color channels, then the pixel is kept unchanged (that is, the pixel in the intermediate image X1 is determined as the pixel in the corrected image Y1).

[0128] For example, suppose the intermediate image X1 includes pixels 1 (a, b) and pixels 2 (c, d); then the original image I is scaled down. n The coordinates of pixel 3 corresponding to pixel 1 (a, b) in the upsampled image X1 are (a, b), and the coordinates of pixel 4 corresponding to pixel 2 (c, d) are (c, d). Let the RGB values ​​of pixel 1 be: R = 155, G = 86, B = 133; the RGB values ​​of pixel 2 be: R = 165, G = 186, B = 33; the RGB values ​​of pixel 3 be: R = 153, G = 83, B = 136; and the RGB values ​​of pixel 4 be: R = 155, G = 176, B = 180. The first difference threshold is 50. Since the difference between the color channel values ​​of pixel 1 and pixel 3 in the R, G, and B color channels is less than 50, pixel 1 in the intermediate image X1 remains unchanged. Since the difference between the color channel values ​​of pixel 2 and pixel 4 in the B color channel is greater than 50, pixel 2 in the intermediate image X1 is replaced with the original image I. n The upsampled image of pixel 4 is used to determine the intermediate image X1 after optimizing all pixels in the intermediate image X1 according to the above method, and the intermediate image X1 is determined as the corrected image Y1.

[0129] In another implementation, based on the thumbnail original image I n Correcting the intermediate image X1 means: based on the pixel information of each pixel in the intermediate image X1, filtering out the second target pixel (i.e., outlier) in the intermediate image X1, and then correcting it by shrinking the original image I. n The upsampled image is used to optimize the second target pixel in the intermediate image X1 to obtain the corrected image Y1. Specifically, the computer device determines the second target pixel in the intermediate image X1 based on the color channel values ​​of each pixel in the three color channels; the second target pixel is then replaced with a thumbnail of the original image Y1. nThe corrected image Y1 is obtained by finding the pixels in the upsampled image that correspond to the second target pixel.

[0130] In one embodiment, the computer device determines the second target pixel in the intermediate image X1 based on the color channel values ​​of each pixel in the three color channels as follows: It is determined whether the difference between the maximum and minimum color channel values ​​of the q-th pixel in the intermediate image X1 is greater than a second difference threshold; if the difference between the maximum and minimum color channel values ​​of the q-th pixel in the intermediate image X1 is greater than the second difference threshold, then the q-th pixel is determined as the second target pixel; wherein the intermediate image X1 includes at least one pixel, and the q-th pixel is any one of the at least one pixels. Assuming the second difference threshold is 15, and the coordinates of the q-th pixel are (x, y), then when Max(X... 1R (x, y), X 1G (x, y), X 1B (x, y))-Min(X) 1R (x, y), X 1G (x, y), X 1B When (x, y))>15, the corrected image Y1 can be expressed as: Y1(x, y)=upsample(I n '(x, y)); when Max(X) 1R (x, y), X 1G (x, y), X 1B (x, y))-Min(X) 1R (x, y), X 1G (x, y), X 1B When (x, y) ≤ 15, the corrected image Y1 can be expressed as: Y1(x, y) = X1(x, y). Where X... 1R (x, y) represents the color channel value of pixel (x, y) in the red (R) channel of the intermediate image X1; X 1G (x, y) represents the color channel value of pixel (x, y) in the green (G) channel of the intermediate image X1; X 1B (x, y) represents the color channel value of pixel (x, y) in the blue (B) channel of the intermediate image X1. Y1(x, y) = upsample(I n '(x, y)) means replacing the pixel (x, y) in the corrected image Y1 with the thumbnail of the original image I. n The pixel (x, y) in the upsampled image is Y1(x, y); Y1(x, y) = X1(x, y) means replacing the pixel (x, y) in the corrected image Y1 with the pixel (x, y) in the intermediate image X1.

[0131] For example, suppose the intermediate image X1 includes pixels 1 (a, b) and pixels 2 (c, d); then the original image I is scaled down. n The coordinates of pixel 3 corresponding to pixel 1 (a, b) in the upsampled image X1 are (a, b), and the coordinates of pixel 4 corresponding to pixel 2 (c, d) are (c, d). Let the RGB values ​​of pixel 1 be R = 155, G = 145, B = 148; the RGB values ​​of pixel 2 be R = 165, G = 186, B = 150; and the second difference threshold be 15. Since the difference between the maximum and minimum color channel values ​​in each color channel of pixel 1 is less than 15, pixel 1 in the intermediate image X1 remains unchanged. Since the difference between the maximum and minimum color channel values ​​in each color channel of pixel 2 is greater than 15, pixel 2 in the intermediate image X1 is replaced with the original image I. n The upsampled image of pixel 4 is used to determine the intermediate image X1 after optimizing all pixels in the intermediate image X1 according to the above method, and the intermediate image X1 is determined as the corrected image Y1.

[0132] It should be noted that the two implementation methods described above can be applied simultaneously in the correction process of the intermediate image X1.

[0133] S306. Based on the corrected image Y1, the thumbnail-enhanced images E1' to E1' in the thumbnail-enhanced image set. n-2 'Based on the feature information at the corresponding scale and the original image I, the corrected image Y corresponding to the original image I is obtained.' n .

[0134] Computer equipment based on corrected image Y1 and thumbnail enhanced image E n-2 Following the implementation method in step S305, the corrected image Y2 can be obtained; similarly, let the number of fusions be j. When j>1 and less than or equal to n-1, the computer device obtains the corrected image Y. j The process is as follows: Obtain the corrected image Y j-1 Correcting image Y j-1 Based on the thumbnail original image I n-j+2 The upsampled image is used to fuse the intermediate image X obtained by the (j-1)th fusion process. j-1 The result after correction is shown in step S305; the specific correction method will not be repeated here. For the corrected image Y... j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The second image to be fused; the second image to be fused, and the thumbnail enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. jBased on the thumbnail original image I n-j+1 The upsampled image of ' to the intermediate image X j The correction is performed to obtain the corrected image Y. j .

[0135] When j = n-1, the computer device processes the intermediate image X based on the original image I. n-1 The correction is performed to obtain the corrected image Y. n-1 For specific calibration methods, please refer to step S305, which will not be repeated here.

[0136] Figure 4b This is a schematic diagram of an image processing flow provided in an embodiment of this application. Figure 4b As shown, the computer device acquires the original image I and the corresponding enhanced image E. It then performs n downsampling operations on both the original image I and the enhanced image E, where n is a positive integer. Using the set of thumbnail enhanced images obtained by performing n downsampling operations on the enhanced image E, the feature information of each thumbnail enhanced image at its corresponding scale is extracted. The specific method can be found in step S303. The thumbnailed original image I is obtained after performing n downsampling operations on the original image I. n The upsampled image of ' and each thumbnail-enhanced image E in the set of thumbnail-enhanced images. n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1, which is based on the thumbnail original image I. n The upsampled image of X1 is used to correct the intermediate image X1, resulting in the corrected image Y1; the upsampled image of the corrected image Y1, and each thumbnail-enhanced image E in the thumbnail-enhanced image set are then compared. n-2 'At scale P n-2 The feature information below is fused to obtain the intermediate image X2. Similarly, the intermediate image X can be obtained using the same method. n-1 Based on the original image I, the intermediate image X n-1 After correction, the fused image corresponding to the original image I is obtained.

[0137] It should be noted that the above method can be encapsulated in an image enhancement model, which can output a fused image corresponding to the original image I based on the acquired original image I and the enhanced image E corresponding to the original image I.

[0138] In this embodiment, the original image I and the enhanced image E corresponding to the original image I are obtained; the original image I is downsampled n times to obtain the scale P. n The original image below is a thumbnail. n '; and perform n downsampling operations on the enhanced image E to obtain a set of thumbnail enhanced images; extract the feature information of each thumbnail enhanced image in the set at the corresponding scale; and perform thumbnail original image In The upsampled image of the original image I and the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at their respective scales are fused to obtain the fused image corresponding to the original image I. It can be seen that by fusing the feature information of the thumbnail-enhanced images at different scales, the original thumbnail image I can be fused. n Enhancement is performed to improve the image enhancement effect, making the enhanced image more realistic. Furthermore, feature information at the corresponding scale of each thumbnail enhanced image is extracted from two adjacent thumbnail enhanced images; and outliers in the intermediate image are optimized to further improve the image enhancement effect, making the enhanced image more realistic.

[0139] The methods of the embodiments of this application have been described in detail above. In order to facilitate better implementation of the above solutions of the embodiments of this application, the apparatus of the embodiments of this application is provided below.

[0140] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. The apparatus can be mounted on a computer device, specifically... Figure 1 The server 102 shown can also be a terminal device 101. Figure 5 The image processing apparatus shown can be used to perform the above. Figure 2 and Figure 3 Some or all of the functionality described in the method embodiments. Please refer to [link / reference]. Figure 5 The detailed descriptions of each unit are as follows:

[0141] The acquisition unit is used to acquire the original image I and the enhanced image E corresponding to the original image I;

[0142] The processing unit is used to perform n downsampling operations on the original image I to obtain the scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images, which contains thumbnail enhanced images at n scales, where n is a positive integer.

[0143] And features for extracting the features of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale;

[0144] And for the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I.

[0145] In one implementation, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are represented as E1'…En ', thumbnail enhanced image E i The corresponding scale is P i , where i is a positive integer less than or equal to n; the number of fusion processes is n; when j = 1, the process of the j-th fusion process includes:

[0146] For the thumbnail original image I n ', and thumbnail-enhanced image E n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1.

[0147] In one implementation, when j>1 and less than or equal to n-1, the j-th fusion process includes:

[0148] Obtain the intermediate image X obtained from the (j-1)th fusion process. j-1 ;

[0149] For the intermediate image X j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The first image to be fused below;

[0150] For the first image to be fused, and the thumbnail-enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j .

[0151] In one embodiment, the processing unit 502 is further configured to:

[0152] Based on the original thumbnail image I n The upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1.

[0153] In one embodiment, the processing unit 502 is configured to, based on the thumbnail original image I n The upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1, which is specifically used for:

[0154] Based on the positional relationship of each pixel in the image, determine the relationship between each pixel in the intermediate image X1 and the original thumbnail image I. n The difference information of corresponding pixels in the upsampled image;

[0155] The first target pixel in the intermediate image X1 is determined based on the difference information;

[0156] Replace the first target pixel with a thumbnail of the original image I. n The corrected image Y1 is obtained by taking the pixels corresponding to the first target pixel in the upsampled image.

[0157] In one implementation, the difference information for each pixel in the intermediate image X1 includes: the difference between that pixel and the original thumbnail image I. n The difference between the color channel values ​​of the corresponding pixel in the upsampled image of ' in the three color channels;

[0158] Processing unit 502 is used to determine the first target pixel in the intermediate image X1 based on the difference information, specifically for:

[0159] If the k-th pixel in the intermediate image X1 is similar to the original image I... n If the difference between the color channel values ​​of the corresponding pixel in the upsampled image of ' is greater than the first difference threshold, then the k-th pixel is determined as the first target pixel.

[0160] The intermediate image X1 includes at least one pixel, and the k-th pixel is any one of the at least one pixels.

[0161] In one embodiment, the processing unit 502 is configured to, based on the thumbnail original image I n The upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1, which is specifically used for:

[0162] The second target pixel in the intermediate image X1 is determined based on the color channel values ​​of each pixel in the three color channels.

[0163] Replace the second target pixel with a thumbnail of the original image I. n The corrected image Y1 is obtained by finding the pixels in the upsampled image that correspond to the second target pixel.

[0164] In one embodiment, the processing unit 502 is configured to determine a second target pixel in the intermediate image X1 based on the color channel values ​​of each pixel in the three color channels, specifically configured to:

[0165] If the difference between the maximum and minimum color channel values ​​of the qth pixel in the intermediate image X1 is greater than the second difference threshold, then the qth pixel is determined as the second target pixel.

[0166] The intermediate image X1 includes at least one pixel, and the q-th pixel is any one of the at least one pixels.

[0167] In one implementation, when j>1 and less than or equal to n-1, the j-th fusion process includes:

[0168] Obtain the corrected image Y j-1 Correcting image Yj-1 The intermediate image X obtained from the (j-1)th fusion process j-1 The result was obtained after correction;

[0169] For the corrected image Y j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The second image to be fused below;

[0170] For the second image to be fused, and the thumbnail-enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j ;

[0171] For the intermediate image X j The correction is performed to obtain the corrected image Y. j .

[0172] In one implementation, when j = n-1, the processing unit 502 is used to process the intermediate image X. j The correction is performed to obtain the corrected image Y. j Specifically used for:

[0173] Based on the original image I, the intermediate image X n-1 The correction is performed to obtain the corrected image Y. n-1 .

[0174] In one implementation, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are represented as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i Where i is a positive integer less than or equal to n; the processing unit 502 is used to extract the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale, specifically for:

[0175] For thumbnail-enhanced image E i 'Perform upsampling to obtain a scale of P' i-1 The sampled image;

[0176] Calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the original image is used to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 The following feature information.

[0177] According to one embodiment of this application, Figure 2 and Figure 3 The image processing method shown may involve some steps that can be derived from... Figure 5 The image processing apparatus shown is executed by each unit. For example, Figure 2Step S201 shown can be performed by Figure 5 The acquisition unit 501 shown is executed, and steps S202-S204 can be performed by... Figure 5 The processing unit 502 shown executes the operation; Figure 3 Step S301 shown can be performed by Figure 5 The acquisition unit 501 shown is executed, and steps S302-S306 can be performed by... Figure 5 The processing unit 502 shown is executed. Figure 5 The various units in the image processing apparatus shown can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the image processing apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0178] According to another embodiment of this application, the following can be executed by running on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM). Figure 2 and Figure 3 The computer program (including program code) for each step involved in the corresponding method shown, to construct such... Figure 5 The image processing apparatus shown herein, and the image processing method for implementing the embodiments of this application, are described. A computer program may be recorded on, for example, a computer-readable recording medium, loaded onto the aforementioned computing device via the computer-readable recording medium, and executed therein.

[0179] Based on the same inventive concept, the principle and beneficial effects of the image processing apparatus provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the image processing method in the embodiments of this application in solving the problem. For the sake of brevity, the principle and beneficial effects of the method implementation can be referred to.

[0180] Please see Figure 6 , Figure 6 This application provides a schematic diagram of the structure of a computer device, as shown in the embodiment of the present application. Figure 6As shown, the computer device includes at least a processor 601, a communication interface 602, and a memory 603. The processor 601, communication interface 602, and memory 603 can be connected via a bus or other means. The processor 601 (or Central Processing Unit, CPU) is the computing and control core of the terminal. It can parse various instructions within the terminal and process various types of data. For example, the CPU can parse power-on / off commands sent by the user to the terminal and control the terminal to perform power-on / off operations; it can also transmit various interactive data between internal structures within the terminal, and so on. The communication interface 602 can optionally include standard wired interfaces and wireless interfaces (such as Wi-Fi, mobile communication interfaces, etc.), and can be used to send and receive data under the control of the processor 601; the communication interface 602 can also be used for data transmission and interaction within the terminal. The memory 603 is the memory device in the terminal, used to store programs and data. It can be understood that the memory 603 here can include the terminal's built-in memory, or it can include extended memory supported by the terminal. The memory 603 provides storage space for storing the terminal's operating system, which may include, but is not limited to, Android, iOS, Windows Phone, etc. This application does not limit this.

[0181] This application embodiment also provides a computer-readable storage medium (Memory), which is a memory device in a terminal for storing programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal and extended storage media supported by the terminal. The computer-readable storage medium provides storage space that stores the terminal's processing system. Furthermore, the storage space also stores one or more instructions suitable for loading and execution by the processor 601, which can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer-readable storage medium located remotely from the aforementioned processor.

[0182] In one implementation, the computer device may specifically be Figure 1 The server 102 shown, or the terminal device 101 equipped with an image processing unit, performs the following operations by running executable program code stored in memory 603:

[0183] The original image I and the corresponding enhanced image E are obtained through the communication interface 602.

[0184] The original image I is downsampled n times to obtain the scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images, which contains thumbnail enhanced images at n scales, where n is a positive integer.

[0185] Extract feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale;

[0186] For the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I.

[0187] As an optional embodiment, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are represented as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i , where i is a positive integer less than or equal to n; the number of fusion processes is n; when j = 1, the process of the j-th fusion process includes:

[0188] For the thumbnail original image I n ', and thumbnail-enhanced image E n-1 'At scale P n-1 The feature information below is fused to obtain the intermediate image X1.

[0189] As an optional embodiment, when j>1 and less than or equal to n-1, the j-th fusion process includes:

[0190] Obtain the intermediate image X obtained from the (j-1)th fusion process. j-1 ;

[0191] For the intermediate image X j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The first image to be fused below;

[0192] For the first image to be fused, and the thumbnail-enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j .

[0193] As an optional embodiment, the processor 601, by running executable program code in the memory 603, also performs the following operations:

[0194] Based on the original thumbnail image I n The upsampled image is used to correct the intermediate image X1, resulting in the corrected image Y1.

[0195] As an optional embodiment, processor 601 is based on the thumbnail original image I n The upsampled image X1 is used to correct the intermediate image X1, resulting in the corrected image Y1. A specific implementation example is as follows:

[0196] Based on the positional relationship of each pixel in the image, determine the relationship between each pixel in the intermediate image X1 and the original thumbnail image I. n The difference information of corresponding pixels in the upsampled image;

[0197] The first target pixel in the intermediate image X1 is determined based on the difference information;

[0198] Replace the first target pixel with a thumbnail of the original image I. n The corrected image Y1 is obtained by taking the pixels corresponding to the first target pixel in the upsampled image.

[0199] As an optional embodiment, the difference information for each pixel in the intermediate image X1 includes: the pixel's appearance compared to the thumbnail original image I. n The difference between the color channel values ​​of the corresponding pixel in the upsampled image of ' in the three color channels;

[0200] The specific implementation of the processor 601 determining the first target pixel in the intermediate image X1 based on the difference information is as follows:

[0201] If the k-th pixel in the intermediate image X1 is similar to the original image I... n If the difference between the color channel values ​​of the corresponding pixel in the upsampled image of ' is greater than the first difference threshold, then the k-th pixel is determined as the first target pixel.

[0202] The intermediate image X1 includes at least one pixel, and the k-th pixel is any one of the at least one pixels.

[0203] As an optional embodiment, processor 601 is based on the thumbnail original image I n The specific implementation of correcting the intermediate image X1 by using the upsampled image is as follows:

[0204] The second target pixel in the intermediate image X1 is determined based on the color channel values ​​of each pixel in the three color channels.

[0205] Replace the second target pixel with a thumbnail of the original image I.n The corrected image Y1 is obtained by finding the pixels in the upsampled image that correspond to the second target pixel.

[0206] As an optional embodiment, the processor 601 determines the second target pixel in the intermediate image X1 based on the color channel values ​​of each pixel in the three color channels. A specific embodiment of this is as follows:

[0207] If the difference between the maximum and minimum color channel values ​​of the qth pixel in the intermediate image X1 is greater than the second difference threshold, then the qth pixel is determined as the second target pixel.

[0208] The intermediate image X1 includes at least one pixel, and the q-th pixel is any one of the at least one pixels.

[0209] As an optional embodiment, when j>1 and less than or equal to n-1, the j-th fusion process includes:

[0210] Obtain the corrected image Y j-1 Correcting image Y j-1 The intermediate image X obtained from the (j-1)th fusion process j-1 The result was obtained after correction;

[0211] For the corrected image Y j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The second image to be fused below;

[0212] For the second image to be fused, and the thumbnail-enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j ;

[0213] For the intermediate image X j The correction is performed to obtain the corrected image Y. j .

[0214] As an optional embodiment, when j = n-1, the processor 601 processes the intermediate image X. j The correction is performed to obtain the corrected image Y. j The specific implementation is as follows:

[0215] Based on the original image I, the intermediate image X n-1 The correction is performed to obtain the corrected image Y. n-1 .

[0216] As an optional embodiment, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are represented as E1'…E n', thumbnail enhanced image E i The corresponding scale is P i Where i is a positive integer less than or equal to n; a specific embodiment of the processor 601 extracting feature information of each thumbnail-enhanced image at the corresponding scale from the set of thumbnail-enhanced images is as follows:

[0217] For thumbnail-enhanced image E i 'Perform upsampling to obtain a scale of P' i-1 The sampled image;

[0218] Calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the original image is used to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 The following feature information.

[0219] Based on the same inventive concept, the principle and beneficial effects of the computer device provided in the embodiments of this application in solving the problem are similar to the principle and beneficial effects of the image processing method in the embodiments of this application in solving the problem. Please refer to the principle and beneficial effects of the implementation of the method. For the sake of brevity, they will not be repeated here.

[0220] This application also provides a computer-readable storage medium storing one or more instructions adapted for loading by a processor and executing the image processing method described in the above-described method embodiments.

[0221] This application also provides a computer program product containing instructions that, when run on a computer, causes the computer to execute the image processing method described in the above method embodiments.

[0222] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing method described above.

[0223] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0224] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0225] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc.

[0226] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art will understand that all or part of the processes for implementing the above embodiments and equivalent variations made in accordance with the claims of this application are still within the scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: Obtain the original image I and the enhanced image E corresponding to the original image I; The original image I is downsampled n times to obtain scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images, which contains thumbnail enhanced images at n scales, where n is a positive integer. Extract the feature information of each thumbnail-enhanced image in the thumbnail-enhanced image set at the corresponding scale; For the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I; Wherein, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are denoted as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i , i is a positive integer less than or equal to n; the number of fusion processes is n, and any fusion process is represented as the j-th fusion process, where j is a positive integer less than or equal to n-1; when j=1, the process of the j-th fusion process includes: processing the thumbnail original image I n 'and thumbnail enhanced image E n-1 'At scale P n-1 The feature information is fused to obtain an intermediate image X1; when j>1 and less than or equal to n-1, the j-th fusion process includes: obtaining the intermediate image X1 obtained from the (j-1)-th fusion process. j-1 ; for the intermediate image X j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The first image to be fused; and the thumbnail enhanced image E. n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j ; wherein, the intermediate image X j This refers to the image after the j-th fusion process, where the fused image corresponding to the original image I includes the intermediate image X. n-1 Or for the intermediate image X n-1 The corrected image Y obtained by correction n-1 .

2. The method as described in claim 1, characterized in that, The method further includes: Based on the thumbnail original image I n The upsampled image of the intermediate image X1 is used to correct the image Y1, resulting in the corrected image Y1.

3. The method as described in claim 2, characterized in that, The original image based on the thumbnail I n The upsampled image X1 is used to correct the intermediate image X1 to obtain the corrected image Y1, including: Based on the positional relationship of each pixel in the image, determine the relationship between each pixel in the intermediate image X1 and the original thumbnail image I. n The difference information of corresponding pixels in the upsampled image; The first target pixel in the intermediate image X1 is determined based on the difference information; Replace the first target pixel with the thumbnail original image I. n The corrected image Y1 is obtained by taking the pixels corresponding to the first target pixel in the upsampled image of '.

4. The method as described in claim 3, characterized in that, The difference information of each pixel in the intermediate image X1 includes: the difference between the pixel and the original thumbnail image I. n The difference between the color channel values ​​of the corresponding pixel in the upsampled image X1 across the three color channels; determining the first target pixel in the intermediate image X1 based on the difference information includes: If the k-th pixel in the intermediate image X1 is similar to the original thumbnail image I... n If the difference between the color channel values ​​of the corresponding pixel in the upsampled image is greater than the first difference threshold, then the kth pixel is determined as the first target pixel. The intermediate image X1 includes at least one pixel, and the k-th pixel is any one of the at least one pixel.

5. The method as described in claim 2, characterized in that, The original image based on the thumbnail I n The upsampled image is used to correct the intermediate image X1 to obtain the corrected image Y1, which includes: The second target pixel in the intermediate image X1 is determined based on the color channel values ​​of each pixel in the three color channels. Replace the second target pixel with the original thumbnail image I. n The corrected image Y1 is obtained by taking the pixels corresponding to the second target pixel in the upsampled image.

6. The method as described in claim 5, characterized in that, The step of determining the second target pixel in the intermediate image X1 based on the color channel values ​​of each pixel in the three color channels includes: If the difference between the maximum and minimum color channel values ​​of the qth pixel in the intermediate image X1 is greater than the second difference threshold, then the qth pixel is determined as the second target pixel. The intermediate image X1 includes at least one pixel, and the q-th pixel is any one of the at least one pixel.

7. The method as described in claim 2, characterized in that, When j>1 and less than or equal to n-1, the j-th fusion process includes: Obtain the corrected image Y j-1 The corrected image Y j-1 The intermediate image X obtained from the (j-1)th fusion process j-1 The result was obtained after correction; For the corrected image Y j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The second image to be fused below; For the second image to be fused, and the thumbnail enhanced image E n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j ; For the intermediate image X j The correction is performed to obtain the corrected image Y. j .

8. The method as described in claim 7, characterized in that, When j=n-1, the intermediate image X j The correction is performed to obtain the corrected image Y. j ,include: Based on the original image I, the intermediate image X n-1 The correction is performed to obtain the corrected image Y. n-1 .

9. The method as described in claim 1, characterized in that, The step of extracting feature information of each thumbnail-enhanced image in the set of thumbnail-enhanced images at the corresponding scale includes: For the thumbnail enhanced image E i 'Perform upsampling to obtain a scale of P' i-1 The sampled image; Calculate the thumbnail-enhanced image E i-1 The difference between the sampled image and the original image is used to obtain the thumbnail-enhanced image E. i-1 'At scale P i-1 The following feature information.

10. An image processing apparatus, characterized in that, The image processing device includes: The acquisition unit is used to acquire the original image I and the enhanced image E corresponding to the original image I; The processing unit is used to perform n downsampling operations on the original image I to obtain scale P. n The original image below is a thumbnail. n The enhanced image E is downsampled n times to obtain a set of thumbnail enhanced images, which contains thumbnail enhanced images at n scales, where n is a positive integer. And for extracting feature information of each thumbnail-enhanced image in the set of thumbnail-enhanced images at the corresponding scale; and for the thumbnail original image I n The upsampled image of ' and the feature information of each thumbnail enhanced image in the set of thumbnail enhanced images at the corresponding scale are fused to obtain the fused image corresponding to the original image I; Wherein, the thumbnail-enhanced images at n scales in the thumbnail-enhanced image set are denoted as E1'…E n ', thumbnail enhanced image E i The corresponding scale is P i , i is a positive integer less than or equal to n; the number of fusion processes is n, and any fusion process is represented as the j-th fusion process, where j is a positive integer less than or equal to n-1; when j=1, the process of the j-th fusion process includes: processing the thumbnail original image I n 'and thumbnail enhanced image E n-1 'At scale P n-1 The feature information is fused to obtain an intermediate image X1; when j>1 and less than or equal to n-1, the j-th fusion process includes: obtaining the intermediate image X1 obtained from the (j-1)-th fusion process. j-1 ; for the intermediate image X j-1 Upsampling is performed to obtain the result at scale P. n-j+2 The first image to be fused; and the thumbnail enhanced image E. n-j+2 'At scale P n-j+2 The feature information below is fused to obtain the intermediate image X. j ; wherein, the intermediate image X j This refers to the image after the j-th fusion process, where the fused image corresponding to the original image I includes the intermediate image X. n-1 Or for the intermediate image X n-1 The corrected image Y obtained by correction n-1 .

11. A computer device, characterized in that, include: Storage devices and processors; A memory, wherein a computer program is stored; A processor for loading the computer program to implement the image processing method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-9.

13. A computer program product, characterized in that, The computer program product includes a computer program adapted to be loaded by a processor and execute the image processing method as described in any one of claims 1-9.

Images