Image processing method, apparatus, device, storage medium and program product

Abstract

The application discloses an image processing method and device, equipment, storage medium and program product, and relates to the technical field of image processing. The method comprises the following steps: acquiring a first image; acquiring a preset mapping curve; performing feature analysis on the first image to obtain a curve correction coefficient corresponding to the first image; correcting the curve parameters of the preset mapping curve by using the curve correction coefficient to obtain a target mapping curve; and performing color parameter adjustment on the first image based on the target mapping curve to obtain a second image corresponding to the first image. Since the correction coefficient is calculated instead of the entire mapping curve when the color parameter adjustment is performed on the first image, the calculation amount is greatly reduced, and thus the efficiency of the image color enhancement processing is improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, storage medium, and program product. Background Technology

[0002] When taking pictures with a mobile device, the captured images may suffer from problems such as partial exposure, grayish image, and pale colors due to factors such as equipment, environment, and shooting technique. These images often require color enhancement processing to obtain optimized images.

[0003] In related technologies, by training a deep neural network in a deep bilateral learning algorithm, the mapping curve between images can be directly fitted. By inputting the target image that needs color enhancement into the trained deep bilateral learning algorithm, an optimized image can be obtained.

[0004] However, the fitting process of the mapping curve in the above algorithm is quite difficult, requiring the deep neural network to contain a large number of parameters, and the computational load of the algorithm is relatively large, which makes the target image color enhancement processing slow and the color enhancement processing efficiency low. Summary of the Invention

[0005] This application provides an image processing method, apparatus, device, storage medium, and program product, which can improve the efficiency of color enhancement processing. The technical solution is as follows:

[0006] On the one hand, an image processing method is provided, the method comprising:

[0007] Acquire a first image, which is the image whose color parameters are to be adjusted;

[0008] Obtain a preset mapping curve, which corresponds to a preset curve parameter;

[0009] Perform feature analysis on the first image to obtain the curve correction coefficient corresponding to the first image;

[0010] The curve parameters of the preset mapping curve are corrected using the curve correction coefficient to obtain the target mapping curve;

[0011] The color parameters of the first image are adjusted based on the target mapping curve to obtain a second image corresponding to the first image. The second image is an image with adjusted color parameters based on the first image.

[0012] On the other hand, an image processing apparatus is provided, the apparatus comprising:

[0013] The acquisition module is used to acquire a first image, which is an image whose color parameters are to be adjusted.

[0014] The acquisition module is also used to acquire a preset mapping curve, which corresponds to a preset curve parameter;

[0015] The analysis module is used to perform feature analysis on the first image to obtain the curve correction coefficient corresponding to the first image;

[0016] The correction module is used to correct the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve;

[0017] The adjustment module is used to adjust the color parameters of the first image based on the target mapping curve to obtain a second image corresponding to the first image, wherein the second image is an image after the color parameters have been adjusted based on the first image.

[0018] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement any of the image processing methods described in the embodiments of this application.

[0019] On the other hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored in the computer-readable storage medium, the at least one piece of program code being loaded and executed by a processor to implement the image processing method described in any of the embodiments of this application.

[0020] On the other hand, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the image processing methods described in the embodiments of this application.

[0021] The beneficial effects of the technical solutions provided in this application include at least the following:

[0022] By performing feature analysis on the first image (the image to be adjusted for color parameters) to obtain curve correction coefficients, the preset mapping curve (a mapping curve with pre-set curve parameters) is corrected based on these curve correction coefficients. The color parameters of the first image are then adjusted based on the adjusted preset mapping curve. Since only the correction coefficients need to be calculated when adjusting the color parameters of the first image, instead of calculating the entire mapping curve, the amount of computation is greatly reduced, thereby improving the efficiency of image color enhancement processing. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a gamma transform curve provided in an exemplary embodiment of this application;

[0025] Figure 2 This is a schematic diagram illustrating the process of obtaining the corrected mapping curve provided in an exemplary embodiment of this application;

[0026] Figure 3 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0027] Figure 4 This is a flowchart of an image processing method provided in an exemplary embodiment of this application;

[0028] Figure 5 This is a flowchart of the model training process provided in an exemplary embodiment of this application;

[0029] Figure 6 This is a flowchart of an image processing method provided in another exemplary embodiment of this application;

[0030] Figure 7 This is a schematic diagram illustrating the process of obtaining prediction correction coefficients provided in an exemplary embodiment of this application;

[0031] Figure 8 This is a schematic flowchart of an image processing method provided in an exemplary embodiment of this application;

[0032] Figure 9 This is a flowchart of an image processing method provided in another exemplary embodiment of this application;

[0033] Figure 10 This is a schematic diagram of a parallel processing flow for image content provided in an exemplary embodiment of this application;

[0034] Figure 11 This is an image of the object evaluation result of an image processing method provided in an exemplary embodiment of this application;

[0035] Figure 12 This is a speed comparison diagram of an image processing method and sample method three provided in an exemplary embodiment of this application;

[0036] Figure 13 This is a structural block diagram of an image processing apparatus provided in an exemplary embodiment of this application;

[0037] Figure 14 This is a structural block diagram of an image processing apparatus provided in another exemplary embodiment of this application;

[0038] Figure 15 This is a structural block diagram of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0040] In this application, the terms "first" and "second" are used to distinguish between identical or similar items that have essentially the same function. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there any limitation on the quantity or execution order.

[0041] First, a brief introduction to the terms used in the embodiments of this application:

[0042] Pixel: A pixel is the smallest unit that makes up an image; it can be understood as a small square of different colors. After an image is digitized, it is actually a digital matrix in a computer. Pixels in a digital matrix have at least two attributes: position and pixel value. Position is represented by rows and columns, and the pixel value represents the average brightness of the pixel square; in a grayscale image, the grayscale value is the pixel value.

[0043] Color space: Images can be represented in computers by mapping to different color spaces. In a color image, each pixel corresponds to multiple channels. Common multi-channel color spaces include: RGB (Red Green Blue) space and HSV (Hue Saturation Value) space.

[0044] Image color enhancement: Digital image processing refers to the process of processing and analyzing images after they have been digitized using computers. Digital image processing includes image transformation, image color enhancement, image restoration, image segmentation, and image recognition and classification. Image color enhancement is a type of digital image processing that mainly involves adjusting the color parameters of an image to optimize image display and enhance the subjective visual experience. For example, it can increase image saturation to make the subject stand out from the background, thus highlighting the main subject of the image.

[0045] In some embodiments, the color parameters of an image can be manually adjusted directly on the camera or image processing software. With the development of computer technology, automatic adjustment of image parameters can also be achieved through image enhancement algorithms. Common image enhancement algorithms include: histogram equalization-based image enhancement algorithms, Laplacian operator-based image enhancement algorithms, logarithmic transform-based image enhancement algorithms, gamma transform-based image enhancement algorithms, etc.

[0046] This application primarily uses an image enhancement algorithm based on gamma transform as an example for illustration. Gamma transform is mainly used for image correction: adjusting the grayscale of images with excessively high grayscale (overexposed) or excessively low grayscale (underexposed), thereby improving the display effect of brightness in various parts of the image. The transformation formula for gamma transform is shown in Formula 1 below:

[0047] Formula 1: S1 = cr γ

[0048] Where r is the initial gray level, which needs to be scaled from 0 to 255 to 0 to 1 during gamma transformation, and then restored to the original dynamic range after gamma transformation. Therefore, the value range of r is [0,1]. S1 is the gray level after gamma transformation. c is the gray level coefficient, which is used to stretch the gray level of the image as a whole, and is usually set to 1. γ is the gamma factor, whose value determines the gray level mapping method between the input image and the output image.

[0049] Please refer to Figure 1 It shows a set of gamma transform curves 100, where the horizontal axis represents the input gray level and the vertical axis represents the output gray level (both input and output gray levels are scaled). The number above each curve represents the size of the gamma factor γ. For example, the gamma factor γ of gamma transform curve 101 is 0.1.

[0050] In related technologies, image color enhancement methods mainly employ the following approaches:

[0051] (1) The image that needs color enhancement is retouched by a professional designer. This process is time-consuming and labor-intensive, the efficiency of image color enhancement is low, and it is not universally applicable.

[0052] (2) An image with good color performance is automatically synthesized through an image color enhancement algorithm. For illustration, the image color enhancement algorithm is trained using the original image and a labeled image (an optimized image obtained based on the original image). Then, the image to be color enhanced is input into the image color enhancement algorithm, which uses its deep neural network to directly fit the mapping curve between the images to obtain the optimized image, thus achieving the color enhancement effect. As can be seen from the above description, the image color enhancement algorithm needs to adjust the parameters of all pixels in the image. This fitting process is quite difficult and requires the deep neural network to have a large number of parameters. Moreover, the computational load in this image enhancement process is large, resulting in low efficiency.

[0053] This application provides an image processing method, which is illustrative; please refer to the following examples. Figure 2 First, a mapping curve is selected as the preset mapping curve 201, and its curve parameters can be set. Second, a convolutional network is used to extract and fuse image features from the first image 202, thereby estimating the curve correction coefficient X of the first image 202 relative to the preset mapping curve 201. Based on this curve correction coefficient X, the preset mapping curve 201 is corrected to obtain the corrected mapping curve 203. Finally, the color parameters of the first image 202 are adjusted using the corrected mapping curve 203. As can be seen from the above process, when adjusting the color parameters of the first image 202, only the curve correction coefficient X of the preset mapping curve 201 needs to be calculated, without calculating the entire mapping curve, greatly reducing the computational load and thus improving the efficiency of image color enhancement processing.

[0054] Optionally, the convolutional layer network in the above method can be replaced with a fully connected layer network or a Transformer layer network. It is worth noting that the computation speed of the fully connected layer network or Transformer layer network scheme will be slower than that of the convolutional layer network scheme.

[0055] Figure 3 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application, such as... Figure 3 As shown, the implementation environment includes a terminal 310, a server 320, and a communication network 330. The terminal 310 and the server 320 are connected through the communication network 330. In some optional embodiments, the communication network 330 can be a wired network or a wireless network. This embodiment does not limit this.

[0056] In some optional embodiments, terminal 310 may be a smartphone, tablet computer, laptop computer, desktop computer, smart home appliance, smart vehicle terminal, smart speaker, digital camera, etc., but is not limited to these. Taking a smartphone implementation as an example, optionally, terminal 310 may have a target application installed. Indicatively, this target application may be a traditional application, a cloud application, a mini-program or application module within a host application, or a web platform; this embodiment does not limit this. Optionally, the target application may provide image color parameter adjustment functions. Indicatively, when it is necessary to adjust the color parameters of an image, such as... Figure 3 As shown, terminal 310 uploads the first image (i.e., the image that needs to be adjusted in color parameters) to server 320. Server 320 adjusts the color parameters of the first image to obtain the second image (i.e., the image after adjusting the color parameters based on the first image) and sends the second image back to terminal 310.

[0057] In some optional embodiments, server 320 is used to provide image color parameter adjustment services for a target application installed on terminal 310. Optionally, server 320 is provided with a curve correction model and a preset mapping curve, wherein the preset mapping curve is a mapping curve with pre-set curve parameters. Illustratively, after receiving a first image, server 320 inputs the first image into the curve correction model, first extracts the image feature representation of the first image, then performs feature analysis on the image feature representation to obtain the curve correction coefficient corresponding to the first image. Based on the curve correction coefficient, server 320 corrects the preset mapping curve to obtain a target mapping curve. Based on the target mapping curve, server 320 adjusts the color parameters of the first image to obtain a second image, and finally feeds the second image back to terminal 310. After receiving the second image, terminal 310 displays the second image through the target application.

[0058] In some alternative embodiments, the curve correction model can also be deployed on the terminal 310 side, and the terminal 310 can adjust the image color parameters locally without the need for the server 320. This embodiment does not limit this.

[0059] It is worth noting that Server 320 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0060] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to a backend system for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing. Optionally, server 320 can also be implemented as a node in a blockchain system.

[0061] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the sample images and reference images involved in this application were obtained with full authorization.

[0062] The image processing method provided in this application embodiment can be applied to at least the following application scenarios:

[0063] I. When using a mobile device (e.g., a smartphone) to capture images, the resulting images often suffer from problems such as localized overexposure and excessively dark colors due to factors such as the shooting environment, shooting techniques, and shooting equipment. Such images fail to meet people's image quality requirements. Manually adjusting the parameters of the captured images is time-consuming and labor-intensive, and it is difficult for non-professionals to achieve optimal adjustments. The image processing method provided in this application can automatically adjust the color parameters of captured photos to obtain optimized images. Moreover, the adjustment process does not directly calculate the mapping curve between the image to be adjusted and the optimized image, but instead calculates the correction coefficient of a pre-set mapping curve, which greatly reduces the amount of calculation, improves the efficiency of image color enhancement processing, and meets the needs of real-time image processing on mobile devices.

[0064] Second, when watching short videos on short video applications, the video quality varies, and the video picture may have problems such as dull colors and uneven brightness distribution. It is also difficult to improve the visual effect of the video picture by manually adjusting the color parameters. The image processing method provided in this application can automatically adjust the color parameters of the video frames, thereby improving the video picture quality. Moreover, in the adjustment process, instead of directly calculating the mapping curve between the picture to be adjusted and the picture to be optimized, it calculates the correction coefficient of the pre-set mapping curve, which greatly reduces the amount of calculation and improves the efficiency of image color enhancement processing. While improving the video viewing experience, it also ensures the smoothness of the video playback process.

[0065] It is worth noting that the above application scenarios are merely illustrative examples, and the embodiments of this application do not limit other application scenarios of the image processing method.

[0066] Based on the above introduction and implementation environment, the image processing method provided in the embodiments of this application will be described. Figure 4 This is a flowchart of an image processing method provided in an embodiment of this application, such as... Figure 4 As shown, the method includes:

[0067] Step 401: Obtain the first image.

[0068] The first image is the one whose color parameters need to be adjusted.

[0069] Optionally, the first image may include at least one of the following: a standalone image, an area within an image that requires color parameter adjustment, or a frame from a video.

[0070] Optionally, adjusting the color parameters of the first image refers to adjusting the color parameters of the pixels in the first image.

[0071] Optionally, color parameters refer to parameters that affect the color performance of an image. Indicatively, color parameters include at least one of parameters such as contrast, brightness, saturation, hue, and grayscale.

[0072] Step 402: Obtain the preset mapping curve.

[0073] Optionally, the preset mapping curve is used to make a baseline adjustment to the color parameters in the image. For example, the process of the preset mapping curve adjusting the gray level of the pixels in the image can be as follows: obtain the gray level corresponding to each pixel in the image, input the obtained gray level corresponding to each pixel into the preset mapping curve, and output the gray level corresponding to each pixel mapped on the preset mapping curve.

[0074] Optionally, the preset mapping curve is a baseline color parameter adjustment curve, such as a logarithmic transform curve or a gamma transform curve. The logarithmic transform curve can expand the low-grayscale portion of an image, revealing more detail in this area, while compressing the high-grayscale portion, reducing detail in the high-grayscale portion, thus emphasizing the low-grayscale portion of the image. The gamma transform curve is mainly used for image correction, correcting images with excessively high or low grayscale levels, thereby improving overexposure issues.

[0075] The preset mapping curve corresponds to pre-defined curve parameters.

[0076] Optionally, the curve parameters are used to adjust the shape of the preset mapping curve. Different shapes of preset mapping curves can have different adjustment effects on the first image. For example, when the preset mapping curve is implemented as a gamma transform curve, such as... Figure 1 As shown, the gamma factor γ is one of the curve parameters in the gamma transform curve. Different gamma factors γ correspond to different shapes of the gamma transform curve, where:

[0077] (1) When γ>1, the grayscale of the brighter parts of the image can be mapped to a smaller grayscale, while the grayscale of the darker parts changes less, and the image as a whole looks darker.

[0078] (2) When γ<1, the grayscale of the darker parts of the image can be mapped to a larger grayscale, while the grayscale of the brighter parts changes less, and the image as a whole looks brighter;

[0079] (3) When γ=1, the original input image is not changed.

[0080] Step 403: Perform feature analysis on the first image to obtain the curve correction coefficient corresponding to the first image.

[0081] The curve correction coefficient refers to the coefficient used to correct the curve parameters of the preset mapping curve.

[0082] In some optional embodiments, performing feature analysis on the first image to obtain the curve correction coefficient corresponding to the first image means inputting the first image into the curve correction model and outputting the curve correction coefficient corresponding to the first image.

[0083] Optionally, the curve correction model is a model trained using sample images.

[0084] The sample image pair includes a sample image and a reference image that make up the image pair. The reference image is the sample image after color parameter adjustment.

[0085] The illustrative, reference, and sample images are obtained in at least one of the following ways:

[0086] (1) Obtain the image after it has been edited by a professional designer as a reference image, and use the image before editing as a sample image;

[0087] (2) High-quality images are automatically synthesized using the benchmark deep learning image color enhancement algorithm. Then, the images with reasonable color distribution are selected by the annotators as reference images, and the images before processing are used as sample images.

[0088] The methods for obtaining the above-mentioned reference images and sample images are merely illustrative examples, and the embodiments of this application do not limit them.

[0089] This is illustrative; please refer to it. Figure 5 This illustrates the training process of a curve correction model provided in an exemplary embodiment of this application, such as... Figure 5 As shown, the training data for the curve correction model consists of sample image 501 and reference image 502. Therefore, the training steps for the curve correction model are as follows:

[0090] Step 1: The sample image 501 is amplified to increase its diversity via the image amplification module 503. Optionally, the image amplification module 503 includes multiple amplification methods, and the amplification method of the sample image includes at least one of the following:

[0091] (1) Image random rotation method, which randomly rotates the image within a certain angle range;

[0092] (2) Image random scaling method, which scales the image within a certain resolution;

[0093] (3) Image random edge patching method: black edges are added around the image, and the pixel width value of the black edges is random.

[0094] (4) Image random horizontal flipping method, which flips the image horizontally at a random angle with a certain probability;

[0095] (5) Image random vertical flipping method: flip the image vertically at a random angle with a certain probability.

[0096] The amplification method of the above sample images is merely an illustrative example, and the embodiments of this application do not limit it.

[0097] Step 2: Add image noise to the amplified sample image.

[0098] Adding image noise refers to generating a random number and then adding this random number to the pixel value of the image.

[0099] Illustratively, image noise includes at least one of the following: Gaussian noise, salt and pepper noise, Rayleigh noise, exponentially distributed noise, and uniformly distributed noise.

[0100] It is worth noting that steps one and two above are optional parallel steps. Step one can be executed alone, step two can be executed alone, or both steps one and two can be executed simultaneously.

[0101] Step 3: Input the amplified sample image 501 with added image noise into the curve correction model 504 to obtain the first prediction correction coefficient. Adjust the preset mapping curve using the first prediction correction coefficient, and apply the adjusted preset mapping curve to the sample image 501 to obtain the predicted image I. n The amplified sample image 501 without added image noise is input into the curve correction model 504 to obtain the second prediction correction coefficient. The preset mapping curve is adjusted using the first prediction correction coefficient, and the adjusted preset mapping curve is applied to the sample image 501 to obtain the predicted image I. o .

[0102] Step 4: The reference image 502 is also amplified using the image amplification module to obtain the amplified reference image I. t The amplification method is the same as that used for the sample image in step one, and will not be repeated here.

[0103] The sample image 501 and the reference image 502 may be amplified using the same method or different methods; this embodiment does not limit this.

[0104] Step 5: Predict the image I n Predicted Image I o and the amplified reference image I t The loss is calculated in the loss calculation module 505, and the curve correction model 504 is trained based on the loss.

[0105] Optionally, the loss calculated in the loss calculation module includes at least one of the following losses:

[0106] (1) Pixel consistency loss, used to constrain the predicted image I o and the amplified reference image I t Pixel values ​​are consistent;

[0107] (2) Noise-robust loss, used to constrain the predicted image I n and predicted image I o The pixel values ​​are consistent.

[0108] Schematic, the curve correction model 504 is trained based on pixel consistency loss, or on noise robustness loss, or on a joint loss of pixel consistency loss and noise robustness loss.

[0109] If the curve correction model 504 is trained using a joint loss based on pixel consistency loss and noise robustness loss, alternatively, the joint loss obtained by weighted summation of pixel consistency loss and noise robustness loss can be used to train the curve correction model 504.

[0110] Step 404: Correct the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve.

[0111] Optionally, the curve correction coefficient can modify the curve parameters of the preset mapping curve to change the shape of the preset mapping curve and obtain the target mapping curve.

[0112] Indicatively, the curve correction coefficient is combined with the curve parameters of the preset mapping curve to obtain the target mapping curve. As can be seen from the above introduction, the formula for the gamma transformation curve is Formula 1. Optionally, the curve correction coefficient is combined with the gray coefficient c or the gamma factor γ. The combination method includes at least one of the calculation methods such as summation and product.

[0113] In some optional embodiments, the process of obtaining the target mapping curve described above further includes:

[0114] The target mapping curve is obtained by weighted summation of the curve correction coefficient and the preset mapping curve.

[0115] To illustrate, if the weighting factor of the curve correction coefficient and the preset mapping curve is 1, then the target mapping curve is:

[0116] S2=cr γ +a

[0117] Where r is the initial gray level; S2 is the gray level after the corrected gamma transform; c is the gray level coefficient; and a is the curve correction coefficient.

[0118] Step 405: Adjust the color parameters of the first image based on the target mapping curve to obtain the second image corresponding to the first image.

[0119] The second image is an image with color parameters adjusted based on the first image.

[0120] To illustrate, a preset mapping curve is implemented as a gamma transform curve. The corrected preset mapping curve, i.e. the target mapping curve, is set as S3. First, the gray level of each pixel in the first image is obtained. The gray level of each pixel is input into the S3 curve to obtain the corrected gray level. Based on the corrected gray level, the pixels of the first image are modified to obtain the second image.

[0121] In summary, the image processing method provided in this application embodiment obtains curve correction coefficients by performing feature analysis on a first image (the image to be adjusted for color parameters), corrects a preset mapping curve (a mapping curve with pre-set curve parameters) using these curve correction coefficients, and adjusts the color parameters of the first image based on the adjusted preset mapping curve. Since only the correction coefficients need to be calculated when adjusting the color parameters of the first image, instead of calculating the entire mapping curve, the amount of computation is greatly reduced, thereby improving the efficiency of image color enhancement processing.

[0122] Figure 6 This is a flowchart of an image processing method provided in an embodiment of this application, such as... Figure 6 As shown, the method includes:

[0123] Step 601: Obtain the first image.

[0124] The first image is the one whose color parameters need to be adjusted.

[0125] Optionally, the method of acquiring the first image includes at least one of the following:

[0126] First, when an image is captured using an image capturing tool, the resulting image may have issues such as localized overexposure or excessively dark colors due to factors such as the shooting environment, shooting techniques, and shooting equipment. In such cases, color parameter adjustments are required, and the captured image is taken as the first image. The image capturing tool includes at least one of digital cameras, mobile phones, tablets, etc.

[0127] Second, when scanning images on printed materials or physical photographs using a scanning tool, the scanned images may have problems such as overexposure or dimness due to factors such as the object being scanned, the scanning environment, and the scanning angle. Color parameters need to be adjusted, and the scanned image is then used as the first image. The scanning tool can be a scanner or a target application with scanning function installed on a mobile phone or tablet.

[0128] Third, capture images from the screen (e.g., capture images from the phone screen using the phone's screenshot function). Since the image quality of different device screens is different, the captured images may have problems such as dull colors, requiring color parameter adjustments. In this case, the captured image is used as the first image.

[0129] The method of acquiring the first image described above is merely an illustrative example, and the embodiments of this application do not limit it.

[0130] Step 602: Obtain the preset mapping curve.

[0131] The preset mapping curve has pre-defined curve parameters.

[0132] Optionally, the preset mapping curve mentioned above is a quantized curve.

[0133] Quantization refers to approximating a continuous curve with a finite number of mapping values. In terms of images, this means processing a smooth curve into a discrete image composed of a finite number of points. This discrete image is the quantized preset mapping curve.

[0134] Step 603: Input the first image into the curve correction model and extract the image feature representation of the first image.

[0135] The training process of the curve correction model has been explained in detail in step 403, and will not be repeated here.

[0136] Optionally, the image feature representation of the first image is used to indicate the image features of the first image.

[0137] Optionally, the image feature representation of the first image is extracted through a feature extraction network. The curve correction model may include one feature extraction network or multiple feature extraction networks.

[0138] It is worth noting that the feature extraction network includes at least one of convolutional layer networks, fully connected layer networks, and Transformer layer networks, but this embodiment does not limit it.

[0139] Optionally, the image features of the first image mentioned above include at least one of texture features, global features, local features, and semantic features. Texture features describe the surface properties of the entire image or a specific region within the image; for example, if the image contains water, texture features include features representing water ripples. Global features describe the overall attributes of the image, and types of global features include color features and shape features. Local features refer to features extracted from sub-image regions within the image, including edges, corners, lines, curves, etc. Semantic features describe the semantic meaning of the image's content; for example, if an image contains a cat, then "cat" is a semantic feature of the image.

[0140] Optionally, the image feature representation of the first image can be a single image feature representation or a fusion feature representation of two or more image feature representations. Therefore, the extraction of the image feature representation of the first image includes at least one of the following methods:

[0141] (1) Extract the texture feature representation of the first image, which is used to indicate the image texture features of the first image; extract the global feature representation of the first image based on the texture feature representation, which is used to indicate the overall image features of the first image; extract the local feature representation of the first image based on the texture feature representation, which is used to indicate the image features of the sub-image regions in the first image; extract the semantic feature representation of the first image based on the texture feature representation, which is used to indicate the semantic content contained in the first image; fuse the texture feature representation, global feature representation, local feature representation and semantic feature representation to obtain the image feature representation of the first image.

[0142] Optionally, the process of obtaining the image feature representation of the first image includes: inputting the first image into a first feature extraction network to extract the texture feature representation of the first image; inputting the texture feature representation and the first image into a second feature extraction network to extract the global feature representation of the first image; inputting the texture feature representation and the first image into a third feature extraction network to extract the local feature representation of the first image; inputting the texture feature representation and the first image into a fourth feature extraction network to extract the semantic feature representation of the first image; and finally, fusing the texture feature representation, global feature representation, local feature representation, and semantic feature representation to obtain the image feature representation of the first image.

[0143] (2) Extract the texture feature representation of the first image; extract the global feature representation of the first image; extract the local feature representation of the first image; extract the semantic feature representation of the first image; fuse the texture feature representation, global feature representation, local feature representation and semantic feature representation to obtain the image feature representation of the first image.

[0144] Optionally, the process of obtaining the image feature representation of the first image includes: inputting the first image into a fifth feature extraction network to extract the texture feature representation of the first image; inputting the first image into a sixth feature extraction network to extract the global feature representation of the first image; inputting the first image into a seventh feature extraction network to extract the local feature representation of the first image; inputting the first image into an eighth feature extraction network to extract the semantic feature representation of the first image; and finally, fusing the texture feature representation, global feature representation, local feature representation, and semantic feature representation to obtain the image feature representation of the first image.

[0145] (3) Extract the global feature representation of the first image as the image feature representation of the first image.

[0146] (4) Extract the local feature representation of the first image as the image feature representation of the first image.

[0147] (5) Extract the texture feature representation of the first image as the image feature representation of the first image.

[0148] (6) Extract the semantic feature representation of the first image as the image feature representation of the first image.

[0149] The above-described method of extracting image features from the first image is merely an illustrative example, and the embodiments of this application do not limit it.

[0150] Step 604: Perform feature analysis on the image feature representation to obtain the curve correction coefficients corresponding to the first image.

[0151] In illustrative terms, the image feature representation of the first image is extracted in method (1). Please refer to [reference needed]. Figure 7 It illustrates the process of obtaining the curve correction coefficient, such as... Figure 7 As shown:

[0152] First, the texture features 702 of the first image 701 are extracted. Then, based on the texture features 702, the global features 703, semantic features 704, and local features 705 of the first image 701 are extracted respectively. Finally, the texture features 702, global features 703, semantic features 704, and local features 705 of the first image 701 are fused to obtain the fused feature 706. Feature analysis is performed on the fused feature 706 to obtain the prediction correction coefficient 707, which is also the curve correction coefficient.

[0153] Step 605: Correct the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve.

[0154] In some optional embodiments, the curve correction coefficient includes n sub-coefficients, where n is an integer greater than 1. The method for obtaining the target mapping curve includes:

[0155] The preset mapping curve is segmented to obtain n segmented mapping curves; the n segmented mapping curves are then corrected using n sub-coefficients to obtain the target mapping curve.

[0156] The k-th piecewise mapping curve is corrected by the k-th sub-coefficient, where k is a positive integer and k≤n.

[0157] Optionally, the method of segmenting the preset mapping curve to obtain n segmented mapping curves includes at least one of the following:

[0158] 1. Divide the preset mapping curve into segments according to the horizontal coordinate of the coordinate axis where the preset mapping curve is located, to obtain n segmented mapping curves.

[0159] 2. Divide the preset mapping curve into segments according to the ordinate of the coordinate axis where the preset mapping curve is located, to obtain n segmented mapping curves.

[0160] Optionally, the method for obtaining the n sub-coefficients in the curve correction coefficients includes at least one of the following:

[0161] 1. Divide the preset mapping curve into segments according to the horizontal coordinate of the coordinate axis where the preset mapping curve is located, to obtain n segmented mapping curves. Take the kth segmented mapping curve as the preset mapping curve and obtain its correction coefficient, that is, the kth sub-coefficient. For illustration, assume that the preset mapping curve is divided into two segments: the first preset mapping curve and the second preset mapping curve. Then, it is necessary to obtain the sub-coefficient A corresponding to the first preset mapping curve and the sub-coefficient B corresponding to the second preset mapping curve.

[0162] Indicatively, the process of obtaining the sub-coefficient A includes: obtaining a first preset mapping curve; inputting the first image into the curve correction model 1 to obtain the sub-coefficient A corresponding to the first image, wherein the sub-coefficient A is used to adjust the first preset mapping curve, and the curve correction model 1 is a pre-trained model.

[0163] Schematic, the process of obtaining the sub-coefficient B includes: obtaining a second preset mapping curve; inputting the first image into the curve correction model 2 to obtain the sub-coefficient B corresponding to the first image, wherein the sub-coefficient B is used to adjust the second preset mapping curve, and the curve correction model 1 is a pre-trained model.

[0164] Optionally, the acquisition processes of sub-coefficient A and sub-coefficient B can be performed synchronously or asynchronously.

[0165] 2. Divide the preset mapping curve into segments according to the ordinate of the coordinate axis where the preset mapping curve is located, to obtain n segmented mapping curves. Take the kth segmented mapping curve as the preset mapping curve and obtain its correction coefficient, which is the kth sub-coefficient. The process of obtaining the sub-coefficient is the same as the description of the process of obtaining sub-coefficient A and sub-coefficient B in Method 1 above.

[0166] This is illustrative; please refer to it. Figure 1 The gamma transform curve 101 is divided into 5 segments according to the horizontal axis: [0,0.2], (0.2,0.4], (0.4,0.6], (0.6,0.8], and (0.8,1). Curve correction coefficients are obtained for each of these 5 segments. These 5 curve correction coefficients are then used to correct each of the 5 segments to obtain the target mapping curve. The k-th segment of the gamma transform curve is corrected using the k-th coefficient, where k is a positive integer and k≤5.

[0167] Step 606: Perform pixel-level feature transformation on the first image to obtain a full-resolution feature map corresponding to the first image.

[0168] Optionally, a full-resolution feature map corresponding to the first image is obtained by performing pixel-level feature transformation on the first image through a feature transformation network. The feature transformation network includes at least one of convolutional neural networks, fully connected networks, and Transformer networks, and this embodiment does not limit it.

[0169] Optionally, pixel-level feature transformation refers to performing feature transformation on the pixels in the input first image, schematically converting pixels in HSV space to pixels in RGB space.

[0170] Step 607: Apply the target mapping curve to the full-resolution feature map and obtain pixel-level mapping coefficients through linear interpolation.

[0171] Optionally, if the preset mapping curve is a quantized curve, then the target mapping curve is a quantized curve. Thus, the above linear interpolation process refers to expanding the target mapping curve. Schematic, linear interpolation process refers to using existing curve points to calculate other curve points to expand the curve.

[0172] The target mapping curve is applied to the full-resolution feature map, and linear interpolation is performed on the target mapping curve to obtain pixel-level mapping coefficients.

[0173] Step 608: Apply pixel-level mapping coefficients to the first image to obtain the second image.

[0174] Optionally, the pixel-level mapping coefficient corresponding to each pixel in the first image is applied to each pixel itself to obtain the second image.

[0175] In summary, the image processing method provided in this application embodiment obtains curve correction coefficients by performing feature analysis on a first image (the image to be adjusted for color parameters), corrects a preset mapping curve (a mapping curve with pre-set curve parameters) based on the curve correction coefficients, and adjusts the color parameters of the first image based on the adjusted preset mapping curve. Since only the correction coefficients need to be calculated when adjusting the color parameters of the first image, instead of calculating the entire mapping curve, the amount of computation is greatly reduced, thereby improving the efficiency of image color enhancement processing.

[0176] The image processing method provided in this application embodiment obtains curve correction coefficients by performing feature analysis on the image feature representation of a first image. The image feature representation includes a fusion representation of multiple image features, which makes the obtained curve correction coefficients more accurate.

[0177] This is illustrative; please refer to it. Figure 8 It shows a complete flowchart of an image processing method provided in an embodiment of this application, such as Figure 8 As shown:

[0178] The specific steps for obtaining the second image include:

[0179] Step 1: Obtain the preset mapping curve 801.

[0180] Optionally, you can first select a mapping curve that can enhance the color of the image, and then set the curve parameters.

[0181] Step 2: Input the first image 802 into the curve correction model for feature extraction and fusion processing, thereby obtaining the curve correction coefficient 803 of the preset mapping curve 801.

[0182] Step 3: Correct the preset mapping curve 801 using the curve correction coefficient 803 to obtain the corrected mapping curve 804.

[0183] Step 4: Input the first image 802 into a deep convolutional neural network to perform pixel-level feature transformation to obtain a full-resolution feature map 805.

[0184] Step 5: Apply the corrected mapping curve 804 to the full-resolution feature map 805, and obtain the pixel-level mapping coefficients 806 through linear interpolation.

[0185] Step 6: Apply the pixel-level mapping coefficient 806 to the first image 802 to obtain the image 807 after image quality enhancement.

[0186] It is worth noting that the "one" and "two" in the above steps one and two do not represent the order of the steps; they are merely illustrative. Step one and step four can be performed simultaneously.

[0187] Figure 9 This is a flowchart of an image processing method provided in an embodiment of this application, such as... Figure 9 As shown, the method includes:

[0188] Step 901: Obtain the first image.

[0189] The first image is the one whose color parameters need to be adjusted.

[0190] The process of acquiring the first image has been described in detail in step 601, and will not be repeated here.

[0191] Step 902: Obtain the preset mapping curve.

[0192] The preset mapping curve has pre-defined curve parameters.

[0193] The specific process of obtaining the preset mapping curve has been explained in step 402 and will not be repeated here.

[0194] Step 903: Divide the first image into blocks to obtain at least two image blocks.

[0195] Optionally, before performing block processing on the first image, the method further includes adjusting the first image to increase its resolution, thereby making the first image a high-resolution image.

[0196] Optionally, the method for dividing the first image into blocks to obtain at least two image blocks includes at least one of the following methods:

[0197] 1. Divide the first image into equal blocks according to the preset number of blocks to obtain at least two image blocks.

[0198] Optionally, the first image is a rectangular image, and the preset number of blocks is an even number. The first image is divided into at least two image blocks of the same size according to the preset number of blocks.

[0199] 2. Divide the first image into random blocks to obtain at least two image blocks.

[0200] Optionally, the first image can be cut by several random line segments to obtain at least two image blocks of different sizes.

[0201] Optionally, the positional relationship of the aforementioned random line segments can be either intersecting or parallel.

[0202] This example illustrates how to divide the first image into at least two image blocks according to a preset number of blocks. Please refer to [link / reference]. Figure 10 The first image 1001 is processed into a grid to obtain several image blocks 1002, wherein each image block in the several image blocks 1002 has the same size.

[0203] Step 904: Perform feature analysis on at least two image blocks respectively to obtain at least two block correction coefficients corresponding to the at least two image blocks.

[0204] Here, the i-th image block corresponds to the i-th block correction coefficient, and the curve correction coefficient is a set of at least two block correction coefficients.

[0205] Optionally, the process of obtaining at least two block correction coefficients specifically includes:

[0206] Step 1: Input at least two image patches into the curve correction model and extract the image patch feature representations corresponding to each image patch in the at least two image patches in parallel.

[0207] To illustrate, the types of image feature representation include global feature representation, local feature representation, semantic feature representation, and texture feature representation.

[0208] Optionally, the image block feature representation of each of the at least two image blocks includes one image block feature representation or two or more image block feature representations.

[0209] The method for extracting the image block feature representation corresponding to each of the at least two image blocks is roughly the same as the method for extracting the image feature representation of the first image. It has been specifically explained in step 603 and will not be repeated here.

[0210] Step 2: Perform feature analysis on the image block feature representation corresponding to each of the at least two image blocks to obtain at least two block correction coefficients corresponding to the at least two image blocks.

[0211] This is illustrative; please refer to it. Figure 10 The image blocks 1002 obtained by meshing are input into the curve correction model. Through the curve correction model, the global features, local features, semantic features and texture features corresponding to each image block in the image blocks 1002 are extracted in parallel, and these features are fused to obtain the image block feature representation corresponding to each image block in the image blocks 1002. Feature analysis is performed on the image block feature representation of each image block in the image blocks 1002 to obtain the mapping curve correction coefficient corresponding to each image block in the image blocks 1002.

[0212] Step 905: Correct the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve.

[0213] The curve correction coefficient is obtained directly from the first image. The process of obtaining the target mapping curve through this curve correction coefficient has been explained in detail in step 404, and will not be repeated here.

[0214] Optionally, the curve correction coefficients are implemented as a set of at least two block correction coefficients, and the process of obtaining the target mapping curve through the set of at least two block correction coefficients includes:

[0215] Applying at least two block correction coefficients to a preset mapping curve yields a block target mapping curve corresponding to at least two block correction coefficients.

[0216] The correction coefficient of the i-th block corresponds to the target mapping curve of the i-th block, and the target mapping curve is a set of at least two target mapping curves.

[0217] For illustrative purposes, please refer to the following: Figure 10The mapping curve correction coefficients corresponding to each image block in the plurality of image blocks 1002 are applied to the preset mapping curve 1003 to obtain the corrected mapping curves corresponding to each image block in the plurality of image blocks 1002. The corrected mapping curves corresponding to each image block in the plurality of image blocks 1002 correspond one-to-one with the content of each image block in the plurality of image blocks 1002, and are unique mapping curves for each image block in the plurality of image blocks 1002.

[0218] Optionally, when the preset mapping curve is implemented as a relatively complex mapping curve, each image patch is labeled, and a deep convolutional neural network is trained through the labeled image patches to achieve smooth connections between image patches.

[0219] Step 906: Adjust the color parameters of the first image based on the target mapping curve to obtain the second image corresponding to the first image.

[0220] The second image is an image with color parameters adjusted based on the first image.

[0221] The target mapping curve is implemented as a target mapping curve directly obtained from the first image. The process of obtaining the second image through this target mapping curve has been specifically explained in step 405 and will not be repeated here.

[0222] Optionally, the target mapping curve is implemented as a set of at least two block target mapping curves, and the process of obtaining the second image through the set of at least two block target mapping curves includes:

[0223] Based on at least two block target mapping curves, the color parameters of at least two corresponding image blocks are adjusted to obtain the second image corresponding to the first image.

[0224] The target mapping curve of the i-th block adjusts the i-th image block.

[0225] In summary, the image processing method provided in this application embodiment obtains curve correction coefficients by performing feature analysis on a first image (the image to be adjusted for color parameters), corrects a preset mapping curve (a mapping curve with pre-set curve parameters) based on the curve correction coefficients, and adjusts the color parameters of the first image based on the adjusted preset mapping curve. Since only the correction coefficients need to be calculated when adjusting the color parameters of the first image, instead of calculating the entire mapping curve, the amount of computation is greatly reduced, thereby improving the efficiency of image color enhancement processing.

[0226] The image processing method provided in this application improves the parallelism of image processing by dividing the first image into blocks, and independently obtains the block target mapping curve for each image block, and then applies the block target mapping curve to the corresponding image block to adjust the color parameters of the image block, thereby achieving a more intelligent color enhancement effect.

[0227] Figure 11 This is an object evaluation result of an image processing method provided in an exemplary embodiment of this application, such as... Figure 11 As shown: Among the 105 samples surveyed, 54 samples (approximately 51%) believed that method 1101 was more effective; 38 samples (approximately 36%) believed that method 1102 was more effective; and 25 samples (approximately 24%) believed that method 1103 was more effective.

[0228] from Figure 11 It is easy to see from the data 1100 shown that, under subjective evaluation, the image enhancement effect of the image processing method provided in the exemplary embodiment of this application is better and the evaluation is higher.

[0229] It is worth noting that the evaluation results mentioned above were obtained after the object was authorized.

[0230] Figure 12 This is a speed comparison diagram between an image processing method and sample method three provided in an exemplary embodiment of this application. Figure 12 The image processing speed comparison at 1080x1920 resolution is shown. The black bar chart represents method 1201 proposed in this application, while the white bar chart represents sample method 1202.

[0231] from Figure 12 As can be seen from the data 1200 shown, the method 1201 proposed in this application has a significant advantage in processing speed compared to the sample method 3 1202, and can provide a performance acceleration of 40% to 60% on different models.

[0232] It is worth noting that the speed comparison data mentioned above was obtained after object authorization.

[0233] Please refer to Figure 13 The diagram illustrates a structural block diagram of an image processing apparatus provided in an exemplary embodiment of this application, the apparatus comprising the following modules:

[0234] The acquisition module 1310 is used to acquire a first image, which is an image whose color parameters are to be adjusted.

[0235] The acquisition module 1310 is also used to acquire a preset mapping curve, the preset mapping curve corresponding to a preset curve parameter;

[0236] Analysis module 1320 is used to perform feature analysis on the first image to obtain curve correction coefficients corresponding to the first image;

[0237] The correction module 1330 is used to correct the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve;

[0238] The adjustment module 1340 is used to adjust the color parameters of the first image based on the target mapping curve to obtain a second image corresponding to the first image, wherein the second image is an image after the color parameters have been adjusted based on the first image.

[0239] In some alternative embodiments, please refer to Figure 14 The analysis module 1320 includes:

[0240] The processing submodule 1321 is used to input the first image into the curve correction model and output the curve correction coefficients corresponding to the first image. The curve correction model is a model trained by sample images.

[0241] The sample image pair includes a sample image and a reference image that constitute the image pair. The reference image is the sample image after color parameter adjustment.

[0242] In some optional embodiments, the processing submodule 1321 includes:

[0243] Feature extraction unit 1322 is used to input the first image into the curve correction model and extract the image feature representation of the first image;

[0244] The feature analysis unit 1323 is used to perform feature analysis on the image feature representation to obtain the curve correction coefficient corresponding to the first image.

[0245] In some optional embodiments, the feature extraction unit 1322 is further configured to extract a texture feature representation of the first image, the texture feature representation indicating the image texture features of the first image; the feature extraction unit 1322 is further configured to extract a global feature representation of the first image based on the texture feature representation, the global feature representation indicating the overall image features of the first image; the feature extraction unit 1322 is further configured to extract a local feature representation of the first image based on the texture feature representation, the local feature representation indicating the image features of a sub-image region in the first image; the feature extraction unit 1322 is further configured to extract a semantic feature representation of the first image based on the texture feature representation, the semantic feature representation indicating the semantic content contained in the first image; the feature extraction unit 1322 further includes:

[0246] The feature fusion subunit 1324 is used to fuse the texture feature representation, the global feature representation, the local feature representation and the semantic feature representation to obtain the image feature representation of the first image.

[0247] In some optional embodiments, the analysis module 1320 includes:

[0248] The block segmentation submodule 1325 is used to perform block segmentation processing on the first image to obtain at least two image blocks;

[0249] The analysis module 1320 is further configured to perform feature analysis on the at least two image blocks respectively to obtain at least two block correction coefficients corresponding to the at least two image blocks, wherein the i-th image block corresponds to the i-th block correction coefficient, and the curve correction coefficient is the set of the at least two block correction coefficients.

[0250] In some optional embodiments, the segmentation submodule 1325 is used to divide the first image into average blocks according to a preset number of blocks to obtain the at least two image blocks; or, it is used to randomly divide the first image into blocks to obtain the at least two image blocks.

[0251] In some optional embodiments, the correction module 1330 is further configured to directly correct the preset mapping curve using the curve correction coefficient to obtain the target mapping curve.

[0252] In some optional embodiments, the curve correction coefficient includes n sub-coefficients, where n is an integer greater than 1; the correction module 1330 includes:

[0253] Segmentation submodule 1331 is used to segment the preset mapping curve to obtain n segmented mapping curves;

[0254] The correction module 1330 is further configured to correct the n segmented mapping curves respectively using n sub-coefficients to obtain the target mapping curve, wherein the kth segmented mapping curve is corrected using the kth sub-coefficient, where k is a positive integer and k≤n.

[0255] In some optional embodiments, the segmentation submodule 1331 is used to perform average segmentation of the preset mapping curve according to the horizontal coordinate of the coordinate axis where the preset mapping curve is located, to obtain n segmented mapping curves; or, it is used to perform average segmentation of the preset mapping curve according to the vertical coordinate of the coordinate axis where the preset mapping curve is located, to obtain n segmented mapping curves.

[0256] In some optional embodiments, the adjustment module 1340 includes:

[0257] The feature transformation submodule 1341 is used to perform pixel-level feature transformation on the first image to obtain the full-resolution feature map corresponding to the first image.

[0258] The linear interpolation submodule 1342 is used to apply the target mapping curve to the full-resolution feature map and obtain pixel-level mapping coefficients through linear interpolation.

[0259] The action submodule 1343 is used to apply the pixel-level mapping coefficients to the first image to obtain the second image.

[0260] In summary, the image processing apparatus provided in this application obtains curve correction coefficients by performing feature analysis on a first image (the image to be adjusted for color parameters), corrects a preset mapping curve (a mapping curve with pre-set curve parameters) based on the curve correction coefficients, and adjusts the color parameters of the first image based on the adjusted preset mapping curve. Since only the correction coefficients need to be calculated when adjusting the color parameters of the first image, instead of calculating the entire mapping curve, the amount of computation is greatly reduced, thereby improving the efficiency of image color enhancement processing.

[0261] It should be noted that the image processing apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image processing apparatus and image processing method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0262] Figure 15 This illustration shows a schematic diagram of a server provided in an exemplary embodiment of this application. The server may be as follows: Figure 3The server shown here. Specifically, it includes the following structure:

[0263] Server 1500 includes a Central Processing Unit (CPU) 1501, a system memory 1504 including Random Access Memory (RAM) 1502 and Read Only Memory (ROM) 1503, and a system bus 1505 connecting the system memory 1504 and the CPU 1501. Server 1500 also includes a mass storage device 1506 for storing an operating system 1513, application programs 1514, and other program modules 1515.

[0264] Mass storage device 1506 is connected to central processing unit 1501 via a mass storage controller (not shown) connected to system bus 1505. Mass storage device 1506 and its associated computer-readable media provide non-volatile storage for server 1500. That is, mass storage device 1506 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.

[0265] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1504 and mass storage device 1506 described above can be collectively referred to as memory.

[0266] According to various embodiments of this application, server 1500 can also be connected to a remote computer on a network, such as the Internet. That is, server 1500 can be connected to network 1512 via network interface unit 1511 connected to system bus 1505, or it can also use network interface unit 1511 to connect to other types of networks or remote computer systems (not shown).

[0267] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0268] Embodiments of this application also provide a computer device that can be implemented as follows: Figure 3 The terminal or server shown. The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the image processing methods provided in the above-described method embodiments.

[0269] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the image processing method provided in the above-described method embodiments.

[0270] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image processing methods provided in the above-described method embodiments.

[0271] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0272] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0273] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image processing method, characterized in that, The method includes: The first image is input into the curve correction model to generate an image feature representation of the first image. The image feature representation includes at least one of texture feature representation, global feature representation, local feature representation, and semantic feature representation. The curve correction model is a neural network model trained using sample image pairs as training samples and based on training loss. The training loss includes at least one of pixel consistency loss and noise robustness loss. The sample image pair includes a sample image and a reference image. The reference image is an image obtained by adjusting the color parameters of the sample image. The curve correction model is used to perform feature analysis on the image feature representation to obtain the curve correction coefficients corresponding to the first image. The curve parameters of the preset mapping curve are corrected by the curve correction coefficient to obtain the target mapping curve; The color parameters of the first image are adjusted based on the target mapping curve to obtain the second image.

2. The method according to claim 1, characterized in that, The step of inputting the first image into the curve correction model and generating an image feature representation of the first image includes: The first image is input into the curve correction model to generate a texture feature representation of the first image, which is used to indicate the image texture features of the first image; A global feature representation of the first image is generated based on the texture feature representation, and the global feature representation is used to indicate the overall image features of the first image; Based on the texture feature representation, a local feature representation of the first image is generated, wherein the local feature representation is used to indicate the image features of a sub-image region in the first image; A semantic feature representation of the first image is generated based on the texture feature representation, and the semantic feature representation is used to indicate the semantic content contained in the first image; The texture feature representation, the global feature representation, the local feature representation, and the semantic feature representation are fused to obtain the image feature representation of the first image.

3. The method according to claim 1 or 2, characterized in that, The step of inputting the first image into the curve correction model and generating an image feature representation of the first image includes: The first image is divided into blocks to obtain at least two image blocks; The at least two image patches are input into the curve correction model to generate an image patch feature representation corresponding to each of the at least two image patches; The step of performing feature analysis on the image feature representation using the curve correction model to obtain the curve correction coefficients corresponding to the first image includes: Feature analysis is performed on the image block feature representations corresponding to the at least two image blocks respectively to obtain the block correction coefficients corresponding to the at least two image blocks respectively, and the curve correction coefficients are a set of at least two block correction coefficients.

4. The method according to claim 3, characterized in that, The step of dividing the first image into blocks to obtain at least two image blocks includes: The first image is divided into at least two image blocks by dividing it into an average number of blocks according to a preset number of blocks; or... The first image is randomly divided into blocks to obtain at least two image blocks.

5. The method according to claim 1 or 2, characterized in that, The step of correcting the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve includes: The target mapping curve is obtained by weighted summation of the curve correction coefficient and the preset mapping curve.

6. The method according to claim 1 or 2, characterized in that, The curve correction coefficient includes n sub-coefficients, where n is an integer greater than 1; The step of correcting the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve includes: The preset mapping curve is segmented to obtain n segmented mapping curves; The target mapping curve is obtained by correcting the n segmented mapping curves with n sub-coefficients, wherein the k-th segmented mapping curve is corrected by the k-th sub-coefficient, where k is a positive integer and k≤n.

7. The method according to claim 6, characterized in that, The process of segmenting the preset mapping curve to obtain n segmented mapping curves includes: The preset mapping curve is divided into segments according to the horizontal coordinate of the coordinate axis where the preset mapping curve is located, resulting in n segmented mapping curves. or, The preset mapping curve is divided into segments according to the ordinate of the coordinate axis where the preset mapping curve is located, resulting in n segmented mapping curves.

8. The method according to claim 1 or 2, characterized in that, The step of adjusting the color parameters of the first image based on the target mapping curve to obtain the second image includes: Pixel-level feature transformation is performed on the first image to obtain a full-resolution feature map corresponding to the first image; The target mapping curve is applied to the full-resolution feature map, and pixel-level mapping coefficients are obtained through linear interpolation. The pixel-level mapping coefficients are applied to the first image to obtain the second image.

9. An image processing apparatus, characterized in that, The device includes: The analysis module is used to input the first image into the curve correction model to generate an image feature representation of the first image. The image feature representation includes at least one of texture feature representation, global feature representation, local feature representation, and semantic feature representation. The curve correction model is a neural network model trained using sample image pairs as training samples and based on training loss. The training loss includes at least one of pixel consistency loss and noise robustness loss. The sample image pair includes a sample image and a reference image, and the reference image is an image obtained by adjusting the color parameters of the sample image. The curve correction model is used to perform feature analysis on the image feature representation to obtain the curve correction coefficients corresponding to the first image. The correction module is used to correct the curve parameters of the preset mapping curve using the curve correction coefficient to obtain the target mapping curve; The adjustment module is used to adjust the color parameters of the first image based on the target mapping curve to obtain the second image.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one program, which is loaded and executed by the processor to implement the image processing method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the image processing method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the image processing method as described in any one of claims 1 to 8.

Images