Method, apparatus and computer storage medium for image correction
By using a convolutional neural network correction model to perform personalized color level correction on images, the problems of image distortion and low sharpness after correction are solved, achieving high-quality image correction results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE INFORMATION TECHNOLOGY CO LTD
- Filing Date
- 2022-12-26
- Publication Date
- 2026-07-03
Smart Images
Figure CN116071254B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing, and specifically to an image correction method, apparatus, and computer storage medium. Background Technology
[0002] During image acquisition, color temperature changes, shooting scene switching, or deviations in camera white balance adjustment can cause color casts in the image, resulting in poor image realism.
[0003] To improve the realism of images, image correction is performed. However, existing image correction methods often result in distorted or low-resolution images.
[0004] Application content
[0005] This application provides an image correction method, apparatus, and computer storage medium, aiming to improve the problem of low image clarity after correction.
[0006] The first aspect of this application provides an image correction method, including:
[0007] The color-skewed image is input into a convolutional neural network correction model for calculation, and the color level correction curve parameters of each pixel in the color-skewed image are obtained.
[0008] Each pixel in the color-skewed image is corrected based on the color level correction curve parameters of each pixel in the color-skewed image to obtain the corrected target image.
[0009] According to the first aspect of this application, a color-skewed image is input into a convolutional neural network correction model for calculation to obtain the color level correction curve parameters for each pixel of the color-skewed image, including:
[0010] The pixel value of each pixel in the color-skewed image is normalized to obtain the transformed pixel value of each pixel in the normalized color-skewed image.
[0011] The converted pixel value of each pixel in the color-skewed image is input into the convolutional neural network correction model for calculation, and the color level correction curve parameters of each pixel in the color-skewed image are obtained.
[0012] According to any of the foregoing embodiments of the first aspect of this application, each pixel in the color-skewed image is corrected based on the color level correction curve parameters of each pixel in the color-skewed image to obtain a corrected target image, including:
[0013] Each pixel in the color-skewed image is corrected based on the color level correction curve parameters of each pixel to obtain the corrected pixel value;
[0014] The corrected pixel values are then denormalized to obtain the corrected target image.
[0015] According to any of the foregoing embodiments of the first aspect of this application, before inputting the color-skewed image into a convolutional neural network correction model for calculation to obtain the color level correction curve parameters of each pixel of the color-skewed image, the method includes:
[0016] Each pixel of the training image is input into the convolutional neural network model for calculation to obtain the color level curve parameters of each pixel in the training image.
[0017] Each pixel in the training image is corrected based on the color level curve parameters of each pixel in the training image to obtain a color-corrected image;
[0018] Image evaluation calculations are performed on the color cast corrected image to obtain the image evaluation results;
[0019] The convolutional neural network model is trained based on the image evaluation results to obtain the convolutional neural network correction model.
[0020] According to any of the foregoing embodiments of the first aspect of this application, the convolutional neural network model includes a convolutional layer and an activation layer; inputting each pixel of the training image into the convolutional neural network model for calculation to obtain the color level curve parameters of each pixel of the training image, further comprising:
[0021] The pixels corresponding to multiple pixel channels of the training image are input into the convolutional layer of the convolutional neural network model to perform convolution calculation, and the convolution values of the pixels corresponding to multiple pixel channels are obtained.
[0022] The convolution values of pixels corresponding to multiple pixel channels are input into the activation layer for processing to obtain the convolution values of pixels corresponding to each pixel channel in the multiple pixel channels.
[0023] The convolution value of each pixel corresponding to each pixel channel is determined as the color level curve parameter of each pixel channel.
[0024] According to any of the foregoing embodiments of the first aspect of this application, a pixel channel includes a first pixel channel, a second pixel channel, and a third pixel channel; the convolution values of pixels corresponding to the multiple pixel channels are input into an activation layer for processing to obtain the convolution value of pixels corresponding to each pixel channel in the multiple pixel channels, including:
[0025] The convolution values of the pixels corresponding to the first pixel channel, the second pixel channel, and the third pixel channel are input into the activation layer for processing to obtain the first parameter group, the second parameter group, and the third parameter group. The first parameter group, the second parameter group, and the third parameter group correspond to the convolution values of the pixels corresponding to the first pixel channel, the second pixel channel, and the third pixel channel, respectively.
[0026] According to any of the foregoing embodiments of the first aspect of this application, the first parameter group, the second parameter group, or the third parameter group includes i color level curve parameters, and the color cast correction image is calculated using the following formula:
[0027] X j =X j-1 +R j *(X j-1 2 -X j-1 ),
[0028] Where j is a positive integer, j≤i, X j X represents the j-th output value of the above formula. j-1 X represents the (j-1)th output value of the above formula. i R represents the transformed pixel value of each pixel in the color cast correction image, X0 represents the transformed pixel value of each pixel in the training image, and R represents the transformed pixel value of each pixel in the training image. j This represents the j-th color level curve parameter in the first, second, or third parameter group.
[0029] According to any of the foregoing embodiments of the first aspect of this application, an image evaluation calculation is performed on a color cast corrected image to obtain an image evaluation result, including:
[0030] The color cast evaluation of the color cast corrected image is calculated based on the first loss function, and the calculation result of the first loss function is obtained.
[0031] The sharpness evaluation of the color cast corrected image is calculated based on the second loss function, and the calculation result of the second loss function is obtained.
[0032] The image evaluation result is obtained based on the calculation results of the first loss function and the second loss function.
[0033] An embodiment of the second aspect of this application provides an image correction apparatus, the apparatus comprising:
[0034] The acquisition module is used to input the color-skewed image into the convolutional neural network correction model for calculation, and obtain the color level correction curve parameters of each pixel in the color-skewed image.
[0035] The correction module is used to correct each pixel in the color-skewed image according to the color level correction curve parameters of each pixel in the color-skewed image, so as to obtain the corrected target image.
[0036] An embodiment of the third aspect of this application provides a computer storage medium, characterized in that the computer storage medium stores computer program instructions, which, when executed by a processor, implement the image correction method of any of the above embodiments.
[0037] According to the image correction method provided in this application, the color-skewed image is first input into a convolutional neural network correction model for calculation to obtain the color level correction curve parameters of each pixel in the color-skewed image. That is, by using the convolutional neural network to make personalized adjustments to each pixel, a better visual experience can be maintained, and the realism of the target image can be improved. Then, each pixel in the color-skewed image is corrected according to the color level correction curve parameters of each pixel in the color-skewed image to obtain the corrected target image. That is, by using the mapping calculation of the color level correction curve parameters, the target image can be guaranteed to have good clarity. Attached Figure Description
[0038] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings, in which the same or similar reference numerals denote the same or similar features, and the drawings are not drawn to scale.
[0039] Figure 1 A schematic flowchart illustrating the image correction method provided in this application embodiment;
[0040] Figure 2 A schematic flowchart illustrating an image correction method provided in another embodiment of this application;
[0041] Figure 3 A schematic diagram of the structure of the image correction apparatus provided in the embodiments of this application;
[0042] Figure 4 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation
[0043] The features and exemplary embodiments of various aspects of this application will now be described in detail. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain this application and are not configured to limit this application. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.
[0044] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.
[0045] This application provides an image correction method, apparatus, and computer storage medium. The following description, in conjunction with the accompanying drawings, will illustrate various embodiments of the image correction method, apparatus, and computer storage medium.
[0046] Please see Figure 1 , Figure 1 This is a flowchart illustrating the image correction method provided in an embodiment of this application.
[0047] like Figure 1 As shown, the image correction method provided in this application embodiment may include the following steps:
[0048] Step 110: Input the color-skewed image into the convolutional neural network correction model for calculation to obtain the color level correction curve parameters for each pixel of the color-skewed image.
[0049] Step 120: Correct each pixel in the color-skewed image according to the color level correction curve parameters of each pixel in the color-skewed image to obtain the corrected target image.
[0050] According to the image correction method provided in this application, the color-skewed image is first input into a convolutional neural network correction model for calculation to obtain the color level correction curve parameters of each pixel in the color-skewed image. That is, by using the convolutional neural network to make personalized adjustments to each pixel, a better visual experience can be maintained, and the realism of the target image can be improved. Then, each pixel in the color-skewed image is corrected according to the color level correction curve parameters of each pixel in the color-skewed image to obtain the corrected target image. That is, by using the mapping calculation of the color level correction curve parameters, the target image can be guaranteed to have good clarity.
[0051] In some embodiments, step 110 includes:
[0052] The pixel value of each pixel in the color-skewed image is normalized to obtain the transformed pixel value of each pixel in the normalized color-skewed image.
[0053] The converted pixel value of each pixel in the color-skewed image is input into the convolutional neural network correction model for calculation, and the color level correction curve parameters of each pixel in the color-skewed image are obtained.
[0054] For example, in the step of normalizing the pixel value of each pixel in the color-skewed image to obtain the converted pixel value of each pixel in the normalized color-skewed image, the pixel value of each pixel can be any value between 0 and 255. Normalization is to divide the pixel value of each pixel by 255 to obtain the converted pixel value in the range (0, 1).
[0055] In these embodiments, the pixel value of each pixel in the color-skewed image is normalized, ensuring that the pixel value of each pixel is between (0,1), resulting in a smaller value that facilitates subsequent calculations and improves computation speed. The transformed pixel value of each pixel in the color-skewed image is then input into a convolutional neural network correction model. This allows the model to personalize the adjustment for each pixel, meaning that the transformed pixel value of each pixel is recalculated to generate color correction curve parameters. The use of the convolutional neural network correction model maintains a good visual quality and improves the realism of the corrected image.
[0056] In some alternative embodiments, step 120 includes:
[0057] Each pixel in the color-skewed image is corrected based on the color level correction curve parameters of each pixel to obtain the corrected pixel value.
[0058] The corrected pixel values are then denormalized to obtain the corrected target image.
[0059] For example, the corrected pixel values are reverse normalized, that is, the corrected pixel value of each pixel is multiplied by 255 to obtain any value between 0 and 255, thereby obtaining the corrected target image.
[0060] In these optional embodiments, the color correction curve parameters of each pixel in the color-biased image are used to correct each pixel through mapping calculations to obtain a corrected pixel value. That is, correcting each pixel using the color correction curve parameters ensures that the corrected target image has good clarity. Since the pixel values of each pixel were normalized in the aforementioned steps, the corrected pixel values are then denormalized to obtain the pixel values of the actual target image, thus obtaining the actual target image.
[0061] Please see Figure 2 , Figure 2 This is a schematic flowchart of an image correction method provided in another embodiment of this application.
[0062] like Figure 2 As shown, in some embodiments, prior to step 110, the image correction method may further include:
[0063] Step 130: Input each pixel of the training image into the convolutional neural network model for calculation to obtain the color level curve parameters of each pixel of the training image.
[0064] Step 140: Correct each pixel in the training image according to the color level curve parameters of each pixel in the training image to obtain a color-corrected image.
[0065] Step 150: Perform image evaluation calculation on the color cast corrected image to obtain the image evaluation result.
[0066] Step 160: Train the convolutional neural network model based on the image evaluation results to obtain the convolutional neural network correction model.
[0067] In these embodiments, each pixel of the training image is first input into a convolutional neural network model for calculation, thereby obtaining the color level curve parameters for each pixel of the training image. Then, the color level curve parameters are used to map and calculate each pixel of the training image, resulting in a color-corrected image. Before correcting the color-corrected image in a practical application, the training image is used for simulated correction to obtain the required convolutional neural network correction model for correcting the color-corrected image. During the simulated correction of the training image, by using the convolutional neural network model and the mapping calculation of the color level curve parameters, it is possible to both personalize the adjustment of each pixel through the convolutional neural network to maintain a good visual experience and improve the realism of the corrected image, and ensure good image sharpness through the mapping calculation of the color level correction curve parameters. Step 150 performs image evaluation calculation on the color-corrected image to obtain the color cast and sharpness loss of the color-corrected image. Finally, the convolutional neural network is trained based on the image evaluation results, namely the color cast and the loss of sharpness. For example, if the image evaluation calculation results are not ideal, the parameters in the convolutional neural network model need to be adjusted. After adjustment, the parameters of each pixel in the training image are calculated again to obtain new color level curve parameters. Then, a new color-corrected image is obtained through mapping calculation, and image evaluation calculation is performed on the new color-corrected image. If the image evaluation results meet the requirements, the convolutional neural network model obtained in this stage is the actual convolutional neural network correction model required. If the image evaluation results do not meet the requirements, the above process is repeated.
[0068] In some embodiments, step 130 includes:
[0069] The pixel value of each pixel in the training image is normalized to obtain the transformed pixel value of each pixel in the normalized training image.
[0070] The converted pixel value of each pixel in the training image is input into the convolutional neural network model for calculation to obtain the color level curve parameters of each pixel in the training image.
[0071] In these embodiments, normalizing the pixel value of each pixel in the training image ensures that the pixel value of each pixel is between (0,1), resulting in a smaller value that facilitates subsequent calculations and improves computation speed. The transformed pixel value of each pixel in the training image is then input into the convolutional neural network correction model, allowing the model to personalize adjustments for each pixel. Specifically, the transformed pixel value of each pixel is recalculated to generate color gradation curve parameters. The use of the convolutional neural network model maintains a good visual quality and improves the realism of the color-corrected image.
[0072] In some embodiments, the convolutional neural network model includes convolutional layers and activation layers; step 130 further includes:
[0073] The pixels corresponding to multiple pixel channels of the training image are input into the convolutional layer of the convolutional neural network model for convolution calculation, and the convolution values of the pixels corresponding to multiple pixel channels are obtained.
[0074] The convolution values of pixels corresponding to multiple pixel channels are input into the activation layer for processing to obtain the convolution value of pixels corresponding to each pixel channel in the multiple pixel channels.
[0075] The convolution value of each pixel corresponding to each pixel channel is determined as the color level curve parameter of each pixel channel.
[0076] In these optional embodiments, the convolutional layers of a convolutional neural network model are first used to calculate the convolutional values for the pixels corresponding to multiple channels of the training image, obtaining the convolutional values for each pixel channel. The use of a convolutional neural network model maintains good visual quality and improves the realism of the corrected image. The convolutional values are then input to an activation layer, which controls the convolutional values obtained from the convolutional layer within a certain range, facilitating subsequent mapping calculations. For example, the activation layer controls the output of the convolutional layer within the range (-1, 1).
[0077] In some embodiments, the convolutional neural network model calculates in the following order: a first convolutional layer, a second convolutional layer, a third convolutional layer, a fourth convolutional layer, and an activation layer. Each pixel of the training image is input to the first convolutional layer, the first convolutional layer calculates and inputs to the second convolutional layer, the second convolutional layer calculates and inputs to the third convolutional layer, the third convolutional layer calculates and inputs to the fourth convolutional layer, and the fourth convolutional layer calculates and inputs to the activation layer, thereby obtaining the color level curve parameters.
[0078] For example, the first, second, and third convolutional layers are all two-dimensional convolutional layers with a kernel size of (3,3), a stride of (1,1), a ReLU activation function, and an output dimension of 32. The fourth convolutional layer is also a two-dimensional convolutional layer with a kernel size of (3,3), a stride of (1,1), and an output dimension of 24. Of the 24 output dimensions, the 24th dimension represents the 24 output values of the fourth convolutional layer. The first 8 output values of the fourth convolutional layer serve as the color curve parameters for the red channel, the 9th to 16th output values serve as the color curve parameters for the green channel, and the 17th to 24th output values serve as the color curve parameters for the blue channel. Activation layers are used to control the output between (-1, 1).
[0079] In some embodiments, a pixel channel includes a first pixel channel, a second pixel channel, and a third pixel channel; the convolution values of pixels corresponding to multiple pixel channels are input into an activation layer for processing to obtain the convolution values of pixels corresponding to each pixel channel in the multiple pixel channels, including:
[0080] The convolution values of the pixels corresponding to the first pixel channel, the second pixel channel, and the third pixel channel are input into the activation layer for processing to obtain the first parameter group, the second parameter group, and the third parameter group. The first parameter group, the second parameter group, and the third parameter group correspond to the convolution values of the pixels corresponding to the first pixel channel, the second pixel channel, and the third pixel channel, respectively.
[0081] In these embodiments, the first pixel channel obtains a first parameter set through a convolutional neural network model, the second pixel channel obtains a second parameter set through a convolutional neural network model, and the third pixel channel obtains a third parameter set through a convolutional neural network model. Each pixel channel obtains its corresponding parameter set, which serves as the color level curve parameters for each pixel channel of each pixel point. The pixel values corresponding to each pixel channel of the color-corrected image are calculated through the mapping of the color level curve parameters, ultimately yielding the color-corrected image.
[0082] Optionally, the first pixel channel, the second pixel channel, and the third pixel channel correspond to one of the red pixel channel, the green pixel channel, and the blue pixel channel, respectively.
[0083] In some embodiments, the first parameter group, the second parameter group, or the third parameter group includes i color level curve parameters, and the color cast correction image is calculated using the following formula:
[0084] X j =X j-1 +R j *(X j-1 2 -X j-1 Equation (1)
[0085] Where j is a positive integer, j≤i, X j X represents the j-th output value of the above formula. j-1 X represents the (j-1)th output value of the above formula. i R represents the transformed pixel value of each pixel in the color cast correction image, X0 represents the transformed pixel value of each pixel in the training image, and R represents the transformed pixel value of each pixel in the training image. j This represents the j-th color level curve parameter in the first, second, or third parameter group.
[0086] In these embodiments, the color correction image is calculated using Equation (1). For example, when the output dimension of the fourth convolutional layer is 24, each pixel of the training image has 8 color level curve parameters for each pixel channel. Taking the first pixel channel as an example, these are denoted as (R1, R2, R3, R4, R5, R6, R7, R8). The iterative calculation process is obtained according to Equation (1):
[0087] X1 = X0 + R1 * (X0) 2 -X0) Equation (1-1)
[0088] X2=X1+R2*(X1 2 -X1) Equation (1-2)
[0089] X3=X2+R3*(X2) 2 -X2) Equation (1-3)
[0090] X4 = X3 + R4 * (X3) 2 -X3) Equation (1-4)
[0091] X5 = X4 + R5 * (X4) 2 -X4) Equation (1-5)
[0092] X6 = X5 + R6 * (X5) 2 -X5) Equation (1-6)
[0093] X7 = X6 + R7 * (X6) 2 -X6) Equation (1-7)
[0094] X8 = X7 + R8 * (X7) 2 -X7) Equation (1-8)
[0095] In this context, X1, X2, X3, X4, X5, X6, and X7 are all process variables with no practical significance. X8 is the converted pixel value of the first pixel channel of each pixel in the color-corrected image. Based on the same calculation method, the converted pixel values of the second and third pixel channels of each pixel in the color-corrected image can be calculated, thus obtaining the color-corrected image.
[0096] In some embodiments, step 150 includes:
[0097] The color cast evaluation of the color cast corrected image is calculated based on the first loss function, and the calculation result of the first loss function is obtained.
[0098] The sharpness evaluation of the color cast corrected image is calculated based on the second loss function, and the calculation result of the second loss function is obtained.
[0099] The image evaluation result is obtained based on the calculation results of the first loss function and the second loss function.
[0100] In these embodiments, by calculating the value of a first loss function for the color-corrected image, a color cast evaluation can be performed based on this value. For example, firstly, the mean color cast values mr, mg, and mb corresponding to the first, second, and third pixel channels of the color-corrected image are calculated respectively. Then, the value of the first loss function is calculated according to equation (2):
[0101] loss_color=((mr-mg) 2 +(mg-mb) 2 + (mb-mr) 2 ) 0.5 Equation (2)
[0102] Wherein, loss_color is the first loss function. After obtaining the mean color cast values mr, mg and mb corresponding to the first pixel channel, second pixel channel and third pixel channel of the color-corrected image, the mean color cast values mr, mg and mb are substituted into equation (2) to calculate the value of the first loss function. The color cast of the color-corrected image is evaluated based on the value of the first loss function. The smaller the value of the first loss function, the smaller the color cast of the color-corrected image.
[0103] By calculating the value of the second loss function of the color-corrected image, the sharpness of the color-corrected image can be evaluated based on this value. The image evaluation result can be determined by combining the calculation results of the first and second loss functions.
[0104] In some embodiments, calculating the sharpness evaluation of the color-corrected image based on the second loss function includes:
[0105] Create the first convolutional model to calculate the difference between left and right adjacent pixels in the image.
[0106] Create a second convolutional model to calculate the difference between adjacent pixels in the image.
[0107] The contrast of the training image is obtained by calculating the contrast of adjacent pixels on the left and right sides and the contrast of adjacent pixels on the top and bottom sides based on the first and second convolution models.
[0108] The contrast of the color-corrected image is obtained by calculating the contrast of adjacent pixels on the left and right sides and the contrast of adjacent pixels on the top and bottom sides based on the first and second convolution models.
[0109] The value of the second loss function is determined based on the difference between the contrast of the training image and the contrast of the color-corrected image.
[0110] For example, a first convolutional model Conv_lr is created: this model contains only one two-dimensional convolutional layer with kernels [[0, 0, 0], [-1, 1, 0], [0, 0, 0]], stride 1, and padding 'same', used to calculate the difference between the left and right adjacent pixels of the image. A second convolutional model Conv_ud is created: this model contains only one two-dimensional convolutional layer with kernels [[0, -1, 0], [0, 1, 0], [0, 0, 0]], stride 1, and padding 'same', used to calculate the difference between the upper and lower adjacent pixels of the image. The contrast ratios D1 and D2 of the left and right adjacent pixels and the upper and lower adjacent pixels of the training image are calculated based on the first convolutional model Conv_lr and the second convolutional model Conv_ud. The contrast ratios D3 and D4 of the left and right adjacent pixels and the upper and lower adjacent pixels of the color-corrected image are calculated based on the first convolutional model Conv_lr and the second convolutional model Conv_ud. The value of the second loss function is calculated according to equation (3):
[0111] loss_definition=(D1-D3) 2 +(D2-D4) 2 Equation (3)
[0112] Among them, loss_definition is the second loss function. The value of the second loss function is calculated by equation (3) so as to evaluate the sharpness of the color-corrected image.
[0113] In these embodiments, the image evaluation calculation result is denoted as loss = loss_color + loss_definition, which is the sum of the first loss function value and the second loss function value. When the image evaluation calculation result loss approaches 0, the convolutional neural network model obtained in this stage is the required convolutional neural network correction model to correct the color cast image.
[0114] When the image evaluation calculation result loss is large, meaning the color cast and sharpness of the color-corrected image are poor, the gradient of the color level curve parameters of each pixel in the training image can be calculated, and the parameters of the convolutional neural network model can be adjusted accordingly. Therefore, in some embodiments, step 160 includes:
[0115] The gradient of the color level curve parameters for each pixel in the training image is calculated.
[0116] The parameters of the convolutional neural network model are adjusted based on the gradient of the color level curve parameters of each pixel in the training image to obtain the convolutional neural network correction model.
[0117] In these embodiments, the gradient of the color level curve parameters is calculated, and the parameters of the convolutional neural network model are adjusted according to the gradient value to obtain a new convolutional neural network model. The new convolutional neural network model is used to repeatedly train the training images using the aforementioned image correction method until the image evaluation of the calculated color-corrected image meets the requirements. The final new convolutional neural network model is the convolutional neural network correction model, which is then applied to the color-corrected image to obtain the target image.
[0118] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of the image correction apparatus provided in the embodiments of this application.
[0119] like Figure 3 As shown, this application embodiment provides an image correction apparatus, the apparatus comprising:
[0120] The acquisition module 201 is used to input the color-skewed image into the convolutional neural network correction model for calculation, and obtain the color level correction curve parameters of each pixel in the color-skewed image.
[0121] The correction module 202 is used to correct each pixel in the color-skewed image according to the color level correction curve parameters of each pixel in the color-skewed image, so as to obtain the corrected target image.
[0122] According to the image correction apparatus of this application embodiment, on the one hand, the acquisition module 201 inputs a color-skewed image into a convolutional neural network correction model for calculation, obtaining the color level correction curve parameters of each pixel in the color-skewed image. That is, by using the convolutional neural network to perform personalized calculations and adjustments for each pixel, a better visual experience can be maintained, improving the realism of the target image. On the other hand, the correction module 202 corrects each pixel in the color-skewed image according to the color level correction curve parameters of each pixel in the color-skewed image, obtaining the corrected target image. That is, by using the mapping calculation of the color level correction curve parameters, the target image can be guaranteed to have good clarity.
[0123] In some embodiments, the acquisition module 201 is further configured to input each pixel of the training image into a convolutional neural network model for calculation to obtain the color level curve parameters of each pixel of the training image. The correction module 202 is further configured to correct each pixel in the training image according to the color level curve parameters of each pixel of the training image to obtain a color-corrected image.
[0124] In some embodiments, the apparatus further includes an image evaluation module and a training module. The image evaluation module is used to perform image evaluation calculations on the color cast correction image to obtain an image evaluation result; the training module is used to train a convolutional neural network model based on the image evaluation result to obtain a convolutional neural network correction model.
[0125] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application.
[0126] like Figure 4 As shown, this application provides an electronic device, which includes a processor 301 and a memory 302 storing computer program instructions; the processor 301 reads and executes the computer program instructions to implement the image correction method of any of the above embodiments.
[0127] For example, the program can be divided into one or more modules / units, one or more of which are stored in memory 302 and executed by processor 301 to complete this application. The one or more modules / units can be a series of program instruction segments capable of performing a specific function, which describe the execution process of the program in the device.
[0128] Specifically, the processor 301 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.
[0129] Memory 302 may include mass storage for data or instructions. For example, and not limitingly, memory 302 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 302 may include removable or non-removable (or fixed) media. Where appropriate, memory 302 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 302 is non-volatile solid-state memory.
[0130] Memory 302 may also include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, typically, memory 302 includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to the first aspect of this application.
[0131] The processor 301 implements any of the methods described in the first aspect embodiment by reading and executing program instructions stored in the memory 302.
[0132] In some embodiments, the electronic device may further include a communication interface 303 and a bus 310. The processor 301, memory 302, and communication interface 303 are connected via the bus 310 and communicate with each other.
[0133] The communication interface 303 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.
[0134] Bus 310 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 310 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.
[0135] Furthermore, in conjunction with the methods in the above embodiments, this application embodiment can provide a computer storage medium for implementation. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the methods in the above embodiments.
[0136] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.
[0137] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on machine-readable media or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.
[0138] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.
[0139] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.
[0140] The embodiments described above are not exhaustive, nor do they limit the application to the specific embodiments described herein. Clearly, many modifications and variations can be made based on the above description. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of this application, thereby enabling those skilled in the art to effectively utilize this application and its modifications. This application is limited only by the claims and their full scope and equivalents.
Claims
1. A method of image correction, characterized by, include: The color-skewed image is input into a convolutional neural network correction model for calculation to obtain the color level correction curve parameters of each pixel in the color-skewed image. Each pixel includes multiple pixel channels, and each pixel channel corresponds to a set of color level correction curve parameters. Each pixel in the color-skewed image is corrected according to the color level correction curve parameters of each pixel in the color-skewed image to obtain a corrected target image. The correction is used to eliminate the color cast of the color-skewed image and maintain the clarity of the color-skewed image.
2. The image correction method according to claim 1, characterized in that, The step of inputting the color-skewed image into a convolutional neural network correction model for calculation to obtain the color level correction curve parameters for each pixel of the color-skewed image includes: The pixel value of each pixel in the color-skewed image is normalized to obtain the normalized converted pixel value of each pixel in the color-skewed image. The converted pixel value of each pixel in the color-skewed image is input into the convolutional neural network correction model for calculation to obtain the color level correction curve parameters of each pixel in the color-skewed image.
3. The image correction method according to claim 2, characterized in that, The step of correcting each pixel in the color-skewed image according to the color level correction curve parameters of each pixel in the color-skewed image to obtain the corrected target image includes: Each pixel in the color-skewed image is corrected according to the color level correction curve parameters of each pixel in the color-skewed image to obtain the corrected pixel value; The corrected pixel values are then denormalized to obtain the corrected target image.
4. The image correction method according to claim 1, characterized in that, Before inputting the color-skewed image into the convolutional neural network correction model for calculation to obtain the color level correction curve parameters for each pixel of the color-skewed image, the method includes: Each pixel of the training image is input into a convolutional neural network model for calculation to obtain the color level curve parameters of each pixel of the training image. Each pixel in the training image is corrected based on the color level curve parameters of each pixel to obtain a color-corrected image; Image evaluation calculations are performed on the color-corrected image to obtain image evaluation results; The convolutional neural network model is trained based on the image evaluation results to obtain the convolutional neural network correction model.
5. The image correction method according to claim 4, characterized in that, The convolutional neural network model includes convolutional layers and activation layers; the step of inputting each pixel of the training image into the convolutional neural network model for calculation to obtain the color level curve parameters of each pixel of the training image includes: The pixels corresponding to multiple pixel channels of the training image are input into the convolutional layer of the convolutional neural network model for convolution calculation to obtain the convolutional values of the pixels corresponding to multiple pixel channels. The convolution values of the pixels corresponding to the multiple pixel channels are input into the activation layer for processing to obtain the convolution value of the pixels corresponding to each of the multiple pixel channels. The convolution value of each pixel corresponding to each pixel channel is determined as the color level curve parameter of each pixel channel.
6. The image correction method according to claim 5, characterized in that, The pixel channels include a first pixel channel, a second pixel channel, and a third pixel channel; the step of inputting the convolution values of the corresponding pixels of the multiple pixel channels into the activation layer for processing to obtain the convolution value of the pixel corresponding to each of the multiple pixel channels includes: The convolution values of the pixels corresponding to the first pixel channel, the second pixel channel, and the third pixel channel are input into the activation layer for processing to obtain a first parameter group, a second parameter group, and a third parameter group. The first parameter group, the second parameter group, and the third parameter group respectively correspond to the convolution values of the pixels corresponding to the first pixel channel, the second pixel channel, and the third pixel channel.
7. The image correction method according to claim 6, characterized in that, The first parameter group, the second parameter group, or the third parameter group includes i color level curve parameters, and the color cast correction image is calculated using the following formula: ， Where j is a positive integer, j≤i, This represents the j-th output value of the above formula. This represents the (j-1)th output value of the above formula. This represents the transformed pixel value for each pixel in the color cast corrected image. This represents the transformed pixel value for each pixel in the training image. This represents the j-th color level curve parameter in the first parameter group, the second parameter group, or the third parameter group.
8. The image correction method according to claim 6, characterized in that, The step of performing image evaluation calculations on the color-corrected image to obtain image evaluation results includes: The color cast evaluation of the color cast corrected image is calculated based on the first loss function to obtain the calculation result of the first loss function; The sharpness evaluation of the color cast corrected image is calculated based on the second loss function, and the calculation result of the second loss function is obtained. Based on the calculation results of the first loss function and the second loss function, the image evaluation result is obtained.
9. An image correction apparatus, characterized in that, The device includes: The acquisition module is used to input the color-skewed image into the convolutional neural network correction model for calculation, and to acquire the color level correction curve parameters of each pixel of the color-skewed image. Each pixel includes multiple pixel channels, and each pixel channel corresponds to a set of color level correction curve parameters. The correction module is used to correct each pixel in the color-skewed image according to the color level correction curve parameters of each pixel in the color-skewed image to obtain a corrected target image. The correction is used to eliminate the color cast of the color-skewed image and maintain the clarity of the color-skewed image.
10. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the image correction method as described in any one of claims 1-8.