Image processing method and device, storage medium, electronic equipment and chip

Abstract

This disclosure provides an image processing method, apparatus, storage medium, electronic device, and chip, relating to the field of image processing technology. The method includes: firstly, determining the RGB response values ​​of a camera capturing a standard color chart under a calibrated light source, and determining the standard RGB values ​​perceived by the human eye under a target light source; then, based on the RGB response values ​​of the camera capturing the standard color chart under the calibrated light source and the standard RGB values ​​perceived by the human eye under the target light source, accurately calibrating a three-dimensional lookup table, thereby applying the calibrated three-dimensional lookup table to the color correction of images captured by the camera with higher precision, significantly improving the accuracy of color correction of images captured by the camera.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, apparatus, storage medium, electronic device and chip. Background Technology

[0002] The colors in captured images often differ from the colors of the environment as perceived by the human eye. Therefore, it is necessary to perform color restoration on captured images to make the colors in captured images closer to what the human eye perceives.

[0003] Currently, color correction matrices are typically used to convert the color information of camera-captured images into a standard color space. However, color correction matrices are usually simple linear matrices, which have limited accuracy and thus reduce the color correction accuracy of camera-captured images. Summary of the Invention

[0004] In view of this, the present disclosure provides an image processing method, apparatus, storage medium, electronic device and chip, the main purpose of which is to solve the problem that related technologies reduce the accuracy of color correction of images captured by a camera.

[0005] According to a first aspect of this disclosure, an image processing method is provided, comprising: Determine the RGB response value of the camera when shooting the standard color chart under a calibrated light source, and determine the standard RGB value of the standard color chart perceived by the human eye under a target light source; Based on the RGB response value and the standard RGB value, a three-dimensional lookup table is calibrated, wherein the calibrated three-dimensional lookup table is used for color correction of the image captured by the camera.

[0006] In some possible implementations, determining the RGB response values ​​of the camera taken with a standard color chart under a calibrated light source includes: Acquire the camera response curve of the camera, the reflectance of the standard color chart, and the spectral data of the calibration light source; Based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera when shooting the standard color chart under the calibration light source is obtained by integration.

[0007] In some possible implementations, determining the standard RGB values ​​perceived by the human eye under the target light source as the standard color chart includes: Acquire the human eye response curve and the spectral data of the target light source; Based on the human eye response curve, the reflectance of the standard color chart, and the spectral data of the target light source, the CIE XYZ tristimulus values ​​are obtained by integration; The CIE XYZ tristimulus values ​​are converted to values ​​in the standard RGB space and used as the standard RGB values ​​perceived by the human eye under the target light source.

[0008] In some possible implementations, before calibrating the three-dimensional lookup table based on the RGB response values ​​and the standard RGB values, the method further includes: The RGB response values ​​are preprocessed sequentially with white balance and brightness alignment to obtain preprocessed RGB response values; The calibration of the three-dimensional lookup table based on the RGB response value and the standard RGB value includes: The three-dimensional lookup table is calibrated based on the standard RGB values ​​and the preprocessed RGB response values.

[0009] In some possible implementations, the RGB response values ​​are preprocessed for white balance, including: Select the grayscale color block corresponding to the standard color card as the reference color block; Determine the first average brightness value of the R / G channel and the second average brightness value of the B / G channel in the camera response of the reference color patch; White balance gain analysis is performed based on the first average brightness value and the second average brightness value to obtain the white balance gain coefficient; The RGB response value is compensated for using the white balance gain coefficient to obtain the compensated RGB response value.

[0010] In some possible implementations, the RGB response values ​​undergo brightness alignment preprocessing, including: The third average brightness value of the compensated G channel response value after the combined effect of the non-standard gamma curve and the standard inverse gamma curve is determined, and the fourth average brightness value of the G channel in the standard RGB value of the corresponding color block is determined. The ratio of the fourth average brightness value to the third average brightness value is determined as the brightness correction coefficient; The brightness correction coefficient is used to perform brightness correction on the compensated RGB response value so that the compensated RGB response value is aligned with the standard RGB value.

[0011] In some possible implementations, calibrating the three-dimensional lookup table based on the standard RGB values ​​and the preprocessed RGB response values ​​includes: Obtain the objective function, which includes constraints on the accuracy of color reproduction, the smoothness of color transition, and the stability of white point visual perception. Based on the preprocessed RGB response values ​​and the standard RGB values, the objective function is used to calibrate the three-dimensional lookup table.

[0012] In some possible implementations, the step of calibrating the three-dimensional lookup table using the objective function based on the preprocessed RGB response values ​​and the standard RGB values ​​includes: Substitute the preprocessed RGB response value and the standard RGB value into the objective function, and obtain the parameter information for updating the grid point values ​​in the three-dimensional lookup table by calculating the minimum value of the objective function; The grid values ​​in the three-dimensional lookup table are updated using the parameter information to calibrate the three-dimensional lookup table.

[0013] In some possible implementations, the accuracy constraint of color reproduction is used to limit the error between the output value obtained by interpolating the preprocessed RGB response value through the three-dimensional lookup table and the standard RGB value to be minimal; The color transition smoothness constraint is used to limit the second derivative relationship between grid points in the three-dimensional lookup table to ensure that the color transition between grid points in the three-dimensional lookup table is smooth. The stability constraint of the white point vision is used to ensure that the visual representation of the white point remains unchanged after correction.

[0014] According to a second aspect of this disclosure, an image processing apparatus is provided, comprising: The determination module is used to determine the RGB response value of the camera to the standard color chart under the calibrated light source, and to determine the standard RGB value perceived by the human eye to the standard color chart under the target light source. The calibration module is used to calibrate the three-dimensional lookup table based on the RGB response value and the standard RGB value, wherein the calibrated three-dimensional lookup table is used for color correction of the image captured by the camera.

[0015] In some possible implementations, the determining module is used to acquire the camera response curve of the camera, the reflectance of the standard color chart, and the spectral data of the calibration light source; based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera shooting the standard color chart under the calibration light source is obtained by integration.

[0016] In some possible implementations, the determining module is further configured to perform white balance and brightness alignment preprocessing on the RGB response values ​​sequentially to obtain preprocessed RGB response values; The calibration module is used to calibrate the three-dimensional lookup table based on the standard RGB values ​​and the preprocessed RGB response values.

[0017] In some possible implementations, the calibration module is used to obtain an objective function, which includes constraints on the accuracy of color reproduction, the smoothness of color transition, and the stability of white point vision; based on the preprocessed RGB response values ​​and the standard RGB values, the objective function is used to calibrate the three-dimensional lookup table.

[0018] According to a third aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image processing method described in the first aspect.

[0019] According to a fourth aspect of this disclosure, an electronic device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the computer program to implement the image processing method described in the first aspect.

[0020] Fifthly, this disclosure provides a chip including one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the image processing method described in the first aspect.

[0021] By employing the above technical solutions, the image processing method, apparatus, storage medium, electronic device, and chip disclosed herein can specifically first determine the RGB response values ​​of a camera capturing a standard color chart under a calibrated light source, and determine the standard RGB values ​​perceived by the human eye under a target light source; then, based on the RGB response values ​​of the camera capturing a standard color chart under a calibrated light source, and the standard RGB values ​​perceived by the human eye under a target light source, the accurate calibration of a three-dimensional lookup table can be achieved. Furthermore, the calibrated three-dimensional lookup table can be applied to the color correction of images captured by the camera, resulting in higher precision and significantly improving the accuracy of color correction for images captured by the camera.

[0022] The above description is merely an overview of the technical solution disclosed herein. In order to better understand the technical means of this disclosure and to implement it in accordance with the contents of the specification, and to make the above and other objectives, features and advantages more obvious and understandable, specific embodiments of this disclosure are described below. Attached Figure Description

[0023] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0024] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a schematic flowchart of an image processing method provided in an embodiment of the present disclosure; Figure 2 A flowchart illustrating a method for calibrating a three-dimensional lookup table according to an embodiment of this disclosure; Figure 3 An example diagram of two-dimensional grid regression provided in this embodiment of the disclosure; Figure 4 This is a schematic diagram illustrating an example method for calibrating a three-dimensional lookup table, provided in an embodiment of this disclosure. Figure 5 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of the present disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0026] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0027] Numerous specific details are set forth in the following description to provide a full understanding of this disclosure. However, this disclosure can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this disclosure. Therefore, this disclosure is not limited to the specific implementations disclosed below.

[0028] The terminology used in one or more embodiments of this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this disclosure. The singular forms “a,” “the,” and “the” as used in one or more embodiments of this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this disclosure refers to and includes any or all possible combinations of one or more associated listed items.

[0029] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this disclosure, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this disclosure, and similarly, second may also be referred to as first. Depending on the context, the word “if” as used herein may be interpreted as “when”, “in response to a determination”, or “when…”.

[0030] In some embodiments, camera profiling aims to convert the RGB response values ​​captured by the camera into display-relevant standard RGB values, where standard RGB spaces include sRGB, Display-P3, Rec.2020, etc. As an optional approach, this conversion can be achieved using a color correction matrix, which linearly transforms the RGB response values ​​captured by the camera to obtain standard RGB values. Essentially, the accuracy of the color correction matrix is ​​crucial for the camera to accurately capture colors that match human visual perception. However, this color correction matrix typically uses a simple linear matrix, which has limited accuracy and consequently reduces the color correction accuracy of the images captured by the camera.

[0031] Therefore, to solve the above problems, this disclosure provides an image processing method, giving a calibration scheme for a three-dimensional lookup table, and then applying the calibrated three-dimensional lookup table to the color correction of images captured by a camera, such as... Figure 1 As shown, the method includes: Step 101: Determine the RGB response value of the camera when shooting the standard color chart under the calibrated light source, and determine the standard RGB value perceived by the human eye under the target light source.

[0032] The standard color chart serves as a reference tool for color measurement, calibration, and evaluation. It may contain a series of known color samples, each possessing known color attributes such as hue, brightness, and saturation. In some examples, the data points of the standard color chart should cover the RGB three-dimensional structural numerical range as comprehensively as possible. Therefore, in this embodiment, the standard color chart may be a color chart with a rich color palette, such as the Munsell or Pantone color charts.

[0033] For the embodiments of this disclosure, a spectral calibration method can be used. Specifically, a monochromator can be used to measure the response curve of the camera module, a reflectance spectral analyzer can be used to measure the reflectance of each color patch on the standard color chart, and the spectra of the calibration light source and the target light source can be measured simultaneously. Combined with the human eye response curve, and based on the principles of colorimetry, the RGB response value of the camera capturing the standard color chart under the calibration light source, and the standard RGB value perceived by the human eye under the target light source are obtained through an integration method. These RGB response values ​​and standard RGB values ​​are used for the calibration of the three-dimensional lookup table, i.e., the process shown in step 102 is executed.

[0034] Step 102: Based on the RGB response values ​​of the camera shooting the standard color chart under the calibrated light source, and the standard RGB values ​​perceived by the human eye under the target light source, the three-dimensional lookup table is calibrated. The calibrated three-dimensional lookup table can be used for color correction of images captured by the camera.

[0035] This embodiment of the disclosure can calibrate a three-dimensional lookup table by calculating the corresponding RGB response values ​​of a standard color chart captured by a camera under a calibrated light source, and the standard RGB values ​​perceived by the human eye under a target light source, using an optimization method. For example, the size of the three-dimensional lookup table used may be such that the number of grid points in each dimension is 17.

[0036] For example, each light source corresponds to an independent calibration process. This involves first measuring the spectral data of the light source, then combining it with the camera response curve, standard color chart reflectance, and human eye response curve. Through integration, the camera's RGB response value and the standard RGB value perceived by the human eye are calculated. Then, an optimization method is used to calculate the corresponding 3D lookup table for that light source. When using the 3D lookup table, light source identification is performed first. For example, by analyzing the RGB channel response ratio, white point coordinates, and scene brightness distribution characteristics of the image data collected by the camera sensor, the type of light source in the current shooting environment is quickly determined. Then, among several pre-stored calibrated 3D lookup tables, the corresponding 3D lookup table is called based on the identified light source type. Finally, the original RGB data of the currently captured image is used to achieve color restoration that conforms to human eye perception through the called 3D lookup table.

[0037] The technical solution provided in this disclosure first determines the RGB response value of the camera shooting the standard color chart under a calibrated light source, and determines the standard RGB value perceived by the human eye under a target light source; then, based on the RGB response value of the camera shooting the standard color chart under a calibrated light source, and the standard RGB value perceived by the human eye under a target light source, the three-dimensional lookup table is accurately calibrated, and the calibrated three-dimensional lookup table can be applied to the color correction of the camera-captured images with higher precision, which can significantly improve the color correction accuracy of the camera-captured images.

[0038] Furthermore, as a refinement and extension of the above embodiments, a method for calibrating a three-dimensional lookup table is provided, such as... Figure 2 As shown, the method includes the following steps: Step 201a: Obtain the camera response curve of the camera, the reflectance of the standard color chart, and the spectral data of the calibration light source.

[0039] As an optional method for obtaining camera response curves, the camera module's (such as a photosensitive element) response intensity to light can be measured wavelength by wavelength using a monochromator to obtain the camera response curve of the camera module. The camera response curve can be the relationship curve between the response value output by the camera under different exposures and the actual response value.

[0040] As an option, the reflectance of each color patch on a standard color chart can be measured using a reflectance spectroscopy measuring device, which can be a specialized instrument used to measure the reflectance characteristics of an object's surface.

[0041] Step 202a: Based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera when shooting the standard color chart under the calibration light source is obtained by integration.

[0042] In some embodiments, based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera captured by the standard color chart under the calibration light source is obtained by integral calculation using colorimetric principles, as shown in the following formula:

[0043] In the formula, Can represent Channel response value, Can represent Channel response value, Can represent Channel response value, It can represent the spectrum of a light source. It can represent reflectivity. , , It can represent the camera response curve. It can represent the normalization coefficient.

[0044] Step 203a: Perform white balance and brightness alignment preprocessing on the RGB response values ​​in sequence to obtain the preprocessed RGB response values.

[0045] White balance can be used to correct color deviations caused by the camera when shooting under different light sources, so that white objects in the photo still look white, unaffected by ambient light.

[0046] As an optional approach, white balance preprocessing of the RGB response values ​​can be performed, which may include: first, selecting a grayscale color block corresponding to a standard color chart as a reference color block; then determining the first average brightness value of the R / G channel and the second average brightness value of the B / G channel in the camera response of the reference color block; then performing white balance gain analysis based on the first and second average brightness values ​​to obtain the white balance gain coefficient; and then using the white balance gain coefficient to perform white balance compensation on the RGB response values ​​to obtain the compensated RGB response values.

[0047] The formula for calculating the white balance gain coefficient is as follows:

[0048]

[0049]

[0050]

[0051] In the formula, This can represent the average brightness of the first R / G channel. This can represent the average second brightness value of the B / G channels. , It can represent the white balance gain coefficient. It can represent the maximum gain value.

[0052] The white balance compensation of RGB response values ​​using the white balance gain coefficient can be expressed by the following formula:

[0053] In the formula, Can represent Channel response value, Can represent Channel response value, Can represent Channel response value, It can represent the result after white balance compensation. Channel response value, It can represent the result after white balance compensation. Channel response value, It can represent the result after white balance compensation. Channel response value.

[0054] In the image processing pipeline of a camera, a gamma function is often applied after the color correction module. The gamma function can be used to convert linear data (i.e., the raw brightness data captured by the camera sensor, which is proportional to the light intensity) into non-linear data to better match the perceptual characteristics of the human visual system.

[0055] When an image is sent to a display device, the display device can apply an inverse gamma transform (i.e., the inverse function of the gamma function) to convert the nonlinear data back into linear data so that the image can be displayed correctly.

[0056] If the gamma transform at the camera end and the inverse gamma transform at the display end are inversely related, the image colors will not change. However, in practical applications, the gamma function in a camera pipeline often does not use the standard gamma curve form, but rather makes certain adjustments to brightness and contrast to optimize image quality.

[0057] Therefore, the actual process must take color calibration into account in advance. Applying the camera's response to a non-standard gamma curve, and then applying it to the standard inverse gamma curve (EOTF), is the OOTF (Optical-to-Face Transfer Function) process.

[0058] Because a non-standard gamma curve is used, the brightness alignment during the preprocessing stage needs to be adjusted in the OOTF process to ensure that the brightness and contrast of the image can still be correctly displayed on the display device after applying the non-standard gamma curve and the standard inverse gamma curve.

[0059] As an optional approach, preprocessing the RGB response values ​​for brightness alignment can specifically include: first, determining the third average brightness value of the compensated G channel response value after the combined action of a non-standard gamma curve and a standard inverse gamma curve, and determining the fourth average brightness value of the G channel in the standard RGB value of the corresponding color block; then, determining the ratio of the fourth average brightness value to the third average brightness value as the brightness correction coefficient; and then using the brightness correction coefficient to perform brightness correction on the compensated RGB response values ​​so that the compensated RGB response values ​​are aligned with the standard RGB values.

[0060] After calculating the white balance effect of the reference color block The average value after applying OOTF, and the average value of the corresponding color patch standard value G. The brightness correction coefficient is shown below:

[0061] In the formula, It can represent the brightness correction coefficient. Standards that can represent corresponding color blocks The fourth average brightness value of the channel response. It can represent the result after white balance compensation. The channel response value is the third average brightness value after being processed by a combination of a non-standard gamma curve and a standard inverse gamma curve.

[0062] The brightness of the compensated RGB response value is corrected using a brightness correction factor, as shown below:

[0063] In the formula, It can represent the result after white balance compensation. Channel response value, It can represent the result after white balance compensation. Channel response value, It can represent the result after white balance compensation. Channel response value, It can represent the brightness correction coefficient. It can represent the brightness correction Channel response value, It can represent the brightness correction Channel response value, It can represent the brightness correction Channel response value.

[0064] Step 201b, which is parallel to step 201a, involves acquiring the human eye response curve and the spectral data of the target light source.

[0065] Among them, the human eye response curve can be the curve of human eye's perception of light of different wavelengths.

[0066] Step 202b: Based on the human eye response curve, the reflectance of the standard color chart, and the spectral data of the target light source, obtain the CIE XYZ tristimulus values ​​through integration.

[0067] In some embodiments, based on the human eye response curve, the reflectance of a standard color chart, and the spectral data of the target light source, the CIE XYZ tristimulus values ​​can be obtained through integral calculations based on colorimetric principles, as shown in the following formula:

[0068] In the formula, , , It can represent the CIE XYZ tristimulus value. It can represent the normalization coefficient. It can represent the spectrum of a light source. It can represent reflectivity. It can represent the human eye response curve, and is generally approximated by numerical integration.

[0069] Step 203b: Convert the CIE XYZ tristimulus values ​​to values ​​in the standard RGB space, which will be used as the standard RGB values ​​perceived by the human eye under the target light source.

[0070] For example, a conversion matrix from CIE XYZ tristimulus values ​​to standard RGB values ​​can be used to convert the CIE XYZ tristimulus values ​​to values ​​in the standard RGB space, which can then be used as the standard RGB values ​​perceived by the human eye under the target light source.

[0071] After executing the above steps, the standard RGB values ​​perceived by the human eye under the target light source and the RGB response values ​​of the camera under the calibrated light source can be obtained. After the RGB response values ​​are preprocessed for white balance and brightness alignment, the three-dimensional lookup table can be calibrated based on the standard RGB values ​​and the preprocessed RGB response values. Specifically, the process shown in step 204 can be executed.

[0072] Step 204: Based on the standard RGB values ​​and the preprocessed RGB response values, calibrate the three-dimensional lookup table using an objective function.

[0073] The embodiments of this disclosure can construct the objective function based on optimization methods, such as using grid regression combined with regularization terms as the optimization objective function. For any color, color correction is achieved using three-dimensional lookup table interpolation. Here, the interpolation can be trilinear interpolation or tetrahedral interpolation, etc., and the embodiments of this disclosure are not limited thereto.

[0074] The objective function may include constraints on color reproduction accuracy, color transition smoothness, and white point visual stability. Specifically, the color reproduction accuracy constraint can be used to minimize the error between the output value obtained by interpolating the preprocessed RGB response value using a 3D lookup table and the standard RGB value; the color transition smoothness constraint can be used to constrain the second derivative relationship between grid points in the 3D lookup table to ensure smooth color transitions between grid points; and the white point visual stability constraint can be used to ensure that the white point visual representation remains unchanged after correction.

[0075] In some embodiments, step 204 may specifically include: substituting the preprocessed RGB response value and the standard RGB value into the objective function, obtaining parameter information for updating the grid point values ​​in the three-dimensional lookup table by calculating the minimum value of the objective function; and then using the parameter information to update the grid point values ​​in the three-dimensional lookup table to calibrate the three-dimensional lookup table, so as to better leverage the advantages of the three-dimensional lookup table and avoid some color reproduction defects that often occur due to the large degree of freedom of the three-dimensional lookup table, such as uneven transitions.

[0076] For embodiments of this disclosure, an example describing grid regression in a two-dimensional case is provided, such as... Figure 3 As shown, for any input point Interpolation weights between its adjacent grid points The output result is Then grid point value It is the result that makes the following first function expression true.

[0077]

[0078] In the formula, It can represent grid point values. It can represent any result of the grid point value. It can represent a 1-dimensional real number px. The number of data points can be represented by m, where m represents the number of grid points. It can represent the result of interpolation operations. This can represent the value corresponding to the i-th input point. This can represent the interpolation weights between the grid points corresponding to the i-th input point. It can represent the j-th grid point.

[0079] The three-dimensional case is similar to the two-dimensional case, except for the interpolation method. To ensure smoothness between grid points, certain constraints must be added. Here, a graph Hessian regularization term is introduced, which is to solve for the sum of squares of the second derivatives of the three dimensions of the grid points. The specific solution process is shown in the second function expression below:

[0080] In the formula, It can represent dimensions; for RGB, d=3. It can represent the positions of adjacent grid points. , , Indicates the corresponding grid point value The graph can be represented by a Hessian matrix. It can represent a matrix composed of grid points, with each column representing a grid point data.

[0081]

[0082] In the formula, It can represent the Hessian matrix of the graph after regularization. It can represent the identity matrix.

[0083]

[0084] In the formula: , , Each column of B represents a grid point, each column of W represents the interpolation weights of m grid points, and each column of Y represents a data value.

[0085] Characterization can include two parts: white balance and color correction. Color correction generally aims to minimize the impact on the white point, thus keeping the white point output unchanged as another regularization term. The white point constraint is similar to the ordinary input, except that the input equals the output. The specific solution process is shown in the third function expression below:

[0086] In the formula, It can represent the coefficient matrix corresponding to the white dots. It can represent a matrix composed of white point output values. , It can represent the regularization coefficient. It can represent the number of data points corresponding to the white dots.

[0087] A complete example flow for the embodiments of this disclosure can be found as follows: Figure 4 As shown, the camera can photograph a standard color chart under a calibrated light source and determine the camera's RGB response value based on the camera's response curve and the reflectance of each color patch on the standard color chart. The standard RGB value can be determined based on the human eye response curve, the reflectance of each color patch on the standard color chart, and the spectrum of the target light source. The RGB response values ​​can then be preprocessed, including white balance processing and brightness alignment processing. White balance can correct color deviations caused by the camera shooting under different light sources, ensuring that white objects in the photo still appear white. Brightness alignment processing aligns the camera's RGB values ​​with the standard RGB values. Then, based on the preprocessed camera RGB values ​​and the standard RGB values, a three-dimensional lookup table is calibrated using an objective function. The three-dimensional lookup table can be used to interpolate the preprocessed camera RGB response values ​​to obtain the interpolation result, which is then subjected to OOTF (i.e., joint processing of non-standard gamma curves and standard inverse gamma curves).

[0088] This disclosure provides a complete three-dimensional lookup table calibration method to better leverage the advantages of three-dimensional lookup tables and avoid color reproduction defects, such as uneven transitions, that often occur due to the high degree of freedom of three-dimensional lookup tables. Subsequent color correction can be performed based on the three-dimensional lookup table with higher accuracy.

[0089] Based on the above Figures 1 to 4 In addition to the specific implementation of the method shown, this disclosure also provides an image processing apparatus, such as... Figure 5 As shown, the device includes: a determination module 31 and a calibration module 32.

[0090] The determination module 31 is used to determine the RGB response value of the camera when shooting the standard color card under the calibrated light source, and to determine the standard RGB value of the standard color card perceived by the human eye under the target light source. The calibration module 32 is used to calibrate the three-dimensional lookup table based on the RGB response value and the standard RGB value, wherein the calibrated three-dimensional lookup table is used for color correction of the image captured by the camera.

[0091] In some embodiments of this disclosure, the determining module 31 is used to acquire the camera response curve of the camera, the reflectance of the standard color chart, and the spectral data of the calibration light source; based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera shooting the standard color chart under the calibration light source is obtained by integration.

[0092] In some embodiments of this disclosure, the determining module 31 is used to acquire the human eye response curve and the spectral data of the target light source; based on the human eye response curve, the reflectance of the standard color card and the spectral data of the target light source, the CIE XYZ tristimulus value is obtained by integration; the CIE XYZ tristimulus value is converted to a value in the standard RGB space as the standard RGB value perceived by the human eye on the standard color card under the target light source.

[0093] In some embodiments of this disclosure, the determining module 31 is further configured to perform white balance and brightness alignment preprocessing on the RGB response values ​​sequentially to obtain preprocessed RGB response values; The calibration module 32 is used to calibrate the three-dimensional lookup table based on the standard RGB values ​​and the preprocessed RGB response values.

[0094] In some embodiments of this disclosure, the determining module 31 is used to select a grayscale color block corresponding to the standard color card as a reference color block; determine the first average brightness value of the R / G channel and the second average brightness value of the B / G channel in the camera response of the reference color block; perform white balance gain analysis based on the first average brightness value and the second average brightness value to obtain a white balance gain coefficient; and use the white balance gain coefficient to perform white balance compensation on the RGB response value to obtain a compensated RGB response value.

[0095] In some embodiments of this disclosure, the determining module 31 is used to determine the third average brightness value of the compensated G channel response value after the combined action of a non-standard gamma curve and a standard inverse gamma curve, and to determine the fourth average brightness value of the G channel in the standard RGB value of the corresponding color block; the ratio of the fourth average brightness value to the third average brightness value is determined as a brightness correction coefficient; the brightness correction coefficient is used to perform brightness correction on the compensated RGB response value so that the compensated RGB response value is aligned with the standard RGB value.

[0096] In some embodiments of this disclosure, calibration module 32 is used to obtain an objective function, which includes constraints on the accuracy of color reproduction, the smoothness of color transition, and the stability of white point vision; based on the preprocessed RGB response value and the standard RGB value, the objective function is used to calibrate the three-dimensional lookup table.

[0097] In some embodiments of this disclosure, the calibration module 32 is used to substitute the preprocessed RGB response value and the standard RGB value into the objective function, calculate the minimum value of the objective function, obtain parameter information for updating the grid point values ​​in the three-dimensional lookup table, and use the parameter information to update the grid point values ​​in the three-dimensional lookup table to achieve the calibration of the three-dimensional lookup table.

[0098] In some embodiments of this disclosure, the accuracy constraint of color restoration is used to limit the error between the output value obtained by interpolation of the preprocessed RGB response value through the three-dimensional lookup table and the standard RGB value to be minimal; the smoothness constraint of color transition is used to limit the second derivative relationship between grid points in the three-dimensional lookup table to ensure that the color transition between grid points in the three-dimensional lookup table is smooth; the stability constraint of white point vision is used to limit the white point visual presentation to remain unchanged after correction.

[0099] It should be noted that other corresponding descriptions of the functional units involved in the image processing apparatus provided in this disclosure embodiment can be found by referring to... Figures 1 to 4 The corresponding descriptions of the Chinese methods will not be repeated here.

[0100] Based on the above, Figures 1 to 4 Accordingly, this disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. Figures 1 to 4 The method shown.

[0101] Based on the above, Figures 1 to 4 Accordingly, this disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method. Figures 1 to 4 The method shown.

[0102] Based on this understanding, the technical solution disclosed herein can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive) and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods of various implementation scenarios of this disclosure.

[0103] Based on the above, Figures 1 to 4 The method shown, and Figure 5The virtual device embodiment shown in this disclosure also provides a chip, including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, it causes the electronic device to perform the above-described... Figures 1 to 4 The method shown.

[0104] Figure 6 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0105] like Figure 6 As shown, device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in ROM (Read-Only Memory) 1002 or loaded from storage unit 1008 into RAM (Random Access Memory) 1003. RAM 1003 may also store various programs and data required for the operation of device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. I / O (Input / Output) interface 1005 is also connected to bus 1004.

[0106] Multiple components in device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of monitors, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0107] The computing unit 1001 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as the methods mentioned earlier. For example, in some embodiments, the methods mentioned earlier can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1008. In some embodiments, part or all of the computer program can be loaded and / or installed on device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by the computing unit 1001, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to perform the aforementioned method by any other suitable means (e.g., by means of firmware).

[0108] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0109] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0112] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0113] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0114] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0115] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0116] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, characterized in that, include: Determine the RGB response value of the camera when shooting the standard color chart under a calibrated light source, and determine the standard RGB value of the standard color chart perceived by the human eye under a target light source; Based on the RGB response value and the standard RGB value, a three-dimensional lookup table is calibrated, wherein the calibrated three-dimensional lookup table is used for color correction of the image captured by the camera.

2. The method according to claim 1, characterized in that, Determining the RGB response values ​​of the camera when shooting a standard color chart under a calibrated light source includes: Acquire the camera response curve of the camera, the reflectance of the standard color chart, and the spectral data of the calibration light source; Based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera when shooting the standard color chart under the calibration light source is obtained by integration.

3. The method according to claim 2, characterized in that, Determining the standard RGB values ​​perceived by the human eye under the target light source for the standard color chart includes: Acquire the human eye response curve and the spectral data of the target light source; Based on the human eye response curve, the reflectance of the standard color chart, and the spectral data of the target light source, the CIE XYZ tristimulus values ​​are obtained by integration; The CIE XYZ tristimulus values ​​are converted to values ​​in the standard RGB space and used as the standard RGB values ​​perceived by the human eye under the target light source.

4. The method according to claim 1, characterized in that, Before calibrating the three-dimensional lookup table based on the RGB response value and the standard RGB value, the method further includes: The RGB response values ​​are preprocessed sequentially with white balance and brightness alignment to obtain preprocessed RGB response values; The calibration of the three-dimensional lookup table based on the RGB response value and the standard RGB value includes: The three-dimensional lookup table is calibrated based on the standard RGB values ​​and the preprocessed RGB response values.

5. The method according to claim 4, characterized in that, Preprocessing the RGB response values ​​for white balance includes: Select the grayscale color block corresponding to the standard color card as the reference color block; Determine the first average brightness value of the R / G channel and the second average brightness value of the B / G channel in the camera response of the reference color patch; White balance gain analysis is performed based on the first average brightness value and the second average brightness value to obtain the white balance gain coefficient; The RGB response value is compensated for using the white balance gain coefficient to obtain the compensated RGB response value.

6. The method according to claim 5, characterized in that, Preprocessing for brightness alignment of the RGB response values ​​includes: The third average brightness value of the compensated G channel response value after the combined effect of the non-standard gamma curve and the standard inverse gamma curve is determined, and the fourth average brightness value of the G channel in the standard RGB value of the corresponding color block is determined. The ratio of the fourth average brightness value to the third average brightness value is determined as the brightness correction coefficient; The brightness correction coefficient is used to perform brightness correction on the compensated RGB response value so that the compensated RGB response value is aligned with the standard RGB value.

7. The method according to any one of claims 4 to 6, characterized in that, The calibration of the three-dimensional lookup table based on the standard RGB values ​​and the preprocessed RGB response values ​​includes: Obtain the objective function, which includes constraints on the accuracy of color reproduction, the smoothness of color transition, and the stability of white point visual perception. Based on the preprocessed RGB response values ​​and the standard RGB values, the objective function is used to calibrate the three-dimensional lookup table.

8. The method according to claim 7, characterized in that, The step of calibrating the three-dimensional lookup table using the objective function based on the preprocessed RGB response values ​​and the standard RGB values ​​includes: Substitute the preprocessed RGB response value and the standard RGB value into the objective function, and obtain the parameter information for updating the grid point values ​​in the three-dimensional lookup table by calculating the minimum value of the objective function; The grid values ​​in the three-dimensional lookup table are updated using the parameter information to calibrate the three-dimensional lookup table.

9. The method according to claim 7, characterized in that, The accuracy constraint of color reproduction is used to limit the error between the output value obtained by interpolation of the preprocessed RGB response value through the three-dimensional lookup table and the standard RGB value to be minimal; The color transition smoothness constraint is used to limit the second derivative relationship between grid points in the three-dimensional lookup table to ensure that the color transition between grid points in the three-dimensional lookup table is smooth. The stability constraint of the white point vision is used to ensure that the visual representation of the white point remains unchanged after correction.

10. An image processing apparatus, characterized in that, include: The determination module is used to determine the RGB response value of the camera to the standard color chart under the calibrated light source, and to determine the standard RGB value perceived by the human eye to the standard color chart under the target light source. The calibration module is used to calibrate the three-dimensional lookup table based on the RGB response value and the standard RGB value, wherein the calibrated three-dimensional lookup table is used for color correction of the image captured by the camera.

11. The apparatus according to claim 10, characterized in that, The determining module is used to acquire the camera response curve of the camera, the reflectance of the standard color chart, and the spectral data of the calibration light source; based on the camera response curve, the reflectance of the standard color chart, and the spectral data of the calibration light source, the RGB response value of the camera shooting the standard color chart under the calibration light source is obtained by integration.

12. The apparatus according to claim 10, characterized in that, The determining module is further configured to perform white balance and brightness alignment preprocessing on the RGB response values ​​sequentially to obtain preprocessed RGB response values; The calibration module is used to calibrate the three-dimensional lookup table based on the standard RGB values ​​and the preprocessed RGB response values.

13. The apparatus according to claim 12, characterized in that, The calibration module is used to obtain an objective function, which includes constraints on the accuracy of color reproduction, the smoothness of color transition, and the stability of white point vision. Based on the preprocessed RGB response values ​​and the standard RGB values, the objective function is used to calibrate the three-dimensional lookup table.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 9.

15. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 9.

16. A chip, characterized in that, The device includes one or more interface circuits and one or more processors; the interface circuits are configured to receive signals from the memory of the electronic device and send the signals to the processors, the signals including computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device performs the method of any one of claims 1 to 9.

Images