Image processing method, color correction information calibration method, device and equipment

Abstract

The invention relates to an image processing method, and a color correction information calibration method, device and equipment, and belongs to the technical field of image processing. The method comprises the following steps: in response to a shooting instruction, determining each piece of actual response information of a camera module for a shooting object; determining a first target hue interval to which any piece of actual response information belongs from the candidate hue intervals; and performing color correction on the corresponding actual response information based on color correction information of the camera module for the first target hue interval under the reference light source to obtain target response information. Therefore, different color correction information can be adopted in different candidate tone intervals, color correction can be performed in different regions, the diversity of the color correction information is improved, the color correction precision is improved, for example, the overall color accuracy and naturalness are remarkably improved, the problems of color distortion and the like caused by using single color correction information can be avoided, and the color correction efficiency is improved. And the visual experience is optimized.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image processing method, a method for calibrating color correction information, an apparatus, an electronic device, a chip, and a storage medium. Background Technology

[0002] Currently, color correction is widely used in photography, film and television production, and driving, with the main purpose of making the colors displayed by electronic devices as close as possible to the colors perceived by the human eye, thereby optimizing the visual experience. It is a key step in image processing. However, the color correction methods in related technologies suffer from low color correction accuracy, resulting in a poor visual experience. Summary of the Invention

[0003] This disclosure provides an image processing method, a color correction information calibration method, an apparatus, an electronic device, a chip, and a storage medium to at least solve the problem of low color correction accuracy in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, an image processing method is provided, comprising: in response to a shooting command, determining each actual response information of a camera module for a shooting object; determining a first target tone range to which any of the actual response information belongs from each candidate tone range; and performing color correction on the corresponding actual response information based on color correction information of the camera module for the first target tone range under a reference light source to obtain target response information.

[0004] According to a second aspect of the present disclosure, a method for calibrating color correction information is provided, comprising: determining each original response information of a camera module for a reference object under a calibrated light source; determining each reference response information of the camera module for the reference object under a target light source; and determining color correction information of the camera module for each candidate hue range under the calibrated light source based on each original response information and each reference response information; wherein the color correction information of any candidate hue range is used to perform color correction on the actual response information belonging to the corresponding candidate hue range.

[0005] According to a third aspect of the present disclosure, an image processing apparatus is provided, comprising: a first determining module configured to determine, in response to a shooting command, actual response information of a camera module for a shooting object; a second determining module configured to determine, from each candidate tone range, a first target tone range to which any of the actual response information belongs; and a processing module configured to perform color correction on the corresponding actual response information based on color correction information of the camera module for the first target tone range under a reference light source, so as to obtain target response information.

[0006] According to a fourth aspect of the present disclosure, a calibration apparatus for color correction information is provided, comprising: a first determining module configured to determine original response information of a camera module for a reference object under a calibrated light source; a second determining module configured to determine reference response information of the camera module for the reference object under a target light source; and a third determining module configured to determine color correction information of the camera module for each candidate hue range under the calibrated light source based on each of the original response information and each of the reference response information; wherein the color correction information of any candidate hue range is used to perform color correction on the actual response information belonging to the corresponding candidate hue range.

[0007] According to a fifth aspect of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the image processing method described in the first aspect of the present disclosure, and / or implements the steps of the color correction information calibration method described in the first aspect of the present disclosure.

[0008] According to a sixth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the image processing method described in the first aspect of the present disclosure, and / or implement the steps of the color correction information calibration method described in the first aspect of the present disclosure.

[0009] According to a seventh aspect of the present disclosure, a chip is provided, the chip including an interface circuit and a processing circuit coupled to each other, the interface circuit being used to input or output signals, and the processing circuit being configured to implement the steps of the image processing method of the first aspect of the present disclosure, and / or to implement the steps of the calibration method for color correction information of the first aspect of the present disclosure.

[0010] According to an eighth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the image processing method described in the first aspect of the present disclosure, and / or implements the steps of the color correction information calibration method described in the first aspect of the present disclosure.

[0011] The technical solution provided by the embodiments of this disclosure brings at least the following beneficial effects: In response to a shooting command, the actual response information of the camera module to the shooting object is determined, a first target tone range to which any actual response information belongs is determined from each candidate tone range, and color correction is performed on the corresponding actual response information based on the color correction information of the camera module for the first target tone range under a reference light source to obtain the target response information. Thus, the tone can be divided into candidate tone ranges, and each candidate tone range can be set with independent color correction information. That is, different color correction information can be adopted under different candidate tone ranges, and color correction can be performed by region, improving the diversity of color correction information and helping to improve color correction accuracy. For example, it significantly improves the overall color accuracy and naturalness, avoids color distortion and other problems caused by using a single color correction information, and optimizes the visual experience.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0014] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0015] Figure 2 This is a flowchart illustrating a method for calibrating color correction information according to an exemplary embodiment.

[0016] Figure 3 This is a flowchart illustrating a method for calibrating color correction information according to another exemplary embodiment.

[0017] Figure 4 This is a flowchart illustrating a method for calibrating color correction information according to another exemplary embodiment.

[0018] Figure 5 This is a flowchart illustrating a method for calibrating color correction information according to another exemplary embodiment.

[0019] Figure 6 This is a schematic diagram illustrating a method for calibrating color correction information according to an exemplary embodiment.

[0020] Figure 7 This is a schematic flowchart of an image processing apparatus according to an exemplary embodiment.

[0021] Figure 8 This is a schematic diagram of the structure of a color correction information calibration device according to an exemplary embodiment.

[0022] Figure 9 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment.

[0023] Figure 10 This is a schematic diagram of the structure of a chip according to an exemplary embodiment. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] The following description, with reference to the accompanying drawings, describes an image processing method, a color correction information calibration method, an apparatus, an electronic device, a chip, and a storage medium according to embodiments of the present disclosure.

[0027] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment, such as... Figure 1 As shown, the image processing method of this disclosure includes the following steps.

[0028] S101, in response to the shooting command, determines the actual response information of the camera module to the subject being shot.

[0029] It should be noted that the image processing method in this embodiment is executed by an electronic device, such as a camera, terminal device, vehicle, server, chip, etc. Terminal devices include mobile phones, wearable devices (such as smartwatches, smart glasses), laptops, etc. Vehicles include vehicle terminals, vehicle controllers, etc., and chips include ISPs (Image Signal Processors), etc.

[0030] The image processing method of this disclosure embodiment can be executed by the image processing device of this disclosure embodiment. The image processing device of this disclosure embodiment can be configured in any electronic device to execute the image processing method of this disclosure embodiment.

[0031] No specific restrictions are placed on any type of response information, such as RAW information and response information in various color spaces. RAW information includes the brightness value of each pixel in a single color, and various color spaces include RGB, HSV, HSL, Lab, YCbCr, YUV, XYZ, etc. The XYZ color space is a standard color space based on the characteristics of human visual perception.

[0032] There are no strict limitations on the subjects being filmed. For example, in photography, film production, and live streaming scenarios, subjects include faces, landscapes, and buildings. In driving scenarios, subjects include the external environment of the vehicle (such as the road in front of the vehicle) and the interior of the vehicle (such as the driver's area). In smart home scenarios, subjects include the interior of the residence (such as the living room) and the exterior of the residence (such as the area outside the door). In remote robot control scenarios, subjects include the environment surrounding the robot.

[0033] There are no strict limitations on shooting commands. For example, a shooting command can be triggered based on an operation on at least one of the camera, terminal device, or vehicle. There are also no strict limitations on the operations used to trigger shooting commands, which may include mechanical operations, touch operations, voice interaction operations, etc.

[0034] For example, the method also includes generating a shooting command in response to a click on the shooting button on the application page of the camera application.

[0035] For example, the method also includes responding to voice interaction operations, collecting voice information, and performing voice recognition on the voice information to generate shooting instructions.

[0036] S102, determine the first target tone range to which any actual response information belongs from each candidate tone range.

[0037] S103, based on the color correction information of the camera module for the first target tone range under the reference light source, performs color correction on the corresponding actual response information to obtain the target response information.

[0038] Currently, color correction is widely used in photography, film and television production, and driving, with the main purpose of making the colors displayed by electronic devices as close as possible to the colors perceived by the human eye, thereby optimizing the visual experience. It is a key step in image processing. However, the color correction methods in related technologies suffer from low color correction accuracy, resulting in a poor visual experience.

[0039] In this disclosure, the hue can be divided into candidate hue ranges, and each candidate hue range is set with independent color correction information. That is, different color correction information can be used under different candidate hue ranges, and color correction can be performed in different areas, which improves the diversity of color correction information and helps to improve color correction accuracy. For example, it can significantly improve the overall color accuracy and naturalness, avoid color distortion and other problems caused by using a single color correction information, and optimize the visual experience.

[0040] It should be noted that there are no excessive restrictions on color correction information, such as including CCM (Color Correction Matrix), LUT (Look-Up Table), polynomial mapping function, color space transformation matrix, adaptive correction parameters, and model parameters of the color correction model.

[0041] There are no restrictions on the method or number of candidate color ranges. The length of each candidate color range can be the same or different. That is, each candidate color range can be divided evenly or unevenly.

[0042] For example, the H (Hue) component in the HSV color space, ranging from [0° to 360°), can be evenly divided into 24 intervals to obtain 24 candidate hue intervals. In this embodiment, the interval length of each candidate hue interval is 15°. For example, the value range of the first candidate hue interval is [0°, 15°), the value range of the second candidate hue interval is [15°, 30°), the value range of the third candidate hue interval is [30°, 45°), and so on, with the value range of the 24th candidate hue interval being [345°, 360°].

[0043] Determining the first target color interval to which any actual response information belongs from each candidate color interval includes converting any actual response information to the HSV color space to determine the H component in the HSV information, and determining the candidate color interval to which the H component belongs from each candidate color interval as the first target color interval.

[0044] For example, if the actual response information includes actual RGB information, any actual RGB information can be converted from the RGB color space to the HSV color space to determine the H component in the HSV information.

[0045] In some possible implementations, the method further includes determining a calibration light source from among the calibration light sources that matches the actual light source during camera module shooting, as a reference light source. Thus, each calibration light source is assigned an independent set of color correction information, and each set of color correction information includes color correction information for each candidate hue range, allowing for color correction by region and by light source, further improving the diversity and accuracy of color correction information.

[0046] It should be noted that the calibration light source is not subject to many limitations, and includes natural light sources (such as the sun and moon), lighting fixtures (such as incandescent lamps and fluorescent lamps), and optoelectronic devices (such as photodiodes and optical fibers). The calibration process of the camera module for the color correction information of the first target tone range under various calibration light sources can be found in the following embodiments, and will not be repeated here.

[0047] For example, the calibration light sources include the sun, the moon, incandescent lamps, and fluorescent lamps. The color correction matrix of the camera module under the sun for 24 candidate tone ranges can be pre-calibrated, the color correction matrix of the camera module under the moon for 24 candidate tone ranges can be pre-calibrated, the color correction matrix of the camera module under incandescent lamps for 24 candidate tone ranges can be pre-calibrated, and the color correction matrix of the camera module under fluorescent lamps for 24 candidate tone ranges can be pre-calibrated.

[0048] In response to the shooting command, the camera module determines the actual RGB information of the subject, selects the first target RGB range from 24 candidate RGB ranges, and selects a calibration light source that matches the actual light source of the camera module at the time of shooting from the sun, moon, incandescent lamp and fluorescent lamp as a reference light source.

[0049] Taking the sun as the reference light source and the first target hue range to which any actual RGB information belongs as the first candidate hue range as an example, the actual RGB information can be color-corrected based on the color correction information of the camera module for the first candidate hue range under the sun, so as to obtain the target response information.

[0050] In some possible implementations, the method further includes at least one of the following operations: Operation 1: Generate target images based on the response information of each target, and visualize the target images.

[0051] Operation 2: Send the target image to the terminal device; the target image is used by the terminal device for visualization display.

[0052] In some possible implementations, a target image is generated based on the target response information, including processing at least one of white balance correction, brightness alignment, gamma correction, and inverse gamma correction on any target response information to generate the target image.

[0053] Therefore, target images can be generated based on the response information of each target, and the target images can be visualized to show the processed images to the user.

[0054] And / or, send the target image to the terminal device to display the processed image on the terminal device.

[0055] It should be noted that this disclosure does not impose any restrictions on the execution sequence of steps S101-S103. Figure 1 The example only demonstrates the execution of steps S101-S103 in sequence.

[0056] The image processing method provided in the embodiments of this disclosure, in response to a shooting command, determines the actual response information of the camera module to the subject, determines a first target tone range to which any actual response information belongs from each candidate tone range, and performs color correction on the corresponding actual response information based on the color correction information of the camera module for the first target tone range under a reference light source to obtain the target response information. Thus, the tone can be divided into candidate tone ranges, and each candidate tone range can be set with independent color correction information. That is, different color correction information can be used under different candidate tone ranges, and color correction can be performed by region, improving the diversity of color correction information and helping to improve color correction accuracy. For example, it significantly improves the overall color accuracy and naturalness, avoids color distortion caused by using a single color correction information, and optimizes the visual experience.

[0057] Figure 2 This is a flowchart illustrating a method for calibrating color correction information according to an exemplary embodiment, such as... Figure 2 As shown, the color correction information calibration method of this disclosure includes the following steps.

[0058] S201, determine the original response information of the camera module to the reference object under the calibrated light source.

[0059] It should be noted that the subject executing the color correction information calibration method in this embodiment is an electronic device, such as a terminal device, vehicle, camera, camera testing platform, server, chip, etc. The terminal device includes mobile phone, wearable device (such as smartwatch, smart glasses), laptop, etc. The vehicle includes vehicle terminal, vehicle controller, etc. The chip includes ISP (Image Signal Processor), etc.

[0060] The color correction information calibration method of this disclosure embodiment can be executed by the color correction information calibration device of this disclosure embodiment. The color correction information calibration device of this disclosure embodiment can be configured in any electronic device to execute the color correction information calibration method of this disclosure embodiment.

[0061] The reference object is not subject to many restrictions; it may include standard color charts, etc. Standard color charts can be Munsell color charts, Pantone color charts, etc.

[0062] In some possible implementations, the original response information of the camera module to the reference object under a calibrated light source is determined, including based on the spectral sensitivity function of the camera module, the reflectivity of the reference object, and the spectrum of the calibrated light source. This allows for the comprehensive determination of the original response information by considering the spectral sensitivity function of the camera module, the reflectivity of the reference object, and the spectrum of the calibrated light source, eliminating the need for the camera module to photograph the reference object under the calibrated light source, making it simple and easy to implement. Furthermore, color correction information can be calibrated before the camera is packaged or tested, and calibration conditions such as the calibrated light source and reflectivity can be freely changed.

[0063] For example, taking a standard color chart as a reference, the original RGB information of the camera module for any color patch in the standard color chart under a calibrated light source can be determined using the following formula:

[0064] Where R is the R component (i.e., the pixel value in the red channel) of the original RGB information of the corresponding color block under the calibrated light source, Gr is the Gr component (i.e., the pixel value in the Gr channel) of the original RGB information of the corresponding color block under the calibrated light source, Gb is the Gb component (i.e., the pixel value in the Gb channel) of the original RGB information of the corresponding color block under the calibrated light source, and B is the B component (i.e., the pixel value in the blue channel) of the original RGB information of the corresponding color block under the calibrated light source. G = (Gr + Gb) / 2, where G is the G component (i.e., the pixel value in the green channel) of the original RGB information of the corresponding color block under the calibrated light source.

[0065] To calibrate the spectrum of the light source, This represents the reflectance (also called the spectral reflectance function) of the corresponding color patch. This is a function of the camera module's spectral sensitivity in the red channel. This is the spectral sensitivity function of the camera module in the first green channel. This is the spectral sensitivity function of the camera module in the second green channel. This represents the spectral sensitivity function of the camera module in the blue channel. The spectral range of any type of spectral sensitivity function is from 380 nm to 780 nm.

[0066] S202, determine the reference response information of the camera module for the reference object under the target light source.

[0067] It should be noted that there are no strict restrictions on the target light source, such as including the D65 light source, which is a standard daylight source.

[0068] In some possible implementations, the camera module determines various reference response information for a reference object under the target light source, including based on the standard observer function, the reflectivity of the reference object, and the spectrum of the target light source. This allows for a comprehensive consideration of the standard observer function, the reflectivity of the reference object, and the spectrum of the target light source to determine the reference response information, making the process simple and easy.

[0069] For example, taking a standard color chart as a reference object, the reference response information of the camera module for any color patch in the standard color chart under the target light source can be determined by the following formula:

[0070] Wherein, X is the X component (i.e., the stimulus value of the virtual red primary color, used to quantify the color composition in the red-green direction) in the reference XYZ information of the camera module for the corresponding color block under the target light source, Y is the Y component (i.e., the brightness value) in the reference XYZ information of the camera module for the corresponding color block under the target light source, and Z is the Z component (i.e., the stimulus value of the virtual blue primary color, used to quantify the color composition in the blue-violet direction) in the reference XYZ information of the camera module for the corresponding color block under the target light source.

[0071] The spectrum of the target light source, The reflectance of the corresponding color block. For the standard observer function in the X channel, For the standard observer function in the Y channel, This represents the standard observer function in the Z channel. The spectral range of any type of standard observer function is from 380 nm to 780 nm.

[0072] The reference XYZ information can be converted from the XYZ color space to the RGB color space to determine the reference RGB information of the camera module for the corresponding color block under the target light source.

[0073] S203, based on each original response information and each reference response information, determine the color correction information of the camera module for each candidate tone range under the calibrated light source; wherein, the color correction information of any candidate tone range is used to perform color correction on the actual response information belonging to the corresponding candidate tone range.

[0074] In this disclosure, considering both the original response information and the reference response information, the color correction information of the camera module for each candidate hue range under the calibrated light source can be calibrated. This allows the hue to be divided into candidate hue ranges, and each candidate hue range can be set with independent color correction information. In other words, different color correction information can be calibrated for different candidate hue ranges, thereby enabling color correction by region. This increases the diversity of color correction information and helps improve color correction accuracy, such as significantly improving overall color accuracy and naturalness. It can also avoid color distortion caused by using a single color correction information, thus optimizing the visual experience.

[0075] In some possible implementations, based on each original response information and each reference response information, the color correction information of the camera module for each candidate tone range under the calibrated light source is determined, including determining the color correction information for each candidate tone range based on the difference information between each original response information and each reference response information.

[0076] In some possible implementations, color correction information for each candidate tone range is determined based on the difference information between each original response information and each reference response information. This includes determining the color correction information for the candidate tone range to which the corresponding original response information belongs based on the difference information between any original response information and the reference response information associated with the corresponding original response information.

[0077] It is understandable that the region of the reference object corresponding to any original response information is consistent with the region of the reference object corresponding to the reference response information associated with the original response information. For example, taking a standard color chart as an example, the original response information corresponding to any color block is associated with the reference response information corresponding to the corresponding color block. For instance, the original response information corresponding to color block i is associated with the reference response information corresponding to color block i.

[0078] The details of steps S201-S203 can be found in the above embodiments and will not be repeated here.

[0079] It should be noted that this disclosure does not impose any restrictions on the execution sequence of steps S201-S203. Figure 2 The example only demonstrates the execution of steps S201-S203 in sequence.

[0080] The color correction information calibration method provided in this disclosure determines the original response information of the camera module to a reference object under a calibrated light source, determines the reference response information of the camera module to the reference object under a target light source, and determines the color correction information of the camera module for each candidate hue range under the calibrated light source based on the original response information and the reference response information. The color correction information of any candidate hue range is used to perform color correction on the actual response information belonging to the corresponding candidate hue range. Therefore, by considering the original response information and the reference response information, the color correction information of the camera module for each candidate hue range under the calibrated light source can be calibrated, thus dividing the hue into candidate hue ranges. Each candidate hue range can be set with independent color correction information, meaning different color correction information can be calibrated for different candidate hue ranges. This allows for regional color correction, increasing the diversity of color correction information and improving color correction accuracy, such as significantly improving overall color accuracy and naturalness. It avoids color distortion caused by using a single color correction information, thus optimizing the visual experience.

[0081] Figure 3 This is a flowchart illustrating a method for calibrating color correction information according to another exemplary embodiment, such as... Figure 3 As shown, the color correction information calibration method of this disclosure includes the following steps.

[0082] S301, determine the original response information of the camera module to the reference object under the calibrated light source.

[0083] S302, determine the reference response information of the camera module for the reference object under the target light source.

[0084] The details of steps S301-S302 can be found in the above embodiments and will not be repeated here.

[0085] S303, based on each original response information and each reference response information, determine the conversion relationship between the original response information and the reference response information in each candidate tone range.

[0086] S304, based on the conversion relationship, determines the color correction information for each candidate hue range.

[0087] In this embodiment, the conversion relationship between the original response information and the reference response information in each candidate tone range can be determined by taking into account each original response information and each reference response information, so as to calibrate the color correction information of each candidate tone range.

[0088] It should be noted that there are no strict restrictions on the transformation relationship. For example, the transformation relationship can be represented by functions, matrices, lookup tables, etc.

[0089] In some possible implementations, based on each original response information and each reference response information, the conversion relationship between the original response information and the reference response information in each candidate tone range is determined, including determining the conversion relationship of the candidate tone range to which the corresponding original response information belongs based on any original response information and the reference response information associated with the corresponding original response information.

[0090] In some possible implementations, color correction information for each candidate hue range is determined based on the conversion relationship, including converting the conversion relationship into color correction information for each candidate hue range.

[0091] In some possible implementations, color correction information is represented in the form of a color correction matrix. Based on each original response information and each reference response information, the conversion relationship between the original and reference response information in each candidate hue interval is determined, including determining an objective function based on each original and reference response information; wherein the objective function is used to indicate the conversion relationship, and the independent variables of the objective function include the color correction matrix for each candidate hue interval.

[0092] Based on the transformation relationship, the color correction information of each candidate tone range is determined, including solving the independent variables of the objective function to determine the color correction matrix of each candidate tone range.

[0093] Therefore, taking into account the original response information and the reference response information, the objective function can be determined to indicate the transformation relationship, and the independent variables of the objective function can be solved to calibrate the color correction matrix of each candidate hue range.

[0094] In some possible implementations, the objective function is solved for independent variables to determine the color correction matrix for each candidate hue range. This includes optimizing the objective function to obtain the color correction matrix when the objective function is minimized, which is then used as the final color correction matrix.

[0095] It should be noted that this disclosure does not impose any restrictions on the execution sequence of steps S301-S304. Figure 3 The example only demonstrates the execution of steps S301-S304 in sequence.

[0096] The color correction information calibration method provided in the embodiments of this disclosure determines the conversion relationship between the original response information and the reference response information in each candidate hue interval based on each original response information and each reference response information, and determines the color correction information for each candidate hue interval based on the conversion relationship. Therefore, considering each original response information and each reference response information, the conversion relationship between the original response information and the reference response information in each candidate hue interval can be determined to calibrate the color correction information for each candidate hue interval.

[0097] Figure 4This is a flowchart illustrating a method for calibrating color correction information according to another exemplary embodiment, such as... Figure 4 As shown, the color correction information calibration method of this disclosure includes the following steps.

[0098] S401, determine the original response information of the camera module to the reference object under the calibrated light source.

[0099] S402, determine the reference response information of the camera module for the reference object under the target light source.

[0100] The details of steps S401-S402 can be found in the above embodiments and will not be repeated here.

[0101] S403, perform image processing on any raw response information to obtain processed response information; wherein, the image processing includes at least color correction.

[0102] In some possible implementations, image processing also includes at least one of white balance correction, brightness alignment, gamma correction, and inverse gamma correction.

[0103] In some possible implementations, image processing is performed on any original response information to obtain processed response information. This includes performing white balance correction on any original response information to obtain first response information, performing brightness correction on the first response information to obtain second response information, determining a second target tone range to which the second response information belongs from each candidate tone range, performing color correction on the second response information based on the color correction matrix of the second target tone range to obtain third response information, and performing gamma correction and inverse gamma correction on the third response information to obtain processed response information. Thus, white balance correction and brightness correction can be performed on any original response information to obtain second response information, and color correction can be performed on the second response information based on the color correction matrix of the second target tone range to which the second response information belongs to to obtain third response information. Gamma correction and inverse gamma correction can then be performed on the third response information to obtain processed response information. In other words, white balance correction, brightness correction, color correction, gamma correction, and inverse gamma correction are performed on any original response information to obtain processed response information.

[0104] In some possible implementations, white balance correction is performed on any original response information to obtain first response information. This includes, when the reference object includes a standard color chart, using grayscale patches in the standard color chart as reference patches, determining white balance correction information based on the original response information corresponding to the reference patches, and performing white balance correction on any original response information based on the white balance correction information to obtain the first response information. Thus, grayscale patches in the standard color chart can be used as reference patches, and white balance correction information can be determined by considering the original response information corresponding to the reference patches, to perform white balance correction on any original response information.

[0105] For example, white balance correction can be performed on any raw response information to obtain the first response information, which can be achieved using the following formula:

[0106]

[0107]

[0108]

[0109]

[0110] in, The R component is the first RGB information of the camera module for the corresponding color patch under a calibrated light source. This refers to the G component in the first RGB information of the camera module for the corresponding color patch under a calibrated light source. This refers to the B component in the first RGB information of the camera module for the corresponding color block under a calibrated light source.

[0111] For reference, the R component in the original RGB information corresponding to the color patch. For reference, the G component in the original RGB information corresponding to the color patch. For reference, the B component in the original RGB information corresponding to the color patch. This is the function for finding the maximum value.

[0112] This is the white balance correction matrix.

[0113] for example, This is the average ratio of the R component to the G component in the original RGB information corresponding to the reference color patch. This is the average ratio of the B component to the G component in the original RGB information corresponding to the reference color patch.

[0114] In some possible implementations, brightness correction is performed on the first response information to obtain second response information. This includes performing gamma correction and inverse gamma correction on the first response information corresponding to the reference color patch to obtain fourth response information corresponding to the reference color patch. Based on the fourth response information and the reference response information corresponding to the reference color patch, brightness correction information is determined. Based on the brightness correction information, brightness correction is performed on the first response information to obtain the second response information. Thus, gamma correction and inverse gamma correction can be performed on the first response information corresponding to the reference color patch to obtain fourth response information corresponding to the reference color patch. Based on the fourth response information and the reference response information corresponding to the reference color patch, brightness correction information is determined to perform brightness correction on the first response information.

[0115] For example, brightness correction can be applied to the first response information to obtain the second response information, which can be achieved using the following formula:

[0116]

[0117] in, The R component is the second RGB information of the camera module for the corresponding color patch under a calibrated light source. This refers to the G component in the second RGB information of the camera module for the corresponding color patch under a calibrated light source. This refers to the B component in the second RGB information of the camera module for the corresponding color block under a calibrated light source.

[0118] The average value of the G component in the reference RGB information corresponding to the reference color patch. The average value of the G component in the fourth RGB information corresponding to the reference color patch. This is the brightness correction factor.

[0119] S404, determine the first difference information between any processing response information and the reference response information associated with the corresponding processing response information.

[0120] The first difference information is not subject to many restrictions, such as including the average color difference under the Lab color space.

[0121] In some possible implementations, determining a first difference information between any processing response information and a reference response information associated with the corresponding processing response information includes converting any processing response information to the Lab color space to determine processing Lab information, converting the reference response information associated with the corresponding processing response information to the Lab color space to determine reference Lab information, and determining the average color difference between the processing Lab information and the reference Lab information as the first difference information.

[0122] It is understandable that the region of the reference object corresponding to any processing response information is consistent with the region of the reference object corresponding to the reference response information associated with the corresponding processing response information. For example, taking a standard color chart as an example, the processing response information corresponding to any color block is associated with the reference response information corresponding to the corresponding color block. For instance, the processing response information corresponding to color block i is associated with the reference response information corresponding to color block i.

[0123] S405, Determine the objective function based on the first difference information.

[0124] S406, solve the independent variables of the objective function to determine the color correction matrix for each candidate hue range.

[0125] In some possible implementations, the objective function is determined based on each first difference information, including weighted summation of each first difference information to determine the objective function.

[0126] For example, taking a reference object including a standard color chart, which contains N color patches, and the hues divided into 24 candidate hue intervals, the objective function can be solved by finding the independent variables to determine the color correction matrix for each candidate hue interval. This can be achieved using the following formula:

[0127] in, This is the first difference information (e.g., average color difference) corresponding to the j-th color block. Let j be the weight of the j-th color block. This is the color correction matrix for the camera module under calibrated lighting conditions for 24 candidate tone ranges. j is a positive integer not greater than N, and N is a positive integer.

[0128] Let be the objective function. Used to characterize the search for an optimal set of color correction matrices, such that Minimum.

[0129] It should be noted that this disclosure does not impose any restrictions on the execution sequence of steps S401-S406. Figure 4 The example only demonstrates the execution of steps S401-S406 in sequence.

[0130] The color correction information calibration method provided in the embodiments of this disclosure performs image processing on any original response information to obtain processed response information; wherein, the image processing includes at least color correction, determining first difference information between any processed response information and a reference response information associated with the corresponding processed response information, and determining an objective function based on each first difference information. Thus, at least color correction can be performed on any original response information to obtain processed response information, and the objective function can be determined by taking into account the first difference information between any processed response information and its associated reference response information.

[0131] Figure 5 This is a flowchart illustrating a method for calibrating color correction information according to another exemplary embodiment, such as... Figure 5 As shown, the color correction information calibration method of this disclosure includes the following steps.

[0132] S501, determine the original response information of the camera module to the reference object under the calibrated light source.

[0133] S502, determine the reference response information of the camera module for the reference object under the target light source.

[0134] S503, perform image processing on any raw response information to obtain processed response information; wherein, the image processing includes at least color correction.

[0135] S504, determine the first difference information between any processing response information and the reference response information associated with the corresponding processing response information.

[0136] The relevant content of steps S501-S504 can be found in the above embodiments, and will not be repeated here.

[0137] S505, based on the relationship between the upper or lower limits of each candidate hue range, groups the color correction matrices to determine at least one set of color correction matrices.

[0138] It should be noted that any set of color correction matrices includes at least two color correction matrices.

[0139] In some possible implementations, the color correction matrices are grouped based on the size relationship between the upper or lower limits of each candidate hue interval to determine at least one set of color correction matrices. This includes sorting each candidate hue interval according to the upper or lower limit, and taking the color correction matrices of two adjacent candidate hue intervals, and / or the color correction matrix of the first candidate hue interval and the color correction matrix of the last candidate hue interval as any set of color correction matrices.

[0140] For example, taking the hue as divided into 24 candidate hue intervals, the value range of the first candidate hue interval is [0°, 15°), the value range of the second candidate hue interval is [15°, 30°), the value range of the third candidate hue interval is [30°, 45°), and so on, with the value range of the 24th candidate hue interval being [345°, 360°).

[0141] The 24 candidate color tones can be sorted from smallest to largest according to their upper limit values ​​to determine the target order, which is the 1st candidate color tones, the 2nd candidate color tones, the 3rd candidate color tones, up to the 24th candidate color tones.

[0142] If the color correction matrix of the i-th candidate hue range is Then you can , As the first color correction matrix, when i is a positive integer greater than 1 and less than or equal to 24, , As the i-th color correction matrix, for example, , As the second color correction matrix, , As the third color correction matrix, and so on, , As the 24th color correction matrix.

[0143] In some possible implementations, the color correction matrices are grouped based on the relationship between the upper or lower limits of each candidate hue interval to determine at least one set of color correction matrices. This includes taking the s-th candidate hue interval and the q-th candidate hue interval as any set of color correction matrices when the upper limit of the s-th candidate hue interval is the same as the lower limit of the q-th candidate hue interval.

[0144] In some possible implementations, the color correction matrices are grouped based on the relationship between the upper or lower limits of each candidate hue interval to determine at least one set of color correction matrices. This includes using the s-th candidate hue interval and the q-th candidate hue interval as any set of color correction matrices if the difference between the upper limit of the s-th candidate hue interval and the lower limit of the q-th candidate hue interval is less than a set threshold.

[0145] S506, determine second difference information between at least one set of color correction matrices.

[0146] It should be noted that the second type of difference information is not subject to many restrictions; for example, it may include scaling difference information, rotation difference information, element difference information, cosine similarity, etc. Element difference information includes Euclidean distance, Manhattan distance, mean squared error, mean absolute error, etc.

[0147] In some possible implementations, determining second difference information between at least one set of color correction matrices includes determining a transformation matrix between any set of color correction matrices, performing singular value decomposition on the transformation matrix to determine a left singular vector matrix, a singular value matrix, and a right singular vector matrix, determining scaling difference information between corresponding sets of color correction matrices based on the singular value matrix, and determining rotation difference information between corresponding sets of color correction matrices based on the left and right singular vector matrices.

[0148] For example, taking a color tone divided into 24 candidate color tone ranges as an example, the second difference information between at least one set of color correction matrices can be determined by the following formula:

[0149]

[0150]

[0151]

[0152]

[0153]

[0154]

[0155]

[0156]

[0157] in, Let i be the transformation matrix between the i-th group of color correction matrices. for The left singular vector matrix obtained by performing singular value decomposition. for The singular value matrix obtained by performing singular value decomposition. for The right singular vector matrix obtained by performing singular value decomposition. for The element in the first row and first column of the middle, for The element in the 2nd row and 2nd column, for The element in the 3rd row and 3rd column.

[0158] This represents the scaling difference information in the x-direction between the i-th group of color correction matrices. This represents the scaling difference information in the y-direction between the i-th group of color correction matrices. This represents the scaling difference information in the z-direction between the i-th group of color correction matrices. This provides scaling difference information between the i-th group of color correction matrices.

[0159] for The transpose of the matrix, for The sum of the elements of the comparison line, This represents the rotational difference information between the i-th group of color correction matrices.

[0160] It is a singular value decomposition function. To find the minimum value function, It is an inverse cosine function. This is the function for taking the absolute value.

[0161] S507, Determine the objective function based on each first difference information and each second difference information.

[0162] S508 solves for the independent variables of the objective function to determine the color correction matrix for each candidate hue range.

[0163] In this disclosure, the color correction matrices can be grouped by taking into account the size relationship between the upper or lower limits of each candidate color tone range to determine at least one set of color correction matrices, determine the second difference information between at least one set of color correction matrices, and comprehensively consider the first difference information and the second difference information to determine the objective function. This can ensure the smoothness and consistency of the color correction process, help avoid color abrupt changes, and improve the overall quality of color correction.

[0164] In some possible implementations, the objective function is determined based on each first difference information and each second difference information, including weighted summation of each first difference information to determine the error term of the objective function, weighted summation of each second difference information to determine the regularization term of the objective function, and determination of the objective function based on the error term and the regularization term.

[0165] For example, taking the hue division into 24 candidate hue intervals as an example, the weighted summation of each second difference information to determine the regularization term of the objective function can be achieved through the following formula:

[0166]

[0167]

[0168] in, This provides information on the overall scaling differences between the 24 color correction matrices. This provides information on the overall scaling differences between the 24 color correction matrices. For the regularization term of the objective function, for The weight, for The weight.

[0169] The color correction matrix for each candidate hue range is determined by solving the objective function for its independent variables, using the following formula:

[0170] in, For coefficients, Let be the objective function. Used to characterize the search for an optimal set of color correction matrices, such that Minimum.

[0171] It should be noted that this disclosure does not impose any restrictions on the execution sequence of steps S501-S508. Figure 5 The example only demonstrates the execution of steps S501-S508 in sequence.

[0172] The color correction information calibration method provided in the embodiments of this disclosure can take into account the size relationship between the upper or lower limits of each candidate hue range, group each color correction matrix to determine at least one set of color correction matrices, determine the second difference information between at least one set of color correction matrices, and comprehensively consider each first difference information and each second difference information to determine the objective function. This can ensure the smoothness and consistency of the color correction process, help avoid color abrupt changes, and improve the overall quality of color correction.

[0173] For ease of understanding, an exemplary embodiment is provided: like Figure 6 As shown, taking a reference object including a standard color chart and 24 candidate color ranges as an example, the original RGB information is determined based on the spectral sensitivity function of the camera module, the reflectance of the standard color chart and the spectrum of the calibration light source, and the reference RGB information is determined based on the standard observer function, the reflectance of the standard color chart and the spectrum of the target light source.

[0174] Perform white balance correction on any original RGB information to obtain first RGB information, and perform brightness correction on the first RGB information to obtain second RGB information.

[0175] The second target color range to which the second RGB information belongs is determined from 24 candidate color ranges. The second RGB information is then color-corrected based on the color correction matrix of the second target color range to obtain the third RGB information.

[0176] Gamma correction and inverse gamma correction are performed on the third RGB information to obtain processed RGB information.

[0177] Based on the processed RGB information and the reference RGB information, each first difference information is determined. Based on each first difference information and each second difference information, the objective function is determined. The objective function is used to indicate the transformation relationship between the original response information and the reference response information in 24 candidate hue intervals. The independent variables of the objective function include the color correction matrix of the 24 candidate hue intervals.

[0178] The objective function is optimized to obtain 24 color correction matrices that minimize the objective function, which are then used as the final 24 color correction matrices.

[0179] Figure 7 This is a schematic diagram of the structure of an image processing apparatus according to an exemplary embodiment.

[0180] Reference Figure 7 The image processing apparatus 700 of this embodiment includes: a first determining module 701, a second determining module 702, and a processing module 703.

[0181] The first determining module 701 is configured to determine, in response to a shooting command, the actual response information of the camera module to the shooting object; The second determining module 702 is configured to determine, from each candidate tone range, the first target tone range to which any of the actual response information belongs; The processing module 703 is configured to perform color correction on the corresponding actual response information based on the color correction information of the camera module for the first target tone range under the reference light source, so as to obtain the target response information.

[0182] In some possible implementations, the second determining module 702 is further configured to: determine from each calibration light source a calibration light source that matches the actual light source when the camera module takes a picture, and use it as the reference light source.

[0183] In some possible implementations, the processing module 703 is further configured to perform at least one of the following operations: A target image is generated based on the target response information, and the target image is visualized. The target image is sent to the terminal device; wherein the target image is used by the terminal device for visualization display.

[0184] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0185] The image processing apparatus provided in the embodiments of this disclosure, in response to a shooting command, determines the actual response information of the camera module to the subject, determines a first target tone range to which any actual response information belongs from each candidate tone range, and performs color correction on the corresponding actual response information based on the color correction information of the camera module for the first target tone range under a reference light source to obtain the target response information. Thus, the tone can be divided into candidate tone ranges, and each candidate tone range can be set with independent color correction information. That is, different color correction information can be used under different candidate tone ranges, and color correction can be performed by region, improving the diversity of color correction information and helping to improve color correction accuracy. For example, it significantly improves the overall color accuracy and naturalness, avoids color distortion and other problems caused by using a single color correction information, and optimizes the visual experience.

[0186] Figure 8 This is a schematic diagram of the structure of a color correction information calibration device according to an exemplary embodiment.

[0187] Reference Figure 8 The color correction information calibration device 800 of this embodiment includes: a first determination module 801, a second determination module 802 and a third determination module 803.

[0188] The first determining module 801 is configured to determine the original response information of the camera module to the reference object under the calibrated light source; The second determining module 802 is configured to determine the reference response information of the camera module for the reference object under the target light source; The third determining module 803 is configured to determine the color correction information of the camera module for each candidate tone range under the calibrated light source based on each of the original response information and each of the reference response information; wherein, the color correction information of any candidate tone range is used to perform color correction on the actual response information belonging to the corresponding candidate tone range.

[0189] In some possible implementations, the third determining module 803 is further configured to: determine the conversion relationship between the original response information and the reference response information in each of the candidate hue intervals based on each of the original response information and each of the reference response information; and determine the color correction information for each of the candidate hue intervals based on the conversion relationship.

[0190] In some possible implementations, the color correction information is represented in the form of a color correction matrix; The third determining module 803 is further configured to: determine an objective function based on each of the original response information and each of the reference response information; wherein the objective function is used to indicate the transformation relationship, and the independent variables of the objective function include the color correction matrix of each of the candidate hue intervals; and solve the objective function for the independent variables to determine the color correction matrix of each of the candidate hue intervals.

[0191] In some possible implementations, the third determining module 803 is further configured to: perform image processing on any of the original response information to obtain processed response information; wherein the image processing includes at least color correction; determine first difference information between any of the processed response information and reference response information associated with the corresponding processed response information; and determine the objective function based on each of the first difference information.

[0192] In some possible implementations, the third determining module 803 is further configured to: group each color correction matrix based on the size relationship between the upper or lower limits of each candidate hue range to determine at least one set of color correction matrices; determine second difference information between at least one set of color correction matrices; and determine the objective function based on each of the first difference information and each of the second difference information.

[0193] In some possible implementations, the third determining module 803 is further configured to: sort each of the candidate hue intervals according to an upper limit or a lower limit; and take the color correction matrices of two adjacent candidate hue intervals, and / or the color correction matrix of the first candidate hue interval and the color correction matrix of the last candidate hue interval as any set of color correction matrices.

[0194] In some possible implementations, the third determining module 803 is further configured to: determine a transformation matrix between any set of color correction matrices; perform singular value decomposition on the transformation matrix to determine a left singular vector matrix, a singular value matrix, and a right singular vector matrix; determine scaling difference information between corresponding sets of color correction matrices based on the singular value matrix; and determine rotation difference information between corresponding sets of color correction matrices based on the left singular vector matrix and the right singular vector matrix.

[0195] In some possible implementations, the third determining module 803 is further configured to: perform a weighted summation on each of the first difference information to determine the error term of the objective function; perform a weighted summation on each of the second difference information to determine the regularization term of the objective function; and determine the objective function based on the error term and the regularization term.

[0196] In some possible implementations, the image processing further includes at least one of white balance correction, brightness alignment, gamma correction, and inverse gamma correction.

[0197] In some possible implementations, the third determining module 803 is further configured to: perform white balance correction on any of the original response information to obtain first response information; perform brightness correction on the first response information to obtain second response information; determine a second target tone range to which the second response information belongs from each of the candidate tone ranges; perform color correction on the second response information based on the color correction matrix of the second target tone range to obtain third response information; and perform gamma correction and inverse gamma correction on the third response information to obtain the processed response information.

[0198] In some possible implementations, the third determining module 803 is further configured to: when the reference object includes a standard color chart, use the grayscale color block in the standard color chart as a reference color block; determine white balance correction information based on the original response information corresponding to the reference color block; and perform white balance correction on any of the original response information based on the white balance correction information to obtain first response information.

[0199] In some possible implementations, the third determining module 803 is further configured to: perform gamma correction and inverse gamma correction on the first response information corresponding to the reference color block to obtain the fourth response information corresponding to the reference color block; determine brightness correction information based on the fourth response information corresponding to the reference color block and the reference response information corresponding to the reference color block; and perform brightness correction on the first response information based on the brightness correction information to obtain the second response information.

[0200] In some possible implementations, the first determining module 801 is further configured to: determine each of the original response information based on the spectral sensitivity function of the camera module, the reflectivity of the reference object, and the spectrum of the calibration light source; The second determining module 802 is further configured to: determine each of the reference response information based on the standard observer function, the reflectivity of the reference object, and the spectrum of the target light source.

[0201] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0202] The color correction information calibration apparatus provided in the embodiments of this disclosure determines the original response information of the camera module to a reference object under a calibrated light source, determines the reference response information of the camera module to the reference object under a target light source, and determines the color correction information of the camera module for each candidate hue range under the calibrated light source based on the original response information and the reference response information. The color correction information for any candidate hue range is used to perform color correction on the actual response information belonging to the corresponding candidate hue range. Therefore, by considering the original response information and the reference response information, the color correction information of the camera module for each candidate hue range under the calibrated light source can be calibrated, thus dividing the hue into candidate hue ranges. Each candidate hue range can be set with independent color correction information, meaning different color correction information can be calibrated for different candidate hue ranges. This allows for regional color correction, increasing the diversity of color correction information and improving color correction accuracy, such as significantly improving overall color accuracy and naturalness. It avoids color distortion caused by using a single color correction information, thus optimizing the visual experience.

[0203] To implement the above embodiments, this disclosure also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps of the image processing method provided in this disclosure.

[0204] Figure 9 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment. For example, the electronic device 900 may be a vehicle, mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0205] Reference Figure 9 The electronic device 900 may include one or more of the following components: processing component 902, memory 904, power component 906, multimedia component 908, audio component 910, input / output (I / O) interface 912, sensor component 914, and communication component 920.

[0206] Processing component 902 typically controls the overall operation of electronic device 900, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 902 may include one or more processors 920 to execute instructions to complete all or part of the steps of the image processing method described above. Furthermore, processing component 902 may include one or more modules to facilitate interaction between processing component 902 and other components. For example, processing component 902 may include a multimedia module to facilitate interaction between multimedia component 908 and processing component 902.

[0207] Memory 904 is configured to store various types of data to support the operation of electronic device 900. Examples of this data include instructions for any application or method operating on electronic device 900, contact data, phonebook data, messages, pictures, videos, etc. Memory 904 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0208] Power component 906 provides power to various components of electronic device 900. Power component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 900.

[0209] Multimedia component 908 includes a screen that provides an output interface between electronic device 900 and user. In some embodiments, the screen may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen may be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors may sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 908 includes a front-facing camera and / or a rear-facing camera. When electronic device 900 is in an operating mode, such as a shooting mode or video mode, the front-facing camera and / or rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0210] Audio component 910 is configured to output and / or input audio signals. For example, audio component 910 includes a microphone (MIC) configured to receive external audio signals when electronic device 900 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 904 or transmitted via communication component 916. In some embodiments, audio component 910 also includes a speaker for outputting audio signals.

[0211] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0212] Sensor assembly 914 includes one or more sensors for providing state assessments of various aspects of electronic device 900. For example, sensor assembly 914 can detect the on / off state of electronic device 900, the relative positioning of components such as the display and keypad of electronic device 900, changes in position of electronic device 900 or a component of electronic device 900, the presence or absence of user contact with electronic device 900, orientation or acceleration / deceleration of electronic device 900, and temperature changes of electronic device 900. Sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 914 may also include an optical sensor, such as a complementary metal-oxide-semiconductor (CMOS) or charge-coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, sensor assembly 914 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0213] Communication component 916 is configured to facilitate wired or wireless communication between electronic device 900 and other devices. Electronic device 900 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 916 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 916 also includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), Bluetooth, and other technologies.

[0214] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the steps of the image processing method described above.

[0215] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, which can be executed by a processor 920 of an electronic device 900 to complete the image processing method described above. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0216] To implement the above embodiments, this disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the steps of the image processing method provided in this disclosure.

[0217] Alternatively, the computer-readable storage medium may be ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0218] To implement the above embodiments, this disclosure also proposes a chip including an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to implement the steps of the image processing method provided in this disclosure.

[0219] Figure 10 This is a schematic diagram illustrating the structure of a chip according to an exemplary embodiment. See also... Figure 10 The diagram shown is a schematic representation of the structure of chip 1000, but it is not limited to this.

[0220] Chip 1000 includes processing circuit 1001, which is configured to perform the steps of any of the above image processing methods.

[0221] In some embodiments, the chip 1000 further includes one or more interface circuits 1002. In some possible implementations, the interface circuit 1002 is connected to the memory 1003, and the interface circuit 1002 can be used to receive signals from the memory 1003 or other devices, and the interface circuit 1002 can be used to send signals to the memory 1003 or other devices. For example, the interface circuit 1002 can read instructions stored in the memory 1003 and send the instructions to the processing circuit 1001.

[0222] In some embodiments, the interface circuit 1002 performs at least one of the communication steps such as sending and / or receiving in the above method, and the processing circuit 1001 performs other steps.

[0223] In some embodiments, the terms interface circuit, interface, transceiver pin, transceiver, etc., can be used interchangeably.

[0224] In some embodiments, chip 1000 further includes one or more memories 1003 for storing instructions. In some possible implementations, all or part of the memories 1003 may be located outside of chip 1000.

[0225] To implement the above embodiments, this disclosure also proposes a computer program product, including a computer program that, when executed by a processor, implements the steps of the image processing method provided in this disclosure.

[0226] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0227] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. An image processing method, characterized in that, include: In response to the shooting command, determine the actual response information of the camera module to the subject being shot; Determine the first target tone range to which any of the actual response information belongs from each candidate tone range; Based on the color correction information of the camera module for the first target tone range under the reference light source, the corresponding actual response information is color corrected to obtain the target response information.

2. The method according to claim 1, characterized in that, The method further includes: A calibration light source that matches the actual light source during the shooting process of the camera module is selected from among the calibration light sources and used as the reference light source.

3. The method according to claim 1 or 2, characterized in that, The method further includes at least one of the following operations: A target image is generated based on the target response information, and the target image is visualized. The target image is sent to the terminal device; wherein the target image is used by the terminal device for visualization display.

4. A method for calibrating color correction information, characterized in that, include: Determine the original response information of the camera module to the reference object under the calibrated light source; Determine the reference response information of the camera module for the reference object under the target light source; Based on the original response information and the reference response information, the color correction information of the camera module for each candidate tone range under the calibrated light source is determined; wherein, the color correction information of any candidate tone range is used to perform color correction on the actual response information belonging to the corresponding candidate tone range.

5. The method according to claim 4, characterized in that, The step of determining the color correction information of the camera module for each candidate tone range under the calibrated light source based on the original response information and the reference response information includes: Based on the original response information and the reference response information, the conversion relationship between the original response information and the reference response information in each candidate tone range is determined; Based on the transformation relationship, color correction information for each of the candidate hue ranges is determined.

6. The method according to claim 5, characterized in that, The color correction information is represented in the form of a color correction matrix; The step of determining the conversion relationship between the original response information and the reference response information in each of the candidate tone ranges based on each of the original response information and each of the reference response information includes: Based on the original response information and the reference response information, an objective function is determined; wherein the objective function is used to indicate the transformation relationship, and the independent variable of the objective function includes the color correction matrix of each candidate hue range; The determination of color correction information for each candidate hue range based on the transformation relationship includes: The objective function is solved by finding the independent variables to determine the color correction matrix for each of the candidate hue ranges.

7. The method according to claim 6, characterized in that, The step of determining the objective function based on each of the original response information and each of the reference response information includes: Image processing is performed on any of the original response information to obtain processed response information; wherein the image processing includes at least color correction; Determine the first difference information between any of the processing response information and the reference response information associated with the corresponding processing response information; The objective function is determined based on each of the first difference information.

8. The method according to claim 7, characterized in that, Determining the objective function based on each of the first difference information includes: Based on the relationship between the upper or lower limits of each candidate hue range, the color correction matrices are grouped to determine at least one set of color correction matrices; Determine the second difference information between at least one set of color correction matrices; The objective function is determined based on each of the first difference information and each of the second difference information.

9. The method according to claim 8, characterized in that, The process of grouping the color correction matrices based on the relationship between the upper or lower limits of each candidate hue range to determine at least one set of color correction matrices includes: The candidate color tone ranges are sorted according to their upper or lower limits. The color correction matrices of two adjacent candidate hue intervals, and / or the color correction matrix of the first candidate hue interval and the color correction matrix of the last candidate hue interval, are used as any set of color correction matrices.

10. The method according to claim 8, characterized in that, Determining the second difference information between at least one set of color correction matrices includes: Determine the transformation matrix between any set of color correction matrices; Singular value decomposition is performed on the transformation matrix to determine the left singular vector matrix, the singular value matrix, and the right singular vector matrix; Based on the singular value matrix, determine the scaling difference information between the corresponding groups of color correction matrices; Based on the left singular vector matrix and the right singular vector matrix, the rotation difference information between the corresponding color correction matrices is determined.

11. The method according to claim 8, characterized in that, The step of determining the objective function based on each of the first difference information and each of the second difference information includes: The first difference information is weighted and summed to determine the error term of the objective function; The second difference information is weighted and summed to determine the regularization term of the objective function; The objective function is determined based on the error term and the regularization term.

12. The method according to any one of claims 7-11, characterized in that, The image processing also includes at least one of white balance correction, brightness alignment, gamma correction, and inverse gamma correction.

13. The method according to claim 12, characterized in that, The step of performing image processing on any of the original response information to obtain processed response information includes: White balance correction is performed on any of the original response information to obtain the first response information; The first response information is subjected to brightness correction to obtain the second response information; Determine the second target tone range to which the second response information belongs from each of the candidate tone ranges; The second response information is color-corrected based on the color correction matrix of the second target hue range to obtain the third response information; The third response information is subjected to gamma correction and inverse gamma correction to obtain the processed response information.

14. The method according to claim 13, characterized in that, The step of performing white balance correction on any of the original response information to obtain the first response information includes: When the reference object includes a standard color chart, the grayscale color block in the standard color chart is used as the reference color block; Based on the original response information corresponding to the reference color block, determine the white balance correction information; Based on the white balance correction information, white balance correction is performed on any of the original response information to obtain the first response information.

15. The method according to claim 14, characterized in that, The step of performing brightness correction on the first response information to obtain the second response information includes: Gamma correction and inverse gamma correction are performed on the first response information corresponding to the reference color block to obtain the fourth response information corresponding to the reference color block; Based on the fourth response information corresponding to the reference color block and the reference response information corresponding to the reference color block, the brightness correction information is determined. Based on the brightness correction information, the first response information is brightness corrected to obtain the second response information.

16. The method according to any one of claims 4-11, characterized in that, The determination of the camera module's original response information to the reference object under the calibrated light source includes: Based on the spectral sensitivity function of the camera module, the reflectivity of the reference object, and the spectrum of the calibration light source, the original response information is determined. Determining the reference response information of the camera module for the reference object under the target light source includes: The reference response information is determined based on the standard observer function, the reflectivity of the reference object, and the spectrum of the target light source.

17. An image processing apparatus, characterized in that, include: The first determining module is configured to determine the actual response information of the camera module to the subject being photographed in response to the shooting command. The second determining module is configured to determine the first target tone range to which any of the actual response information belongs from each candidate tone range; The processing module is configured to perform color correction on the corresponding actual response information based on the color correction information of the camera module for the first target tone range under the reference light source, so as to obtain the target response information.

18. The apparatus according to claim 17, characterized in that, The second determining module is further configured to: A calibration light source that matches the actual light source during the shooting process of the camera module is selected from among the calibration light sources and used as the reference light source.

19. The apparatus according to claim 17 or 18, characterized in that, The processing module is also configured to perform at least one of the following operations: A target image is generated based on the target response information, and the target image is visualized. The target image is sent to the terminal device; wherein the target image is used by the terminal device for visualization display.

20. A calibration device for color correction information, characterized in that, include: The first determining module is configured to determine the original response information of the camera module to the reference object under the calibrated light source; The second determining module is configured to determine the reference response information of the camera module for the reference object under the target light source; The third determining module is configured to determine the color correction information of the camera module for each candidate tone range under the calibrated light source based on each of the original response information and each of the reference response information; wherein, the color correction information of any candidate tone range is used to perform color correction on the actual response information belonging to the corresponding candidate tone range.

21. The apparatus according to claim 20, characterized in that, The third determining module is further configured to: Based on the original response information and the reference response information, the conversion relationship between the original response information and the reference response information in each candidate tone range is determined; Based on the transformation relationship, color correction information for each of the candidate hue ranges is determined.

22. The apparatus according to claim 21, characterized in that, The color correction information is represented in the form of a color correction matrix; The third determining module is further configured to: Based on the original response information and the reference response information, an objective function is determined; wherein the objective function is used to indicate the transformation relationship, and the independent variable of the objective function includes the color correction matrix of each candidate hue range; The objective function is solved by finding the independent variables to determine the color correction matrix for each of the candidate hue ranges.

23. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the steps of the method according to any one of claims 1-16.

24. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-16.

25. A chip, characterized in that, The chip includes an interface circuit and a processing circuit coupled to each other. The interface circuit is used to input or output signals, and the processing circuit is configured to implement the steps of the method according to any one of claims 1-16.

26. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-16.

Images