Methods, apparatus and systems for determining and correcting color errors
By determining the color difference and generating a color correction matrix, the problem of existing technologies being unable to accurately describe human visual preferences is solved, achieving a more accurate color correction effect.
Patent Information
- Application Number
- CN202480090845.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-04-12
- Publication Date
- 2026-11-17
AI Technical Summary
Existing color difference measurement methods cannot accurately describe human visual preferences, leading to inaccurate color correction.
By obtaining the color difference between the first image and the second image, the color error is determined using the first function Ψ, and a color correction matrix (CCM) is generated based on this. During the iterative optimization process, the CCM is adjusted to better simulate human visual preferences.
It achieves a more accurate simulation of human visual preferences, improves the effect of color correction, and makes the corrected image appear the same as the original image to the human eye.
Smart Images

Figure CN123125891A_ABST
Abstract
Description
Technical Field
[0001] This invention generally relates to the field of image processing, and more particularly to a method, apparatus and system for determining and correcting color errors, as well as a computer-readable storage medium. Background Technology
[0002] measure It was proposed by the International Commission on Illumination (CIE) in 1976 to describe color difference. It is defined on the LAB color space, which takes into account the non-uniformity perceived by humans. However, This is insufficient to describe human visual preferences. Therefore, an improved metric is desired to define color difference in terms of human visual preferences.
[0003] The purpose of providing this background information is to disclose information that the applicant believes may be relevant to the present invention, and it is not intended to acknowledge, nor should any of the foregoing information be construed as prior art in relation to the present invention. Summary of the Invention
[0004] This invention provides a method for determining color error and color correction.
[0005] According to a first aspect, a method for determining color errors is provided. The method includes: obtaining a first color difference between a first color of a first pixel in a first image and a second color of a second pixel in a second image. Tonal difference and color difference The second color is the color to be corrected corresponding to the first color; based on , and The value of a first function Ψ is determined, wherein the first function is used to determine the difference between the first color and the second color.
[0006] In this way, accurate color differences based on human perception can be obtained.
[0007] In some embodiments, the first color is a baseline true value color.
[0008] In this case, the first function is used to determine the difference between the reference true value color and the color to be corrected corresponding to the reference true value. In other words, the first function is used to determine the error between the color to be corrected and its reference true value.
[0009] In some embodiments, after determining the value of Ψ, the method further includes: determining a color correction matrix (CCM) based on the value of Ψ.
[0010] In some embodiments, determining the CCM includes: optimizing the CCM if the value of Ψ is not greater than a threshold.
[0011] In some embodiments, tuning the CCM includes using iterative optimization to tune the CCM, and the iterative optimization uses Ψ as the target loss.
[0012] In this way, the CCM can be fine-tuned to obtain an ideal CCM.
[0013] In some embodiments, determining the CCM includes: outputting the CCM if the value of Ψ is greater than the threshold.
[0014] In this way, the acquired CCM can provide ideal results during color correction.
[0015] In some embodiments, the method further includes: capturing a third image using a camera; and processing the third image using the CCM to obtain a fourth image.
[0016] In this way, the fourth image, which appears to the human eye as the same as the third image, can be obtained.
[0017] In some embodiments, the first function is a piecewise function having multiple segmented parts, the multiple segmented parts depending on... , or At least one of them; the first function is expressed as: Where U represents the upper bound and L represents the lower bound. Representation parameters; based on the , and Determining the value of the first function Ψ includes: exist , or If at least one of them is within a first range, the first function satisfies ; exist , or In the case of at least one of the following within the second range, the first function satisfies ; exist , or If at least one of them is within the third range, then the first function satisfies ; exist , or If at least one of them is within the fourth range, then the first function satisfies ; exist , or If at least one of them is within the fifth range, then the first function satisfies ; exist , or If at least one of them is within the sixth range, then the first function satisfies .
[0018] In this way, it can be based on , or At least one of them is used to obtain the first function.
[0019] In some embodiments, the first range, the second range, the third range, the fourth range, the fifth range, and the sixth range correspond to the first region, the second region, the third region, the fourth region, the fifth region, and the sixth region in a color space, respectively.
[0020] In this way, it can be based on , or At least one of the following can be used to obtain different regions in the color space. Furthermore, an ideal color correction patch can be obtained based on the region it belongs to.
[0021] In some embodiments, in the color space, the first region is represented by a first approximate ellipse, which is located inside a second approximate ellipse; the second region, the third region, the fourth region, and the fifth region are located outside the first approximate ellipse and inside the second approximate ellipse; the sixth region is represented by a region in the color space other than the region inside the second approximate ellipse.
[0022] In some embodiments, In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; in, express The first threshold, express The second threshold, express The threshold.
[0023] In this way, it can be based on , or At least one of them is used to obtain the first function.
[0024] In some embodiments, z=0 or z=1.
[0025] In some embodiments, the method further includes: dividing the color space according to a plurality of hues to obtain a plurality of partitions.
[0026] In this way, different partitions in the color space can be obtained, and ideal color correction patches can be obtained based on the partition to which they belong.
[0027] In some embodiments, z=1 when the first color and the second color are in the same partition; z=0 when the first color and the second color are not in the same partition.
[0028] In this way, the value of z can indicate whether the first color and the second color are in the same partition. Furthermore, an ideal color-corrected patch that is in the same partition as the reference true value can be obtained.
[0029] In some embodiments, It is represented as: ,in ;or ;or ,in ;or ,in .
[0030] In this way, It can be represented by a function whose graph is continuous or smooth, whose attenuation rate is adjustable, and which can connect two given points.
[0031] In some embodiments, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; .
[0032] In this way, The attenuation rate can be limited in an appropriate manner.
[0033] In some embodiments, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; .
[0034] In this way, provide The rule of thumb value.
[0035] In some embodiments, L1=U2=U3=U4=U5, and U6=L2=L3=L4=L5.
[0036] In one implementation, L1=U2=U3=U4=U5=0.9; U6=L2=L3=L4=L5=0.4; U1=1; L6=0.
[0037] In some embodiments, the color space is the LAB color space.
[0038] In some embodiments, the sum of each row of the CCM is equal to a value.
[0039] In this way, hard constraints can be satisfied.
[0040] In one implementation, the value is 1024 to preserve white.
[0041] According to a second aspect, a method for color correction is provided. The method includes: capturing an image of a first image using a camera; processing the first image using a color correction matrix (CCM) to obtain a second image; wherein the CCM is obtained based on a first function Ψ, wherein the first function Ψ is used to determine a difference between a first color and a second color; the first color is the color of a first pixel in the first image, and the second color is the color of a second pixel in the second image; and a color difference is determined based on a first color difference between the first color and the second color in the second image. Tonal difference and color difference To obtain the first function Ψ.
[0042] In this way, accurate color differences based on human perception can be obtained. Furthermore, the acquired CCM can provide ideal results during color correction.
[0043] In some embodiments, the first color is a baseline true value color.
[0044] In this case, the first function is used to determine the difference between the reference true value color and the color to be corrected corresponding to the reference true value. In other words, the first function is used to determine the error between the color to be corrected and its reference true value.
[0045] In some embodiments, the CCM is tuned if the value of Ψ is not greater than a threshold.
[0046] In this way, the acquired CCM can provide ideal results during color correction.
[0047] In one implementation, iterative optimization is used to tune the CCM, whereby the iterative optimization uses Ψ as the target loss.
[0048] In this way, the CCM can be fine-tuned to obtain an ideal CCM.
[0049] In some embodiments, the CCM is output when the value of Ψ is greater than the threshold.
[0050] In some embodiments, the method further includes: capturing a third image using a camera; and processing the third image using the CCM to obtain a fourth image.
[0051] In this way, the fourth image, which appears to the human eye as the same as the third image, can be obtained.
[0052] In some embodiments, the first function is a piecewise function having multiple segmented parts, the multiple segmented parts depending on... , or At least one of them; The first function is represented as: Where U represents the upper bound and L represents the lower bound. Indicates parameters; based on , and Determining the value of the first function Ψ includes: exist , or If at least one of them is within a first range, the first function satisfies ; exist , or In the case of at least one of the following within the second range, the first function satisfies ; exist , or If at least one of them is within the third range, then the first function satisfies ; exist , or If at least one of them is within the fourth range, then the first function satisfies ; exist , or If at least one of them is within the fifth range, then the first function satisfies ; exist , or If at least one of them is within the sixth range, then the first function satisfies .
[0053] In this way, it can be based on , or At least one of them is used to obtain the first function.
[0054] In some embodiments, the first range, the second range, the third range, the fourth range, the fifth range, and the sixth range correspond to the first region, the second region, the third region, the fourth region, the fifth region, and the sixth region in a color space, respectively.
[0055] In this way, it can be based on , or At least one of the following can be used to obtain different regions in the color space. Furthermore, an ideal color correction patch can be obtained based on the region it belongs to.
[0056] In some embodiments, in the color space, the first region is represented by a first approximate ellipse, which is located inside a second approximate ellipse; the second region, the third region, the fourth region, and the fifth region are located outside the first approximate ellipse and inside the second approximate ellipse; the sixth region is represented by a region in the color space other than the region inside the second approximate ellipse.
[0057] In this way, ideal color correction patches can be obtained based on the region to which they belong.
[0058] In some embodiments, In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; in, express The first threshold, express The second threshold, express The threshold.
[0059] In this way, it can be based on , or At least one of them is used to obtain the first function.
[0060] In some embodiments, z=0 or z=1.
[0061] In some embodiments, the method further includes: dividing the color space according to a plurality of hues to obtain a plurality of partitions.
[0062] In this way, different partitions in the color space can be obtained, and ideal color correction patches can be obtained based on the partition to which they belong.
[0063] In some embodiments, z=1 when the first color and the second color are in the same partition; z=0 when the first color and the second color are not in the same partition.
[0064] In this way, the value of z can indicate whether the first color and the second color are in the same partition. Furthermore, an ideal color-corrected patch that is in the same partition as the reference true value can be obtained.
[0065] In some embodiments, It is represented as: ,in ;or ;or ,in ;or ,in .
[0066] In this way, It can be represented by a function whose graph is continuous or smooth, whose attenuation rate is adjustable, and which can connect two given points.
[0067] In some embodiments, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; .
[0068] In this way, The attenuation rate can be limited in an appropriate manner.
[0069] In some embodiments, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; .
[0070] In this way, provide The rule of thumb value.
[0071] In some embodiments, L1=U2=U3=U4=U5, and U6=L2=L3=L4=L5.
[0072] In one implementation, L1=U2=U3=U4=U5=0.9; U6=L2=L3=L4=L5=0.4; U1=1; L6=0.
[0073] In some embodiments, the color space is the LAB color space.
[0074] In some embodiments, the sum of each row of the CCM is equal to a value.
[0075] In this way, hard constraints can be satisfied.
[0076] In one implementation, the value is 1024 to preserve white.
[0077] According to a third aspect, a chip is provided. The chip includes: a logic circuit; and a power supply circuit. The power supply circuit is used to supply power to the logic circuit. The logic circuit is used to perform the steps of the method according to the first or second aspect.
[0078] According to a fourth aspect, an apparatus is provided. The apparatus includes: at least one processor; and at least one memory coupled to the at least one processor. The at least one memory is used to store at least a portion of instructions, which, when executed by the at least one processor, cause the at least one processor to perform steps of the method according to a first or second aspect.
[0079] According to a fifth aspect, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium includes computer software instructions that, when executed on a computer device, cause the computer device to perform the steps of the method according to the first or second aspect.
[0080] According to a sixth aspect, a computer program product is provided, stored on a non-transitory computer-readable storage medium. When the computer program product is run on a computer, it causes the computer to perform the steps of the method according to the first or second aspect.
[0081] According to a seventh aspect, a chip system is provided. The chip system includes: processing circuitry; and a storage medium storing computer program instructions, which, when executed by the processing circuitry, cause the chip system to implement the method according to either the first or second aspect.
[0082] The advantages of any of the designs in aspects three through seven can be referred to in aspect one or two, and will not be elaborated further here.
[0083] Based on the implementation methods provided above, the present invention can provide more implementation methods through further combinations. Attached Figure Description
[0084] Figure 1 Examples of reference truth values and color patches according to some embodiments of the present invention are shown; Figure 2 An environment in which embodiments of the present invention can be implemented is shown; Figure 3 A block diagram of an exemplary electronic device is shown; Figure 4 A flowchart of a method performed by an ED according to some embodiments of the present invention is shown; Figure 5Another environment in which embodiments of the present invention can be implemented is shown; Figure 6 A flowchart is shown of another method performed by an ED according to some embodiments of the present invention; Figures 7 to 9 Each illustrates a regionalized color space according to some embodiments of the present invention; Figure 10 This illustrates regionalized and partitioned color spaces according to some embodiments of the present invention; Figures 11 to 14 Each illustrates some embodiments according to the present invention. The decay rate at different values; Figure 15 A flowchart for generating a color correction matrix according to some embodiments of the present invention is shown; Figure 16 A schematic diagram for generating a color correction matrix according to some embodiments of the present invention is shown; Figure 17 A flowchart of a method performed by a terminal device according to some embodiments of the present invention is shown. Detailed Implementation
[0085] The principles of this invention will be described in conjunction with embodiments thereof. These embodiments are described merely for illustrative purposes and to assist those skilled in the art in understanding and implementing the invention, and are not intended to limit the scope of the invention. The embodiments described herein can be implemented in various ways, and are not limited to those described below.
[0086] In this invention, references to "an embodiment," "an exemplary embodiment," "some embodiments," etc., indicate that the described embodiments may include specific features, structures, or characteristics, but not every embodiment must include specific features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same embodiment. Moreover, when a specific feature, structure, or characteristic is described in connection with an embodiment, it is assumed that those skilled in the art will understand how to combine such features, structures, or characteristics with other embodiments, whether explicitly described or not.
[0087] While this document may use terms such as "first," "second," etc., preceding one or more nouns to describe various elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another, and they do not restrict the order of one or more nouns. For example, a first element may be referred to as a second element without departing from the scope of the embodiments, and similarly, a second element may be referred to as a first element. The term "and / or" as used herein includes any and all combinations of one or more of the listed items.
[0088] As used herein, “at least one of the following: ” and “at least one of ” and similar wording mean at least any one of the elements, or at least any two or more of the elements, or at least all the elements, wherein the list of two or more elements is connected by “and” or “or”.
[0089] The terminology used herein is for describing particular embodiments only and is not intended to limit the exemplary embodiments. Unless the context clearly indicates otherwise, the singular forms “a” and “described” as used herein include the plural meaning. It should also be understood that the terms “comprising” and / or “having” as used herein specify the presence of the stated features, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.
[0090] Because the human eye is more sensitive to certain colors than others, it was proposed that... This describes a color difference that is generally consistent with human perception.
[0091] However, This is insufficient to describe human visual preferences. Specifically, for a given baseline true color, although two colors have the same visual characteristics compared to the baseline true color, they are not necessarily compatible. Value or similar The human eye perceives colors G1 and G2 as having the same ΔE value as G, but still prefers one over the other. For example, given a baseline true color G, although colors G1 and G2 have the same ΔE value as color G, the human eye still prefers color G1 over color G2 (i.e., the human eye perceives color G1 as more similar to color G than G2). Therefore, It is insufficient to describe the color difference that reflects human visual perception and human visual preferences.
[0092] Figure 1 An example of this observation is shown. Among the three with the same or similar values... In a color block, human perception may glimpse the color block on the left rather than the color block on the right.
[0093] In the example, for a given baseline truth color A0, color A1 is compared to color A0. , and The color, and color A2 is compared to color A0. , and The color. In such cases, the human eye may prefer color A1 over color A2.
[0094] In another example, for a given baseline truth color B0, color B1 is compared to color B0. , and The color, and color B2 is compared to color B0. , and The color. In such cases, the human eye may prefer color B1 over color B2.
[0095] In yet another example, for a given baseline truth color C0, color C1 is compared to color C0. , and The color, and color C2 is compared to color C0. , and The color. In such cases, the human eye may prefer color C1 over color C2.
[0096] Color correction (CC) is the task of converting the camera's raw sensor readings into RGB values for the display pixels. CC is an essential component of digital cameras. Figure 2 As shown, after a camera (e.g., a new generation of mobile phones) captures an image, the raw image is obtained from the sensor. Color correction is then performed by an image signal processing (ISP) pipeline. The ISP pipeline includes sharpening, raw noise filtering, color correction, YUV noise filtering, contrast gain, temporal noise reduction, etc. Color correction is performed using its corresponding color correction matrix (CCM). The CCM is used to convert the raw sensor values into display values. The matrix composed of the raw sensor values is multiplied by the CCM to generate a matrix composed of the display values.
[0097] CC optimization work can be categorized into four types: CC target loss: This target represents the numerical difference between the true reference value and the color correction prediction. Early methods minimized the squared error or angular difference in the RGB color space. However, recent studies have shown that the ΔE error (especially ΔE00 defined in the LAB color space) can produce better results. This is likely due to the superior accuracy of E00 in characterizing color differences.
[0098] Raw sensor input features: Many feature representations have been developed for color correction optimization, such as linear methods, lookup tables, and neural networks. Linear methods are used in this work, but they can be used with any other feature. Linear color correction (LCC) optimizes a 3×3 linear matrix for conversion between the camera and the standard color space. Its performance can be improved using polynomial and root polynomial features (PCC, RPCC), which include higher-order terms as well as first-order linear terms in LCC.
[0099] Optimization methods: Different optimization methods can be used depending on the target loss to be optimized. For example, when using squared error in RGB space, a closed-form solution can be used to accomplish the task simply. In more complex cases (e.g., ΔE error), there are different optimization methods, such as Nedler-Mead and pattern search, which are already used in the prior art.
[0100] Lighting independence: Many existing techniques are used to estimate the lighting level in a scene. In this work, we assume that the lighting level is given.
[0101] To more accurately simulate human perceptual preferences, this invention proposes a new metric for describing color differences. .based on and calculate In some embodiments, It can be used to optimize CCM.
[0102] Figure 3 An exemplary device is shown that can implement the methods and teachings according to the present invention.
[0103] like Figure 3 As shown, the electronic device (ED) 110 includes at least one processing unit 200. The processing unit 200 implements various processing operations of the ED 110. For example, the processing unit 200 may perform data processing, signal encoding, power control, input / output processing, or any other function. The processing unit 200 may also be used to implement some or all of the functions and / or embodiments detailed herein. Each processing unit 200 includes any suitable processing or computing device for performing one or more operations. Each processing unit 200 may, for example, include a microprocessor, microcontroller, digital signal processor, field-programmable gate array, or application-specific integrated circuit.
[0104] ED 110 may also include at least one optional transceiver 202. Transceiver 202 is used to modulate data or other content for transmission by at least one antenna or network interface controller (NIC) 204. Transceiver 202 is also used to demodulate data or other content received by at least one antenna 204. Each transceiver 202 includes any suitable structure for generating signals for wireless or wired transmission and / or for processing signals received wirelessly or wiredly. Each antenna 204 includes any suitable structure for transmitting and / or receiving wireless or wired signals. One or more transceivers 202 may be used in ED 110. One or more antennas 204 may be used in ED 110. Although shown as a single functional unit, transceiver 202 may also be implemented using at least one transmitter and at least one separate receiver.
[0105] ED 110 may also include one or more optional input / output devices 206 or interfaces (e.g., wired interfaces connected to the Internet). Input / output devices 206 support interaction with a user or other devices. Each input / output device 206 includes any suitable structure for providing or receiving information from a user, such as a speaker, microphone, keypad, keyboard, display, or touchscreen, including network interface communication.
[0106] In addition, ED 110 may include at least one optional memory 208. Memory 208 stores instructions and data used, generated, or collected by ED 110. For example, memory 208 may store software instructions or modules for implementing some or all of the functions and / or embodiments described above and executed by one or more processing units 200. Each memory 208 includes any suitable one or more volatile and / or non-volatile storage and retrieval devices. Any suitable type of memory can be used, such as random access memory (RAM), read-only memory (ROM), hard disk, optical disk, subscriber identity module (SIM) card, memory stick, and secure digital (SD) memory card, etc.
[0107] ED 110 may have image processing capabilities. For example, ED 110 may have the ability to correct color and / or calculate color errors. ED 110 may include, but is not limited to, mobile phones, cellular phones, smartphones, Voice over IP (VoIP) phones, wireless local loop phones, tablets, wearable terminal devices, personal digital assistants (PDAs), portable computers, desktop computers, image capture terminal devices (e.g., digital cameras), gaming terminal devices, music storage and playback devices, in-vehicle wireless terminal devices, wireless endpoints, smart devices, wireless customer-premises equipment (CPE), Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, devices operating in commercial and / or industrial wireless networks, etc.
[0108] The embodiments described herein illustrate information sufficient to practice the claimed subject matter and explain methods for practicing such subject matter. Those skilled in the art will understand the concepts of the claimed subject matter after reading the following description with reference to the accompanying drawings, and will recognize that the application of these concepts is not specifically mentioned herein. These concepts and applications fall within the scope of this invention and the appended claims.
[0109] As briefly mentioned above, obtaining accurate color differences based on human perception is crucial.
[0110] Various embodiments of the present invention will be described below by way of example.
[0111] Figure 4 A flowchart illustrating a method performed by an electronic device (ED) according to some embodiments of the present invention is shown.
[0112] In step 410, the ED obtains the first color difference between the first color of the first pixel in the first image and the second color of the second pixel in the second image. Tonal difference and color difference The second color is the color to be corrected that corresponds to the first color.
[0113] In some embodiments, the first color is a reference true value color. In such cases, the difference between the reference true value color and the color to be corrected corresponding to the reference true value can be determined accordingly. In other words, the error between the color to be corrected and its reference true value can be determined accordingly.
[0114] In one implementation, the first color of the first pixel in the first image is the color of the color chart, which serves as a reference true value. The second color of the second pixel in the second image can be obtained by capturing an image of the color chart with a camera. Once the camera has captured the image of the color chart, the raw image is acquired from the sensor. The raw image can then be processed (e.g., using an ISP pipeline) and a processed image can be obtained. In this case, the second color of the second pixel in the second image is the same as the second color of the second pixel in the processed image. The second pixel in the processed image corresponds to the first pixel in the color chart.
[0115] For example, such as Figure 5 As shown, the ED 110 captures an image from a color chart. The ED acquires a first image from the sensor, which is the raw image. Then, the ED 110 performs color correction to obtain a second image. The second image is the processed image that can be displayed on the ED 110's screen.
[0116] In step 415, ED 110 is based on , and This determines the value of the first function Ψ. The first function is used to determine the difference between the first color and the second color.
[0117] As described above, by maintaining hue and chroma with By combining these approaches, a new metric is proposed. To more accurately simulate human perceptual preferences, a multi-scale objective function is proposed, which can obtain accurate color differences based on human perception.
[0118] In some embodiments, for the purpose of calculation Reference True Value The surrounding space can be based on , or At least one of them is regionalized. In such cases, step 415 may also include steps 4151 and 4152. Figure 6 A flowchart of the method performed by ED 110 is shown.
[0119] In step 4151, ED according to , or At least one of them is used to regionalize the color space.
[0120] For a given benchmark truth value in LAB space The present invention provides a method for spatial regionalization around a reference truth value.
[0121] In some embodiments, according to Regionalize the space surrounding the baseline truth value. Specifically, according to The threshold will regionalize the space around the benchmark truth.
[0122] For example, express The upper threshold, express The lower threshold. According to... and Regionalize the space surrounding the baseline truth value. For example... Figure 7 As shown, the spatial region surrounding the benchmark truth value is divided into three parts: region Corresponding to Points; regions Corresponding to Points; regions Corresponding to point.
[0123] For example, =2, =6.
[0124] In some cases, human perception preferences Secondly The worst The point in the middle.
[0125] In some embodiments, according to The baseline truth value The surrounding space is regionalized. Specifically, according to The threshold will be the benchmark truth value The surrounding space is regionalized. Hue can be described as the angle between a line from a point to the origin and the positive direction of the x-axis. In other words, every point at the same angle has the same hue. To preserve hue, the following is introduced: .
[0126] like Figure 8A As shown, the color correction swatch is Its corresponding baseline truth value is Then the hue (tone) of the color correction swatch is defined as... Define the baseline truth value as . pointer and The angle between them. Then Defined as .
[0127] To preserve hue (i.e., tone), a hue threshold can be set to regionalize the space around the baseline truth. For example, based on the threshold... Regionalize the space surrounding the baseline truth value. For example... Figure 8B As shown, the baseline truth value The surrounding space is divided into four parts: Less than Color correction color patches correspond to areas and ; Greater than Color correction color patches correspond to areas and .
[0128] In some embodiments, according to The baseline truth value The surrounding space is regionalized. Specifically, according to The threshold will be the benchmark truth value The surrounding space is regionalized.
[0129] Chroma (color intensity) can be described as the distance to the origin in the LAB color space. For example, a color correction swatch is... Its corresponding baseline truth value is . Chroma will be defined as ,and Chroma will be defined as .Then It can be defined as .
[0130] The points are more saturated and appear warmer to human perception. In some cases, human perception prefers higher saturation points rather than lower saturation points; that is, human preference tends to maintain a certain level of saturation.
[0131] To preserve hue (i.e., saturation), a hue threshold can be set to regionalize the space around the baseline truth. For example, based on the threshold... Regionalize the space surrounding the baseline truth value. For example... Figure 9 As shown, the baseline truth value The surrounding space is divided into two parts: region Corresponding to Points; regions Corresponding to The point. In one implementation, =1.
[0132] In the embodiments, according to , and The color space is regionalized. In this case, the first, second, third, fourth, fifth, and sixth ranges correspond to the first, second, third, fourth, fifth, and sixth regions in the color space, respectively.
[0133] In one implementation, in the color space, a first region is represented by a first approximate ellipse, which is located inside a second approximate ellipse; a second region, a third region, a fourth region, and a fifth region are located outside the first approximate ellipse and inside the second approximate ellipse; and a sixth region is represented by a region in the color space other than the region inside the second approximate ellipse.
[0134] like Figure 10 As shown, the square represents the true value of a given benchmark. According to... , and The baseline truth value The surrounding space is regionalized.
[0135] Region R3 corresponds to The point; region R2 corresponds to The point; region R1 corresponds to point.
[0136] You can also based on and Will Regionalization into sub-regions .like Figure 10 As shown, the area and Corresponding to and Points. Area and Corresponding to and The point. From the perspective of preserving hue, the human eye prefers... and The point in the middle, not and The point in the middle.
[0137] In addition, the region Corresponding to point, Corresponding to The point. From the perspective of maintaining high color saturation, the human eye prefers... and The point in the middle, not and The point in the middle.
[0138] In other words, region R3 (an example of the first region) corresponds to The point; region R2.1 (example of the second region) corresponds to The point; region R2.2 (example of the third region) corresponds to The point; region R2.3 (example of the fourth region) corresponds to The point; region R2.4 (example of the fifth region) corresponds to The point; region R1 (example of the sixth region) corresponds to point.
[0139] Overall, for sub-regions It seems that in terms of perception It is the highest priority. It is the least priority, and Priority over Generally speaking, for a given baseline truth value, human perceptual preference: R3 > > > > .
[0140] In this way, regions illustrate human perceptual preferences, which provides more than individual regions. Better measurement.
[0141] In some cases, The same two points can still have different perceptual preferences. For example, given a baseline truth color K, although color K1 (on the ground truth color K) may have different perceptual preferences, the same truth color K1 may have different perceptual preferences. (in the middle) and color K2 (below) (middle) has the same as color K. However, the human eye still prefers color K1 over color K2 (i.e., the human eye perceives color K1 as more similar to color K than color K2). In some embodiments, a partitioning technique is introduced to address this issue.
[0142] In step 4152, the ED divides the color space into multiple partitions based on multiple hues. In this case, the color space is divided into multiple partitions based on multiple hues. Step 4152 is optional.
[0143] As described above, hue can be described as the angle between a point and the positive direction of the x-axis. In other words, every point at the same angle has the same hue.
[0144] like Figure 10As shown on the right, the LAB space is partitioned by lines (e.g., diagonals) starting from the origin. For each diagonal, humans can perceive a significant change in color between the two different partitions. For example, as... Figure 10 As shown, humans can perceive a significant difference between the colors in partition 1 and the colors in partition 2.
[0145] exist When two points with the same value fall into different partitions, the visual preference is for the point that is in the same partition as the baseline true value. For example, for the baseline true value color K mentioned above, color K1 (in the partition mentioned above) is preferred. (in the middle) and color K2 (below) (middle) has the same as color K. However, the human eye prefers color K1 over color K2. This is because color K1 and color K are in the same partition, while K2 and K are in different partitions.
[0146] The color space can be divided into multiple partitions based on a color chart, but this invention does not limit this. In one example, the color space can be divided into 19 partitions based on the McBeth color chart. In another example, the color space can be divided into 24 partitions based on the SG24 color chart. In yet another example, the color space can be divided into 140 partitions based on the SG140 color chart.
[0147] In some embodiments, the first function is a piecewise function having multiple segmented parts, which depend differently on... , or At least one of them.
[0148] In one implementation, the first function is represented as: Where U represents the upper bound and L represents the lower bound. Indicates a parameter.
[0149] In some embodiments, the first range corresponds to a first region. , and In the case of at least one of the following within a first range, the first function satisfies The first range is .
[0150] exist , or At least one of them is within a first range corresponding to a first region in the color space (e.g., In the case of ), Represented as ; exist , or At least one of them is within a second range corresponding to a second region in the color space (e.g., In the case of ), Represented as ; exist , or At least one of them is in a third range corresponding to a third region in the color space (e.g., In the case of ), Represented as ; exist , or At least one of them is in the fourth range corresponding to the fourth region in the color space (e.g., In this case, Represented as ; exist , and At least one of them is within the fifth range corresponding to the fifth region in the color space (e.g., In this case, Represented as ; exist , or At least one of them is in the sixth range corresponding to the sixth region in the color space (e.g., In the case of ), Represented as .
[0151] express The first threshold, express The second threshold, express The threshold.
[0152] In this way, the proposed function Combining the above measures in a multi-scale manner or as well as Therefore, human perceptual preferences will be imitated more accurately.
[0153] The proposed function It is designed with an attenuation rate that can be adjusted based on the behavior or dynamics of the camera's image signal processing. This attenuation rate can be adjusted for each specific brand and hardware to achieve optimal color correction performance.
[0154] function The alternative can be derived as follows: 1. Continuity: A function should be continuous, which means that it has no breaks or jumps in its graph.
[0155] 2. Adjustable decay rate: The decay rate of a function is adjustable, and the decay rate determines how quickly the function decreases.
[0156] 3. Connecting two points: The function should pass through two given points, for example... In some cases, , This invention does not limit this.
[0157] In the first implementation, points in different regions or within the same region can be distinguished by using an exponential decay function. For example, It is represented as: ,in and .
[0158] parameter Define the decay rate in the scoring function (i.e., the first function Ψ).
[0159] Figure 11 It shows The decay rate for different values. For example... Figure 11 As shown, the x-axis represents The value of the y-axis represents The value. Different lines are represented under the following boundaries. Different values: To maintain smooth lines, you can use... Some values. For example... Figure 11 As shown, point A represents , The calculated value of Ψ is 0.6.
[0160] In the second implementation, a polynomial decay function can be used to distinguish points in different regions or within the same region. For example, It is represented as:
[0161] .
[0162] Figure 12 Different values are shown The attenuation rate. For example... Figure 12 As shown, the x-axis represents The value of the y-axis represents The value. Different lines are represented under the following boundaries. Different values: .
[0163] In the third implementation, a logarithmic decay function can be used to distinguish points in different regions or within the same region. In one example, It is represented as: , in , .
[0164] Figure 13 Different values are shown The attenuation rate. For example... Figure 13 As shown, the x-axis represents The value of the y-axis represents The value. Different lines are represented under the following boundaries. Different values: .
[0165] In another example, It is represented as: , in , .
[0166] Figure 14 Different values are shown The attenuation rate. For example... Figure 14 As shown, the x-axis represents The value of the y-axis represents The value. Different lines are represented under the following boundaries. Different values: .
[0167] Understandably, the above examples do not restrict the functions. The design.
[0168] In some embodiments, controlled functions with upper and lower bounds middle, The upper bound can be The lower bound of the integer part. In one implementation, L1, U2, U3, U4, and U5 can have the same value. In the example, L1=U2=U3=U4=U5=0.9. In one implementation, U6, L2, L3, L4, and L5 can have the same value. In the example, U6=L2=L3=L4=L5=0.4. In one implementation, U1=1, L6=0.
[0169] In such cases, Figure 10 In the example shown, the region In The value follows ,in Representing R1, R2 and R3 respectively The value of . Furthermore. The point in They fall into [0, 0.4], [0.4, 0.9] and [0.9, 1.0] respectively.
[0170] The complete definition is as follows:
[0171] , where 0 and 1 represent the worst and perfect scores or values, respectively.
[0172] For example, in In this case, , For example, The value can be calculated from Table 1. and The value can be set to =2, =6.
[0173] It should be noted that the present invention does not limit the values of L1, L2, L3, L4, L5 and L6, or the values of U1, U2, U3, U4, U5 and U6.
[0174] In one implementation, the boolean parameter Used to indicate whether two points are in the same partition. For example, This indicates that the reference true value point and the color correction patch are in the same partition. This indicates that they are not in the same partition. For example, if the first color and the second color are in the same partition, z=1; if the first color and the second color are not in the same partition, z=0.
[0175] In one implementation, as shown below, Table 1 illustrates... Examples of values.
[0176] Table 1
[0177] In the examples in Table 1: ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; .
[0178] Instead, a range of g(α,z) can be provided, rather than an exact value. For example, for each value in the table, 0.8 [Precise Value] g(α,z) 1.2 [Precise value].
[0179] Right now, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; .
[0180] As described above, different metrics are used in a multi-scale manner. , and This paper introduces a unique way to regionalize the LAB space. Furthermore, it defines a new way to divide the LAB space into different partitions based on the dominant hue. Regionalization and partitioning can be used to calculate the proposed metric Ψ. It should be noted that the metric Ψ can be implemented on a portion of the hardware to provide a real-time score reflecting accurate color differences.
[0181] In some embodiments, the proposed metric Ψ can be used to compare the quality of different devices. For example, Ψ1 is the color difference between a color-corrected image (processed by device 1) and its corresponding ground truth value; Ψ2 is the color difference between another color-corrected image (processed by device 2) and its corresponding ground truth value. If Ψ1 is greater than Ψ2, then device 1 is considered superior to device 2. In this way, the quality of different devices can be ranked according to human perception.
[0182] In some embodiments, the proposed metric Ψ can be used to evaluate a CCM or compare different CCMs. For example, Ψ01 is the color difference between a color-corrected image (processed by CCM1) and its corresponding ground truth value; Ψ02 is the color difference between another color-corrected image (processed by CCM2) and its corresponding ground truth value. If Ψ01 is greater than Ψ02, then CCM1 is considered superior to CCM2. In this way, different CCMs can be ranked according to human perception.
[0183] This invention provides a method for color correction. In this case, a multi-scale objective function is used for camera color correction.
[0184] In step 1110, after determining the value of Ψ, the electronic device determines the color correction matrix (CCM) based on the value of Ψ.
[0185] First, initialize the color correction matrix. The matrix can be initialized with any color coefficient values.
[0186] The evolutionary method NSGA can be used, which starts with a set of initial random candidates and then continues the search to find a better solution. The proposed method supports hot start, that is, the process can be started with a set of CCM candidates to facilitate the search at the beginning.
[0187] In step 1115, at each step of the iterative process, the color coefficients in the matrix are adjusted to generate display values. That is, the pixel dataset (e.g., the matrix consisting of the original sensor values) is multiplied by the CCM to generate a color-corrected pixel dataset (e.g., the matrix consisting of display values). In one embodiment, the color correction matrix may be a 3×6 matrix used to convert pixel data into color-corrected pixel data.
[0188] For example, at each step of the search, a set of CCM candidates is generated and passed through a synthesis pipeline or a real mobile phone to use these candidates to generate color-corrected pixel data.
[0189] In step 1120, using based , and To calculate the metric In one implementation, the calculation is performed by taking steps 410 and 415 as described above. .
[0190] In step 1125, an evaluation is performed to determine Is the value greater than a certain threshold? If not, the iteration process returns to step 1110, where the CCM is adjusted to continue additional iterations until the calculated color difference has reached the threshold. In other words, the CCM is tuned if the value of Ψ is not greater than the threshold.
[0191] In one implementation, iterative optimization is used to tune the CCM, with Ψ used as the target loss. Black-box optimization or evolutionary optimization can be used to optimize the CCM. The metric Ψ can be used as the target loss function for optimization. For example, the metric Ψ can be used as the target loss function for black-box optimization (e.g., Bayesian optimization, NeverGrad), and this invention is not limited thereto.
[0192] In one implementation, the evolutionary method NSGA is used. NSGA starts with a set of initial random CCM candidates and then continues the search to find a better solution. The proposed method supports hot start, i.e., the process can be started with a set of CCM candidates to facilitate the search at the beginning.
[0193] In one implementation, the sum of each row of the CCM equals a value. For example, that value is 1024.
[0194] In CCM it is 3 In the example of an N-matrix, the first (N-1) elements of each row can be searched. Then, the last element of each row will be calculated as... .
[0195]
[0196] In this way, hard constraints such as white retention on CCM elements will be satisfied.
[0197] In another implementation, elements in the CCM are within the lower and upper bounds. This way, soft constraints are satisfied.
[0198] In step 1130, if the value of Ψ is greater than the threshold, the final CCM of the candidate light source is output.
[0199] The iterative process will continue until a satisfactory CCM is achieved or the search budget is fully utilized.
[0200] After determining the CCM (Conversion Module), the ED (Electrical Device) captures a third image via the camera; the third image is then processed using the CCM to obtain a fourth image. The third image is the raw image captured by the sensor, and the fourth image is the image processed using the CCM.
[0201] For reference Figure 16 This illustrates a method for generating CCM according to an embodiment of the present invention. Figure 15 A schematic diagram of the steps. Training set 1305 includes a color checkerboard pattern. Image sensor 1310 acquires a raw pixel dataset by capturing images of the checkerboard pattern. The pixel dataset (e.g., a matrix composed of raw sensor values) is multiplied by CCM candidates 1315 to generate a color-corrected pixel dataset (e.g., a matrix composed of display values). CCM candidates 1315 are derived from CCM samples 1335. CCM samples 1335 are iteratively generated by black-box optimization module 1340 until the color difference between the color-corrected pixel dataset and the baseline ground truth dataset is minimized (i.e., the value of Ψ is maximized).
[0202] The CC target loss calculation module 1320 calculates the color difference (e.g., the value of Ψ) between the color-corrected pixel dataset (e.g., the color to be displayed after correction by CCM candidate 1315) and the baseline ground truth dataset (e.g., a checkerboard of colors). If the value of Ψ is not less than a certain threshold, the determination module determines the output target CCM 1330.
[0203] Those skilled in the art will recognize that any optimization algorithm can be used to find the minimum color difference, such as Newton's method, the simplex method, gradient descent, etc. Those skilled in the art will understand that the convergence of an optimization algorithm can depend on how the color correction matrix is initialized.
[0204] This invention provides another method for color correction implemented in a terminal device. The terminal device performs color correction using a color calibration mechanism (CCM). The CCM can be determined or obtained by employing the embodiments described above. For example, a smartphone can perform color correction using a CCM determined by a computing device.
[0205] refer to Figure 17 The diagram illustrates a flowchart of a method performed by a terminal device according to some embodiments of the present invention.
[0206] In step 1410, the terminal device captures an image of the first image using a camera.
[0207] In step 1415, the terminal device processes the first image through CCM to obtain the second image.
[0208] In some embodiments, the color difference (CCM) is obtained based on a first function Ψ, where the first function Ψ is used to determine the difference between a first color and a second color. The first color is a color in a first image, and the second color is a color in a second image. The color difference is determined based on the first color and the second color in the second image. Tonal difference and color difference To obtain the first function Ψ.
[0209] Since the methods for calculating Ψ and determining CCM have been described above, they will not be repeated here.
[0210] In the example, if the image capturing device cannot optimize the CCM (i.e., the CCM is fixed in the image capturing device), it can use an already optimized CCM. The CCM can be an ideal CCM obtained from a computing device. The computing device can optimize the CCM using the methods described above in the previous embodiments.
[0211] In another example, if the image capturing device (e.g., a professional camera) has CC flexibility and is capable of CCM tuning, the ideal CCM can be obtained by real-time adjustment using the method described above in the previous embodiments.
[0212] The term "terminal equipment" refers to any end device capable of wireless communication. By way of example and not limitation, terminal equipment may also be referred to as communication equipment, user equipment (UE), subscriber station (SS), portable subscriber station, mobile station (MS), or access terminal (AT). Terminal devices may include, but are not limited to, mobile phones, cellular phones, smartphones, voice over IP (VoIP) phones, wireless local loop phones, tablets, wearable terminal devices, personal digital assistants (PDAs), portable computers, desktop computers, image capture terminal devices (e.g., digital cameras), gaming terminal devices, music storage and playback devices, in-vehicle wireless terminal devices, wireless endpoints, mobile stations, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), USB dongles, smart devices, customer-premises equipment (CPE), Internet of Things (IoT) devices, watches or other wearable devices, head-mounted displays (HMDs), vehicles, drones, medical devices and applications (e.g., remote surgery), industrial devices and applications (e.g., robots and / or other wireless devices operating in the context of industrial and / or automated processing chains), consumer electronics devices, and devices operating in commercial and / or industrial wireless networks. The terminal device may also correspond to the mobile termination (MT) portion of an IAB node (e.g., a relay node). In the following description, the terms "terminal device," "communication device," "terminal," "user equipment," and "UE" are used interchangeably.
[0213] Table 2 shows the experimental results, in which Ψ and Ψ are used to optimize CCM, respectively.
[0214] Table 2
[0215] In the experimental setup, a camera sensitivity function from the SFU dataset was used; an X-rite SG 140 color chart was used as the training set for tuning the CC matrix; and five different parameter optimization methods were benchmarked according to the proposed CC task formula. A validation set of 1997 objects with different reflectivities was used in the SFU dataset for performance comparison.
[0216] As can be seen from Table 2, the use replace Not only do they achieve equal / better results in different tuning methods Average / maximum, and also significantly reduced hue shift ( Furthermore, relative to the validation results in the second row, Overall, robust color correction was achieved.
[0217] Some embodiments of the present invention provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium). The computer-readable storage medium stores program instructions that, when executed on an ED, cause the ED to perform one or more steps of the method for color error determination and / or color correction described in any of the above embodiments.
[0218] For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical disks (e.g., compact disks, CDs, or DVDs), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives). The various computer-readable storage media described in embodiments of the present invention can represent one or more devices and / or other machine-readable storage media for storing information. The term "computer-readable storage medium" can include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.
[0219] Some embodiments of the present invention also provide a computer program product. The computer program product includes program instructions carried on a non-transitory computer-readable storage medium. When executed on a network device / terminal device, the computer program instructions cause the network device / terminal device to perform one or more steps of the methods for color error determination and / or color correction described in the above embodiments.
[0220] The beneficial effects of the computer-readable storage medium and computer program product are the same as those of the methods for determining and / or correcting color errors described in the above embodiments, and will not be repeated here.
[0221] The foregoing description is merely a specific implementation of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions falling within the technical scope of the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
[0222] In some aspects of the invention, a computer program is provided, comprising instructions. When executed by a processor, the instructions cause the processor to implement the method of the invention.
[0223] In some aspects of the present invention, a chip is provided. The chip includes: a logic circuit; and a power supply circuit. The power supply circuit is used to supply power to the logic circuit. The logic circuit is used to perform the steps of the method of the present invention for determining and / or correcting color errors.
[0224] In some aspects of the invention, a device / chipset system is provided, including components (e.g., at least one processor) for implementing the methods of the invention. The device / chipset system may be a device (i.e., a terminal device or a network device) or a module or component within a device. The at least one processor may execute instructions stored in a computer-readable medium to implement the method.
[0225] A device can be an electronic device or a device implemented in an electronic device. For example, a device implemented in an electronic device can be an integrated circuit, which in some contexts may have other common names such as chip, modem, modem chip, baseband chip, or baseband processor. In some implementations, one or more integrated circuits may be packaged as a system-on-a-chip, system-in-package, or multi-chip module. A device may include one or more integrated circuits, or include one or more integrated circuits and other discrete components.
[0226] The solutions described in this invention are applicable to next-generation (e.g., sixth generation, 6G or above) networks, or traditional (e.g., 5G, 4G, 3G or 2G) networks.
[0227] It should be understood that any module, component, or device disclosing the executable instructions herein may include or otherwise access one or more non-transitory computer / processor-readable storage media for storing information (e.g., computer / processor-readable instructions, data structures, program modules, and / or other data). A non-exhaustive list of examples of non-transitory computer / processor-readable storage media includes magnetic tape cassettes, magnetic tape, disk storage devices or other magnetic storage devices such as compact disc read-only memory (CD-ROM), digital video disc or digital versatile disc (DVD), Blu-ray Disc™ and other optical storage devices, volatile and non-volatile, removable and non-removable media implemented in any method or technology, random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies. Any such non-transitory computer / processor storage medium may be part of a device or apparatus, or may be accessed or connected to a device or apparatus. Computer / processor-readable / executable instructions used to implement the methods, applications, or modules described herein may be stored by such non-transitory computer / processor-readable storage media or otherwise preserved.
[0228] It should be noted that the message in this invention can be replaced with information, and information can be carried in a single message or in more than one separate message.
[0229] The terms “device” and “equipment” are used interchangeably.
[0230] In this invention, when used in conjunction with the term "comprising" in the claims and / or specification, the word "a" may refer to "one," but it also has the same meaning as "one or more," "at least one," and "one or more," unless explicitly stated otherwise. Similarly, the word "another" may refer to at least a second or more, unless explicitly stated otherwise.
[0231] In this invention, when used before the same term (e.g., UE or operation step), the words "first," "second," etc., do not imply an order or sequence of terms. For example, unless otherwise specified, "first UE" and "second UE" refer to two different UEs; similarly, unless otherwise specified, "first step" and "second step" refer to two different operation steps, but this does not mean that the first step must occur before the second step. The actual order depends on the logic of the two steps.
[0232] The terms “coupling” or “connection” as used herein can have several different meanings depending on the context in which they are used. For example, depending on the specific context, the terms “coupling” or “connection” as used herein can mean two elements or devices that are directly connected to each other or connected to each other via one or more intermediate elements or devices through mechanical elements.
[0233] It should be noted that the expression "at least one of A or B" as used herein is interchangeable with the expression "A and / or B". It refers to a list in which either A or B, or both A and B, can be selected. Similarly, "at least one of A, B, or C" as used herein is interchangeable with "A and / or B and / or C" or "A, B, and / or C". It refers to a list in which the following can be selected: A or B or C, or both A and B, or both A and C, or both B and C, or all of A, B, and C. The same principle applies to longer lists with the same format.
[0234] This invention includes various embodiments, not only method embodiments but also other embodiments, such as apparatus embodiments and embodiments relating to non-transitory computer-readable storage media. Embodiments may be incorporated individually or in combination with the features disclosed herein.
[0235] As used herein, the terms “receive,” “detect,” and “decode” can have several different meanings depending on the context in which they are used. For example, without specific indication, the term “receive” can mean that information (e.g., DCI or MAC-CE, RRC signaling, or TB) has been successfully received by the receiving node, meaning that the receiving side correctly detected and decoded the information. In this scenario, “receive” can include both “detect” and “decode,” or it can mean the same thing, such as “receive paging” meaning correctly decoding a paging and successfully acquiring the paging; correspondingly, “received paging” means the receiving side did not detect and / or decode the paging. “Not received paging” means the receiving side attempted to detect and / or decode the paging but failed to acquire it. The term “receive” can sometimes mean that a signal has arrived at the receiving side, but this does not necessarily mean that the information in the signal has been correctly detected and decoded. In this case, the receiving side needs to perform detection and decoding on the signal to obtain the information carried in the signal. In this scenario, “receive,” “detect,” and “decode” can represent different processes of acquiring information at the receiving side. While this invention refers to illustrative embodiments, it is not intended to be interpreted in a limiting sense. Those skilled in the art will understand, upon referring to this description, various modifications and combinations of the illustrative embodiments and other embodiments of the invention. When combining two or more embodiments, not all features of the embodiments to be combined are necessary for the combination.
[0236] Additionally or alternatively, features disclosed herein in the context of any particular embodiment may be implemented in other embodiments. For example, method embodiments may be implemented in apparatus, system, and / or computer program product embodiments. Furthermore, although embodiments are described primarily in the context of methods and apparatus, other implementations are contemplated as instructions stored in one or more non-transitory computer-readable media, etc. Such media may store programs or instructions to perform any of the various methods consistent with the present invention.
Claims
1. A method for determining color error, characterized in that, include: Obtain the first color difference between the first color of the first pixel in the first image and the second color of the second pixel in the second image. Tonal difference and color difference The second color is the color to be corrected that corresponds to the first color; based on , and To determine the value of a first function Ψ, wherein the first function is used to determine the difference between the first color and the second color.
2. The method according to claim 1, characterized in that, The first color is the baseline true value color.
3. The method according to claim 1 or 2, characterized in that, After determining the value of Ψ, the process also includes: The color correction matrix (CCM) is determined based on the value of Ψ.
4. The method according to claim 3, characterized in that, Determining the CCM includes: If the value of Ψ is not greater than the threshold, the CCM is optimized.
5. The method according to claim 3 or 4, characterized in that, Optimizing the CCM includes: The CCM is tuned using iterative optimization, and the iterative optimization uses Ψ as the target loss.
6. The method according to any one of claims 3 to 5, characterized in that, Determining the CCM includes: If the value of Ψ is greater than the threshold, the CCM is output.
7. The method according to claim 6, characterized in that, Also includes: Capture a third image using a camera; The third image is processed by the CCM to obtain the fourth image.
8. The method according to any one of the preceding claims, characterized in that, The first function is a piecewise function with multiple segmented parts, each segment depending on the specific function. , or At least one of them; The first function is represented as: Where U represents the upper bound and L represents the lower bound. Indicates parameters; based on , and Determining the value of the first function Ψ includes: exist , or If at least one of them is within a first range, the first function satisfies ; exist , or In the case of at least one of the following within the second range, the first function satisfies ; exist , or If at least one of them is within the third range, then the first function satisfies ; exist , or If at least one of them is within the fourth range, then the first function satisfies ; exist , or If at least one of them is within the fifth range, then the first function satisfies ; exist , or If at least one of them is within the sixth range, then the first function satisfies .
9. The method according to claim 8, characterized in that, The first range, the second range, the third range, the fourth range, the fifth range, and the sixth range correspond to the first region, the second region, the third region, the fourth region, the fifth region, and the sixth region in the color space, respectively.
10. The method according to claim 8 or 9, characterized in that, In the color space, The first region is represented by a first approximate ellipse, which is located inside a second approximate ellipse; The second region, the third region, the fourth region, and the fifth region are located outside the first approximate ellipse and inside the second approximate ellipse; The sixth region is represented by the region in the color space excluding the region inside the second approximate ellipse.
11. The method according to any one of the preceding claims, characterized in that, exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; in, express The first threshold, express The second threshold, express The threshold.
12. The method according to any one of claims 8 to 11, characterized in that, z=0 or z=1.
13. The method according to any one of the preceding claims, characterized in that, Also includes: The color space is divided into multiple partitions based on multiple hues.
14. The method according to any one of claims 8 to 13, characterized in that, When the first color and the second color are in the same partition, z=1; If the first color and the second color are not in the same partition, z=0.
15. The method according to any one of claims 8 to 14, characterized in that, It is represented as: ,in ;or ;or ,in ;or ,in .
16. The method according to any one of claims 8 to 15, characterized in that, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; 。 17. The method according to claim 16, characterized in that, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; 。 18. The method according to any one of claims 8 to 17, characterized in that, L1=U2=U3=U4=U5=0.9; U6=L2=L3=L4=L5=0.4; U1=1; L6=0。 19. The method according to any one of claims 9 to 18, characterized in that, The color space is the LAB color space.
20. The method according to any one of claims 3 to 19, characterized in that, The sum of each row in the CCM equals a value.
21. The method according to claim 20, characterized in that, The value is 1024.
22. A method for color correction, characterized in that, include: The image of the first image is captured by the camera; The first image is processed using a color correction matrix (CCM) to obtain the second image; The CCM is obtained based on a first function Ψ, which is used to determine the difference between the first color and the second color. The first color is the color of the first pixel in the first image, and the second color is the color of the second pixel in the second image; The first function Ψ is based on the first color difference between the first color and the second color in the second image. Tonal difference and color difference To obtain.
23. The method according to claim 22, characterized in that, The first color is the baseline true value color.
24. The method according to claim 22 or 23, characterized in that, If the value of Ψ is not greater than the threshold, the CCM is optimized.
25. The method according to any one of claims 22 to 24, characterized in that, The CCM is tuned using iterative optimization, and the iterative optimization uses Ψ as the target loss.
26. The method according to any one of claims 22 to 25, characterized in that, If the value of Ψ is greater than the threshold, the CCM is output.
27. The method according to claim 26, characterized in that, Also includes: Capture a third image using a camera; The third image is processed by the CCM to obtain the fourth image.
28. The method according to any one of claims 22 to 27, characterized in that, The first function is a piecewise function with multiple segmented parts, each segment depending on the specific function. , or At least one of them; The first function is represented as: Where U represents the upper bound and L represents the lower bound. Indicates parameters; based on , and Determining the value of the first function Ψ includes: exist , or If at least one of them is within a first range, the first function satisfies ; exist , or In the case of at least one of the following within the second range, the first function satisfies ; exist , or If at least one of them is within the third range, then the first function satisfies ; exist , or If at least one of them is within the fourth range, then the first function satisfies ; exist , or If at least one of them is within the fifth range, then the first function satisfies ; exist , or If at least one of them is within the sixth range, then the first function satisfies .
29. The method according to claim 28, characterized in that, The first range, the second range, the third range, the fourth range, the fifth range, and the sixth range correspond to the first region, the second region, the third region, the fourth region, the fifth region, and the sixth region in the color space, respectively.
30. The method according to claim 28 or 29, characterized in that, In the color space, The first region is represented by a first approximate ellipse, which is located inside a second approximate ellipse; The second region, the third region, the fourth region, and the fifth region are located outside the first approximate ellipse and inside the second approximate ellipse; The sixth region is represented by the region in the color space excluding the region inside the second approximate ellipse.
31. The method according to any one of claims 22 to 30, characterized in that: exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; exist In this case, Represented as ; in, express The first threshold, express The second threshold, express The threshold.
32. The method according to any one of claims 28 to 31, characterized in that, z=0 or z=1.
33. The method according to any one of claims 22 to 32, characterized in that, Also includes: The color space is divided into multiple partitions based on multiple hues.
34. The method according to any one of claims 28 to 33, characterized in that, When the first color and the second color are in the same partition, z=1; If the first color and the second color are not in the same partition, z=0.
35. The method according to any one of claims 28 to 34, characterized in that, It is represented as: ,in ;or ;or ,in ;or ,in .
36. The method according to any one of claims 28 to 35, characterized in that, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; 。 37. The method according to claim 36, characterized in that, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; When the first color and the second color are not in the same partition When the first color and the second color are in the same partition, ; ; ; 。 38. The method according to any one of claims 28 to 37, characterized in that, L1=U2=U3=U4=U5=0.9; U6=L2=L3=L4=L5=0.4; U1=1; L6=0。 39. The method according to any one of claims 29 to 38, characterized in that, The color space is the LAB color space.
40. The method according to any one of claims 22 to 39, characterized in that, The sum of each row in the CCM equals a value.
41. The method according to claim 40, characterized in that, The value is 1024.
42. An apparatus, characterized in that, include: At least one processor; At least one memory coupled to the at least one processor, the at least one memory storing at least a portion of instructions that, when executed by the at least one processor, cause the at least one processor to implement the method according to any one of claims 1 to 21 or claims 22 to 41.
43. A computer-readable storage medium, characterized in that, It stores computer program instructions that, when executed by the computer's processing circuitry, cause the computer to implement the method according to any one of claims 1 to 21 or claims 22 to 41.
44. A computer program product, characterized in that, It has instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 21 or claims 22 to 41.
45. A chip system, characterized in that, include: Processing circuitry; A storage medium, wherein computer program instructions are stored thereon, which, when executed by the processing circuitry, cause the chip system to implement the method according to any one of claims 1 to 21 or 22 to 41.
