Image processing method and device

Abstract

An image processing method includes: performing content analysis on an input frame to generate at least one content analysis result of the input frame; a processing circuit determines an anti-blue light intensity level of the input frame according to the content analysis result of the input frame; and performing color correction on the input frame to generate an output frame in response to the determined anti-blue light intensity level of the input frame.

Description

[0001] Related references

[0002] This application claims priority to U.S. Provisional Application No. 63 / 145,551, filed on February 4, 2021, which is incorporated herein by reference. Technical Field

[0003] The present invention relates to blue light filtering design, and more particularly, to a method and apparatus for applying color correction to an input frame based on an anti-blue light intensity level derived from content analysis of the input frame. Background Art

[0004] Electromagnetic waves can be divided into ultraviolet light (wavelength between 10 nanometers (nm) and 400nm), visible light (wavelength between 400nm and 700nm), and infrared light (wavelength between 700nm and 1000nm). The wavelength range of 415nm-455nm has a short wavelength, high energy, and good penetration into the structure of the human eye. In other words, when blue light with a wavelength between 415nm and 455nm directly irradiates the retina, the structure of the human eye will be harmed by blue light. People operate more fancy and modern electronic products than ever before day after day, and are exposed to the blue light emitted by electronic products. Therefore, the long-term accumulation of blue light can cause eye fatigue, dryness, decreased vision, etc.

[0005] Conventional blue light filtering designs use a fixed anti-blue light intensity level for all frames and introduce significant color shifts. Therefore, an innovative blue light filtering design that can solve the blue light hazard problem with fewer side effects is needed. Summary of the invention

[0006] An object of the present invention is to provide a method and apparatus for color correction of an input frame based on a blue light resistance intensity level derived from content analysis of the input frame.

[0007] According to a first aspect of the present invention, an exemplary image processing method is disclosed. The exemplary image processing method includes: performing content analysis on an input frame to generate at least one content analysis result of the input frame; a processing circuit determines an anti-blue light intensity level of the input frame according to the at least one content analysis result of the input frame; and in response to the determined anti-blue light intensity level of the input frame, performing color correction on the input frame to generate an output frame.

[0008] According to a second aspect of the present invention, an exemplary non-transitory machine-readable medium is disclosed. Program code is stored in the non-transitory machine-readable medium. When the program code is loaded and executed by a processing circuit, the program code instructs the processing circuit to perform the following steps: controlling content analysis of an input frame to obtain at least one content analysis result of the input frame; determining an anti-blue light intensity level of the input frame according to at least one content analysis result of the input frame; and color correcting the input frame to generate an output frame in response to the determined anti-blue light intensity level of the input frame.

[0009] According to a third aspect of the present invention, an exemplary image processing device is disclosed. The exemplary image processing device includes a content analysis circuit, a processing circuit, and a color correction circuit. The content analysis circuit is used to perform content analysis on an input frame to generate at least one content analysis result of the input frame. The processing circuit is coupled to the content analysis circuit to determine the anti-blue light intensity level of the input frame according to the at least one content analysis result of the input frame. The color correction circuit is coupled to the processing circuit to perform color correction on the input frame in response to the determined anti-blue light intensity level of the input frame to generate an output frame.

[0010] These and other objects of the present invention will no doubt become apparent to those of ordinary skill in the art after reading the following detailed description of the preferred embodiments illustrated in the various drawings and figures. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A diagram showing an image processing apparatus according to an embodiment of the present invention.

[0012] Figure 2 A flowchart of an image processing method according to an embodiment of the present invention is shown.

[0013] Figure 3 A diagram showing an example of a spectrum of an input frame obtained by content analysis.

[0014] Figure 4 A diagram showing blue object detection results of an input frame obtained through content analysis.

[0015] Figure 5 is a diagram showing an example of a histogram of an input frame obtained by content analysis.

[0016] Figure 6 A diagram showing a regression line used by a blue light resistance intensity decider according to an embodiment of the present invention.

[0017] Figure 7 This is a diagram illustrating an embodiment of the present invention. Figure 2 The details of step 206 are shown in the flowchart.

[0018] Figure 8A diagram showing a blue light reduction line, a modified target chromaticity locus, a preferred region, and a blackbody radiation color temperature locus line on a chromaticity diagram according to an embodiment of the present invention.

[0019] Fig. 9 A diagram showing another image processing apparatus according to an embodiment of the present invention.

[0020] Fig.10 Graph showing performance comparison between the proposed blue light filter design using the modified target chromaticity locus for color correction and the blue light filter design using the black body radiation color temperature locus for color correction. DETAILED DESCRIPTION

[0021] Throughout the following description and claims, specific terms are used to refer to specific components. As will be appreciated by those skilled in the art, electronic device manufacturers may refer to a component by different names. This document does not intend to distinguish between components that differ in name but function. In the following description and claims, the terms "include" and "comprising" are used in an open-ended manner and should be interpreted as "including but not limited to..." In addition, the term "coupled" is intended to mean an indirect or direct electrical connection. Thus, if one device is coupled to another device, that connection may be through a direct electrical connection, or through an indirect electrical connection via other devices and connections.

[0022] Figure 1 1 is a diagram showing an image processing device according to an embodiment of the present invention. The image processing device 100 includes a processing circuit 102 and a storage device 104. In this embodiment, the processing circuit 102 can be implemented by a processor, a microcontroller unit, or any component with software execution capability. The storage device 104 is a machine-readable medium, which can be a volatile memory, a non-volatile memory, or any component with code storage capability. Please refer to Figure 1 , a program code PROG is stored in the storage device 104. When the program code PROG is loaded and executed by the processing circuit 102, the program code PROG will instruct the processing circuit 102 to execute the proposed intelligent blue light protection function. In other words, the proposed intelligent blue light protection function can be implemented by software.

[0023] Please combine Figure 2 refer to Figure 1 , Figure 2A flow chart of an image processing method according to an embodiment of the present invention is shown. The image processing device 100 can use the image processing method to implement blue light filtering. In step 202, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) performs content analysis on the input frame IMG_IN to generate at least one content analysis result of the input frame IMG_IN, or controls the content analysis of the input frame IMG_IN to obtain at least one content analysis result of the input frame IMG_IN. In step 204, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) determines the anti-blue light intensity level of the input frame IMG_IN according to the content analysis result of the input frame IMG_IN. In step 206, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) performs color correction on the input frame IMG_IN in response to the determined anti-blue light intensity level of the input frame IMG_IN to generate an output frame IMG_OUT.

[0024] When executing step 202, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) acts as at least a part (i.e., part or all) of the content analyzer. For example, but not limited to, the content analysis result obtained in step 202 may include a histogram of the input frame IMG_IN, a blue object detection result of the input frame IMG_IN, a spectrum of the input frame IMG_IN, blue light related information obtained by applying artificial intelligence (AI)-based image processing (e.g., deep learning-based image processing or machine learning-based image processing) to the input frame IMG_IN, or a combination thereof.

[0025] Figure 3A diagram showing an example of a spectrum of an input frame IMG_IN obtained by content analysis. The processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) can control the content analysis of the input frame IMG_IN to obtain the spectrum of the pre-displayed input frame IMG_IN. For example, the image processing device 100 is coupled to a measuring device for measuring the spectrum, which can be a spectrometer, an integrating sphere, or a chroma meter. Therefore, the processing circuit 102 (particularly the program code PROG running on the processing circuit 102) can obtain the spectrum of the input frame IMG_IN from the measuring device after instructing the measuring device to perform spectral measurement on the pre-displayed input frame IMG_IN. In some embodiments of the present invention, the measuring device can further measure the ratio of the harmful blue light spectrum to the full spectrum, the ratio of the harmful blue light spectrum to the blue light spectrum, and / or the ratio of the peak blue light spectrum to the peak non-blue light spectrum through the processing circuit 102 (particularly the program code PROG running on the processing circuit 102). The information related to the blue light spectrum provided by the measuring device under the control of the processing circuit 102 (especially the program code PROG running on the processing circuit 102) can be used to determine the intensity of the blue light hazard in the wavelength range of 415nm-455nm.

[0026] Figure 4 The schematic diagram shows the blue object detection result obtained by the processing circuit 102 (particularly the program code PROG running on the processing circuit 102) performing content analysis on the input frame IMG_IN. For example, a blue object 402, such as a blue car, can be identified from the input frame IMG_IN, and can provide reference information to determine the intensity of the blue light hazard in the wavelength range of 415nm-455nm.

[0027] Figure 5 1 is a diagram showing an example of a histogram of an input frame IMG_IN obtained by content analysis by the processing circuit 102 (particularly, the program code PROG running on the processing circuit 102). The histogram of the input frame IMG_IN consists of a blue channel histogram H_B, a green channel histogram H_G, and a red channel histogram H_R, and the blue channel histogram H_B can provide information for determining the intensity of blue light hazards in the wavelength range of 415nm-455nm.

[0028] When executing step 204, the processing circuit 102 (particularly the program code PROG running on the processing circuit 102) acts as an anti-blue light intensity determiner. In this embodiment, the processing circuit 102 (particularly the program code PROG running on the processing circuit 102) refers to the content analysis result and The authentication determines the blue light hazard index of the input frame IMG_IN, and then determines the anti-blue light intensity level of the input frame IMG_IN according to the blue light hazard index and the mapping function. For example, the mapping function can be represented by a regression line obtained from an experiment on a display panel. Figure 6 A diagram illustrating a regression line RL used by a blue light resistance intensity determiner according to an embodiment of the present invention is shown. The regression line RL defines a mapping between a blue light (BL) hazard index and a blue light resistance intensity level. Regarding the generation of the regression line RL, a large number of images are measured to find Certification indicators (i.e. compliance with Certified Blue Light Hazard Index), and the anti-blue light intensity level is fine-tuned through repeated trials and experiments on the display panel to avoid / mitigate color shift. The certified index and associated fine-tuned blue light intensity level can be recorded and used by Figure 6 The points A1, A2, A3, A4, A5 and A6 shown in FIG. 1 are represented. After obtaining the points A1-A6, the regression line RL used by the anti-blue light intensity determiner can be obtained by regression. For example, the regression line RL can be defined by a polynomial function Y=f(X), where Y represents the anti-blue light intensity level and X represents the blue light hazard index. Figure 6 As shown, the regression line RL is a curve, but the present invention is not limited thereto.

[0029] When executing step 206, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) acts as a blue light filter that adaptively controls the blue light level of each frame. Figure 8 refer to Figure 7 , Figure 7 This is a diagram illustrating an embodiment of the present invention. Figure 2 The details of step 206 are shown in the flowchart. Figure 8 A diagram showing a blue light reduction line on a chromaticity diagram, a modified target chromaticity trajectory, a preferred region, and a blackbody radiation color temperature trajectory line according to an embodiment of the present invention. In this embodiment, step 206 can be viewed as having steps 702-710. Therefore, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) performs steps 702-710 to apply color correction to the input frame IMG_IN to reduce the blue light hazard.

[0030] At step 702, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) determines a blue light reduction line L_BLR. As known to those skilled in the art, the curved edge of the human visual color gamut is called a spectrum locus L_SP, corresponding to monochromatic light (each point represents a pure hue of a single wavelength), where the wavelength is in nanometers. In the present embodiment, the blue light reduction line L_BLR is defined by a straight line passing through the white point P_W of the display panel and the blue light hazard boundary point P_B on the spectrum locus L_SP. For example, but not limited to, the blue light hazard boundary point P_B may have a wavelength of approximately 435nm. In the present embodiment, Figure 1 The processing circuit 102 shown can also receive panel information INF_P, which carries parameters of the display panel, such as panel gamma and three vertex values. Based on the panel information INF_P, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) knows the color gamut of the display panel (given by Figure 8 ) and the white point W_P of the display panel. For example, the white point P_W may have a chromaticity (x, y) = (0.3127, 0.3290). In step 704, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) searches for a first chromaticity point P1 along the blue light reduction line L_BLR according to the anti-blue light intensity level obtained in step 204. In step 706, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) modifies the first chromaticity point P1 to generate a second chromaticity point P2 within the preferred region RP to avoid / mitigate color shift. Figure 8 As shown, the preferred region RP is located between the blue light reduction line L_BLR and the black body radiation color temperature trajectory line L_CT. More specifically, the preferred region RP is delimited by the blue light reduction line L_BLR, the black body radiation color temperature trajectory line L_CT and the color gamut boundary of the display panel. In this embodiment, it is ensured that the second chromaticity point P2 found for different anti-blue light intensity levels (which indicates non-zero blue light filtering) is located on the modified target chromaticity trajectory L_TC within the preferred region RP, where the modified target chromaticity trajectory L_TC can be determined experimentally.

[0031] In one exemplary design, the initial color coordinates (x, y) = (x1, y1) on the chromaticity diagram are determined by finding the first chromaticity point P1 along the blue light reduction line L_BLR, and the final color coordinates (x, y) = (x2, y2) on the chromaticity diagram are determined by directly referencing the first chromaticity point P1 to find the second chromaticity point P2 on the modified target chromaticity trajectory L_TC. That is, the first chromaticity point P1 can be directly mapped to the second chromaticity point P2 without further adjustment.

[0032] In another exemplary design, the first chromaticity point P1 is used to determine the second chromaticity point P2 after a certain modification. For example, the initial color coordinate (x, y) = (x1, y1) on the chromaticity diagram is determined by searching for the first chromaticity point P1 along the blue light reduction line L_BLR, the first modified color coordinate (x, y) = (x3, y3) on the chromaticity diagram is obtained by modifying the initial color coordinate (x, y) = (x1, y1) according to the image color temperature of the input frame IMG_IN, the second modified color coordinate (x, y) = (x4, y4) on the chromaticity diagram is obtained by modifying the first modified color coordinate (x, y) = (x3, y3) according to the fine-tuning offset preferred by the user, and the final color coordinate (x, y) = (x2, y2) is determined by searching for the second chromaticity point P2 on the modified target chromaticity trajectory L_TC with reference to the second modified color coordinate (x, y) = (x4, y4).

[0033] It should be noted that the above description is for illustration only and does not constitute a limitation of the present invention. In practice, any algorithm-based blue light filtering design that derives a second chromaticity point P2 (limited to the preferred region RP) on the modified target chromaticity trajectory L_TC from the first chromaticity point P1 on the blue light reduction line L_BLR falls within the scope of the present invention.

[0034] In step 708, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) determines a color correction matrix according to the relationship between the chromaticity of the white point P_W (i.e., the color coordinates of the white point P_W on the chromaticity diagram) and the chromaticity of the second chromaticity point P2 (i.e., the color coordinates of the second chromaticity point P2 on the chromaticity diagram). For example, the chromaticity (x, y) of the white point P_W is P_W and the chromaticity of the second chromaticity point P2 (x,y) P2 The relationship between them can be expressed by the color correction matrix M corr To describe, that is, (x, y) P2 =M corr *(x,y) P_W .

[0035] In step 710, the processing circuit 102 (specifically, the program code PROG running on the processing circuit 102) performs pixel-by-pixel color correction on the input frame IMG_IN according to the color correction matrix to generate an output frame IMG_OUT. For example, a color-corrected frame with reduced blue light hazard is generated and output as the output frame IMG_OUT, which will be displayed on the display panel.

[0036] In the above embodiment, the proposed intelligent blue light protection function is implemented by software. However, this is only for illustrative purposes and does not mean a limitation of the present invention. Alternatively, the proposed intelligent blue light protection function can also be implemented by pure hardware.

[0037] Fig. 9 A diagram showing another image processing device according to an embodiment of the present invention. The image processing device 900 includes a content analysis circuit 902, a processing circuit 904, and a color correction circuit 906. The content analysis circuit 902 may be dedicated hardware for processing the operation specified in step 202. For example, the content analysis circuit 902 may include a spectral measurement device (e.g., a spectrometer, an integrating sphere, or a colorimeter), a blue object detection circuit, and / or a histogram calculation circuit. The processing circuit 904 may be dedicated hardware for processing the operation specified in step 204. The color correction circuit 906 may be dedicated hardware for processing the operations specified in step 206 and steps 702-710. Since a person of ordinary skill in the art can easily understand the details of the image processing device 900 after reading the above paragraphs, they will not be repeated here for the sake of brevity.

[0038] Fig.10 The performance comparison diagram between the proposed blue light filtering design using the modified target chromaticity trajectory L_TC for color correction and the blue light filtering design using the blackbody radiation color temperature trajectory line L_CT for color correction is shown. Characteristic curve L1 is the original spectrum of the input frame IMG_IN when the blue light is not filtered. Characteristic curve L2 shows the spectrum of the output frame IMG_OUT generated from the blue light filtering design using the blackbody radiation color temperature trajectory line L_CT for color correction. Characteristic curve L3 shows the spectrum of the output frame IMG_OUT generated from the proposed blue light filtering design using the modified target chromaticity trajectory L_TC for color correction. As described above, the embodiment of the present invention limits the second chromaticity point P2 to the preferred area RP between the blue light reduction line L_BLR and the blackbody radiation color temperature trajectory line L_CT. Compared with the blue light filtering design using the blackbody radiation color temperature trajectory line L_CT for color correction, the proposed blue light filtering design using the modified target chromaticity trajectory L_TC for color correction may have lower brightness loss and can be achieved through Certification.

[0039] In short, the proposed blue light filtering design can solve the blue light hazard problem with low color loss, low correlated color temperature variation, low brightness loss, satisfactory color gamut range and / or no additional cost of panel film / filter. In addition, the proposed blue light filtering design uses content analysis to adaptively control the blue light level of each frame. Therefore, the image processing device 100 / 900 can apply color correction with different anti-blue light intensity levels to multiple input frames respectively (i.e., content-aware blue light filtering).

[0040] Those skilled in the art will readily observe that various modifications and changes can be made to the apparatus and methods while retaining the teachings of the present invention.Accordingly, the above disclosure should be construed as being limited only by the metes and bounds of the appended claims.

Claims

1. An image processing method, comprising: Performing content analysis on an input frame to generate at least one content analysis result of the input frame; Determining, by a processing circuit, a blue light resistance intensity level of the input frame according to the at least one content analysis result of the input frame; and responsive to the determined blue light resistance intensity level of the input frame, color correcting the input frame to generate an output frame, The color correction of the input frame to generate the output frame comprises: Determine a blue light reduction line that passes through the white point and the blue light hazard boundary point on the spectrum locus; Finding a first chromaticity point along the blue light reduction line according to the determined blue light resistance intensity level of the input frame; and A color correction matrix used for the color correction is determined according to the first chromaticity point.

2. The image processing method according to claim 1, characterized in that: The at least one content analysis result comprises a histogram of the input frame.

3. The image processing method according to claim 1, characterized in that: The at least one content analysis result includes a blue object detection result of the input frame.

4. The image processing method according to claim 1, wherein: The at least one content analysis result comprises a spectrum of the input frame.

5. The image processing method according to claim 1, wherein: The at least one content analysis result includes Blue-light related information obtained by applying artificial intelligence-based image processing to the input frame.

6. The image processing method according to claim 1, characterized in that: Determining the color correction matrix used for the color correction according to the first chromaticity point includes: modifying the first chromaticity point to produce a second chromaticity point within a preferred region, wherein the preferred region is located between the blue light reduction line and a black body radiation color temperature locus; and The color correction matrix is ​​determined according to a relationship between the chromaticity of the white point and the chromaticity of the second chromaticity point.

7. The image processing method according to claim 1, characterized in that: The input frame is one of a plurality of input frames, and the image processing method applies the color correction having a plurality of different anti-blue light intensity levels to the plurality of input frames respectively.

8. A non-transitory machine-readable medium for storing program code, wherein when loaded and executed by a processing circuit, the program code instructs the processing circuit to perform the following steps: Controlling content analysis of an input frame to obtain at least one content analysis result of the input frame; determining a blue light resistance intensity level of the input frame according to the at least one content analysis result of the input frame; and responsive to the determined blue light resistance intensity level of the input frame, color correcting the input frame to generate an output frame, The step of performing color correction on the input frame to generate an output frame comprises: Determine a blue light reduction line that passes through the white point and the blue light hazard boundary point on the spectrum locus; searching for a first chromaticity point along the blue light reduction line according to the determined blue light resistance intensity level of the input frame; as well as A color correction matrix used for the color correction is determined according to the first chromaticity point.

9. The non-transitory machine-readable medium of claim 8, wherein: The at least one content analysis result comprises a histogram of the input frame.

10. The non-transitory machine-readable medium of claim 8, wherein: The at least one content analysis result includes a blue object detection result of the input frame.

11. The non-transitory machine-readable medium of claim 8, wherein: The at least one content analysis result comprises a spectrum of the input frame.

12. The non-transitory machine-readable medium of claim 8, wherein: The at least one content analysis result includes Blue-light related information obtained by applying artificial intelligence-based image processing to the input frame.

13. The non-transitory machine-readable medium of claim 8, wherein: Determining the color correction matrix used for the color correction according to the first chromaticity point includes: modifying the first chromaticity point to produce a second chromaticity point within a preferred region, wherein the preferred region is located between the blue light reduction line and a black body radiation color temperature locus; and The color correction matrix is ​​determined according to a relationship between the chromaticity of the white point and the chromaticity of the second chromaticity point.

14. The non-transitory machine-readable medium of claim 8, wherein: The input frame is one of a plurality of input frames, and the program code instructs the processing circuit to apply the color correction having a plurality of different anti-blue light intensity levels to the plurality of input frames respectively.

15. An image processing device, comprising: A content analysis circuit, configured to perform content analysis on an input frame to generate at least one content analysis result of the input frame; a processing circuit, coupled to the content analysis circuit, for determining a blue light resistance intensity level of the input frame according to the at least one content analysis result of the input frame; as well as a color correction circuit coupled to the processing circuit, for performing color correction on the input frame in response to the determined blue light resistance intensity level of the input frame to generate an output frame, wherein the color correction circuit is further configured to determine a blue light reduction line passing through a white point and a blue light hazard boundary point on the spectrum locus; Along the blue light reduction line, according to the determined anti-blue light intensity level of the input frame, a first chromaticity point is found; and according to the first chromaticity point, a color correction matrix adopted for the color correction is determined.

16. The image processing device according to claim 15, characterized in that: The at least one content analysis result comprises a histogram of the input frame.

17. The image processing device according to claim 15, wherein: The at least one content analysis result includes a blue object detection result of the input frame.

18. The image processing device according to claim 15, characterized in that: The at least one content analysis result comprises a spectrum of the input frame.

19. The image processing device according to claim 15, wherein: The at least one content analysis result includes Blue-light related information obtained by applying artificial intelligence-based image processing to the input frame.

20. The image processing device according to claim 15, wherein: The color correction circuit is also used to modify the first chromaticity point to produce a second chromaticity point within a preferred area, wherein the preferred area is located between the blue light reduction line and the blackbody radiation color temperature trajectory line; and to determine the color correction matrix based on the relationship between the chromaticity of the white point and the chromaticity of the second chromaticity point.

21. The image processing device according to claim 15, wherein: The input frame is one of a plurality of input frames, and the processing circuit applies the color correction having a plurality of different anti-blue light intensity levels to the plurality of input frames respectively.

Images