Color break-up degree quantification method and apparatus

Abstract

The present invention provides a color break-up degree quantification method and apparatus. The method comprises: determining an eye movement feature on the basis of an eye image acquired in real time, the eye movement feature comprising a real-time eye movement speed; extracting a plurality of image features of each pixel point of an input image of a display device, determining, on the basis of the quantitative relationship between each image feature and color break-up, a color break-up degree corresponding to each image feature, determining a color break-up degree of the pixel point on the basis of the color break-up degrees corresponding to the plurality of image features of the pixel point, and determining a threshold image on the basis of the color break-up degrees of all pixel points; and on the basis of the threshold image and the real-time eye movement speed, determining a color break-up degree of the display device when displaying the input image.

Description

Color separation degree quantification method and device

[0001] Cross-reference to Related Applications

[0002] This application claims priority to Chinese Patent Application No. 202410831881.5, filed on June 25, 2024, the contents of which are incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] Embodiments of the present application relate to the field of display technology, and in particular to a color separation degree quantification method and device. BACKGROUND

[0004] Compared with the traditional spatial color technology (backlight is white, red, green and blue (RGB) is realized by color filter), field sequential color display (backlight is RGB, RGB is refreshed in time sequence) reduces the use of color filter, has the advantages of high light source efficiency, high resolution and low cost, and can be applied to direct view display, projection display and head-mounted display, etc. In field sequential color display, each frame of image is divided into RGB three sub-fields and scanned in sequence, if the human eye rotates, the RGB components of the same point image scanned in time sequence will fall on different positions on the retina in a frame, therefore the viewer will perceive the color separation at the edge of the display target, i.e. color break-up phenomenon (CBU, Color Break-up). Color break-up phenomenon will cause visual fatigue, tension and interfere with visual experience, hindering the further development and application of field sequential display mode.

[0005] How to accurately quantify the subjective color break-up phenomenon is crucial for the elimination and evaluation of color break-up phenomenon. SUMMARY

[0006] Embodiments of the present application provide a color separation degree quantification method and device, which are used to solve the problem of how to accurately quantify the subjective color break-up phenomenon.

[0007] In order to solve the above technical problems, the present application is implemented as follows:

[0008] In a first aspect, embodiments of the present application provide a color separation degree quantification method, comprising:

[0009] According to the real-time collected eye image, determine the eye movement feature, the eye movement feature includes real-time eye movement speed;

[0010] extracting a plurality of image features of each pixel point of an input image of a display device, determining a color separation degree corresponding to each of the image features according to a quantitative relationship between each of the image features and color separation, determining a color separation degree of the pixel point according to the color separation degrees corresponding to the plurality of image features of the pixel point, and determining a threshold image according to the color separation degrees of all the pixel points;

[0011] determining a color separation degree of the display device when displaying the input image according to the threshold image and the real-time eye movement speed.

[0012] Optionally, the eye movement feature further includes an eye movement direction; and the method further includes:

[0013] determining a mask image according to the eye movement direction, the mask image including a gaze area and a non-gaze area, the gaze area being determined according to the eye movement direction;

[0014] wherein the determining of the color separation degree of the display device when displaying the input image according to the threshold image and the real-time eye movement speed includes:

[0015] determining a gaze area threshold image according to the threshold image and the mask image;

[0016] determining the color separation degree of the display device when displaying the input image according to the color separation degree of each pixel point of the gaze area threshold image and the real-time eye movement speed.

[0017] Optionally, the determining of the eye movement feature according to the real-time eye images includes:

[0018] determining whether a position of a pupil changes and the positions of the pupil before and after the change according to a plurality of real-time eye images;

[0019] determining a movement angle of an eyeball according to the positions of the pupil before and after the change;

[0020] determining a real-time eye movement speed of a user according to the movement angle of the eyeball.

[0021] Optionally, the determining of the eye movement feature according to the real-time eye images includes:

[0022] determining whether a position of a pupil changes and the positions of the pupil before and after the change according to a plurality of real-time eye images;

[0023] determining an eye movement direction according to the positions of the pupil before and after the change;

[0024] wherein the determining of the mask image according to the eye movement direction includes:

[0025] determining a target axis corresponding to the eye movement direction, the target axis having a starting point being a projection of the position of the pupil before the change on a plane where a display screen of the display device is located and a terminal point being a projection of the position of the pupil after the change on the plane where the display screen of the display device is located;

[0026] determining a plurality of circles with a preset radius and with an arbitrary point on the target axis as a center of each circle;

[0027] determining a gaze region of the mask image according to the plurality of circles, and determining a non-gaze region of the mask image according to a remaining region of the mask image.

[0028] Optionally, the determining the color separation degree of the pixel point according to the color separation degrees corresponding to the plurality of image features of the pixel point comprises:

[0029] determining a minimum value of the color separation degrees corresponding to the plurality of image features of the pixel point as the color separation degree of the pixel point.

[0030] Optionally, the image features comprise at least one of the following: contrast, saturation, hue and a target size in the input image.

[0031] Optionally, the quantitative relationship between each image feature and color separation is determined by the following method:

[0032] For each image feature, the following test steps are performed:

[0033] obtaining a plurality of test videos, each of the test videos corresponding to a specified eye movement speed;

[0034] obtaining a target test video in the plurality of test videos when the image feature is a specified value, and determining an eye movement threshold corresponding to the image feature being the specified value as an eye movement speed corresponding to the target test video, wherein color separation of a test image in the target test video is observed to be in a critical situation between visible and invisible when a user's eye is located at an out-pupil position;

[0035] changing a size of the value of the image feature and obtaining a corresponding eye movement threshold;

[0036] obtaining a quantitative relationship between the image feature and the eye movement threshold according to all values of the image feature and the eye movement threshold corresponding to each value;

[0037] obtaining a quantitative relationship between the image feature and color separation according to the quantitative relationship between the image feature and the eye movement threshold, wherein the eye movement threshold is negatively correlated with the color separation degree.

[0038] Optionally, when the test step is performed for one of the image features, other image features remain unchanged.

[0039] Optionally, the obtaining a plurality of test videos comprises:

[0040] generating a test video corresponding to a specified eye movement speed, wherein a position of a moving target in the test video in a test image of the nth frame satisfies the following relationship between the position and the specified eye movement speed:

[0041] frame_pos=f(FOV,L’,EP,D,w,RF,n,β,pitch)

[0042] wherein frame_pos is the position of the moving target in the test video in the test image of the nth frame;

[0043] FOV is a field of view of the display device;

[0044] L’ is an image distance of the display device;

[0045] EP is an exit pupil distance;

[0046] D is an eye-antenna distance;

[0047] W is the specified eye movement speed;

[0048] RF is a maximum supportable refresh rate of the display device;

[0049] n is a serial number of the test image in the test video;

[0050] β is a magnification of the display device;

[0051] pitch is a pixel pitch of the display device.

[0052] Optionally, when the hardware features of the display device are different, the quantitative relationship between the corresponding image features and the color separation is different.

[0053] Optionally, the image features include contrast, and the quantitative relationship between the contrast and the color separation comprises: the contrast and the color separation are positively correlated;

[0054] and / or

[0055] the image features include saturation, and the quantitative relationship between the saturation and the color separation comprises: the saturation and the color separation are negatively correlated;

[0056] and / or

[0057] The image features include a target size, and the quantitative relationship between the target size and the color separation includes that an influence of the target size on the color separation is related to an eye movement speed, when the eye movement speed is less than a preset threshold, the target size does not influence the color separation, and when the eye movement speed is greater than the preset threshold, the color separation degree decreases with the increase of the target size.

[0058] Optionally, the determining the gaze region threshold image according to the threshold image and the mask image comprises:

[0059] dividing values of each corresponding pixel point in the threshold image and the mask image to obtain the gaze region threshold image.

[0060] Optionally, the determining the color separation degree of the display device when displaying the input image according to the color separation degree of each pixel point in the gaze region threshold image and the real-time eye movement speed comprises:

[0061] calculating a difference between the real-time eye movement speed and an eye movement threshold corresponding to the color separation degree of the pixel point;

[0062] determining the color separation degree of the display device when displaying the input image according to the difference, wherein the greater the difference is, the higher the color separation degree of the input image is.

[0063] Optionally, the method further comprises:

[0064] extracting a color separation region or performing color separation suppression adjustment on the input image according to the color separation degree of the input image.

[0065] In a second aspect, an embodiment of the present application provides a color separation degree quantification device, comprising:

[0066] a first determining module configured to determine eye movement features according to a real-time collected eye image, wherein the eye movement features include a real-time eye movement speed;

[0067] a second determining module configured to extract a plurality of image features of each pixel point of an input image of a display device, determine a color separation degree corresponding to each image feature according to a quantitative relationship between each image feature and the color separation, determine a color separation degree of the pixel point according to color separation degrees corresponding to a plurality of image features of the pixel point, and determine a threshold image according to color separation degrees of all pixel points;

[0068] a third determining module configured to determine a color separation degree of the display device when displaying the input image according to the threshold image and the real-time eye movement speed.

[0069] In a third aspect, an electronic device is provided, which comprises a processor, a memory, and a program stored in the memory and capable of running on the processor. When the program is executed by the processor, the steps of the color separation degree quantification method according to the first aspect are implemented.

[0070] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the color separation degree quantification method according to the second aspect are implemented.

[0071] In a fifth aspect, a computer program product is provided, which comprises computer instructions. When the computer instructions are executed by a processor, the steps of the color separation degree quantification method according to the first aspect are implemented.

[0072] In the embodiments of the present application, the effects of image features and eye movement characteristics on color separation are comprehensively considered, and a comprehensive and accurate objective evaluation method for color separation degree is provided. BRIEF DESCRIPTION OF DRAWINGS

[0073] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments and are not intended to limit the scope of the application. Moreover, the same reference numerals are used throughout the drawings to designate similar or equivalent components. In the drawings:

[0074] FIG. 1 is a flowchart of a color separation degree quantification method according to an embodiment of the present application;

[0075] FIG. 2 is another flowchart of a color separation degree quantification method according to an embodiment of the present application;

[0076] FIG. 3 is a schematic diagram of a mask image determination method according to an embodiment of the present application;

[0077] FIG. 4 is a flowchart of a method for determining the quantitative relationship between each image feature and color separation according to an embodiment of the present application;

[0078] FIG. 5 is a schematic diagram of an eye tracking movement target rotating from P1 to P2;

[0079] FIG. 6 is a schematic diagram of an image after color separation region extraction according to the color separation degree of the image according to an embodiment of the present application;

[0080] FIG. 7 is a schematic diagram of an image before and after color separation suppression adjustment;

[0081] FIG. 8 is a schematic diagram of a color separation degree quantification device according to an embodiment of the present application;

[0082] FIG. 9 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0083] The technical solutions in the embodiments of the present application will be clearly and completely described with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of the present application.

[0084] How to accurately quantify the subjective color separation phenomenon is crucial for the elimination and evaluation of the color separation phenomenon. The specific reasons are as follows: (1) color separation only occurs when the eye movement, image, and refresh frequency meet certain conditions, and the degree of color separation is different when the values of the three change. In technical and product development, we want to obtain the degree of actual color separation to the vision in advance to proceed with the next step of work; (2) the existing methods to improve the color separation phenomenon include color gamut fitting, color gamut compression, and contrast self-adaptation. Among them, color gamut fitting is to calculate the minimum color gamut triangle containing all color points on the image. This method can weaken color separation without sacrificing picture color and increasing the number of fields, but it can only effectively suppress the color separation of images with small color gamut and the algorithm process is complex. Other methods (color gamut compression, contrast self-adaptation, etc.) sacrifice the color information of the image to achieve the purpose of eliminating color separation. How to balance the picture color and color separation elimination is the key to the application of the scheme; (3) increasing the refresh frequency can also weaken the color separation, but considering the hardware limitation and power consumption, a specific frequency increase needs to be made for a specific color separation degree. Therefore, whether it is the development of field sequential display related technology and products or the color separation suppression processing, obtaining real-time and accurate evaluation results of color separation is crucial.

[0085] Referring to FIGS. 1 and 2, the embodiment of the present application provides a color separation degree quantification method, which comprises:

[0086] Step 11: determining the eye movement characteristics according to the real-time collected eye image, wherein the eye movement characteristics include real-time eye movement speed;

[0087] Color separation occurs in the case of eye movement, and for the same hardware characteristics and image characteristics, the greater the eye movement speed, the more obvious the color separation. Therefore, it is necessary to detect the real-time eye movement characteristics to accurately quantify the actual color separation degree.

[0088] In the embodiment of the present application, referring to FIG. 3, a camera can be used to collect the eye image of the user in real time.

[0089] In the embodiment of the present application, the eye region in the eye image can be extracted using the eye feature information, and the eye region is taken as a region of interest for subsequent processing, and other regions outside the region of interest are not processed subsequently; the eye feature edge information in the eye region is extracted in combination with the eye feature, and the eye feature edge information includes the eyelid and sclera edge, the sclera and iris edge, the iris and pupil edge, etc., and each region of the eye is segmented according to the above eye feature edge information to obtain the pupil region (the position change of the pupil in the collected multiple eye images can indicate whether the eyeball moves or not); and the eye movement direction and real-time eye movement speed are determined according to the position change of the pupil in the adjacent multiple eye images.

[0090] The eye movement direction is the direction of the eyeball change, and the eye movement speed is the speed of the eyeball change towards the eye movement direction.

[0091] Optionally, when the eye movement speed is calculated, the movement angle of the eyeball (the movement angle of the eyeball is twice the visual angle C in FIG. 3) can be determined through the geometric relationship between the image coordinate system and the eyeball coordinate system, and the eye movement speed is the differential of the movement angle with respect to time. The visual angle C can be determined according to the positions of the pupil before and after the change. In FIG. 3, A is the projection of the pupil before the change on the display screen of the display device, B is the projection of the pupil after the change on the display screen of the display device, and the visual angle C can be determined according to the positions A and B.

[0092] That is, the eye movement feature is determined according to the real-time collected eye images, and the method comprises the following steps.

[0093] The positions of the pupil before and after the change are determined according to the real-time collected multiple eye images.

[0094] The movement angle of the eyeball is determined according to the positions of the pupil before and after the change.

[0095] The real-time eye movement speed of the user is determined according to the movement angle of the eyeball.

[0096] Step 12: multiple image features of each pixel point of the input image of the display device are extracted, the color separation degree corresponding to each image feature is determined according to the quantitative relationship between each image feature and color separation, the color separation degree of the pixel point is determined according to the color separation degrees corresponding to the multiple image features of the pixel point, and the threshold image is determined according to the color separation degrees of all pixel points.

[0097] Optionally, in the embodiment of the present application, the image features include at least one of the following: contrast, saturation, hue and target size in the input image.

[0098] The hue can be defined as the relative light and dark degree of the image, and is expressed as a color on a color image. The saturation can be defined as the chroma divided by the lightness, and is the same as the chroma, which represents the degree of color deviation from the same brightness gray.

[0099] Referring to FIG. 2, the above-mentioned "determining the color separation degree corresponding to each image feature according to the quantitative relationship between each image feature and color separation" can also be referred to as a feature space conversion step.

[0100] Optionally, determining the color separation degree of the pixel point according to the color separation degrees corresponding to the multiple image features of the pixel point comprises: taking the minimum value in the color separation degrees corresponding to the multiple image features of the pixel point as the color separation degree of the pixel point.

[0101] Optionally, determining the threshold image according to the color separation degrees of all pixel points comprises: obtaining the color separation degrees of all pixel points in the input image and performing denormalization processing to obtain the threshold image. Wherein, the denormalization refers to the transformation process from the color separation degree (the value range is 0 to 1) to the threshold image (the value range is about 0-1000 degrees / second).

[0102] It should be noted that in the embodiment of the present application, the quantitative relationship between the image features and the color separation corresponding to the hardware features of the display device is different.

[0103] Step 13: determining the color separation degree of the display device when displaying the input image according to the threshold image and the real-time eye movement speed.

[0104] In the embodiment of the present application, the influence of image features and eye movement characteristics on color separation is comprehensively considered, and a comprehensive and accurate objective evaluation method for color separation degree is given.

[0105] Optionally, in the embodiment of the present application, the eye movement features further include an eye movement direction; the method further comprises: determining a mask image according to the eye movement direction, the mask image comprising a fixation area and a non-fixation area, and the fixation area being determined according to the eye movement direction.

[0106] Optionally, determining the color separation degree of the display device when displaying the input image according to the threshold image and the real-time eye movement speed comprises:

[0107] Determining a fixation area threshold image according to the threshold image and the mask image;

[0108] Determining the color separation degree of the display device when displaying the input image according to the color separation degree of each pixel point of the fixation area threshold image and the real-time eye movement speed.

[0109] The color separation is not only related to the eye movement speed, but also related to the identification ability of the eye, such as color resolution and spatial resolution, and when the color separation occurs and the eye cannot identify, the color separation does not interfere with us. The resolution of the human eye is related to the viewing angle, and the mask image is the viewing angle area with high resolution of the human eye, that is, the fixation area (also called high-definition area or eye movement area along the eye movement direction), and only the color separation degree in the area is concerned.

[0110] In the embodiment of the application, as shown in FIG. 3, whether the eye movement occurs can be determined according to the position change of the pupil, and the eye movement direction can be determined according to the positions of the pupil before and after the change.

[0111] That is, the eye movement feature is determined according to the real-time collected eye image, and the method comprises the following steps:

[0112] The position of the pupil is determined according to the real-time collected multiple eye images, whether the position of the pupil changes, and the positions of the pupil before and after the change;

[0113] The eye movement direction is determined according to the positions of the pupil before and after the change;

[0114] The mask image is determined according to the eye movement direction, and the method comprises the following steps:

[0115] A target axis corresponding to the eye movement direction is determined, the starting point of the target axis is the projection of the position of the pupil before the change on the plane of the display screen of the display device (that is, A in FIG. 3), and the ending point is the projection of the position of the pupil after the change on the plane of the display screen of the display device (that is, B in FIG. 3);

[0116] Multiple circles are determined, which are formed with a preset radius as the center and the radius of any point on the target axis; wherein the preset value can be determined according to the characteristics of the human eye.

[0117] The fixation area of the mask image is determined according to the multiple circles, and the remaining area of the mask image is taken as the non-fixation area.

[0118] The fixation area can also be called a high-definition area in the process of eye movement (general movement, including fixation process), that is, the color and spatial resolution ability of the area is strong, and away from the area is weakened. This is determined by the distribution of cells on the retina of the eye.

[0119] In the embodiment of the application, optionally, in the mask image, the pixel value of the fixation area can be set to 1, and the pixel value of the non-fixation area can be set to 0.

[0120] Optionally, the threshold image of the fixation area = threshold image / mask image.

[0121] That is, the gaze region threshold image is determined according to the threshold image and the mask image, including: dividing the value of each corresponding pixel point in the threshold image and the mask image to obtain the gaze region threshold image.

[0122] Assuming that in the mask image, the pixel value of the gaze region is set to 1, and the pixel value of the non-gaze region is set to 0, the value of the corresponding non-gaze region pixel point of the gaze region threshold image obtained after the threshold image and the mask image are divided is 0, the non-gaze region does not participate in the evaluation calculation of the color separation degree, and actually, the data in the corresponding "threshold image" of the non-gaze region is discarded, only the data of the gaze region participates in the evaluation of the color separation degree, and the evaluation result is more in line with the actual situation considering the color resolution ability difference of the human eye under different viewing angles.

[0123] In the embodiment of the application, the eye movement threshold is negatively correlated with the color separation degree; and the color separation degree of the display device when displaying the input image is determined according to the color separation degree of each pixel point of the gaze region threshold image and the real-time eye movement speed, including:

[0124] The difference between the real-time eye movement speed and the eye movement threshold corresponding to the color separation degree of the pixel point is calculated;

[0125] The color separation degree of the display device when displaying the input image is determined according to the difference, wherein the greater the difference, the higher the color separation degree of the input image.

[0126] In the embodiment of the application, the color separation degree quantification method is not only suitable for natural images, but also compatible with any color target (except black) on a white background and high-saturation red, green and blue (color separation does not occur, but general methods cannot achieve simultaneous compatibility). And for the same image, when the eye's region of interest is different, the color separation degree quantification result is also different.

[0127] In the embodiment of the application, the color separation degree quantification method further includes determining the quantitative relationship between each image feature and color separation.

[0128] In the embodiment of the application, the quantitative relationship between each image feature and color separation is determined, including:

[0129] For each image feature, the following test steps are performed:

[0130] Step 41: Obtain a plurality of test videos, each of which corresponds to a specified eye movement speed;

[0131] The test video in the embodiment of the present application may be, for example, a video of a moving white bar on a black background, or a video of a moving black bar on a white background, etc.

[0132] Step 42: When the image feature is a specified value, a target test video in the plurality of test videos is obtained, and an eye movement speed corresponding to the target test video is taken as an eye movement threshold corresponding to the image feature being the specified value; wherein, when the user's eye is located at an exit pupil position, color separation of a test image in the target test video is observed to be in a critical condition between visible and invisible;

[0133] The aperture stop of an optical system forms an image in the image space of the optical system, which is called the "exit pupil" of the system. The exit pupil position (represented by the exit pupil distance) and diameter (represented by the exit pupil diameter) represent the position and aperture of the exit beam.

[0134] Step 43: The size of the value of the image feature is changed, and the corresponding eye movement threshold is obtained. According to all values of the image feature and the eye movement threshold corresponding to each value, a quantitative relationship between the image feature and the eye movement threshold is obtained.

[0135] Step 44: According to the quantitative relationship between the image feature and the eye movement threshold, a quantitative relationship between the image feature and the color separation is obtained, wherein the eye movement threshold is negatively correlated with the color separation degree.

[0136] In the embodiment of the present application, the law that the eye movement threshold is negatively correlated with the color separation degree is used to represent the current subjective color separation degree, that is, the smaller the eye movement threshold, the greater the color separation degree.

[0137] In the embodiment of the present application, when the test step is performed for one image feature (variable), other image features (variables) remain unchanged, thereby avoiding the influence of other variables, that is, the control variable method.

[0138] In the embodiment of the present application, ① Unlike the subjective scoring evaluation of color separation degree in existing research (the influence degree of color separation phenomenon (CBU) on visual effect is divided into 4 levels (1 not perceptible, 2 perceptible but not annoying, 3 perceptible and annoying, 4 obvious)), the subjective phenomenon is converted into the objective variable "eye movement threshold" to represent it, which is more intuitive and reduces individual subjective differences, and can be directly used for subsequent quantitative calculation of color separation degree; ② Multiple factors (multiple image features) affecting the color separation degree are considered, and the result is more accurate and more universal.

[0139] In the above embodiment, it is mentioned that the quantitative relationship between the image feature and the color separation is different when the hardware features of the display device are different. Therefore, when the above test step is tested, certain hardware features need to be set.

[0140] The hardware features include, for example, at least one of resolution, pixel pitch, image distance L' of the imaging system, exit pupil distance EP, magnification β, field of view angle FOV, maximum supportable refresh rate RF, color field sequence, etc.

[0141] In some embodiments, the above test step can have the following test conditions: the experiment is performed using a field-sequential liquid crystal near-eye display device with a resolution of 4k*4k, the pixel pitch of the field-sequential liquid crystal near-eye display device is known, the image distance L' of the imaging system, the exit pupil distance EP, the magnification β, the field of view angle FOV are known, the maximum supportable refresh rate of the field-sequential liquid crystal near-eye display device is RF, the color field sequence is fixed (such as RGB cycle), and the human eye anteroposterior diameter D is known.

[0142] In the embodiments of the present application, optionally, the obtaining a plurality of test videos comprises: generating a test video corresponding to a specified eye movement speed, and a position of a moving target in the test video in an nth frame of test image satisfies the following relationship between the specified eye movement speed:

[0143] frame_pos=f(FOV,L’,EP,D,w,RF,n,β,pitch)

[0144] Wherein, frame_pos is the position of the moving target in the test video in the nth frame of test image;

[0145] FOV is the field of view angle of the display device;

[0146] L' is the image distance of the display device;

[0147] EP is the exit pupil distance;

[0148] D is the human eye anteroposterior diameter;

[0149] W is the specified eye movement speed;

[0150] RF is the maximum supportable refresh rate of the display device;

[0151] n is the serial number of the test image in the test video;

[0152] β is the magnification of the display device;

[0153] pitch is the pixel pitch of the display device.

[0154] Wherein, n is an integer greater than or equal to 1.

[0155] That is, in the embodiment of the present application, the eye movement speed is controlled by testing the speed of the moving target in the video, and according to the relationship between the eye movement speed w and the position frame_pos of the moving target in the nth test image, the test video corresponding to the moving target corresponding to any eye movement speed w can be generated, and when the eye follows the moving target to move, the real-time eye movement speed can be known during the experiment. That is, the real-time passive induced eye movement speed can be obtained without the eyetracking technology. Please refer to FIG. 5, which shows a schematic diagram of eye tracking of a moving target rotating from P1 to P2, wherein the moving target moves from position ① to position ②, L' is the image distance of the display device, EP is the exit pupil distance, and D is the anteroposterior diameter of the human eye.

[0156] In the embodiment of the present application, optionally, the image feature includes contrast, and the quantitative relationship between the contrast and the color separation includes that the contrast and the color separation are positively correlated.

[0157] Please refer to Table 1:

[0158] Table 1

[0159] The values of the variables are shown in Table 1, the target contrast C is adjusted, and the eye movement threshold ω is recorded th , to obtain N(ω th )=aln(C)+b, wherein a and b are constants, N(ω th ) represents the normalized subjective color separation degree for ω th , and ln is the natural logarithm with e as the base. The results show that the contrast and the color separation are positively correlated.

[0160] In the embodiment of the present application, optionally, the image feature includes saturation, and the quantitative relationship between the saturation and the color separation includes that the saturation and the color separation are negatively correlated.

[0161] Please refer to Table 2:

[0162] Table 2

[0163] The values of the variables are shown in Table 2, the target saturation S under different hues is adjusted, and the eye movement threshold ω is recorded th , to obtain N(ω th )=a1S 4 +a2S 3 +a3S 2 +a4S+a5, S≥S0, wherein a1, a2, a3, a4, and a5 are constants, and N(ω th ) represents the normalized subjective color separation degree for ω th . The results show that when the saturation is greater than S0, the saturation and the color separation are negatively correlated.

[0164] Optionally, in the embodiment of the present application, the image feature includes a target size, and the quantitative relationship between the target size and the color separation includes that the influence of the target size on the color separation is related to the eye movement speed, when the eye movement speed is less than a preset threshold, the target size does not affect the color separation, and when the eye movement speed is greater than the preset threshold, the color separation degree decreases with the increase of the target size.

[0165] Please refer to Table 3:

[0166] Table 3

[0167] The values of the variables are shown in Table 3, and the target size (in the embodiment of the present application, the horizontal size of the target is focused on because the human eye is more sensitive in the horizontal direction) T is adjusted size and the eye movement threshold ω is recorded th , and the following is obtained where b0, ω0, k, and const are constants, and N(ω th ) represents the normalized subjective color separation degree for ω th . The results show that the influence of the target width on the color separation is related to the eye movement, when the eye movement speed is less than the value ω0, the target size does not affect the color separation, and when the eye movement speed is greater, the color separation decreases with the increase of the target size.

[0168] In the embodiment of the present application, compared with the existing sub-frame offset color separation simulation method, the present method directly calculates in the RGB space according to the experimental results, and has a smaller time complexity; it is compatible with any image and any eye movement condition, and makes up for the shortcomings of the existing research that cannot be applied to all images and only targets at fixed eye movement paths.

[0169] Optionally, in the embodiment of the present application, the color separation degree quantification method further includes extracting a color separation region of the input image according to the color separation degree of the input image. Please refer to FIG. 6, which is a schematic diagram of the color separation region extraction of the image after the color separation degree of the image is extracted. Since when observing the color separation phenomenon, some specific regions in the image are obvious, and other regions have no color separation, the color separation region extraction result can quickly locate the region where the color separation occurs, which can be used whether for actual observation of color separation or for color separation suppression.

[0170] Optionally, the color separation degree quantification method further comprises: color separation suppression adjustment of the input image according to the color separation degree of the input image. In the embodiment of the present application, the quantification result of the color separation degree acquired in real time is taken as the basis for color separation suppression adjustment, so as to achieve the purpose of color separation suppression with the least color sacrifice or hardware consumption. The color separation suppression adjustment can be a color gamut fitting, color gamut compression or contrast self-adaptation color separation suppression adjustment method. Please refer to FIG. 7, (1) in FIG. 7 is the original image without color separation suppression adjustment, (2) in FIG. 7 is the image after color separation suppression adjustment of the image according to the color separation degree determined by the present application, and (3) in FIG. 7 is the image after color separation suppression adjustment of the image according to the color separation degree determined by the existing sub-frame offset color separation simulation method. As can be seen from FIG. 7, the present application can achieve the purpose of color separation suppression with the least color sacrifice or hardware consumption.

[0171] Please refer to FIG. 8, the embodiment of the present application further provides a color separation degree quantification device 80, comprising:

[0172] A first determination module 81 is configured to determine an eye movement feature according to an eye image acquired in real time, wherein the eye movement feature comprises a real-time eye movement speed;

[0173] A second determination module 82 is configured to extract a plurality of image features of each pixel point of an input image of a display device, determine a color separation degree corresponding to each image feature according to a quantitative relationship between each image feature and color separation, determine a color separation degree of the pixel point according to the color separation degrees corresponding to the plurality of image features of the pixel point, and determine a threshold image according to the color separation degrees of all pixel points.

[0174] A third determination module 83 is configured to determine a color separation degree of the display device when displaying the input image according to the threshold image and the real-time eye movement speed.

[0175] Optionally, the eye movement feature further comprises an eye movement direction; and the color separation degree quantification device 80 further comprises:

[0176] A fourth determination module is configured to determine a mask image according to the eye movement direction, wherein the mask image comprises a gaze area and a non-gaze area, and the gaze area is determined according to the eye movement direction.

[0177] The third determination module 83 is configured to determine a gaze area threshold image according to the threshold image and the mask image, and determine a color separation degree of the display device when displaying the input image according to the color separation degree of each pixel point of the gaze area threshold image and the real-time eye movement speed.

[0178] Optionally, the first determining module 81 is configured to determine whether the position of the pupil changes and the positions of the pupil before and after the change according to a plurality of eye images collected in real time; determine the movement angle of the eyeball according to the positions of the pupil before and after the change; and determine the real-time eye movement speed of the user according to the movement angle of the eyeball.

[0179] Optionally, the first determining module 81 is configured to determine whether the position of the pupil changes and the positions of the pupil before and after the change according to a plurality of eye images collected in real time; determine the movement angle of the eyeball according to the positions of the pupil before and after the change; and determine the real-time eye movement speed of the user according to the movement angle of the eyeball.

[0180] Optionally, the fourth determining module is configured to determine a target axis corresponding to the eye movement direction, wherein the starting point of the target axis is the projection of the position of the pupil before the change on the plane of the display screen of the display device, and the ending point of the target axis is the projection of the position of the pupil after the change on the plane of the display screen of the display device; determine a plurality of circles formed with a preset radius as the center of any point on the target axis and with the preset radius as the radius; and determine the fixation area of the mask image according to the plurality of circles, and the remaining area of the mask image as the non-fixation area.

[0181] Optionally, the second determining module 82 is configured to take the minimum value in the color separation degrees corresponding to the plurality of image features of the pixel point as the color separation degree of the pixel point.

[0182] Optionally, the image features include at least one of the following: contrast, saturation, hue, and target size in the input image.

[0183] Optionally, the color separation degree quantifying apparatus 80 further comprises:

[0184] A fifth determining module is configured to determine the quantitative relationship between each image feature and color separation.

[0185] Optionally, the fifth determining module is configured to perform the following test steps for each image feature:

[0186] Obtain a plurality of test videos, and each test video corresponds to a specified eye movement speed;

[0187] When the image feature is a specified value, obtain a target test video in the plurality of test videos, take the eye movement speed corresponding to the target test video as the eye movement threshold corresponding to the image feature being the specified value, and wherein the color separation of the test image in the target test video is observed to be in a critical situation between visible and invisible when the user's eye is located at the out-pupil position.

[0188] Change the size of the value of the image feature, and obtain the corresponding eye movement threshold;

[0189] According to all values of the image features and the eye movement threshold corresponding to each value, a quantitative relationship between the image features and the eye movement threshold is obtained;

[0190] According to the quantitative relationship between the image features and the eye movement threshold, a quantitative relationship between the image features and color separation is obtained, wherein the eye movement threshold is negatively correlated with the degree of color separation.

[0191] Optionally, when the test step is performed for one of the image features, other image features remain unchanged.

[0192] Optionally, the obtaining of the plurality of test videos comprises:

[0193] A test video corresponding to a specified eye movement speed is generated, and a position of a moving target in the test video in a test image of the nth frame satisfies the following relationship between the position and the specified eye movement speed: frame_pos=f(FOV, L', EP, D, w, RF, n, β, pitch)

[0194] Wherein, frame_pos is the position of the moving target in the test video in the test image of the nth frame;

[0195] FOV is the field of view of the display device;

[0196] L' is the image distance of the display device;

[0197] EP is the exit pupil distance;

[0198] D is the front-back diameter of the human eye;

[0199] W is the specified eye movement speed;

[0200] RF is the highest supportable refresh rate of the display device;

[0201] n is the serial number of the test image in the test video;

[0202] β is the magnification of the display device;

[0203] pitch is the pixel pitch of the display device.

[0204] Optionally, when the hardware features of the display device are different, the quantitative relationship between the corresponding image features and color separation is different.

[0205] Optionally, the image features include contrast, and the quantitative relationship between the contrast and color separation comprises: the contrast is positively correlated with the color separation;

[0206] and / or

[0207] The image feature comprises saturation, and the quantitative relationship between the saturation and the color separation comprises that the saturation is negatively correlated with the color separation.

[0208] And / or

[0209] The image feature comprises target size, and the quantitative relationship between the target size and the color separation comprises that the influence of the target size on the color separation is related to the eye movement speed, when the eye movement speed is less than a preset threshold, the target size does not affect the color separation, and when the eye movement speed is greater than the preset threshold, the color separation degree decreases with the increase of the target size.

[0210] Optionally, the third determining module 83 is configured to divide the value of each corresponding pixel point in the threshold image and the mask image to obtain the gaze area threshold image.

[0211] Optionally, the third determining module 83 is configured to calculate a difference value between the real-time eye movement speed and the eye movement threshold corresponding to the color separation degree of the pixel point; and determine the color separation degree of the display device when displaying the input image according to the difference value, wherein the greater the difference value is, the higher the color separation degree of the input image is.

[0212] Optionally, the color separation degree quantifying apparatus 80 further comprises:

[0213] An application module configured to perform extraction of a color separation region or color separation suppression adjustment on the input image according to the color separation degree of the input image.

[0214] Please refer to FIG. 9, the embodiment of the present application further provides an electronic device 90, comprising a processor 91, a memory 92, a computer program stored in the memory 92 and executable on the processor 91, which realizes each process of the color separation degree quantifying method embodiment and achieves the same technical effect when the processor 91 executes the computer program, and details are not repeated here to avoid repetition.

[0215] The embodiment of the present application further provides a computer readable storage medium, which stores a computer program, and each process of the color separation degree quantifying method embodiment is realized and the same technical effect is achieved when the processor executes the computer program, and details are not repeated here to avoid repetition. The computer readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0216] The embodiment of the present application further provides a computer program product comprising computer instructions which, when executed by a processor, implement each process of the method embodiment shown in Fig. 1 and achieve the same technical effects. To avoid repetition, details are not described herein.

[0217] It should be noted that in this document, the terms "comprising", "including", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles, or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or inherent to such processes, methods, articles, or devices. Without more limitations, the element defined by the phrase "comprising a" does not exclude the presence of other identical elements in the process, method, article, or device that includes the element.

[0218] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and a necessary general hardware platform, and of course, they can also be realized by hardware, but in many cases, the former is a better embodiment. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a number of instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0219] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not limiting. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope of protection of the claims, and all of them belong to the protection of the present application.

Claims

1. A method for quantifying the degree of color separation, characterized in that, include: Based on real-time acquired eye images, eye movement features are determined, including real-time eye movement velocity; Extract multiple image features of each pixel in the input image of the display device, determine the degree of color separation corresponding to each image feature based on the quantitative relationship between each image feature and color separation, determine the degree of color separation of the pixel based on the degree of color separation corresponding to the multiple image features of the pixel, and determine a threshold image based on the degree of color separation of all pixels. The degree of color separation of the display device when displaying the input image is determined based on the threshold image and the real-time eye movement speed.

2. The method according to claim 1, characterized in that, The eye movement features also include eye movement direction; the method further includes: A mask image is determined based on the eye movement direction, the mask image including a fixation area and a non-fixation area, the fixation area being determined based on the eye movement direction; The determination of the color separation degree of the display device when displaying the input image, based on the threshold image and the real-time eye movement velocity, includes: Determine the gaze region threshold image based on the threshold image and the mask image; The degree of color separation of the display device when displaying the input image is determined based on the degree of color separation of each pixel in the gaze region threshold image and the real-time eye movement speed.

3. The method according to claim 1, characterized in that, The process of determining eye movement features based on real-time acquired eye images includes: Based on multiple real-time captured eye images, determine whether the position of the pupil has changed and the position of the pupil before and after the change; The angle of eye movement is determined based on the position of the pupil before and after the change; The user's real-time eye movement speed is determined based on the eye movement angle.

4. The method according to claim 2, characterized in that, The process of determining eye movement features based on real-time acquired eye images includes: Based on multiple real-time captured eye images, determine whether the position of the pupil has changed and the position of the pupil before and after the change; Determine the direction of eye movement based on the position of the pupil before and after the change; Determining the mask image based on the eye movement direction includes: The target axis corresponding to the eye movement direction is determined. The starting point of the target axis is the projection of the position of the pupil before the change onto the plane where the display screen of the display device is located, and the ending point is the projection of the position of the pupil after the change onto the plane where the display screen of the display device is located. Determine multiple circles centered on any point on the target axis and with a preset radius; The gaze region of the mask image is determined based on the plurality of circles, and the remaining region of the mask image is defined as the non-gaze region.

5. The method according to claim 1, characterized in that, Determining the color separation degree of a pixel based on the color separation degree corresponding to multiple image features of the pixel includes: The minimum value among the color separation degrees corresponding to multiple image features of the pixel is taken as the color separation degree of the pixel.

6. The method according to claim 1, characterized in that, The image features include at least one of the following: contrast, saturation, hue, and target size in the input image.

7. The method according to claim 1 or 6, characterized in that, The quantitative relationship between each of the image features and color separation is determined in the following manner: For each of the aforementioned image features, perform the following test steps: Acquire multiple test videos, each of which corresponds to a specific eye movement rate; When the image feature is obtained as a specified value, the eye movement speed corresponding to the target test video in the plurality of test videos is used as the eye movement threshold when the image feature is the specified value; wherein, when the user's eyes are at the exit pupil position, the color separation of the test image in the target test video is observed to be at the borderline between visible and invisible. Change the value of the image feature and obtain the corresponding eye movement threshold; Based on all the values ​​of the image features and the eye movement threshold corresponding to each value, a quantitative relationship between the image features and the eye movement threshold is obtained; Based on the quantitative relationship between the image features and the eye movement threshold, the quantitative relationship between the image features and color separation is obtained, wherein the eye movement threshold is negatively correlated with the degree of color separation.

8. The method according to claim 7, characterized in that, When the test step is performed on one of the image features, the other image features remain unchanged.

9. The method according to claim 7, characterized in that, The acquisition of multiple test videos includes: Generate a test video corresponding to a specified eye movement velocity, wherein the position of the moving target in the test image of the nth frame of the test video satisfies the following relationship with the specified eye movement velocity: frame_pos=f(FOV, L', EP, D, w, RF, n, β, pitch) Where frame_pos is the position of the moving target in the test video in the test image of the nth frame; FOV is the field of view of the display device; L' is the image distance of the display device; EP is the exit pupil distance; D is the distance from front to back of a person; W represents the specified eye movement velocity; RF represents the highest supported refresh rate of the display device; n is the sequence number of the test image in the test video; β is the magnification of the display device; pitch refers to the pixel spacing of the display device.

10. The method according to claim 1, characterized in that, When the hardware features of the display device are different, the quantitative relationship between the corresponding image features and color separation is different.

11. The method according to claim 7, characterized in that, The image features include contrast, and the quantitative relationship between contrast and color separation includes: there is a positive correlation between contrast and color separation; and / or The image features include saturation, and the quantitative relationship between saturation and color separation includes: there is a negative correlation between saturation and color separation; and / or The image features include target size, and the quantitative relationship between target size and color separation includes: the effect of target size on color separation is related to eye movement speed; when eye movement speed is less than a preset threshold, target size does not affect color separation; when eye movement speed is greater than the preset threshold, the degree of color separation decreases as the target size increases.

12. The method according to claim 2, characterized in that, Determining the gaze region threshold image based on the threshold image and the mask image includes: The threshold image of the gaze region is obtained by dividing the value of each corresponding pixel in the threshold image and the mask image.

13. The method according to claim 2, characterized in that, Determining the degree of color separation of the display device when displaying the input image based on the degree of color separation of each pixel in the gaze region threshold image and the real-time eye movement speed includes: Calculate the difference between the real-time eye movement velocity and the eye movement threshold corresponding to the color separation degree of the pixel; The degree of color separation of the display device when displaying the input image is determined based on the difference value, wherein the larger the difference value, the higher the degree of color separation of the input image.

14. The method according to claim 1, characterized in that, Also includes: Based on the degree of color separation in the input image, the color separation region is extracted or the color separation suppression is adjusted.

15. A color separation degree quantification device, characterized in that, include: The first determining module is used to determine eye movement features based on real-time acquired eye images, wherein the eye movement features include real-time eye movement velocity; The second determining module is used to extract multiple image features of each pixel of the input image of the display device, determine the degree of color separation corresponding to each image feature according to the quantitative relationship between each image feature and color separation, determine the degree of color separation of the pixel according to the degree of color separation corresponding to the multiple image features of the pixel, and determine the threshold image according to the degree of color separation of all pixels. The third determining module is used to determine the degree of color separation of the display device when displaying the input image based on the threshold image and the real-time eye movement speed.

16. An electronic device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the steps of the color separation degree quantification method as described in any one of claims 1 to 14.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the color separation degree quantification method as described in any one of claims 1 to 14.

18. A computer program product, characterized in that, It includes computer instructions that, when executed by a processor, implement the steps of the color separation degree quantification method as described in any one of claims 1 to 14.

Images