Eye tracking system, control method, and display panel

Abstract

The present application discloses an eye tracking system, a control method, and a display panel, wherein an eye tracking system comprises: a first infrared sensing unit and a second infrared sensing unit arranged adjacent to each other, the first infrared sensing unit is used to collect a first signal, and the second infrared sensing unit is used to collect a second signal, wherein the first signal is a mixed signal composed of an ambient infrared signal and an infrared signal fed back by a human eye; the second signal is an ambient infrared signal; the first infrared sensing unit comprises a light control component for assisting in obtaining the infrared signal fed back by the human eye; a control processing module is configured to differentially amplify the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye; a dynamic threshold updating module is configured to periodically obtain the infrared signal fed back by the human eye and generate a dynamic threshold; a control module is configured to determine the position of the human eye in real time according to the amplified signal and the dynamic threshold obtained in real time, so as to realize eye tracking.

Description

Technical Field

[0001] The present application generally relates to the field of AR / VR technology, and specifically to an eye tracking system, a control method, and a display panel. Background Art

[0002] Eye tracking technology, also known as gaze tracking technology, is a technology that uses various detection methods such as mechanical, electronic, and optical to obtain the direction of the subject's current "visual attention". It is widely used in human-computer interaction, medicine, marketing and advertising, ergonomics, virtual reality, and military fields.

[0003] Infrared sensors are widely used due to their simple structure and quick response. Infrared sensors are a common technology for eye tracking, which mainly detects eye movement characteristics by receiving changes in the value of infrared light reflected by the human eye.

[0004] However, in some specific scenarios, such as VR machines, the closed environment not only integrates a display screen but also has a large number of heat sources, which bring a large amount of ambient infrared noise, thus limiting the normal use of infrared sensors.

[0005] Therefore, how to overcome the infrared interference problem is very critical. Summary of the invention

[0006] In view of the above-mentioned defects or deficiencies in the prior art, it is desirable to provide an eye tracking system, a control method, and a display panel, which can improve the accuracy of eye tracking and eliminate the problem of threshold inapplicability caused by changes in usage scenarios.

[0007] In a first aspect, the present application provides an eye tracking system, comprising:

[0008] A first infrared sensor unit and a second infrared sensor unit are arranged adjacent to each other, wherein the first infrared sensor unit is used to collect a first signal, and the second infrared sensor unit is used to collect a second signal, wherein the first signal is a mixed signal composed of an ambient infrared signal and an infrared signal fed back by a human eye; the second signal is an ambient infrared signal; the first infrared sensor unit includes a light control component for assisting in obtaining the infrared signal fed back by a human eye;

[0009] A control processing module is configured to perform differential amplification on the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye;

[0010] A dynamic threshold updating module is configured to periodically obtain the infrared signal fed back by the human eye and generate a dynamic threshold;

[0011] The control module is configured to determine the position of the human eye in real time according to the amplified signal and the dynamic threshold obtained in real time, so as to realize eye tracking.

[0012] Optionally, the control processing module includes a differential amplifier circuit, a first input end of the differential amplifier circuit is connected to the first infrared sensor unit, a second input end of the differential amplifier circuit is connected to the second infrared sensor unit, and an output end of the differential amplifier circuit is coupled to the control module.

[0013] Furthermore, the control processing module also includes:

[0014] a first filter circuit, wherein a first end of the first filter circuit is connected to the first infrared sensor unit, and a second end of the first filter circuit is connected to a first input end of the differential amplifier circuit;

[0015] A second filtering circuit, wherein a first end of the second filtering circuit is connected to the second infrared sensing unit, and a second end of the second filtering circuit is connected to the second input end of the differential amplifier circuit.

[0016] Furthermore, a dynamic screening module is also included, and the dynamic screening module includes:

[0017] a comparison circuit configured to compare the amplified signal with the dynamic threshold to generate a comparison result;

[0018] The switch circuit is configured to switch on and off the connection between the comparison processing module and the control module according to the comparison result.

[0019] Optionally, the comparison circuit comprises a comparator, a first input terminal of the comparator is connected to an output terminal of the comparison processing module, a second input terminal of the comparator is connected to an output terminal of the dynamic threshold updating module, and an input terminal of the comparator is connected to a control terminal of the switch circuit;

[0020] The first end of the switch circuit is connected to the output end of the comparison processing module, and the second end of the switch circuit is connected to the control module.

[0021] In a second aspect, the present application provides a display panel, comprising an eye tracking system as described in any one of the above, wherein the display panel comprises a display area and a non-display area arranged around the display area, wherein the first infrared sensor unit and the second infrared sensor unit are evenly arranged around the non-display area, and the first infrared sensor unit and the second infrared sensor unit are arranged in the same layer and are equal in number.

[0022] Optionally, on one side of the non-display area, a plurality of the first infrared sensor units are arranged in an array along a first direction, a plurality of the second infrared sensor units are arranged one-to-one with the first infrared sensor units along the first direction, and the spacing between each of the first infrared sensor units and each of the second infrared sensor units is equal.

[0023] Optionally, a sensitivity of the control processing module is ≥10%, and the sensitivity is a magnitude of a numerical change between the first signal and the second signal.

[0024] In a second aspect, the present application provides an eye tracking control method, comprising:

[0025] Acquire in real time a first signal collected by a first infrared sensor unit and a second signal collected by a second infrared sensor unit, wherein the first signal is a mixed signal of an ambient infrared signal and an infrared signal fed back by a human eye, and the second signal is an ambient infrared signal; wherein the first infrared sensor unit is adjacent to the second infrared sensor unit, and the first infrared sensor unit includes a light control component for assisting in acquiring an infrared signal fed back by a human eye;

[0026] Performing differential amplification on the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye;

[0027] generating a dynamic threshold based on the periodically acquired amplified signal;

[0028] According to the amplified signal and the dynamic threshold obtained in real time, the position of the human eye is determined in real time to achieve eye tracking.

[0029] Optionally, generating a dynamic threshold based on the periodically acquired amplified signal comprises:

[0030] Selecting several typical positions, and obtaining amplified signals obtained based on the first infrared sensing unit and the second infrared sensing unit at the typical positions;

[0031] Marking the average value of the amplified signal at each of the typical positions as a threshold;

[0032] Based on periodically acquiring the average value of the amplified signal at each of the typical positions, the calibrated threshold is updated to obtain the dynamic threshold.

[0033] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0034] The eye tracking system provided in the embodiment of the present application effectively reduces the influence of environmental infrared noise by setting up an environmental infrared noise filtering circuit of the infrared sensor control system, and realizes the perception and filtering of environmental infrared noise by setting up a control processing module and a dynamic threshold update module, thereby effectively improving the sensing ability of the infrared sensor in the VR machine, eliminating the problem of threshold inapplicability caused by changes in usage scenarios, thereby improving the accuracy of eye tracking and ensuring the overall performance of the VR machine. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0036] Figure 1 A schematic diagram of the structure of an eye tracking system provided in an embodiment of the present application;

[0037] Figure 2 A schematic diagram of the structure of an eye tracking system provided in an embodiment of the present application;

[0038] Figure 3 A connection diagram of an eye tracking system provided in an embodiment of the present application;

[0039] Figure 4 A schematic diagram of the principle of an infrared sensor unit provided in an embodiment of the present application;

[0040] Figure 5 A schematic diagram of the position of an infrared sensor unit in a VR machine provided in an embodiment of the present application;

[0041] Figure 6 A schematic diagram of the principle of an optical component provided in an embodiment of the present application;

[0042] Figure 7 A schematic diagram of an output condition of a comparison circuit provided in an embodiment of the present application;

[0043] Figure 8 A schematic diagram of the structure of a display panel provided in an embodiment of the present application;

[0044] Fig. 9 A flow chart of a control method of an eye tracking system provided in an embodiment of the present application;

[0045] Fig.10 A flowchart of another method for controlling an eye tracking system provided in an embodiment of the present application;

[0046] Fig.11 A schematic diagram of the principle of a position information encoding method provided in an embodiment of the present application;

[0047] Fig.12 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application. DETAILED DESCRIPTION

[0048] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the relevant inventions, rather than to limit the inventions. It should also be noted that, for ease of description, only the parts related to the invention are shown in the accompanying drawings.

[0049] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] In an eye tracking system based on infrared sensors, it is necessary to use an infrared generation system to monitor the gaze of human eyes, then use infrared sensors to sense the feedback from the infrared generation system, and quantify the infrared information based on the photoelectric conversion characteristics, thereby realizing eye tracking within the entire VR machine.

[0051] According to actual tests, in a closed lens barrel without an integrated display screen, changes in the gaze point of the human eye can cause about 10% propagation fluctuations in the infrared generation system. At this time, the infrared sensor can effectively detect this fluctuation and transmit it to the eye tracking module for related calculations. In a closed lens barrel with an integrated display screen, the corresponding value will decay to less than 10%. Photoelectric changes below 10% cannot be effectively identified and transmitted due to the existence of the circuit's own noise. This makes it impossible to effectively guarantee the accuracy of the gaze point calculation of the eye tracking system based on infrared sensors.

[0052] In addition, in the actual application scenarios of VR devices, the threshold used to determine the gaze point will also change accordingly, depending on the different users, the different multimedia content being shown, and the accumulated usage time. In the existing eye tracking system based on infrared sensors, the threshold is fixed, which means that this system can only be used under specific test conditions. Once the relevant conditions change, the accuracy of the system will drop significantly or even fail.

[0053] Please see Figure 1 , an eye tracking system, comprising:

[0054] A first infrared sensor unit 100 and a second infrared sensor unit 110 are arranged adjacent to each other, wherein the first infrared sensor unit 100 is used to collect a first signal, and the second infrared sensor unit 110 is used to collect a second signal, wherein the first signal is a mixed signal composed of an ambient infrared signal and an infrared signal fed back by a human eye; the second signal is an ambient infrared signal; and the first infrared sensor unit 100 includes a light control component for assisting in obtaining the infrared signal fed back by a human eye.

[0055] The control processing module 200 is configured to perform differential amplification on the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye.

[0056] The dynamic threshold updating module 300 is configured to periodically obtain the infrared signal fed back by the human eye and generate a dynamic threshold.

[0057] The control module 400 is configured to determine the position of the human eye in real time according to the amplified signal and the dynamic threshold obtained in real time, so as to realize eye tracking.

[0058] The eye tracking system provided in the embodiment of the present application can effectively eliminate both static ambient infrared noise and dynamic ambient infrared noise through the combination of control processing and dynamic thresholds, so that the eye tracking system based on infrared sensors can adapt to various infrared scenes in real time, thereby achieving accurate and reliable eye tracking.

[0059] In specific implementation, Figure 2 As shown, the control processing module 200 includes a differential amplifier circuit 201, a first input end of the differential amplifier circuit 201 is connected to the first infrared sensor unit 100, a second input end of the differential amplifier circuit 201 is connected to the second infrared sensor unit 110, and an output end of the differential amplifier circuit 201 is coupled to the control module 400.

[0060] Among them, Figure 3 As shown, the control processing module also includes:

[0061] A first filter circuit 202 , wherein a first end of the first filter circuit 202 is connected to the first infrared sensor unit 100 , and a second end of the first filter circuit 202 is connected to a first input end of the differential amplifier circuit 201 .

[0062] The second filter circuit 203 , wherein a first end of the second filter circuit 203 is connected to the second infrared sensor unit 110 , and a second end of the second filter circuit 203 is connected to the second input end of the differential amplifier circuit 201 .

[0063] In an embodiment of the present application, the filtering circuit includes a filter. The pulse signal collected by the sensing unit first passes through the filter, and the filter is used to perform denoising on the pulse signal. The denoising can remove the self-noise in the circuit. The present application does not specifically limit the filter used in the filtering circuit. The filter can also eliminate noise and glitches in the pulse signal, making the waveform of the pulse signal smoother. After the denoising process, a denoised pulse signal is obtained.

[0064] It should be noted that in the embodiment of the present application, the signals collected by the first infrared sensor unit 100 and the second infrared sensor unit 110 are different. In the eye tracking system based on the infrared sensor unit, it is necessary to determine the gaze point of the human eye based on the infrared light fed back by the human eye received by the infrared sensor unit. In the VR machine, due to the influence of many factors on the optical path, such as the existence of lenses, the shape of the eyeballs, and the length of the lens barrel, not every position can effectively receive the infrared light fed back by the human eye.

[0065] Therefore, in the embodiments of the present application, Figure 4 As shown, by configuring the light control component 101 for the first infrared sensor unit 100, it is possible to ensure that the infrared sensor units located around the module can receive the infrared light fed back by the human eye. Figure 4 The second infrared sensor unit 110 is arranged around the first infrared sensor unit 100, and the second infrared sensor unit 110 is not provided with a light control component 101, so as to obtain infrared light noise (ambient infrared light) around the first infrared sensor unit 100. In this way, the detection accuracy can be effectively improved.

[0066] It should be noted that in the embodiment of the present application, a plurality of infrared light source emitting devices 20 are arranged on the light emitting side of the display panel 11, and the light control component 101 includes a plurality of optical lenses 1 arranged between the sensor and the human eye, and the optical lenses 1 are used to receive infrared light reflected by the human eye, such as Figure 5-6 As shown. Therefore, in the implementation of the present application, by evenly distributing the infrared light source emitting device 20, the infrared light source can evenly illuminate the eyes, and through the combination of various optical lenses 1, it can be achieved that the infrared light entering the human eye at all angles can be reflected. The optical lens 1 of the present application can also further increase the infrared light reflected by the human eye by transmitting a fixed wavelength. Of course, the embodiment of the present application also includes other optical elements for adjusting the light to achieve the recognition accuracy of the infrared sensor for the infrared signal fed back by the human eye.

[0067] By reading out the output signals of the two sensing units in parallel, the circuit noise is first filtered out through a filtering circuit, and then the two signal values ​​are subtracted through a differential amplifier circuit 201, so that only the infrared signal fed back by the human eye is retained, and finally the signal is amplified and output to the next module for processing.

[0068] It should also be noted that, in the embodiment of the present application, the differential amplifier circuit 201 performs differential amplification processing on the signals obtained by the two sensing units, and outputs the amplified signal after the differential amplification processing. Since the detection signal after the differential may be only tens of millivolts, it is not easy to obtain a more accurate result if it is directly input into the dynamic threshold update module 300 and the control module 400, so before the detection signal is input into the next module, a pre-signal amplification is required.

[0069] In the embodiment of the present application, the form of the differential amplifier circuit 201 is not specifically limited. Different amplification factors may be provided according to different application devices or application environments. Exemplarily, the differential amplifier circuit 201 includes an operational amplifier, the in-phase input terminal of the operational amplifier is connected to the first signal (ambient infrared light signal + infrared signal fed back by the human eye), and the inverting input terminal is connected to the second signal (ambient infrared light signal), and the amplified signal is obtained after differential amplification by the operational amplifier.

[0070] Therefore, in the light field jointly established by the infrared generation system and the human eye, only the infrared sensing units at certain specific positions can receive infrared light signals that interact with the human eye and can be used for eye tracking with the assistance of the light control system. Moreover, these positions can be precisely controlled by the light control system at the millimeter level. This allows two adjacent infrared sensing units, one of which can receive both ambient infrared noise and effective infrared light feedback, while the other can only receive ambient infrared noise. Based on this, by differentially amplifying the photoelectric signals received by the two, effective infrared photoelectric signals can be extracted, and the comparison filtering system can be effectively implemented.

[0071] In the embodiment of the present application, the eye tracking system further includes a dynamic screening module 500, and the dynamic screening module 500 includes:

[0072] The comparison circuit is configured to compare the amplified signal with a dynamic threshold to generate a comparison result.

[0073] The switch circuit is configured to switch on and off the connection between the comparison processing module 200 and the control module 400 according to the comparison result.

[0074] Among them, the comparison circuit includes a comparator 501, a first input end of the comparator 501 is connected to the output end of the control processing module 200, a second input end of the comparator 501 is connected to the output end of the dynamic threshold update module 300, and an input end of the comparator 501 is connected to the control end of the switch circuit 502.

[0075] The first end of the switch circuit 502 is connected to the output end of the comparison processing module 200, and the second end of the switch circuit 502 is connected to the control module 400. The switch circuit 502 includes a transistor, and the control end of the transistor is connected to the output end of the comparator 501. The high and low levels output by the comparator 501 control the transistor to be turned on or off, so as to control whether the output end of the differential amplifier circuit 201 uploads the amplified signal to the control module 400.

[0076] Since the eye tracking system needs to process the output of several infrared sensor units, and the infrared sensor units around the non-fixation point have no value changes. If all the data are uploaded without screening, the control module 400 will do a lot of invalid calculations, reducing the frequency of eye tracking. Therefore, it is necessary to screen the uploaded data. According to the principle of eye tracking, when all the data are uploaded, it is necessary to screen out the infrared sensor units with the lowest values ​​in each direction to locate the fixation point of the human eye.

[0077] In the embodiment of the present application, the dynamic screening module 500 can set a dynamic threshold to set a judgment condition for whether to upload data, which can greatly reduce the data processing pressure of the control module 400.

[0078] In a specific application, one output path of the infrared sensor unit is connected to the comparator 501 , and the other output path is connected to the switch circuit 502 . Figure 7 An output result of a comparator 501 is illustrated in FIG. ref is the input terminal of the dynamic threshold, U s The input end of the amplified signal is represented by U0, and the output end is represented by U0. For example, when the output value is less than the reference voltage, the comparator 501 outputs a high level, turns on the switch circuit 502, and allows this circuit to be uploaded. On the contrary, if the output value is greater than the reference value, the switch circuit 502 is not turned on and cannot be uploaded.

[0079] However, due to the influence of many factors such as changes in users and changes in display screens, the intensity of infrared light that can be reflected by the human eye will continue to change. Therefore, a fixed threshold will cause the upload screening function to fail or all data to be truncated. Therefore, it is necessary to update the judgment threshold in real time.

[0080] Therefore, in the embodiment of the present application, the output end of the differential amplifier circuit 201 is connected to the first input end of the comparator 501, and the dynamic threshold update module 300 is connected to the second input end of the comparator 501. By comparing the differentially amplified infrared signal with the dynamic threshold, if the differentially amplified signal is less than the dynamic threshold, the signal of the infrared sensor unit and other signals are uploaded. If the differentially amplified signal is greater than the dynamic threshold, it is not uploaded.

[0081] It should be noted that since the human eye has the lowest reflectivity for infrared, the lower the value of the infrared signal, the higher the correlation of the human eye signal obtained by the infrared sensor, and the more accurate the positioning of the human eye's gaze point.

[0082] In the embodiment of the present application, the dynamic threshold update module 300 can obtain the threshold by selecting a series of typical positions in the lens barrel and setting an infrared sensing unit to establish monitoring of the ambient infrared noise, and then setting a hardware circuit and a software algorithm to superimpose the noise-filtered amplified signal to the dynamic threshold update module 300, update the threshold at a certain frequency, realize dynamic threshold judgment, and thereby eliminate the influence of a series of dynamic factors.

[0083] It is worth noting that in the embodiment of the present application, the dynamic threshold update module 300 obtains the dynamic threshold by calling the signal of the infrared sensing unit stored in the control module 400 after differential amplification and the signal value after comparison with the threshold. The accuracy of the signal value is higher than that of other unprocessed signals. Therefore, the accuracy of the eye tracking system can be further improved by improving the accuracy of the dynamic threshold. By means of dynamic thresholds, the decision threshold can be updated in real time according to changes in the environment and usage scenarios, the infrared level can be perceived in real time, and the efficiency and accuracy of the threshold decision can be improved. In addition, the real-time perception of the infrared irradiation level can also be achieved, so that the decision benchmark always maintains the optimal level in the current usage scenario.

[0084] In the embodiment of the present application, the tracking of human eyes is achieved by combining the first infrared sensor unit 100 and the second infrared sensor unit 110 arranged on the AR / VR device. The present application also provides a display panel, including the eye tracking system as described above, wherein the display panel 11 includes a display area AA and a non-display area DA arranged around the display area, wherein the first infrared sensor unit 100 and the second infrared sensor unit are evenly arranged around the non-display area DA, such as Figure 8 shown.

[0085] In the embodiment of the present application, the first infrared sensor unit and the second infrared sensor unit are arranged in the same layer and are equal in number. On one side of the non-display area, a plurality of the first infrared sensor units are arranged in an array along a first direction, and a plurality of the second infrared sensor units are arranged one-to-one with the first infrared sensor units along the first direction, and the spacing between each of the first infrared sensor units and each of the second infrared sensor units is equal.

[0086] On one side of the non-display area DA, a plurality of the first infrared sensor units 100 are arranged in an array along a first direction in the non-display area DA, a plurality of the second infrared sensor units 110 are also arranged in an array along the first direction in the non-display area DA, and the array of the second infrared sensor units 110 is located in a second direction of the array of the first infrared sensor units 100, and the first direction is perpendicular to the second direction.

[0087] It should be noted that, in the embodiment of the present application, the first direction is a direction extending along the four sides of the display panel 11, and the first direction is different on different boundaries of the display panel 11. However, the second direction is a direction perpendicular to the first direction. The second direction can be a direction pointing from the display center to the display edge, or a direction pointing from the display edge to the display center. Therefore, in the embodiment of the present application, the second infrared sensor unit 110 can be set in the direction where the first infrared sensor unit 100 is away from the display center, and can also be set in the direction where the first infrared sensor unit 100 is close to the display center. In addition, different setting schemes can exist on the boundaries of the display panel 11 in different directions.

[0088] In the embodiment of the present application, the sensitivity of the control processing module 200 is not less than 10%, wherein the sensitivity is used to characterize the magnitude of the numerical change between the first signal and the second signal. In the embodiment of the present application, the sensitivity of the control processing module 200 can be affected by setting the arrangement of the first sensing unit and the second infrared sensing unit, and the settings of different functional modules in the control processing module 200, such as the settings of the filter circuit and the amplifier circuit.

[0089] Please refer to Fig. 9 , the present application also provides an eye tracking control method, comprising:

[0090] S02. Acquire in real time a first signal collected by the first infrared sensing unit 100 and a second signal collected by the second infrared sensing unit 110, wherein the first signal is a mixed signal of an ambient infrared signal and an infrared signal fed back by the human eye; and the second signal is an ambient infrared signal.

[0091] The first infrared sensor unit 100 is disposed adjacent to the second infrared sensor unit 110 , and the first infrared sensor unit 100 includes a light control component 101 for assisting in obtaining infrared signals fed back by human eyes.

[0092] In the embodiment of the present application, by configuring the light control component 101 for the first infrared sensor unit 100, it is possible to ensure that the infrared sensor units located around the module can receive the infrared light fed back by the human eye. The second infrared sensor unit 110 is set around the first infrared sensor unit 100, and the second infrared sensor unit 110 is not provided with the light control component 101 to obtain the infrared light noise (ambient infrared light) around the first infrared sensor unit 100. In this way, the detection accuracy can be effectively improved.

[0093] S04. Perform differential amplification on the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye.

[0094] By reading out the output signals of the two sensing units in parallel, the circuit noise is first filtered out through a filtering circuit, and then the two signal values ​​are subtracted through a differential amplifier circuit 201, so that only the infrared signal fed back by the human eye is retained, and finally the signal is amplified and output to the next module for processing.

[0095] S06. Generate a dynamic threshold based on the periodically acquired amplified signal.

[0096] In a specific application, step S06 includes:

[0097] S101, select several typical positions, and obtain the amplified signals obtained based on the first infrared sensor unit 100 and the second infrared sensor unit at the typical positions. In the embodiment of the present application, the typical positions may include several positions closest to the human eye, or positions with the least influence of ambient light signals. Depending on the shielding conditions of different devices, there may be different typical positions.

[0098] S102, calibrating the mean of the amplified signals at each of the typical positions as a threshold; in the embodiment of the present application, the size of the threshold being the mean of the amplified signals at each position is only for exemplary purposes, and in different embodiments, the accuracy of the threshold may also be controlled by setting the weights at different typical positions.

[0099] S103 , based on periodically obtaining the average value of the amplified signal at each of the typical positions, updating the calibrated threshold value to obtain the dynamic threshold value.

[0100] In the embodiment of the present application, a series of typical positions can be selected in the lens barrel to establish monitoring of the ambient infrared noise, and the amplified signal after noise filtering is superimposed on the dynamic threshold update module 300, and the threshold is updated at a certain frequency to implement dynamic threshold judgment, thereby eliminating the influence of a series of dynamic factors.

[0101] In addition, the number of typical positions, the period for updating the calibrated threshold, and the calculation method for obtaining the dynamic threshold in this application are only exemplary. Different settings can be made depending on different devices and application scenarios. There can also be different settings for different APPs for the same device.

[0102] S08. According to the amplified signal and the dynamic threshold obtained in real time, the position of the human eye is determined in real time to achieve eye tracking.

[0103] Wherein, the method further comprises:

[0104] S10, performing data screening on the amplified signal acquired in real time based on the dynamic threshold value, so as to perform at least one of the following:

[0105] A. Determine whether to upload the amplified signal;

[0106] B. determining whether to save the amplified signal; and

[0107] C. Determine whether the amplified signal is used to determine the position of the human eye in real time.

[0108] In the embodiments of the present application, by applying dynamic thresholds to different locations, different effects can be achieved. For example, by setting dynamic thresholds, setting judgment conditions for whether to upload data, the data processing pressure can be greatly reduced. For another example, by setting dynamic thresholds, setting judgment conditions for whether to save data, the data memory can be further reduced. For another example, by setting dynamic thresholds, for real-time judgment of the position of the human eye, it is necessary to screen out the infrared sensor unit with the lowest value in each direction to locate the gaze point of the human eye.

[0109] Among them, Fig.10 As shown, in the step S08, the method of performing data screening on the amplified signal acquired in real time based on the dynamic threshold comprises:

[0110] S110, determining whether the amplified signal acquired in real time is less than the dynamic threshold;

[0111] S120: If less than, determine that the amplified signal is used to perform at least one of the following:

[0112] A. Determine whether to upload the amplified signal;

[0113] B. determining whether to save the amplified signal; and

[0114] C. Determine whether the amplified signal is used to determine the position of the human eye in real time.

[0115] It should be noted that since the human eye has the lowest reflectivity for infrared, the lower the value of the infrared signal, the higher the correlation of the human eye signal obtained by the infrared sensor, and the more accurate the positioning of the human eye's gaze point.

[0116] In the embodiment of the present application, data screening can be performed by means of a physical circuit, and threshold comparison can also be performed by means of program processing of the signal. In some other embodiments, it can also be implemented by a combination of multiple methods. For example, the circuit method can be used to screen and determine whether to upload, and then after receiving the uploaded data, the size of the data can be determined by the program method to determine whether to save it, and the signal obtained after data screening in the two methods is used to determine the position of the human eye.

[0117] It should be noted that in the embodiment of the present application, due to the influence of many factors such as changes in the user and changes in the display screen, the intensity of the infrared light that can be fed back by the human eye will continue to change. Therefore, a fixed threshold value may cause the upload screening function to fail or all data to be truncated. It can be seen that the judgment threshold value needs to be updated in real time. When applied, no matter what method is used, the threshold value used to execute the judgment is the threshold value after dynamic update.

[0118] This application illustrates a circuit method for determining whether to upload a sensor signal using a dynamic threshold. In the application, in step S08, the amplified signal acquired in real time is screened based on the dynamic threshold, and the method includes:

[0119] S201, inputting the amplified signal and the dynamic threshold into two input terminals of the comparator 501, and comparing them to generate a comparison result;

[0120] S202, inputting the comparison result into the switch circuit 502, and implementing data screening by turning on and off the switch circuit 502; wherein,

[0121] S203, when the amplified signal is less than the dynamic threshold, the switch circuit 502 is turned on;

[0122] S204 , when the amplified signal is not less than the dynamic threshold, the switch circuit 502 is disconnected.

[0123] In a specific application, one output path of the infrared sensor unit is connected to the comparator 501 and the other is connected to the switch circuit 502. When the output value is less than the reference voltage, the comparator 501 outputs a high level, turns on the switch circuit 502, and allows this circuit to be uploaded. On the contrary, if the output value is greater than the reference value, the switch circuit 502 is not turned on and cannot be uploaded.

[0124] In an embodiment of the present application, it is possible to establish monitoring of ambient infrared noise, and then set up hardware circuits and software algorithms to quantify the ambient infrared noise level and superimpose it on a threshold decision module, update the threshold at a certain frequency, implement dynamic threshold decision, and thereby eliminate the influence of a series of dynamic factors.

[0125] It should be noted that the dynamic threshold is obtained by calling the stored signal of the infrared sensor unit after differential amplification and comparing it with the threshold. The accuracy of this signal value is higher than that of other unprocessed signals. Therefore, by improving the accuracy of the dynamic threshold, the accuracy of the eye tracking system can be further improved.

[0126] In the embodiment of the present application, the control module 400 includes two domains, one is the infrared sensor information storage domain, and the other is the threshold update calculation domain. Among them, the infrared sensor information storage domain is used to record the position information and output voltage information of each infrared sensor. In the embodiment of the present application, an example of a storage encoding method of infrared signals is given, and a 16-bit storage space is opened for each infrared sensor, wherein the upper eight bits are used to store the infrared sensor position information, and the lower eight bits are used to store the output voltage value of the infrared sensor. The encoding method of the voltage value follows the encoding specification of the AD processor. The upper eight bits are used for position addressing, and the lower eight bits are used to calculate the update threshold. Fig.11 An example of an encoding method for the position of an infrared sensor is shown in FIG. 4 bits are used to group the infrared sensors on the display panel 11 into m groups, each group having n / m infrared sensors, where n is the number of infrared sensors. The other four bits are used to locate the sensors in the group.

[0127] It should be noted that, in the embodiment of the present application, n is the number of the first infrared sensor units 100 or the number of the second infrared sensor units 110. In the embodiment of the present application, the first infrared sensor unit 100 and the second infrared sensor unit 110 are used in combination, and the infrared signal stored is a signal after noise filtering, so n is the number of a type of sensor. The present application is not limited to this for the encoding method, and other encoding methods may also exist in different embodiments.

[0128] Among them, the threshold update domain records the position information of the threshold update infrared sensor. Based on the recorded typical position information, the threshold update calculation program addresses from the infrared sensor information storage domain according to the position information at intervals, records the voltage values ​​of all threshold update infrared sensors, and outputs them to the mean calculation module to update the optimal threshold.

[0129] The present application provides an electronic device, comprising an eye tracking system as described above, and also comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the eye tracking control method as described above is implemented.

[0130] The electronic devices described in this application include but are not limited to wearable devices. There are many types of wearable devices, which basically have some computing functions and are in the form of portable accessories that can be connected to mobile phones and various terminals, such as head-mounted virtual reality glasses and head-mounted augmented reality glasses. The electronic device in the embodiment of this application can also be a TV, or a PC, a smart phone, a tablet computer, a car display device, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III, a dynamic image expert compression standard audio layer) player, an MP4 (Moving Picture Experts Group Audio Layer IV, a dynamic image expert compression standard audio layer) player, a portable computer, and other display devices with display functions.

[0131] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described eye tracking control methods.

[0132] Reference below Fig.12 , Fig.12 A schematic diagram of the structure of a computer system of an electronic device according to an embodiment of the present application.

[0133] like Fig.12 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 909 into a random access memory (RAM) 909. In the RAM 909, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 909 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0134] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0135] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a machine-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present application are executed.

[0136] It should be noted that the computer-readable medium shown in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.

[0137] More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk-read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0138] In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.

[0139] Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0140] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of the code, and the aforementioned module, program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] It should be understood that the orientation or position relationship indicated by terms such as "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", and "outside" are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention.

[0142] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0143] Unless otherwise defined, the technical and scientific terms used herein have the same meaning as those generally understood by those skilled in the art in the technical field of the present invention. The terms used herein are only for describing specific implementation purposes and are not intended to limit the present invention. Terms such as "setting" appearing in this article can indicate that a component is directly attached to another component, or that a component is attached to another component through an intermediate. Features described in one embodiment herein can be applied to another embodiment alone or in combination with other features, unless the feature is not applicable in the other embodiment or otherwise specified.

[0144] The present invention has been described through the above-mentioned embodiments, but it should be understood that the above-mentioned embodiments are only for the purpose of example and illustration, and are not intended to limit the present invention to the described embodiments. It can be understood by those skilled in the art that more variations and modifications can be made according to the teachings of the present invention, and these variations and modifications all fall within the scope of the protection claimed by the present invention.

Claims

1. An eye tracking system, characterized in that: include: A first infrared sensor unit and a second infrared sensor unit are arranged adjacent to each other, wherein the first infrared sensor unit is used to collect a first signal, and the second infrared sensor unit is used to collect a second signal, wherein the first signal is a mixed signal composed of an ambient infrared signal and an infrared signal fed back by a human eye; the second signal is an ambient infrared signal; the first infrared sensor unit includes a light control component for assisting in obtaining the infrared signal fed back by a human eye; A control processing module is configured to differentially amplify the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye; A dynamic threshold updating module is configured to periodically obtain the infrared signal fed back by the human eye and generate a dynamic threshold; wherein the dynamic threshold updating module is specifically configured to: select a number of typical positions, obtain the amplified signals obtained at the typical positions based on the first infrared sensing unit and the second infrared sensing unit; calibrate the average value of the amplified signals at each of the typical positions as a threshold; based on periodically obtaining the average value of the amplified signals at each of the typical positions, update the calibrated threshold to obtain the dynamic threshold; The control module is configured to determine the position of the human eye in real time according to the amplified signal and the dynamic threshold obtained in real time, so as to realize eye tracking.

2. The eye tracking system according to claim 1, characterized in that: The control processing module includes a differential amplifier circuit, a first input end of the differential amplifier circuit is connected to the first infrared sensor unit, a second input end of the differential amplifier circuit is connected to the second infrared sensor unit, and an output end of the differential amplifier circuit is coupled to the control module.

3. The eye tracking system according to claim 2, characterized in that: The control processing module also includes: a first filter circuit, wherein a first end of the first filter circuit is connected to the first infrared sensor unit, and a second end of the first filter circuit is connected to a first input end of the differential amplifier circuit; A second filtering circuit, wherein a first end of the second filtering circuit is connected to the second infrared sensing unit, and a second end of the second filtering circuit is connected to the second input end of the differential amplifier circuit.

4. The eye tracking system according to claim 1, characterized in that: It also includes a dynamic screening module, which includes: a comparison circuit configured to compare the amplified signal with the dynamic threshold to generate a comparison result; The switch circuit is configured to switch on and off the connection between the comparison processing module and the control module according to the comparison result.

5. The eye tracking system according to claim 4, characterized in that: The comparison circuit comprises a comparator, a first input terminal of the comparator is connected to the output terminal of the comparison processing module, a second input terminal of the comparator is connected to the output terminal of the dynamic threshold updating module, and an input terminal of the comparator is connected to the control terminal of the switch circuit; The first end of the switch circuit is connected to the output end of the comparison processing module, and the second end of the switch circuit is connected to the control module.

6. A display panel, characterized in that: An eye tracking system comprising the eye tracking system as described in any one of claims 1 to 5, wherein the display panel comprises a display area and a non-display area arranged around the display area, wherein the first infrared sensor unit and the second infrared sensor unit are evenly arranged around the non-display area, and the first infrared sensor unit and the second infrared sensor unit are arranged in the same layer and are equal in number.

7. The display panel according to claim 6, characterized in that: On one side of the non-display area, a plurality of the first infrared sensor units are arranged in an array along a first direction, a plurality of the second infrared sensor units are arranged one-to-one with the first infrared sensor units along the first direction, and the spacing between each of the first infrared sensor units and each of the second infrared sensor units is equal.

8. The display panel according to claim 7, characterized in that: The sensitivity of the control processing module is ≥10%, and the sensitivity is the amplitude of the numerical change between the first signal and the second signal.

9. A method for controlling eye tracking, characterized in that: include: Acquire in real time a first signal collected by a first infrared sensor unit and a second signal collected by a second infrared sensor unit, wherein the first signal is a mixed signal of an ambient infrared signal and an infrared signal fed back by a human eye, and the second signal is an ambient infrared signal; wherein the first infrared sensor unit and the second infrared sensor unit are arranged adjacent to each other, and the first infrared sensor unit includes a light control component for assisting in acquiring an infrared signal fed back by a human eye; Performing differential amplification on the first signal and the second signal to obtain an amplified signal of the infrared signal fed back by the human eye; Generating a dynamic threshold based on the periodically acquired amplified signal; comprising: Selecting several typical positions, and obtaining amplified signals obtained based on the first infrared sensing unit and the second infrared sensing unit at the typical positions; Marking the average value of the amplified signal at each of the typical positions as a threshold; Based on periodically acquiring the mean value of the amplified signal at each of the typical positions, updating the calibrated threshold value to obtain the dynamic threshold value; According to the amplified signal and the dynamic threshold obtained in real time, the position of the human eye is determined in real time to achieve eye tracking.

Images