An image acquisition method, a light spot positioning method and a gaze information determination method
By controlling the light source module to alternately illuminate the acquired images, the problem of inaccurate positioning caused by the light spot being too close in telemetry eye tracking was solved, achieving accurate positioning of the light spot and improving the precision of the gaze point.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING 7INVENSUN TECH
- Filing Date
- 2024-12-19
- Publication Date
- 2026-06-23
Smart Images
Figure CN122265610A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of eye-tracking technology, and in particular to an image acquisition method, a spot localization method, and a method for determining gaze information. Background Technology
[0002] Telemetry-based eye-tracking solutions require a camera and at least two sets of LEDs, symmetrically positioned on either side of the eye-tracking camera. The camera needs to acquire eye images with at least two Purchin spots. The eye-tracking algorithm calculates the gaze point by locating the position of the spots (Purchin spots) in the image and combining this with the pupil center or other eye features. Existing technologies require precise location (center coordinates) of at least two spots in the image. When product design limits the overall length of the product while supporting long usage distances, the two sets of LEDs being too close together results in the spots being too close in the image, making it impossible to accurately locate their positions and affecting the accuracy of gaze point calculation. Therefore, how to avoid the two spots being too close in the image becomes a problem to be solved. Summary of the Invention
[0003] This application provides an image acquisition method, a spot localization method, and a method for determining gaze information to solve the problem of spot distances being too close in an image.
[0004] According to one aspect of this application, a spot positioning method is provided, comprising:
[0005] In response to receiving a first signal, a second signal is determined, wherein the second signal includes a light source control signal, the light source control signal being used to control the first light source module and the second light source module to alternately light up or alternately turn off;
[0006] Acquire the first image captured by the image acquisition device when the first light source module is lit;
[0007] Acquire a second image captured by the image acquisition device when the second light source module is lit;
[0008] The first image and the second image are used to determine effective ocular spot information.
[0009] According to another aspect of this application, a spot positioning method is provided, comprising:
[0010] A first image and a second image are acquired, wherein the first image and the second image are obtained according to the image acquisition method described in any embodiment of this application;
[0011] A first set of light spots is determined based on the first image, and a second set of light spots is determined based on the second image;
[0012] The effective eyeball spot position is determined based on the first and second spot sets.
[0013] According to another aspect of this application, a method for determining gaze information is provided, comprising:
[0014] The effective eyeball spot position is obtained, wherein the effective eyeball spot position is determined according to the spot positioning method described in any embodiment of this application;
[0015] The user's gaze information is determined based on the effective eyeball spot position.
[0016] According to another aspect of this application, an image acquisition apparatus is provided, comprising:
[0017] A signal determination module is used to determine a second signal in response to receiving a first signal, wherein the second signal includes a light source control signal, the light source control signal being used to control the first light source module and the second light source module to alternately light up or alternately turn off;
[0018] The first image acquisition module is used to acquire the first image captured by the image acquisition device when the first light source module is lit.
[0019] The second image acquisition module is used to acquire the second image captured by the image acquisition device when the second light source module is lit;
[0020] The first image and the second image are used to determine effective ocular spot information.
[0021] According to another aspect of this application, a light spot positioning device is provided, comprising:
[0022] The image acquisition module is used to acquire a first image and a second image, wherein the first image and the second image are obtained according to the image acquisition method described in any embodiment of this application;
[0023] A spot set determination module is used to determine a first spot set based on the first image and a second spot set based on the second image;
[0024] The effective eyeball spot determination module is used to determine the position of the effective eyeball spot based on the first spot set and the second spot set.
[0025] According to another aspect of this application, a device for determining gaze information is provided, comprising:
[0026] An effective position acquisition module is used to acquire the effective eyeball spot position, wherein the effective eyeball spot position is determined according to the spot positioning method described in any embodiment of this application;
[0027] The gaze information determination module is used to determine the user's gaze information based on the effective eyeball spot position.
[0028] According to another aspect of this application, an eye-tracking device is provided, the eye-tracking device comprising:
[0029] The first light source module and the second light source module are used to respond to the control of the light source control signal, wherein the first light source module and the second light source module are alternately lit or alternately turned off;
[0030] An image acquisition device is used to acquire a first image when the first light source module is lit, and a second image when the second light source module is lit.
[0031] At least one processor, and a memory communicatively connected to said at least one processor;
[0032] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the image acquisition method, spot localization method, or gaze information determination method described in any embodiment of this application.
[0033] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the image acquisition method, spot localization method, or gaze information determination method described in any embodiment of this application.
[0034] According to another aspect of this application, a computer program product is provided, the computer program product including a computer program, which, when executed by a processor, implements the image acquisition method, spot localization method, or gaze information determination method described in any embodiment of this application.
[0035] The technical solution of this application embodiment, in response to receiving a first signal, determines a second signal, wherein the second signal includes a light source control signal, which is used to control the first light source module and the second light source module to alternately light up or alternately turn off; acquires a first image captured by an image acquisition device when the first light source module is lit; acquires a second image captured by an image acquisition device when the second light source module is lit; wherein the first image and the second image are used to determine effective eyeball spot information, solving the problem of the spot distance being too close, controlling the image acquisition device to perform image acquisition, determining the second signal through the first signal, the second signal including the light source control signal, sending the light source control signal to control the first light source module and the second light source module to alternately light up or alternately turn off, the image acquisition device acquires the first image when the first light source module is lit, and acquires the second image when the second light source module is lit; since the first light source module and the second light source module alternately light up or alternately turn off, there is only one spot in the acquired first image and the second image, and there will be no spot adhesion due to the light source being too close; by determining the effective eyeball spot information through the first image and the second image, the spot can be accurately located, thereby improving the accuracy of the fixation point.
[0036] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description
[0037] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0038] Figure 1 This is a flowchart of an image acquisition method provided according to Embodiment 1 of this application;
[0039] Figure 2 This is a schematic diagram of an effective ocular spot image provided according to Embodiment 1 of this application;
[0040] Figure 3 This is a schematic diagram of another effective ocular spot image provided according to Embodiment 1 of this application;
[0041] Figure 4 This is a schematic diagram of another effective ocular spot image provided according to Embodiment 1 of this application;
[0042] Figure 5This is a flowchart of a spot positioning method according to Embodiment 2 of this application;
[0043] Figure 6 This is a flowchart of a method for determining gaze information according to Embodiment 3 of this application;
[0044] Figure 7 This is a schematic diagram of the structure of an image acquisition device according to Embodiment 4 of this application;
[0045] Figure 8 This is a schematic diagram of a light spot positioning device according to Embodiment 5 of this application;
[0046] Figure 9 This is a schematic diagram of a gaze information determination device according to Embodiment Six of this application;
[0047] Figure 10 This is a schematic diagram of the structure of an eye-tracking device that implements an embodiment of this application;
[0048] Figure 11 This is a schematic diagram of the structure of another eye-tracking device that implements the embodiments of this application. Detailed Implementation
[0049] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0050] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0051] Example 1
[0052] Figure 1This is a flowchart of an image acquisition method provided in Embodiment 1 of this application. This embodiment is applicable to acquiring eye images with effective light spots. The method can be executed by an image acquisition device, which can be implemented in hardware and / or software, and can be configured in an eye-tracking device. Figure 1 As shown, the method includes:
[0053] S101. In response to receiving the first signal, determine the second signal, the second signal including a light source control signal, the light source control signal being used to control the first light source module and the second light source module to alternately light up or alternately turn off.
[0054] In this embodiment, the first signal can be understood as a trigger signal or an exposure signal, etc., used to indicate the exposure status of the image acquisition device, for example, indicating that the image acquisition device is about to start exposure or that the image acquisition device is about to end exposure. The second signal can be understood as a signal used to control the light source; the light source control signal can be understood as a signal used to control the light source module, for example, controlling the light source in the light source module to turn on or off. The second signal includes the light source control signal, that is, at least one processor can determine the light source control signal to be sent based on the first signal, so as to control the light source module to turn on or off through the light source control signal. The first light source module and the second light source module are both modules composed of at least one light source, and the first light source module and the second light source module are light source modules set in two different locations.
[0055] The first signal can be sent by the image acquisition device, or it can be generated by other devices or other control modules in this execution device. After receiving the first signal, this execution subject determines the exposure state of the image acquisition device and determines that the light source module needs to be controlled, and generates a second signal. The second signal includes a light source control signal, which can be sent to the first light source module and the second light source module to control the first light source module and the second light source module to alternately light up or alternately turn off.
[0056] Optionally, when the first signal is a trigger signal, the second signal may also include an exposure signal, which is used to control the exposure of the image acquisition device and includes an exposure synchronization signal.
[0057] In this embodiment of the application, at least one processor responds to a trigger signal to determine a second signal, which includes a light source control signal and an exposure synchronization signal. The light source control signal controls the first light source module and the second light source module to alternately light up or turn off, and the exposure synchronization signal controls the image acquisition device to perform exposure when the first light source module or the second light source module is lit, so as to ensure that the image acquisition device can acquire images when the light source module is lit.
[0058] Image acquisition devices can be cameras, video recorders, infrared cameras, etc. Exposure synchronization signals are used to control the exposure of the image acquisition device to acquire image data.
[0059] Optionally, when the first signal is an exposure signal, the exposure signal is sent by the image acquisition device. The exposure signal includes an exposure start signal or an exposure end signal.
[0060] In this embodiment, the exposure conditions of the image acquisition device can be preset. When the exposure conditions are met or the image acquisition device needs to be exposed, the image acquisition device generates an exposure signal and sends it to at least one processor. The at least one processor determines a light source control signal based on the exposure signal. For example, the exposure condition is an exposure period, and the exposure signal is sent to the image acquisition device periodically according to the exposure period; or, the exposure condition is a specific user operation, and the exposure signal is sent to the image acquisition device after the specific user operation is detected, and so on.
[0061] When an exposure signal sent by an image acquisition device is detected, at least one processor generates a light source control signal based on the exposure signal, or sends a corresponding light source control signal based on the exposure frequency of the image acquisition device. The light source control signal controls the first and second light source modules to alternately light up or turn off. For example, sending a light source control signal to the first and second light source modules controls their respective lighting and turning them off; that is, controlling the first light source module to light up and controlling the second light source module to turn off, or controlling the first light source module to turn off and controlling the second light source module to light up. Alternatively, a control signal is sent to the power supply controlling the lighting of the light sources in the first and second light source modules to turn the power supply on or off, controlling the light sources in the first and second light source modules to alternately light up, and so on. In other words, the embodiments of this application do not limit the recipient of the light source control signal, as long as it enables the alternating lighting or turning off of the light sources in the first and second light source modules.
[0062] Exposure signals include exposure start signals and exposure end signals. An exposure start signal can be understood as the signal emitted by the image acquisition device when it starts or is about to start exposure, and an exposure end signal can be understood as the signal emitted by the image acquisition device when it ends or is about to end exposure.
[0063] When the exposure signal is the exposure start signal, the processor determines the light source control signal based on the exposure start signal. This light source control signal then controls the first and second light source modules to alternately light up or alternately turn off. For example, when the image acquisition device starts exposure and sends an exposure start signal, the processor determines the light source control signal based on this signal. The light source control signal controls the first light source module to light up and the second light source module to turn off. When the image acquisition device starts exposure again and sends the exposure start signal again, the light source control signal controls the second light source module to light up and the first light source module to turn off. In other words, only one light source module is lit during the exposure time of the image acquisition device.
[0064] When the exposure signal is an exposure end signal, the processor determines the light source control signal based on the exposure end signal, and controls the first light source module and the second light source module to alternately turn off through the light source control signal. In this embodiment, in response to receiving the exposure end signal and determining the light source control signal, the processor further includes: controlling the light source module to light up and the image acquisition device to expose in response to a trigger signal.
[0065] The processor controls the first or second light source module to pre-illuminate based on a trigger signal, and controls the image acquisition device to expose when the light source module is illuminated. After exposure, the image acquisition device generates an exposure end signal, and the processor controls the light source module to shut down based on this signal. For example, if the trigger signal controls the first light source module to illuminate and the image acquisition device to begin exposure, and the image acquisition device sends an exposure end signal after exposure, the processor determines a light source control signal based on this signal, which then controls the first light source module to shut down. Similarly, if the trigger signal controls the second light source module to illuminate and the image acquisition device to begin exposure, and the image acquisition device sends an exposure end signal after exposure, the processor determines a light source control signal based on this signal, which then controls the second light source module to shut down.
[0066] S102. Acquire the first image captured by the image acquisition device when the first light source module is lit.
[0067] In this embodiment, the first image can be understood as the image captured when the first light source module is lit, which can be an image of the human eye captured by the image acquisition device; for example, the first image includes eye data. The image acquisition device performs exposure according to the exposure instruction or when the exposure conditions are met, and captures the first image. For example, the image acquisition device performs exposure after receiving the exposure signal and captures the first image; at this time, the light source module that is lit is the first light source module, and the effective eyeball spot in the first image is formed by the light source in the first light source module shining on the user's eye.
[0068] S103. Acquire the second image captured by the image acquisition device when the second light source module is lit.
[0069] In this embodiment, the second image can be understood as the image captured when the second light source module is lit, which can be an image of the human eye captured by the image acquisition device; for example, the second image includes eye data. The image acquisition device performs exposure according to the exposure instruction or when the exposure conditions are met, and captures the second image. For example, the image acquisition device performs exposure after receiving the exposure signal and captures the second image; at this time, the light source module that is lit is the second light source module, and the effective eyeball spot in the second image is formed by the light source in the second light source module shining onto the user's eye.
[0070] The first and second images are used to determine the effective eyeball spot information.
[0071] In this embodiment, the effective eyeball spot information can be information such as the coordinates, size, and shape of the spot. The effective eyeball spot information includes the effective eyeball spot coordinates (or position), effective eyeball spot shape, and / or effective eyeball spot size. An effective eyeball spot is the spot formed when light emitted from the light source in the light source module shines on the user's eye, is reflected by the eye, and enters the image acquisition device. Spots formed after light is reflected by other objects (such as lenses) and enters the image acquisition device, or spots formed by some light from natural light entering the image acquisition device, are all invalid eyeball spots. Invalid spots can be defined as stray spots, and information about invalid eyeball spots cannot be used for calculating gaze information. The effective eyeball spot can exist in a human eye image, an animal eye image, or even an image of a prosthetic eye. In other words, this embodiment does not limit the type of eye image in which the effective eyeball spot exists.
[0072] Generally, both the first and second images should include at least one valid eyeball spot. The valid eyeball spot information is determined by analyzing the positions of the valid eyeball spots in the first and second images. There should be at least two valid eyeball spots. Alternatively, both the first and second images should include at least one valid eyeball spot, and the valid eyeball spots in different images should have different information. The user's gaze information at a specific moment is analyzed using the valid eyeball spot information in the first and second images; in other words, the valid eyeball spot information needed to analyze the user's gaze information at a specific moment is determined by using different images.
[0073] The first and second light source modules illuminate alternately, forming only one Pulcim spot (effective eyeball spot) corresponding to the illuminated light source module on each frame. Determining the position of the Pulcim spot in each frame individually avoids the problem of inaccurate analysis of the effective eyeball spot position due to different Pulcim spots sticking together. For example, by combining and analyzing two consecutive frames, the accurate positions of the different Pulcim spots formed by the two light source modules can be obtained.
[0074] Figure 2 This is a schematic diagram illustrating an effective eyeball light spot image provided in an embodiment of this application. Figure 2 When the first light source module and the second light source module are lit simultaneously, the image acquisition device captures an eye image with two eyeball light spots that are stuck together. Figure 3 This is a schematic diagram illustrating another effective ocular spot image provided in an embodiment of this application. Figure 3 When the first light source module is lit and the second light source module is turned off, the image acquisition device captures an eye image with an effective eyeball light spot. Figure 4 This is a schematic diagram illustrating another effective ocular spot image provided in an embodiment of this application. Figure 4 When the second light source module is lit and the first light source module is turned off, the image acquisition device captures an eye image with one valid eyeball spot. In other words, when both the first and second light source modules are lit simultaneously, the two eyeball spots in the eye image are intertwined, making it difficult to determine the valid eyeball spot information. When the first and second light source modules are lit alternately, each eye image contains only one valid eyeball spot, making it easier to determine the information for each valid eyeball spot.
[0075] The first or second image can be understood as the Pukin field image captured by the image acquisition device. The Pukin field image is formed by reflections from several optical interfaces of the eye. The image reflected by the cornea is the first Pukin field image. The image reflected from the posterior surface of the cornea is relatively weak and is called the second Pukin field image. The image reflected from the anterior surface of the lens is called the third Pukin field image. The fourth Pukin field image is formed by reflections from the posterior surface of the lens. Generally speaking, the first Pukin field image is brighter than the other Pukin field images, so it is usually chosen to determine the eye's fixation position.
[0076] The order in which the first and second images are acquired depends on the lighting order of the first and second light source modules. When the first light source module is lit first, the first image can be acquired first; when the second light source module is lit first, the second image can be acquired first. Therefore, the first image can be acquired first, followed by the second image, or vice versa.
[0077] This application provides an image acquisition method. In response to receiving a first signal, a second signal is determined. The second signal includes a light source control signal, which controls a first light source module and a second light source module to alternately light up or turn off. The method acquires a first image captured by an image acquisition device when the first light source module is lit, and a second image captured by the image acquisition device when the second light source module is lit. The first and second images are used to determine effective eyeball spot information, solving the problem of inaccurate spot positioning when spots are clustered in the image. The method controls the image acquisition device to acquire images, determines the light source control signal through an exposure signal, and sends the light source control signal to control the first and second light source modules to alternately light up or turn off. The image acquisition device acquires the first image when the first light source module is lit and the second image when the second light source module is lit. Because the first and second light source modules alternately light up or turn off, only one spot is captured in the acquired first and second images, preventing spot clustering due to close proximity of the light sources. By determining effective eyeball spot information through the first and second images, accurate spot positioning can be achieved, thereby improving the accuracy of the fixation point.
[0078] Optionally, the timing of sending the light source control signal is determined based on the exposure frequency of the exposure signal.
[0079] When the first light source module and the second light source module are controlled to alternately light up or alternately turn off by the light source control signal, the timing of sending the light source control signal can be determined according to the exposure frequency corresponding to the image acquisition device. For example, the sending frequency of the light source control signal is the same as the exposure frequency of the exposure signal, that is, a light source control signal is sent to the first light source module or the second light source module every time an exposure signal is sent, or, every time an exposure signal is sent, two light source control signals are sent, one light source control signal is sent to the first light source module and the other light source control signal is sent to the second light source module.
[0080] To ensure accurate image acquisition, the lighting and shutting off of the light source modules can be synchronized with the exposure time of the image acquisition device. Light source control signals are sent to the first and second light source modules before or simultaneously with the image acquisition device sending the exposure signal. For example, taking LED group A as the first light source module and LED group B as the second light source module, when the image acquisition device starts capturing a frame of image exposure or before exposure begins, LED group A is lit and LED group B is turned off. After capturing that frame, LED group A is turned off. When capturing the next frame of image exposure begins or before exposure begins, LED group B is lit and LED group A is turned off. After exposure ends, LED group B is turned off, and the logic for lighting group A or group B during subsequent camera exposures follows the same pattern. In this embodiment, to achieve alternating lighting of the two light source modules and synchronization of the image acquisition device's exposure time, the exposure signal output of the image acquisition device can be used. By detecting the corresponding exposure signal of the image acquisition device in the firmware or program, the alternating lighting logic of the two light source modules can be controlled, or the two light source modules can be controlled to alternately light up before camera exposure according to a preset light source control signal sending frequency and / or sending order.
[0081] Optionally, the light source control signal includes a light source status control signal, which includes a light source turn-on signal and / or a light source turn-off signal;
[0082] The light source status control signal sent to the first light source module is different from the light source status control signal sent to the second light source module. The light source status control signal includes a light source turn-on signal and a light source turn-off signal.
[0083] Alternatively, light source status control signals may be sent alternately to the first light source module and the second light source module to control the first light source module and the second light source module to turn on or off for a preset duration. The light source status control signals include light source turn-on signals and / or light source turn-off signals.
[0084] In this embodiment, the light source state control signal can be understood as a signal used to control the state of the light source module, for example, controlling the light source module to turn on or off. The light source turn-on signal can be understood as a signal used to instruct the light source module to turn on the light source; the light source turn-off signal can be understood as a signal used to instruct the light source module to turn off the light source.
[0085] When light source status control signals are simultaneously sent to the first light source module and the second light source module, the light source status control signals sent to the first light source module and the second light source module are different. The light source status control signals include a light source on signal and a light source off signal. For example, a light source on signal is sent to the first light source module and a light source off signal is sent to the second light source module, or vice versa. By sending different light source status control signals to the first and second light source modules, the first and second light source modules are controlled to alternately light up or alternately turn off, ensuring that only one light source module is lit at any given time.
[0086] Alternatively, light source status control signals can be sent alternately to the first and second light source modules. Specifically, only one light source module or the second light source module receives the control signal at a time. This control signal includes a light source activation signal, which controls either the first or second light source module to illuminate for a preset duration. Upon receiving the activation signal, either the first or second light source module illuminates and remains illuminated for the preset duration before automatically turning off. The preset duration can be pre-set, for example, based on the exposure time of the image acquisition device, ensuring that either the first or second light source module remains illuminated during the exposure process.
[0087] Optionally, the light source control signal may also include light source information, which is used to determine the light source to be lit.
[0088] In this embodiment, light source information can be understood as information describing the light source that needs to be lit. For example, information that uniquely identifies the light source, such as its identifier or serial number, as well as the number and location of the light sources. The light source information can include at least one of the above. The light source to be lit can be determined based on the light source information. The first light source module and the second light source module can include one or more light sources. When only one light source is included, the light source module can only light up this light source. When multiple light sources are included, the light source module can light up all the light sources or only some of the light sources. In the case of lighting up some light sources, the light source to be lit can be randomly selected, or the light source to be lit can be selected according to preset conditions, rules, etc. The relevant information of the light source to be lit is written into the light source information, such as directly writing the light source identifier, the light source location, or the number of light sources, etc., one or more of these.
[0089] Optionally, the method further includes:
[0090] The transmitted light source information is adjusted according to preset switching conditions.
[0091] In this embodiment, the preset switching condition can be understood as a condition used to determine whether the light source to be lit in the light source module needs to be switched. The preset switching condition is set in advance, and the system checks whether the preset switching condition is met. This detection can be performed in real time, periodically, or after receiving a trigger operation or indication information; this embodiment does not limit this. If the condition is met, the transmitted light source information is adjusted; if not, the light source information is not adjusted.
[0092] Optionally, the preset switching conditions include at least one of the following:
[0093] The number of times the light source is continuously lit exceeds a preset threshold.
[0094] The temperature corresponding to the light source exceeds the preset temperature threshold.
[0095] In this embodiment, the preset number of times threshold and the preset temperature threshold can be preset according to the needs of actual application, or determined through experiments and statistical analysis of historical data. Taking the first light source module as an example, the light source that is lit each time in the first light source module is recorded, and the number of consecutive times the light source is lit is counted. If the number of consecutive times the light source is lit exceeds the preset number of times threshold, it is determined that the preset switching condition is met, and the information of the light source to be lit next is adjusted so that a different light source can be lit next time; or, the temperature near the light source is collected. If the temperature exceeds the preset temperature threshold, it is determined that the preset switching condition is met. The temperature near the light source can be collected by installing temperature sensors, temperature control elements, etc., near the light source. The temperature can be collected by temperature sensors, or by temperature control elements sensing the temperature on the side of the light source, or by calculating the temperature using parameters such as current and power.
[0096] For example, when each light source module includes at least two light sources, the alternating flashing can be configured to alternate between light source a1 in the first light source module and light source a2 in the second light source module. After alternating flashing for a period of time, the alternating flashing can switch to alternate between light source b1 in the first light source module and light source b2 in the second light source module. This allows for timely changes in the light-emitting position of the light sources, preventing the heat generated by a light source in one position from emitting light continuously without sufficient dissipation, thus improving the heat dissipation of the device. Alternatively, a temperature control element can be installed. This element senses the temperature on the light source side, and when the temperature rises to a certain threshold, it switches to another light source for alternating flashing.
[0097] By setting preset switching conditions, the light source can be controlled to stop working after working continuously for a period of time, thus avoiding damage to the light source or equipment caused by prolonged operation.
[0098] Optionally, within an alternating control cycle, the light source illumination area controlled by the first light source module through light source information is the same as the light source illumination area controlled by the second light source module through light source information.
[0099] In this embodiment, the light source illumination area can be understood as the area that can be illuminated after the light source is turned on. The first light source module and the second light source module are alternately turned on or off, and one image is acquired each time they are turned on. Each image includes at least one effective eyeball spot, and the two images, with at least two effective eyeball spots, can determine the user's gaze information. In order to determine the user's gaze information, the two images need to be processed. Therefore, in this embodiment, an alternating control cycle refers to a time period in which the first light source module is turned on once and the second light source module is turned on once. Within this cycle, one first image and one second image can be acquired.
[0100] Within an alternating control cycle, the area illuminated by the first light source module through light source information is the same as the area illuminated by the second light source module through light source information. Taking a first image and a second image as an example, this set of images is the image acquired within an alternating control cycle and can be used to determine the user's gaze information. When the first image and the second image in this set of images are acquired, the area illuminated by the first light source module is the same as the area illuminated by the second light source module. For example, the areas illuminated by the first and second light source modules are divided into regions 1, 2, 3, and 4. In the first alternating control cycle, light source 1 in the first light source module is controlled to light up for 5ms, illuminating region 1 and acquiring the first image. After 5ms, light source 1 in the second light source module is controlled to light up for 5ms, illuminating region 1 and acquiring the second image. The first and second images acquired in this instance can be used as a set of images to determine gaze information. In the second alternating control cycle, light source 1 in the first light source module is controlled to light up for 5ms, illuminating region 1 and acquiring the first image. After 5ms, light source 1 in the second light source module is controlled to light up for 5ms, illuminating region 1 and acquiring the second image, thus obtaining a new set of images. In the Nth alternating control cycle, if the preset switching conditions are met, light source 2 in the first light source module is controlled to light up for 5ms, illuminating region 2 and acquiring the first image. After 5ms, light source 2 in the second light source module is controlled to light up for 5ms, illuminating region 2 and acquiring the second image, thus obtaining another set of images... In some embodiments, the areas illuminated by different light sources in the light source module can also be the same area. For example, in the first alternating control cycle, light source 1 in the first light source module is controlled to light up for 5ms, illuminating area 1 and acquiring the first image. After 5ms, light source 1 in the second light source module is controlled to light up for 5ms, illuminating area 1 and acquiring the second image. The first and second images acquired this time can be used as a set of images to determine gaze information. After determining that the preset switching conditions are met, in the second alternating control cycle, light source 2 in the first light source module is controlled to light up for 5ms, illuminating area 1 and acquiring the first image. After 5ms, light source 2 in the second light source module is controlled to light up for 5ms, illuminating area 1 and acquiring the second image, thus obtaining a new set of images...
[0101] Within an alternating control cycle, the area illuminated by the light source controlled by the first light source module through light source information is the same as the area illuminated by the light source controlled by the second light source module through light source information. This ensures the consistency of the light spots in the first and second images and reduces the error in light spot positioning.
[0102] In this embodiment, the distance between the two light source modules is between 1cm and 30cm, or between 1cm and 15cm. The distance between the light source modules can be understood as the distance between the center point of the geometric shape formed by the light sources in one light source module and the center point of the geometric shape formed by the light sources in another light source module.
[0103] This application provides a light spot positioning method that solves the problem of light spot adhesion, enabling accurate spot positioning and preventing inaccurate spot information from affecting the eye-tracking device's calculation of user gaze information. During the alternating flashing of the light source module, the light sources within the module can be used intermittently, avoiding excessive temperature at the same light source location, which is detrimental to heat dissipation. It also addresses interference from speckles that are similar in shape and position to the correct light spot. The image acquisition method provided in this application can accurately acquire light spot images, avoiding spot adhesion and accurately positioning the light spot.
[0104] Example 2
[0105] Figure 5 This is a flowchart of a spot positioning method provided in Embodiment 2 of this application. This embodiment further processes the first and second images obtained in the above embodiments. Figure 5 As shown, the method includes:
[0106] S201. Obtain a first image and a second image, wherein the first image and the second image are obtained according to the image acquisition method provided in any embodiment of this application.
[0107] S202. Determine the first set of light spots based on the first image, and determine the second set of light spots based on the second image.
[0108] In this embodiment, both the first set of light spots and the second set of light spots can be understood as datasets containing light spot information. For example, the first set of light spots and the second set of light spots include the coordinates of the light spots.
[0109] The first image is processed to determine the information of the light spots. For example, filtering methods can be used to process the first image, filtering out light spots or data that are far away, or light spots or data that are obviously not valid eyeball light spots due to reflections from other objects (such as lenses), based on the shape of the light spots and their relative position to the pupil. This filtered data can be considered as not being light spot data, and the filtered light spots are determined as the first set of light spots. Alternatively, all light spots in the first image can be identified without filtering, and all identified light spots are determined as the first set of light spots.
[0110] The coordinates of the first set of light spots are determined by processing the first set of light spots, thus obtaining the first set of light spots. The first set of light spots includes effective eyeball light spots formed by the lit light source and possible ineffective eyeball light spots, also known as stray spots.
[0111] The second image is processed to determine the information of the light spots. For example, filtering can be used to process the second image, filtering out data that is not a light spot based on the shape of the light spot and its relative position to the pupil, or light spots or data that are obviously not valid eyeball light spots due to reflections from other objects (such as lenses), etc. The filtered light spots are then determined as the second set of light spots. Alternatively, all light spots in the second image can be identified without filtering, and all identified light spots are determined as the second set of light spots. The coordinates of the light spots are determined by processing the second set of light spots, resulting in the second set of light spots. The second set of light spots includes valid eyeball light spots formed by the lit light source and possible invalid eyeball light spots, i.e., noise spots.
[0112] S203. Determine the effective eyeball spot position based on the first spot set and the second spot set.
[0113] In this embodiment, the effective eyeball spot position can be understood as the position of the spot formed by the light emitted from the lit light source after reflection by the eye and entering the image acquisition device. The spot information in the first spot set and the second spot set are analyzed and processed respectively to filter out noise and obtain the effective eyeball spot position.
[0114] Optionally, the effective eyeball spot position is determined based on the first spot set and the second spot set, including:
[0115] A1. Determine the intersection of the first and second light spot sets to obtain the third light spot set.
[0116] In this embodiment, the third spot set can be understood as a dataset containing spot information, such as the coordinates of the spots. By analyzing the data in the first and second spot sets, the intersection of the two sets is determined to obtain the third spot set. The spots in the third spot set are considered clutter, meaning that the same spot appears in both the first and second images.
[0117] A2. Determine the effective eyeball spot position based on the first spot set, the second spot set, and the third spot set.
[0118] The effective eyeball spot positions corresponding to the first image are obtained by removing spots that appear in the third spot set from the first spot set. The effective eyeball spot positions corresponding to the second image are obtained by removing spots that appear in the third spot set from the second spot set.
[0119] For example, this application provides a method for determining the effective location of the eyeball light spot:
[0120] 1. Obtain the infrared image image1 (e.g., the first image) of the first light source module.
[0121] 2. Obtain the infrared image image2 (e.g., the second image) of the second light source module.
[0122] 3. Obtain the set of light spot coordinates glints1 of image1 (e.g., the first light spot set). For example, obtain the set of two light spots (one light spot from the first light source module and one stray light spot).
[0123] 4. Obtain the set of light spot coordinates glints2 of image2 (e.g., the second light spot set). For example, obtain the set of two light spots (one light spot from the second light source module and one stray light spot).
[0124] 5. Based on glints1 and glints2, obtain the intersection of the two sets, glints3 (e.g., the third set of light spots).
[0125] 6. Subtract glints3 from glints1 to get the set glints4.
[0126] 7. Subtract glints3 from glints2 to get the set glints5.
[0127] 8. Filter glints4 based on the shape of the light spot and / or its relative position to the pupil to obtain the set glints6.
[0128] 9. Filter glints5 based on the shape of the light spot and / or its relative position to the pupil to obtain the set glints7.
[0129] 10. The light spot formed by the first light source module is in glints6, and the light spot formed by the second light source module is in glints7.
[0130] Steps 8 and 9 can also be performed after steps 3 or 4.
[0131] In some embodiments, the effective eyeball spot position can be determined by other means, such as processing the first and second light spot sets using a deep learning model to determine the effective eyeball spot position. The embodiments of this application do not limit the specific method of determining the effective eyeball spot position based on the first and second light spot sets.
[0132] This application embodiment improves the problem of inaccurate positioning of the Puerchin spot caused by the adhesion of the Puerchin spot by alternately lighting two sets of light source modules. When the light source modules are lit alternately, only one Puerchin spot is formed on one frame of the image. At this time, the positioning of the Puerchin spot will avoid the problem of inaccuracy caused by the adhesion of the Puerchin spot. By combining two consecutive frames of images, the position of the Puerchin spot formed by the two sets of light source modules can be obtained.
[0133] Optionally, the effective eyeball spot position includes the first target spot position and the second target spot position.
[0134] In this embodiment, the first target spot position can be understood as the position of the effective eyeball spot in the first image. The second target spot position can be understood as the position of the effective eyeball spot in the second image. The first target spot position and the second target spot position are different effective eyeball spot information.
[0135] This application provides a spot localization method that solves the problem of inaccurate spot localization, thereby avoiding the situation where inaccurate spot information affects the inaccurate calculation of user gaze information by eye-tracking devices. By alternately turning the first and second light source modules on or off, two images are acquired, preventing spot adhesion; this also solves the interference of speckles that are similar in shape and position to the correct spot. Speckles are filtered out by using the intersection of the first and second spot sets. Since speckles will appear in the image regardless of which light is on, this characteristic of speckles can be used to filter out some speckles. This improves localization accuracy and data processing speed.
[0136] Example 3
[0137] Figure 6 This is a flowchart illustrating a method for determining gaze information according to Embodiment 3 of this application. This embodiment further processes the method based on the determination of the effective eyeball spot position according to the first image and the second image in the above embodiment. The method includes:
[0138] S301. Obtain the effective eyeball spot position, which is determined according to the spot positioning method provided in any embodiment of this application.
[0139] S302. Determine the user's gaze information based on the effective eyeball spot position.
[0140] Analyze the effective eyeball spot positions to determine the user's gaze information. The effective eyeball spot position is the spot position on two different images. The two spot positions can be mapped onto the same image, and the user's gaze information can be analyzed based on the two spot positions on the same image.
[0141] Optionally, the effective eyeball spot position includes the first target spot position and the second target spot position;
[0142] Determining the user's gaze information based on the effective eyeball spot position includes:
[0143] The relative position of the first target light spot is determined based on its position and the pupil position in the first image; the light spot in the first image is mapped onto the second image based on its relative position to obtain the position of the third target light spot; the user's gaze information is determined based on the positions of the second and third target light spots; or...
[0144] The relative position of the second light spot is determined based on the position of the second target light spot and the position of the pupil in the second image; the light spot in the second image is mapped onto the first image based on the relative position of the second light spot to obtain the position of the fourth target light spot; the user's gaze information is determined based on the position of the first target light spot and the position of the fourth target light spot.
[0145] In this embodiment, the first target spot position can be understood as the position of the effective eyeball spot in the first image. The second target spot position can be understood as the position of the effective eyeball spot in the second image. The relative position of the first spot is the relative position of the spot in the first image and the pupil in the image; the third spot position is the position of the effective eyeball spot in the first image mapped onto the second image. The relative position of the second spot is the relative position of the spot in the second image and the pupil in the image; the fourth spot position is the position of the effective eyeball spot in the second image mapped onto the first image. Gaze information can include information such as gaze direction and gaze point.
[0146] Since the two light spots are located in two different images, the user's eye position may shift slightly. To improve the accuracy of gaze analysis, the two light spots are mapped onto the same image for processing. The position of the first target light spot and the pupil position in the first image are analyzed to determine the relative position of the first light spot. This relative position can be a coordinate relationship; for example, the offset of the first target light spot position relative to the pupil position in the first image is the relative position of the first light spot. The pupil position in the second image is then determined. Based on the pupil position and the relative position of the first light spot, the light spot in the first image is mapped onto the second image to obtain the position of the third target light spot. At this point, the second image contains two light spots: the second target light spot position and the third target light spot position. The user's gaze information is then determined based on these two positions. Alternatively, the position of the second target spot can be analyzed, and the pupil position in the second image can be used to determine the relative position of the second spot. The relative position can be a coordinate relative position. For example, with the pupil position in the second image as the center, the offset of the second target spot position relative to the center is the relative position of the first spot. The pupil position in the first image is determined, and the spot in the second image is mapped onto the first image based on the pupil position and the relative position of the second spot to obtain the position of the fourth target spot. At this time, the first image includes two spots, the first target spot position and the fourth target spot position. The user's gaze information is determined based on the first target spot position and the fourth target spot position.
[0147] In this embodiment, the position of the light spot in the image is compensated based on the relative position of the light spot and the pupil. The light spot in the first image is mapped to the second image, or the light spot in the second image is mapped to the first image. Finally, the user's gaze information is determined based on the positions of the two light spots in the same image, which can accurately determine the user's gaze information.
[0148] Example 4
[0149] Figure 7 This is a schematic diagram of the structure of an image acquisition device provided in Embodiment 4 of this application. Figure 7 As shown, the device includes: a signal determination module 41, a first image acquisition module 42, and a second image acquisition module 43.
[0150] The signal determination module 41 is used to determine a second signal in response to receiving a first signal, wherein the second signal includes a light source control signal, which is used to control the first light source module and the second light source module to alternately light up or alternately turn off;
[0151] The first image acquisition module 42 is used to acquire the first image captured by the image acquisition device when the first light source module is lit;
[0152] The second image acquisition module 43 is used to acquire the second image captured by the image acquisition device when the second light source module is lit;
[0153] The first image and the second image are used to determine effective ocular spot information.
[0154] This application provides an image acquisition device that solves the problem of inaccurate spot positioning. The image acquisition device acquires images by determining a light source control signal through a first signal and sending the light source control signal to control the first light source module and the second light source module to alternately light up or alternately turn off. The image acquisition device acquires a first image when the first light source module is lit and acquires a second image when the second light source module is lit. Since the first light source module and the second light source module alternately light up or turn off, there is only one spot in the acquired first image and second image, and the spot will not stick together due to the light source being too close. By determining the effective eyeball spot information through the first image and the second image, the spot can be accurately positioned, thereby improving the accuracy of the fixation point.
[0155] Optionally, when the first signal is a trigger signal, the second signal further includes an exposure signal, which is used to control the exposure of the image acquisition device, and the exposure signal includes an exposure synchronization signal. Optionally, when the first signal is an exposure signal, the exposure signal is sent by the image acquisition device; the exposure signal includes an exposure start signal or an exposure end signal.
[0156] Optionally, the timing of sending the light source control signal is determined based on the exposure frequency of the exposure signal.
[0157] Optionally, the light source control signal includes a light source state control signal;
[0158] The light source status control signal sent to the first light source module is different from the light source status control signal sent to the second light source module;
[0159] Alternatively, light source status control signals can be sent alternately to the first light source module and the second light source module to control the first light source module and the second light source module to be lit for a preset duration.
[0160] Optionally, the light source control signal may further include light source information, which is used to determine the light source to be lit.
[0161] Optionally, the device may also include:
[0162] Light source information adjustment information is used to adjust the transmitted light source information according to preset switching conditions.
[0163] Optionally, the preset switching conditions include at least one of the following:
[0164] The number of times the light source is continuously lit exceeds a preset threshold.
[0165] The temperature corresponding to the light source exceeds the preset temperature threshold.
[0166] Optionally, within an alternating control cycle, the light source illumination area controlled by the first light source module through light source information is the same as the light source illumination area controlled by the second light source module through light source information.
[0167] Optionally, both the first image and the second image include at least one effective eyeball spot, and the effective eyeball spot on different images has different effective eyeball spot information.
[0168] The image acquisition device provided in this application embodiment can execute the image acquisition method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.
[0169] Example 5
[0170] Figure 8 This is a schematic diagram of a light spot positioning device provided in Embodiment 5 of this application. Figure 8 As shown, the device includes: an image acquisition module 51, a light spot set determination module 52, and an effective eyeball light spot determination module 53.
[0171] Image acquisition module 51 is used to acquire a first image and a second image, wherein the first image and the second image are obtained according to the image acquisition method provided in any embodiment of this application;
[0172] The spot set determination module 52 is used to determine a first spot set based on the first image and a second spot set based on the second image;
[0173] The effective eyeball spot determination module 53 is used to determine the position of the effective eyeball spot based on the first spot set and the second spot set.
[0174] This application provides a light spot positioning device that solves the problem of inaccurate light spot positioning, thereby avoiding the situation where inaccurate light spot information affects the inaccurate calculation of user gaze information by eye-tracking devices. By alternately turning the first and second light source modules on or off, two images are acquired, preventing light spot adhesion; interference from speckles that are similar in shape and position to the correct light spot is also addressed. Speckles are filtered out by using the intersection of the first and second light spot sets. Since speckles will appear in the image regardless of which light is on, this characteristic of speckles can be used to filter out some speckles. This improves positioning accuracy and data processing speed.
[0175] Optional, the effective eyeball spot determination module 53 includes:
[0176] The third spot set determination unit is used to determine the intersection of the first spot set and the second spot set to obtain the third spot set;
[0177] An effective eyeball spot determination unit is used to determine the position of the effective eyeball spot based on the first spot set, the second spot set, and the third spot set.
[0178] The spot positioning device provided in this application can execute the spot positioning method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the method.
[0179] Example 6
[0180] Figure 9 This is a schematic diagram of a gaze information determination device provided in Embodiment Six of this application. Figure 9 As shown, the device includes: an effective position acquisition module 61 and a gaze information determination module 62.
[0181] The effective position acquisition module 61 is used to acquire the effective eyeball spot position, wherein the effective eyeball spot position is determined according to the spot positioning method provided in any embodiment of this application;
[0182] The gaze information determination module 62 is used to determine the user's gaze information based on the effective eyeball spot position.
[0183] This application provides a device for determining gaze information, which solves the problem of inaccurate gaze information determination. By obtaining the accurate effective eyeball spot position, the user's gaze information can be determined, thereby improving the accuracy of gaze information.
[0184] Optionally, the effective eyeball spot position includes the first target spot position and the second target spot position, and the gaze information determination module 62 includes:
[0185] A first gaze information determination unit is configured to: determine the relative position of a first light spot based on the position of the first target light spot and the pupil position in a first image; map the light spot in the first image onto a second image based on the relative position of the first light spot to obtain the position of a third target light spot; and determine the user's gaze information based on the position of the second target light spot and the position of the third target light spot; or...
[0186] The second gaze information determination unit is used to determine the relative position of the second spot based on the position of the second target spot and the pupil position in the second image; to map the spot in the second image onto the first image based on the relative position of the second spot to obtain the position of the fourth target spot; and to determine the user's gaze information based on the position of the first target spot and the position of the fourth target spot.
[0187] The gaze information determination device provided in this application embodiment can execute the gaze information determination method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of the execution method.
[0188] Example 7
[0189] Figure 10 A schematic diagram of an eye-tracking device that can be used to implement embodiments of this application is shown. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of this application described and / or claimed herein.
[0190] like Figure 10 As shown, the eye-tracking device includes a first light source module 701, which is illuminated under the control of a light source control signal; and a second light source module 702, which is illuminated under the control of a light source control signal. The first light source module 701 and the second light source module 702 are used to respond to the control of the light source control signal. The first light source module 701 and the second light source module 702 are illuminated alternately. An image acquisition device 703 is used to acquire a first image when the first light source module 701 is illuminated, and a second image when the second light source module 702 is illuminated. The eye-tracking device also includes at least one processor 704 and a memory, such as a read-only memory (ROM) 705 or a random access memory (RAM) 706, communicatively connected to the at least one processor 704. The memory stores a computer program executable by the at least one processor. The processor 704 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 705 or the computer program loaded from the storage unit 711 into the random access memory (RAM) 706. The RAM 706 can also store various programs and data required for the operation of the eye-tracking device. The processor 704, ROM 705, and RAM 706 are interconnected via bus 707. The input / output (I / O) interface 708 is also connected to bus 707.
[0191] For example, Figure 11 This is a schematic diagram of another eye-tracking device used to implement an embodiment of this application. The eye-tracking device includes a first light source module 701, a second light source module 702, and an image acquisition device 703.
[0192] Multiple components in the eye-tracking device are connected to I / O interface 708, including: input unit 709, such as a keyboard, mouse, etc.; output unit 710, such as various types of displays, speakers, etc.; storage unit 711, such as a disk, optical disk, etc.; and communication unit 712, such as a network card, modem, wireless transceiver, etc. The communication unit 712 allows the eye-tracking device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.
[0193] Processor 704 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 704 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 704 performs the various methods and processes described above, such as image acquisition methods, spot localization methods, or methods for determining gaze information.
[0194] In some embodiments, the image acquisition method, the spot localization method, and / or the gaze information determination method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 711. In some embodiments, part or all of the computer program may be loaded into and / or installed onto the eye-tracking device via ROM 705 and / or communication unit 712. When the computer program is loaded into RAM 706 and executed by processor 704, one or more steps of the image acquisition method, spot localization method, and / or gaze information determination method described above may be performed. Alternatively, in other embodiments, processor 704 may be configured to perform the image acquisition method, spot localization method, and / or gaze information determination method by any other suitable means (e.g., by means of firmware).
[0195] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.
[0196] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0197] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements the image acquisition method, spot localization method, or gaze information determination method described in any embodiment of this application.
[0198] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0199] To provide interaction with a user, the systems and techniques described herein can be implemented on an eye-tracking device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the eye-tracking device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).
[0200] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.
[0201] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.
[0202] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.
[0203] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.
Claims
1. An image acquisition method, characterized in that, include: In response to receiving a first signal, a second signal is determined, wherein the second signal includes a light source control signal, the light source control signal being used to control the first light source module and the second light source module to alternately light up or alternately turn off; Acquire the first image captured by the image acquisition device when the first light source module is lit; Acquire a second image captured by the image acquisition device when the second light source module is lit; The first image and the second image are used to determine effective ocular spot information.
2. The method according to claim 1, characterized in that, When the first signal is a trigger signal, the second signal further includes an exposure signal, which is used to control the exposure of the image acquisition device, and the exposure signal includes an exposure synchronization signal.
3. The method according to claim 1, characterized in that, When the first signal is an exposure signal, the exposure signal is sent by the image acquisition device; the exposure signal includes an exposure start signal or an exposure end signal.
4. The method according to claim 3, characterized in that, The timing of sending the light source control signal is determined based on the exposure frequency of the exposure signal.
5. The method according to claim 1, characterized in that, The light source control signal includes a light source status control signal; The light source status control signal sent to the first light source module is different from the light source status control signal sent to the second light source module; Alternatively, light source status control signals can be sent alternately to the first light source module and the second light source module to control the first light source module and the second light source module to turn on or off for a preset duration.
6. The method according to claim 1, characterized in that, The light source control signal also includes light source information, which is used to determine the light source to be lit.
7. The method according to claim 6, characterized in that, Also includes: The transmitted light source information is adjusted according to preset switching conditions.
8. The method according to claim 1, characterized in that, Both the first image and the second image include at least one effective eyeball spot, and the effective eyeball spot on different images has different effective eyeball spot information.
9. A method for locating a light spot, characterized in that, include: A first image and a second image are obtained, wherein the first image and the second image are obtained by any one of the image acquisition methods according to claims 1-8; A first set of light spots is determined based on the first image, and a second set of light spots is determined based on the second image; The effective eyeball spot position is determined based on the first and second spot sets.
10. The method according to claim 9, characterized in that, The determination of the effective ocular spot position based on the first spot set and the second spot set includes: The intersection of the first set of light spots and the second set of light spots is determined to obtain the third set of light spots; The effective eyeball spot position is determined based on the first spot set, the second spot set, and the third spot set. The effective eyeball spot position includes the first target spot position and the second target spot position.
11. A method for determining gaze information, characterized in that, include: The effective eyeball spot position is obtained, wherein the effective eyeball spot position is determined by any one of the spot positioning methods according to claims 9-10; The user's gaze information is determined based on the effective eyeball spot position.
12. The method according to claim 11, characterized in that, The effective eyeball spot position includes the first target spot position and the second target spot position; Determining the user's gaze information based on the effective eyeball spot position includes: The relative position of the first target light spot is determined based on the position of the first target light spot and the pupil position in the first image; the light spot in the first image is mapped onto the second image based on the relative position of the first light spot to obtain the position of the third target light spot; the user's gaze information is determined based on the position of the second target light spot and the position of the third target light spot; or... The relative position of the second light spot is determined based on the position of the second target light spot and the pupil position in the second image; the light spot in the second image is mapped onto the first image based on the relative position of the second light spot to obtain the position of the fourth target light spot; the user's gaze information is determined based on the position of the first target light spot and the position of the fourth target light spot.
13. An image acquisition device, characterized in that, include: A signal determination module is used to determine a second signal in response to receiving a first signal, wherein the second signal includes a light source control signal, the light source control signal being used to control the first light source module and the second light source module to alternately light up or alternately turn off; The first image acquisition module is used to acquire the first image captured by the image acquisition device when the first light source module is lit. The second image acquisition module is used to acquire the second image captured by the image acquisition device when the second light source module is lit; The first image and the second image are used to determine effective ocular spot information.
14. A light spot positioning device, characterized in that, include: The image acquisition module is used to acquire a first image and a second image, wherein the first image and the second image are obtained by any one of the image acquisition methods according to claims 1-8; A spot set determination module is used to determine a first spot set based on the first image and a second spot set based on the second image; The effective eyeball spot determination module is used to determine the position of the effective eyeball spot based on the first spot set and the second spot set.
15. A device for determining gaze information, characterized in that, include: An effective position acquisition module is used to acquire the effective eyeball spot position, wherein the effective eyeball spot position is determined by any one of the spot positioning methods according to claims 9-10; The gaze information determination module is used to determine the user's gaze information based on the effective eyeball spot position.
16. An eye-tracking device, characterized in that, The eye-tracking device includes: The first light source module and the second light source module are used to respond to the control of the light source control signal, wherein the first light source module and the second light source module are alternately lit or alternately turned off; An image acquisition device is used to acquire a first image when the first light source module is lit, and a second image when the second light source module is lit. At least one processor, and a memory communicatively connected to said at least one processor; The memory stores a computer program that can be executed by the at least one processor, which is then executed by the at least one processor to enable the at least one processor to perform the image acquisition method according to any one of claims 1-8, the spot localization method according to any one of claims 9-10, or the gaze information determination method according to any one of claims 11-12.
17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the image acquisition method of any one of claims 1-8, the spot localization method of any one of claims 9-10, or the gaze information determination method of any one of claims 11-12.
18. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the image acquisition method according to any one of claims 1-8, the spot localization method according to any one of claims 9-10, or the method for determining gaze information according to any one of claims 11-12.