Light spot and light source matching method and apparatus, device and storage medium
The eye spot map is obtained through various irradiation methods and the characteristics are extracted, which solves the problem of insufficient matching accuracy between spots and light sources in the prior art, and achieves a more efficient matching effect.
Patent Information
- Application Number
- PCT/CN2024/090259
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-05
- Filing Date
- 2024-04-28
- Publication Date
- 2024-11-14
AI Technical Summary
The existing light spot and light source matching algorithm has accuracy problems when there are fewer light spots or when users wear glasses, and it is impossible to effectively match the light spot and light source.
By controlling multiple light sources to illuminate the eyes in different illumination methods (shape, luminous power, strobe frequency), obtain the eye spot map, extract the spot characteristics, and determine the matching relationship between the spot and the light source based on these characteristics and illumination methods. .
The rate and accuracy of matching the light spot with the light source are improved, ensuring that the light spot and the light source can be correctly matched under different conditions.
Smart Images

Figure CN2024090259_14112024_PF_FP_ABST
Abstract
Description
Light spot and light source matching method, device, equipment and storage medium
[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on May 5, 2023, with application number 202310498678.6. The entire contents of the above application are incorporated by reference into this application. Technical Field
[0002] Embodiments of the present application relate to the field of eye tracking technology, for example, to a method, device, equipment, and storage medium for matching a light spot with a light source. Background Art
[0003] Near-eye eye tracking devices that use pupil-corneal reflection for eye tracking require multiple infrared lamps to form a ring because the eye has a wide range of motion. This ensures that a sufficient number of light spots are visible in any direction to support gaze tracking calculations. When multiple infrared lamps emit light that hits the eye, multiple light spots are formed. Subsequent algorithmic processing requires geometric constraints to match the infrared lamps to the corresponding light spots.
[0004] The light spot light source matching algorithm in the related art cannot obtain correct results when there are few light spots or when the user wears glasses and there is stray light.
[0005] Summary of the Invention
[0006] The embodiments of the present application provide a method, apparatus, device, and storage medium for matching a light spot with a light source, which can improve the speed and accuracy of matching a light spot with a light source.
[0007] The present invention provides a method for matching a light spot with a light source, including:
[0008] Controlling multiple light sources to illuminate the eye in multiple illumination modes to obtain an eye light spot diagram; wherein the eye light spot diagram includes multiple light spots;
[0009] Extracting a light spot feature of each light spot among a plurality of light spots in the eye light spot image;
[0010] A matching relationship between the plurality of light spots and the plurality of light sources is determined based on the light spot characteristics and the illumination mode; wherein one light spot corresponds to one light source.
[0011] The embodiment of the present application further provides a device for matching a light spot with a light source, comprising:
[0012] An eye spot pattern acquisition module is configured to control multiple light sources to illuminate the eye in multiple illumination modes to obtain an eye spot pattern; wherein the eye spot pattern includes multiple light spots;
[0013] A light spot feature extraction module is configured to extract a light spot feature of each of the multiple light spots in the eye light spot image;
[0014] A matching module is configured to determine a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot characteristics and the illumination mode; wherein one light spot corresponds to one light source.
[0015] An embodiment of the present application further provides an electronic device, comprising:
[0016] at least one processor; and
[0017] a memory communicatively connected to the at least one processor; wherein,
[0018] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform the method for matching a light spot and a light source described in the embodiment of the present application.
[0019] An embodiment of the present application further provides a computer-readable storage medium, which stores computer instructions. The computer instructions are used to enable a processor to implement the method for matching a light spot and a light source described in the embodiment of the present application when executed. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] FIG1 is a flow chart of a method for matching a light spot and a light source in Example 1 of the present application;
[0021] FIG2 is a schematic structural diagram of a device for matching a light spot and a light source in a second embodiment of the present application;
[0022] FIG3 is a schematic structural diagram of an electronic device in a third embodiment of the present application. DETAILED DESCRIPTION
[0023] The present application will be described below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application.
[0024] Eye tracking, also known as gaze tracking, is a technique for estimating eye gaze and / or fixation by measuring eye movements.
[0025] The line of sight can be understood as a three-dimensional vector, and the gaze point can be understood as the two-dimensional coordinate of the three-dimensional vector projected on a plane.
[0026] Optical recording method: Use a camera or video camera to record the subject's eye movements, i.e., obtain eye images reflecting eye movements. Eye features are extracted from the images to build a line of sight / gaze point estimation model. Eye features may include pupil position, pupil shape, iris position, iris shape, eyelid position, canthus position, and light spot (also known as Purkinje spot) position.
[0027] In optical recording methods, the eye tracking method is called Pupil Center Corneal Reflection (PCCR), and the specific method is as follows:
[0028] Part 1. Eye Image Acquisition:
[0029] The light source is directed toward the eye, and the image acquisition device photographs the eye, and the corresponding reflection point of the light source on the cornea is photographed, namely the light spot (also called Purkinje spot), thereby obtaining an eye image with the light spot.
[0030] Part 2. Line of sight / gaze estimation:
[0031] As the eyeball rotates, the relative position relationship between the pupil center and the light spot changes accordingly, and the corresponding multiple eye images with light spots collected reflect this position change relationship;
[0032] Line of sight / gaze point estimation is performed based on the position change relationship.
[0033] It can be seen from this that when using the pupil-corneal reflection method to estimate the line of sight / gaze point, it is very important to determine the matching relationship between the light spot and the light source.
[0034] Example 1
[0035] Figure 1 is a flow chart of a method for matching a light spot with a light source provided in Example 1 of the present application. This embodiment is applicable to matching light spots with light sources in eye tracking. The method can be performed by a light spot and light source matching device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, which can be a mobile terminal, a personal computer (PC), or a server. The method for matching a light spot with a light source includes the following steps:
[0036] S110, controlling multiple light sources to illuminate the eyes in different, i.e., multiple, illumination modes to obtain an eye light spot diagram.
[0037] The eye light spot diagram includes multiple light spots. The light source can be an infrared light-emitting diode (LED) lamp or a laser lamp (such as a vertical-cavity surface-emitting laser (VCSEL)). The wavelength range of the light source is between 700nm and 1200nm. In eye tracking devices, because the range of eye movement is very large, multiple LED lamps are required to form an LED light ring so that when the eyes look in any direction, there are enough light spots to support the calculation of line of sight tracking. After multiple LED lamps emit light and illuminate the eyeball, multiple light spots are formed. The image sensor is used to capture the image of the eye where the light spots are formed, thereby obtaining the eye light spot diagram.
[0038] In this embodiment, the different illumination modes may include at least one of the following: different light source shapes, different light emitting powers, and different stroboscopic frequencies.
[0039] Different light source shapes can be understood as modulating the shape of the infrared light source so that multiple light sources illuminate the eye with different light source shapes. For example, assuming there are four light sources A1, A2, A3, and A4, the four light sources illuminate the eye in the shape of A1 triangle, A2 square, A3 diamond, and A4 circle, respectively. The light spot formed by each light source in the eye spot pattern also has the shape corresponding to each light source.
[0040] Different luminous powers can be understood as controlling multiple infrared light sources to irradiate the eyes with different luminous intensities. For example, assuming that there are three light sources B1, B2, and B3, the three light sources illuminate the eyes with luminous powers of P1, P2, and P3 respectively, so that the brightness of the light spots formed by multiple light sources in the eye light spot diagram is different.
[0041] Different strobe frequencies can be understood as controlling multiple infrared light sources to illuminate the eyes at different flashing frequencies.
[0042] For example, assume there are four light sources C1, C2, C3, and C4, where the strobe frequency of light source C1 is 1 Hz (equivalent to one flash per second); the strobe frequency of light source C2 is 0.5 Hz (equivalent to one flash per two seconds); the strobe frequency of light source C3 is 0.25 Hz (equivalent to one flash per four seconds); and the strobe frequency of light source C4 is 0.125 Hz (equivalent to one flash per eight seconds). This results in the light spots formed by each light source appearing at different frequencies in multiple eye spot images.
[0043] In another embodiment, a fixed time period can be set to cause the light sources to flash at specific time points. For example, assume there are four light sources C1', C2', C3', and C4', and a time period of 5 seconds is set to control the light sources to flash in a certain pattern. For example, light source C1' is set to flash at the 1st, 2nd, and 5th seconds; light source C2' is set to flash at the 1st, 3rd, and 4th seconds; light source C3' is set to flash at the 1st and 5th seconds; and light source C4' is set to flash at the 1st, 2nd, 3rd, and 5th seconds. All light sources will only be illuminated at the set time points and will not be illuminated at other times. At the 5th second, the time period ends, and the next new time period starts from the 1st second, controlling the flashing of the light sources until the end of the current time period, and so on. In this embodiment, a common lighting time is set, that is, all four light sources are in the lighting state, which makes it easier for the system to find the correct start time. In this embodiment, the first second is set as the common lighting time. In this way, each light source has a different strobe frequency.
[0044] In another embodiment, 10 seconds can be set as a time period, and the light source C1' can be set to flash at the 1st, 2nd, 3rd, 4th, 5th, 6th, 7th, 8th, 9th, and 10th seconds; the light source C2' can be set to flash at the 1st, 3rd, 5th, 7th, and 9th seconds; the light source C3' can be set to flash at the 1st, 4th, 7th, and 10th seconds; and the light source C4' can be set to flash at the 1st, 5th, and 9th seconds.
[0045] It should be understood that the time period can be set freely as long as the flashing frequencies of the N light sources can be distinguished.
[0046] S120 , extracting a light spot feature of each of the multiple light spots in the eye light spot image.
[0047] The light spot characteristics may include: at least one of the light spot shape, the light spot brightness (or the light spot gray value) and the display period of the light spot.
[0048] Optionally, controlling multiple light sources to illuminate the eye in different illumination modes to obtain the eye spot pattern may be accomplished by controlling multiple light sources to illuminate the eye in different light source shapes to obtain the eye spot pattern. Accordingly, extracting the spot features of each spot in the eye spot pattern may be accomplished by extracting the spot shape of each spot in the eye spot pattern.
[0049] Exemplarily, the process of extracting the spot shape of each spot in the eye spot image may be: extracting pixel points whose brightness exceeds a set threshold in the eye spot image to obtain the spot area; performing shape fitting on the spot area to obtain the shape of each spot.
[0050] Optionally, controlling multiple light sources to illuminate the eye in different illumination modes to obtain the eye spot pattern may be accomplished by controlling multiple light sources to illuminate the eye at different luminous powers to obtain the eye spot pattern. Accordingly, extracting the spot features of each spot in the eye spot pattern may be accomplished by extracting the spot grayscale value of each spot in the eye spot pattern.
[0051] The process of extracting the light spot grayscale value of each light spot in the eye light spot image can be: extracting the pixel points whose brightness exceeds the set threshold in the eye light spot image to obtain multiple light spot areas; then calculating the grayscale average value in each light spot area, and taking the grayscale average value corresponding to each light spot area as the grayscale value of a light spot.
[0052] Optionally, controlling multiple light sources to illuminate the eye in different illumination modes to obtain an eye spot pattern may involve controlling the multiple light sources to illuminate the eye at different stroboscopic frequencies and capturing multiple eye spot patterns at a set acquisition frequency. Accordingly, extracting the characteristics of each spot in the eye spot pattern may involve determining the stroboscopic frequency of each spot based on the multiple eye spot patterns.
[0053] There is no requirement for the set acquisition frequency, as long as the stroboscopic frequency of each light source can be identified. In this embodiment, when the light source is lit, a light spot will be formed on the eye. If the light source is not lit, no light spot will be formed on the eye. Therefore, the stroboscopic frequency of the light source corresponds to the stroboscopic frequency of the light spot.
[0054] The process of determining the stroboscopic frequency of each light spot based on multiple eye light spot images can be: for each eye light spot image, extract the pixel points whose brightness exceeds the set threshold in each eye light spot image to obtain multiple light spots in each eye light spot image; for each light spot, extract the frequency of each light spot appearing in multiple eye light spot images as the stroboscopic frequency of each light spot.
[0055] S130: Determine a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot characteristics and the illumination mode.
[0056] One light spot corresponds to one light source. The process of determining the matching relationship between multiple light spots and multiple light sources based on the light spot characteristics and the illumination mode can be: determining the illumination mode corresponding to each light spot characteristic, determining the light source operating with the illumination mode, and thus obtaining the matching relationship between the multiple light spots and the multiple light sources.
[0057] Optionally, if multiple light sources illuminate the eyes with different light source shapes, the method of determining the matching relationship between multiple light spots and multiple light sources based on the light spot characteristics and the illumination method can be: determining the matching relationship between multiple light spots and multiple light sources based on the light spot shape and the light source shape.
[0058] For example, the light source shape corresponding to the spot shape is determined, and then a light source that operates with that light source shape is determined, thereby obtaining a matching relationship between the light spot and the light source. Taking the example in step S110, four light sources illuminate the eye in the shapes of A1 triangle, A2 square, A3 diamond, and A4 circle. The extracted light spots S1 in the resulting eye spot image are triangles, S2 squares, S3 diamonds, and S4 circles. Therefore, A1 can be matched with S1, A2 with S2, A3 with S3, and A4 with S4.
[0059] Optionally, if multiple light sources illuminate the eyes with different luminous powers, the method for determining the matching relationship between multiple light spots and multiple light sources based on the light spot characteristics and the illumination method can be: determining the matching relationship between multiple light spots and multiple light sources based on the light spot grayscale value and the luminous power of the light source.
[0060] Exemplarily, the luminous power corresponding to the grayscale value of the light spot is determined, and then a light source operating with the luminous power is determined, thereby obtaining a matching relationship between the light spot and the light source.
[0061] Optionally, the method of determining the matching relationship between multiple light spots and multiple light sources based on the light spot grayscale value and the luminous power of the light source can be: determining the corresponding relationship between the light spot grayscale value and the luminous power; and determining the matching relationship between multiple light spots and multiple light sources based on the corresponding relationship.
[0062] The correspondence between the light spot grayscale value and the luminous power can be understood as the correspondence between the grayscale value range and the luminous power. The process of determining the matching relationship between multiple light spots and multiple light sources based on this correspondence can be as follows: determining the grayscale value range within which the light spot grayscale value falls, obtaining the luminous power corresponding to this grayscale value range based on the correspondence, and obtaining a light source operating at this luminous power, thereby obtaining the correspondence between the multiple light spots and the multiple light sources.
[0063] In this embodiment, for sensors with wide dynamic range and high bit quantization, multiple light sources can emit light at different luminous powers. The eye spot diagram uses 10-bit quantized data as an example, and the image quantization order is 1024 bits. The luminous power of the light source is controlled so that the brightness of the light spot on the image is within a resolvable range. For example, taking the three light sources B1, B2, and B3 in step S110 as an example, the luminous power of B1 is P1, and its corresponding grayscale value range is 900-950, the luminous power of B2 is P2, and its corresponding grayscale value range is 800-850, and the luminous power of B3 is P2, and its corresponding grayscale value range is 700-750. In the resulting eye spot diagram, the grayscale value of light spot S5 extracted is 910, the grayscale value of light spot S6 is 820, and the grayscale value of light spot S7 is 740. Therefore, B1 and S5, B2 and S6, and B3 and S7 can be matched.
[0064] Optionally, when multiple light sources illuminate the eyes with different stroboscopic frequencies, the method of determining the matching relationship between the multiple light spots and the multiple light sources based on the light spot characteristics and the illumination method can be: determining the matching relationship between the multiple light spots and the multiple light sources based on the stroboscopic frequencies of the multiple light spots and the stroboscopic frequencies of the multiple light sources.
[0065] Exemplary methods for determining the matching relationship between the multiple light spots and the multiple light sources based on the stroboscopic frequencies of the multiple light spots and the stroboscopic frequencies of the multiple light sources may include: comparing the stroboscopic frequencies of the multiple light spots with the stroboscopic frequencies of the multiple light sources, and matching the light spots with the light sources with the same stroboscopic frequencies. The stroboscopic frequencies of the multiple light spots are obtained, and then the stroboscopic frequencies of the multiple light spots are compared with the stroboscopic frequencies of the multiple light sources, and the light spots with the same stroboscopic frequencies are matched with the light sources, thereby obtaining the matching relationship between the light spots and the light sources.
[0066] In this embodiment, for the dynamic visual sensor, multiple light sources emit light at different stroboscopic frequencies, and multiple eye spot images are collected within a certain period of time. The matching relationship between the light spot and the light source is obtained by analyzing the stroboscopic frequency of each light spot in the multiple eye spot images. For example, taking the four light sources C1, C2, C3, and C4 in step S110 as an example, the acquisition frequency is set to 1HZ, that is, one eye spot image is collected per second, then a total of 8 eye spot images are collected within 8 seconds. It is known that the stroboscopic frequency of C1 is 1Hz, the stroboscopic frequency of C2 is 0.5Hz, the stroboscopic frequency of C3 is 0.25Hz, and the stroboscopic frequency of C4 is 0.125Hz. Therefore, C1 flashed 8 times, C2 flashed 4 times, C3 flashed 2 times, and C4 flashed only 1 time. Spot S8, extracted from the eye spot pattern, appears once every 1 second, so we can determine that the stroboscopic frequency of S8 is 1Hz. Spot S9 appears once every 2 seconds, so we can determine that the stroboscopic frequency of S9 is 0.5Hz. Spot S10 appears once every 4 seconds, so we can determine that the stroboscopic frequency of S10 is 0.25Hz. Spot S11 appears only once, so we can determine that the stroboscopic frequency of S11 is 0.125MHz. We can then match C1 with S8, C2 with S9, C3 with S10, and C4 with S11. In this way, the four lights and the four spots can be distinguished by the stroboscopic frequency, thus matching the spots with the light sources.
[0067] The technical solution of the embodiment of the present application controls multiple light sources to illuminate the eye in different illumination modes to obtain an eye spot diagram; wherein the eye spot diagram includes multiple light spots; extracts the spot characteristics of each of the multiple light spots in the eye spot diagram; and determines the matching relationship between the multiple light spots and the multiple light sources based on the spot characteristics and the illumination mode; wherein each light spot corresponds to one light source. The method for matching light spots and light sources provided in the embodiment of the present application determines the matching relationship between the multiple light spots and the multiple light sources based on the spot characteristics and the illumination mode, which can improve the speed and accuracy of light spot and light source matching.
[0068] Example 2
[0069] FIG2 is a schematic structural diagram of a device for matching a light spot and a light source provided in a second embodiment of the present application. As shown in FIG2 , the device includes:
[0070] The eye light spot map acquisition module 210 is configured to control multiple light sources to illuminate the eyes in multiple illumination modes to obtain an eye light spot map; wherein the eye light spot map includes multiple light spots; the light spot feature extraction module 220 is configured to extract the light spot features of each of the multiple light spots in the eye light spot map; the matching module 230 is configured to determine the matching relationship between the multiple light spots and the multiple light sources based on the light spot features and the illumination mode; wherein, one light spot corresponds to one light source.
[0071] Optionally, the multiple illumination modes include at least one of the following: multiple light source shapes, multiple light emitting powers, and multiple stroboscopic frequencies.
[0072] Optionally, the eye spot pattern acquisition module 210 is configured to control multiple light sources to illuminate the eyes in multiple light source shapes to obtain the eye spot pattern.
[0073] Optionally, the light spot feature extraction module 220 is configured to extract the light spot shape of each light spot in the eye light spot image.
[0074] Optionally, the matching module 230 is configured to determine a matching relationship between a plurality of light spots and a plurality of light sources based on a light spot shape and a light source shape.
[0075] Optionally, the eye light spot pattern acquisition module 210 is configured to control multiple light sources to illuminate the eyes with multiple light powers to obtain the eye light spot pattern.
[0076] Optionally, the light spot feature extraction module 220 is configured to extract the light spot grayscale value of each light spot in the eye light spot image.
[0077] Optionally, the matching module 230 is configured to determine a matching relationship between the plurality of light spots and the plurality of light sources based on the grayscale value of the light spots and the luminous power of the light sources.
[0078] Optionally, the matching module 230 is configured to: determine a corresponding relationship between a light spot grayscale value and a light emitting power; and determine a matching relationship between a plurality of light spots and a plurality of light sources based on the corresponding relationship.
[0079] Optionally, the eye spot image acquisition module 210 is configured to control multiple light sources to illuminate the eyes at multiple stroboscopic frequencies to obtain multiple eye spot images.
[0080] Optionally, the light spot feature extraction module 220 is configured to determine the stroboscopic frequency of each light spot according to a plurality of eye light spot images.
[0081] Optionally, the matching module 230 is configured to determine a matching relationship between the multiple light spots and the multiple light sources based on the stroboscopic frequencies of the multiple light spots and the stroboscopic frequencies of the multiple light sources.
[0082] Optionally, the matching module 230 is configured to compare the stroboscopic frequencies of the multiple light spots with the stroboscopic frequencies of the multiple light sources, and match the light spots with the light sources having the same stroboscopic frequencies.
[0083] The above device can execute the methods provided by all the above embodiments of this application, and has the functional modules and effects corresponding to the execution of the above methods. For technical details not fully described in this embodiment, please refer to the methods provided by all the above embodiments of this application.
[0084] Example 3
[0085] FIG3 shows a block diagram of an electronic device 10 that can be used to implement an embodiment of the present application. The electronic device can be a digital computer in various forms, such as a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also be a mobile device in various forms, such as a personal digital processing, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.
[0086] As shown in FIG3 , the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 and a random access memory (RAM) 13, that is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the ROM 12 or the computer program loaded from the storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.
[0087] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0088] The processor 11 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for matching a light spot with a light source.
[0089] In some embodiments, the method for matching a light spot with a light source may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for matching a light spot with a light source described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for matching a light spot with a light source in any other appropriate manner (e.g., by means of firmware).
[0090] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0091] Computer programs for implementing the methods of the present application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0092] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0093] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD)) configured to display information to the user (a monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be configured to provide interaction with a user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0094] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0095] A computing system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by running computer programs on the respective computers. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a host product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and virtual private server (VPS) services.
[0096] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the multiple steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this application can be achieved. This is not limited herein.
Claims
1. A method for matching a light spot with a light source, include: Controlling multiple light sources to illuminate the eye in multiple illumination modes to obtain an eye light spot diagram; wherein the eye light spot diagram includes multiple light spots; Extracting the light spot feature of each light spot among the multiple light spots in the eye light spot image; A matching relationship between the plurality of light spots and the plurality of light sources is determined based on the light spot characteristics and the illumination mode; wherein one light spot corresponds to one light source.
2. The method according to claim 1, in, The multiple irradiation modes include at least one of the following: multiple light source shapes, multiple light emitting powers, and multiple stroboscopic frequencies.
3. The method according to claim 2, in, Control multiple light sources to illuminate the eyes in a variety of ways to obtain eye spot patterns, including: Controlling multiple light sources to illuminate the eyes in various light source shapes to obtain eye light spot images; Extracting the light spot feature of each light spot in the eye light spot image includes: Extracting the spot shape of each spot in the eye spot image; Determining a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot feature and the illumination mode includes: A matching relationship between the plurality of light spots and the plurality of light sources is determined based on the light spot shape and the light source shape.
4. The method according to claim 2, in, Control multiple light sources to illuminate the eyes in a variety of ways to obtain eye spot patterns, including: Controlling multiple light sources to illuminate the eyes with various light emitting powers to obtain eye light spot images; Extracting the light spot feature of each light spot in the eye light spot image includes: Extracting the spot gray value of each spot in the eye spot image; Determining a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot feature and the illumination mode includes: A matching relationship between the plurality of light spots and the plurality of light sources is determined based on the grayscale value of the light spots and the luminous power of the light sources.
5. The method according to claim 4, in, Determining a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot grayscale value and the luminous power of the light source includes: Determine the corresponding relationship between the light spot gray value and the luminous power; A matching relationship between the plurality of light spots and the plurality of light sources is determined based on the corresponding relationship.
6. The method according to claim 2, in, Control multiple light sources to illuminate the eyes in a variety of ways to obtain eye spot patterns, including: Control multiple light sources to illuminate the eyes at multiple stroboscopic frequencies to obtain multiple eye spot images; Extracting the light spot feature of each light spot in the eye light spot image includes: Determine the stroboscopic frequency of each light spot according to the multiple eye light spot images; Determining a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot feature and the illumination mode includes: A matching relationship between the plurality of light spots and the plurality of light sources is determined based on the stroboscopic frequencies of the plurality of light spots and the stroboscopic frequencies of the plurality of light sources.
7. The method according to claim 6, in, Determining a matching relationship between the plurality of light spots and the plurality of light sources based on the stroboscopic frequencies of the plurality of light spots and the stroboscopic frequencies of the plurality of light sources includes: The stroboscopic frequencies of the multiple light spots are compared with the stroboscopic frequencies of the multiple light sources, and the light spots with the same stroboscopic frequencies are matched with the light sources.
8. A device for matching a light spot with a light source, include: An eye spot image acquisition module is configured to control multiple light sources to illuminate the eyes in multiple illumination modes to obtain an eye spot image; wherein the eye spot image includes multiple spots; A light spot feature extraction module, configured to extract a light spot feature of each light spot in the eye light spot image; A matching module is configured to determine a matching relationship between the plurality of light spots and the plurality of light sources based on the light spot characteristics and the illumination mode; wherein one light spot corresponds to one light source.
9. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method for matching a light spot with a light source according to any one of claims 1 to 7.
10. A computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a processor to implement the method for matching a light spot and a light source according to any one of claims 1 to 7 when the instructions are executed.
Citation Information
Patent Citations
Light source and light spot coupling method and device
CN108510542A
Light spot tracking method and device, electronic equipment and storage medium
CN115965653A
Eye tracking device and intelligent glasses
CN218413524U
Lighting and light spot image data acquisition method and apparatus
WO2021258343A1
Imaging method
WO2022017273A1
