Light source position calibration method, device and storage medium

Abstract

The present application relates to the field of smart wearable devices. Provided are a light source position calibration method, a device and a storage medium. The method comprises: using an image collection apparatus to collect user eye images; extracting from the eye images corresponding light spot two-dimensional coordinates and light spot three-dimensional coordinates of light spots formed by a light source, and, on the basis of the light spot two-dimensional coordinates, determining corresponding light source two-dimensional coordinates of the light source; and, on the basis of the light source two-dimensional coordinates, the light spot three-dimensional coordinates and preset coordinate parameters, determining corresponding light source three-dimensional coordinates of the light source, so as to calibrate the light source position of the light source in a smart wearable device on the basis of the light source three-dimensional coordinates. The present application calculates a position of a light source device on the basis of light spots formed by the light source in cornea areas of eye images, thus reducing the complexity of light source position calibration.

Description

Light source position calibration method, device and storage medium Technical Field

[0001] The present application relates to the field of smart wearable devices, and in particular to a method, device and storage medium for calibrating the position of a light source. Background Art

[0002] AR glasses usually use light source devices (such as infrared LED lamp beads) to track eye movements, and determine the direction of human eye focus based on the light spot formed by the light source device on the cornea. In the process of eye tracking, it is necessary to obtain the position of the light source device in order to calculate the direction of the human eye's gaze based on the position of the light source device. Therefore, whether the obtained position of the light source device is accurate determines the user experience of AR glasses. However, the methods of obtaining the position of the light source device in related technologies are often cumbersome. How to quickly and easily determine the position of the light source device has become an urgent problem to be solved.

[0003] Application Contents

[0004] The main purpose of this application is to provide a light source position calibration method, device and storage medium, aiming to calibrate the light source position in a smart wearable device.

[0005] In a first aspect, the present application provides a method for calibrating a light source position, which is applied to a smart wearable device. The smart wearable device is provided with a light source and an image acquisition device. The method for calibrating the light source position comprises the following steps:

[0006] Using the image acquisition device to acquire an eye image of the user;

[0007] Extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot;

[0008] The three-dimensional coordinates of the light source corresponding to the light source are determined according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

[0009] In a second aspect, the present application also provides a smart wearable device, which includes a processor, a memory, and a computer program stored in the memory and executable by the processor, wherein when the computer program is executed by the processor, the light source position calibration method as described above is implemented.

[0010] In a third aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the method for calibrating the position of a light source as described above is implemented.

[0011] The present application provides a method, device, and computer storage medium for calibrating the position of a light source. The present application uses the image acquisition device to capture an eye image of a user; extracts the two-dimensional coordinates and three-dimensional coordinates of the light spot corresponding to the light spot formed by the light source from the eye image, and determines the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot; determines the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot, and preset coordinate parameters, so as to calibrate the position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source. Compared with the technical solution in the related art that transmits the light spot image through an optical lens, the present solution directly calculates the position of the light source device through the light spot formed by the light source in the cornea area in the eye image, thereby reducing the complexity of the light source position calibration. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] FIG1 is a schematic flow chart of a method for calibrating a light source position according to an embodiment of the present application;

[0014] FIG2 is a schematic diagram of a flow chart of sub-steps of step S102 provided in an embodiment of the present application;

[0015] FIG3 is a schematic diagram of a flow chart of sub-steps of step S103 provided in an embodiment of the present application;

[0016] FIG4 is a schematic block diagram of a light source position calibration device provided in one embodiment of the present application;

[0017] FIG5 is a schematic block diagram of the structure of a smart wearable device according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0019] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined, or partially merged, so the actual execution order may vary depending on the actual situation.

[0020] Embodiments of the present application provide a method, device, and computer-readable storage medium for calibrating a light source position.

[0021] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0022] Please refer to Figure 1, which is a flow chart of a method for calibrating the position of a light source provided in an embodiment of the present application. The method for calibrating the position of a light source can be used in smart wearable devices, and the smart wearable devices can be, for example, AR (Augmented Reality) glasses, and the various steps of the method are executed by the processor in the AR glasses. Of course, it is not limited to this. The method can also be implemented by a terminal or server that is connected to the smart wearable device for communication, wherein the terminal can be an electronic device such as a mobile phone, tablet computer, laptop computer, desktop computer, personal digital assistant and wearable device; the server can be an independent server, a server cluster, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0023] Exemplarily, the smart wearable device is provided with a light source and an image acquisition device.

[0024] As shown in FIG1 , the method for calibrating the light source position includes steps S101 to S103 .

[0025] Step S101: Use the image acquisition device to acquire an eye image of the user.

[0026] Exemplarily, the smart wearable device is provided with an image acquisition device for acquiring eye images, for example, a camera is provided on the frame of AR glasses, and the camera may be an infrared camera for acquiring infrared images of the user's eyes.

[0027] For example, the smart wearable device can identify whether it is currently in the wearing state through sensors, such as through temperature sensors, human body sensors, etc., and start up when it is in the wearing state. Of course, it is not limited to this. It can also determine whether the smart wearable device is in the startup state in response to the startup operation of the user's smart wearable device. This is not limited here.

[0028] For example, the wearable device can also identify the user wearing it through iris recognition. If the relevant data of the user is stored in the database, the relevant data of the user's cornea and light source calibration can be directly determined through the user data in the database, avoiding repeated calculations and improving the efficiency of light source calibration.

[0029] In some embodiments, the step of acquiring the user's eye image using the image acquisition device includes:

[0030] identifying the number of light spots in the eye image;

[0031] Determining the number of target images corresponding to the number of light spots according to the corresponding relationship between the number of light spots and the number of images;

[0032] The image acquisition device is used to acquire eye images corresponding to the target number of images.

[0033] For example, the number of eye images should ensure that there is a unique solution for the three-dimensional coordinates of the light source, and the three-dimensional coordinates of the light source are determined based on the light spots formed by the light source on the user's eyes. Therefore, the number of target images required to determine the three-dimensional coordinates of the light source also needs to be determined based on the number of light spots in the eye image.

[0034] Exemplarily, the light spots in the eye image are identified by an edge detection algorithm to determine the number of light spots in any frame of the eye image. Then, based on the number of light spots, according to a preset target correspondence, the number of target images required to determine the three-dimensional coordinates of the light source under the current number of light spots is determined, and the eye image is obtained according to the number of target images, wherein the target correspondence is the correspondence between the number of light spots and the number of images, and the number of images corresponding to the number of light spots can be determined according to the target correspondence.

[0035] For example, it is also possible to take a number of candidate eye images that is greater than the number of target images, and then select the target number of eye images from the candidate eye images based on the image quality of the eye images to ensure that the acquired eye images have a higher image quality. This is not limited here.

[0036] For example, compared with obtaining eye images through an optical waveguide, the embodiment of the present application directly obtains user eye images through an optical camera, reducing the complexity of light source position calibration.

[0037] Step S102 : extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot corresponding to the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot.

[0038] Illustratively, according to the eye image acquired in step S101 , based on the positional relationship between the light spot in the eye image, the light source, and the corneal center, the three-dimensional coordinates of the light spot and the two-dimensional coordinates of the light source can be determined.

[0039] It can be understood that the coordinate systems of the above-mentioned two-dimensional coordinates and three-dimensional coordinates can be selected arbitrarily without affecting the relationship between the coordinates. For the convenience of calculation, the center position of the eye image can be determined as the coordinate origin of the coordinate system of each two-dimensional coordinate, and the optical center of the camera can be determined as the coordinate origin of the coordinate system of each three-dimensional coordinate. Of course, it is not limited to this and will not be elaborated here.

[0040] Please refer to FIG. 2 , which is a flowchart of sub-steps of step S102 provided in an embodiment of the present application.

[0041] As shown in FIG. 2 , in some embodiments, step S102 extracts the two-dimensional coordinates and the three-dimensional coordinates of the light spot corresponding to the light spot formed by the light source from the eye image, including:

[0042] Step S1021: Identify the light spot in the eye image to obtain the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source;

[0043] Step S1022: Based on the two-dimensional coordinates of the light spot, according to preset corneal radius information and the three-dimensional coordinates of the corneal center, determine the three-dimensional coordinates of the light spot.

[0044] For example, the eye image may be pre-processed, for example, by adjusting the brightness of the eye image and / or increasing the contrast of the eye image to make the light spot more prominent in the eye image.

[0045] Exemplarily, an edge detection algorithm and an image segmentation algorithm are used to identify the pre-processed eye image to obtain the two-dimensional coordinates of the light spot formed by the light source on the cornea.

[0046] In some embodiments, identifying the light spot in the eye image to obtain the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source includes:

[0047] Identifying the light spot in the eye image to obtain the light spot pixel coordinates of the light spot in the eye image;

[0048] The pixel coordinates of the light spot are converted to obtain the two-dimensional coordinates of the light spot in the image space.

[0049] For example, the pixel coordinates (glint_px, glint_py) of the light spot directly obtained in the eye image are the coordinates of the light spot in the image coordinate system. For example, the origin of the image coordinate system is usually the position of the pixel at the upper left corner of the image. Therefore, it is necessary to convert the pixel coordinates (glint_px, glint_py) of the light spot in the image coordinate system into the two-dimensional coordinates (glint_x, glint_y) of the light spot in the two-dimensional coordinate system to achieve standardization for subsequent calculations. For example, the origin of the two-dimensional coordinate system is usually the position of the pixel at the center of the image. According to the coordinate difference between the center pixel of the image and the pixel at the upper left corner of the image, the coordinates can be converted from the image coordinate system to the two-dimensional coordinate system. For example, if the coordinate difference is (a, b), then the coordinates in the two-dimensional coordinate system are obtained by subtracting a from the horizontal coordinate value of the coordinate in the image coordinate system and b from the vertical coordinate value, that is, (glint_px-a, glint_py-b)=(glint_x, glint_y), where the sizes of a and b are determined according to actual conditions.

[0050] As shown in FIG2 , in some embodiments, step S102 determines the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot, and further includes:

[0051] Step S1023 : determining the two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source according to the collinear relationship among the two-dimensional coordinates of the light spot, the two-dimensional coordinates of the cornea center, and the two-dimensional coordinates of the light source.

[0052] For example, taking the light source as an LED lamp bead, assume that the projection of the LED lamp bead in the image space is the LED lamp bead two-dimensional coordinate (led_x, led_y). Since the light spot is formed by the light emitted by the LED lamp bead reflecting on the cornea, there is a collinear relationship between the LED lamp bead two-dimensional coordinate (led_x, led_y), the cornea center two-dimensional coordinate (cornea_x, cornea_y), and the light spot two-dimensional coordinate (glint_x, glint_y). Based on the relationship of equal slope, the equation is constructed: (led_y-glint_y) / (led_x-glint_x)=(glint_y-cornea_y) / (glint_x-cornea_x). It can be understood that the number of equations that can be constructed for each frame of the eye image is equal to the number of light spots in the eye image. Taking the number of light spots as 4 as an example, that is, 4 LED lamp beads are reflected on the cornea to form 4 light spots, then each frame of the eye image can construct 4 equations, and there are a total of 10 unknowns in the 4 equations (including 4 LED lamp beads two-dimensional coordinates and 1 cornea center two-dimensional coordinate). Among them, since the LED lamp beads and the image acquisition device are fixedly set on the smart wearable device (such as the frame of AR glasses), the two-dimensional coordinates of the LED lamp beads in each frame of the eye image are the same, and only the two-dimensional coordinates of the cornea center (cornea_x, cornea_y) are different in the eye images of different frames. Therefore, each additional frame of eye image will increase the two unknowns corresponding to the two-dimensional coordinates of the cornea center. When the number of eye images is 4, the number of unknowns and the number of equations are both 16, and the unknowns in the equations have a unique solution. Therefore, when the number of light spots is 4, 4 frames of eye images are obtained.

[0053] For example, when the number of light spots is other values, the number of eye images acquired can also be other values. For example, when the number of light spots is 3, 6 frames of eye images are acquired, so that the number of unknowns and the number of equations are both 18; when the number of light spots is 6, 3 frames of eye images are acquired, so that the number of unknowns and the number of equations are both 18.

[0054] In some embodiments, determining the three-dimensional coordinates of the light spot based on the two-dimensional coordinates of the light spot according to preset corneal radius information and the three-dimensional coordinates of the corneal center includes:

[0055] Based on preset camera intrinsic parameter data, the two-dimensional coordinates of the light spot are converted into normalized coordinates of the light spot;

[0056] Based on the normalized light spot coordinates, determining the light spot depth information of the three-dimensional light spot coordinates according to the equivalent relationship between the distance between the three-dimensional light spot coordinates and the three-dimensional coordinates of the corneal center and the corneal radius information;

[0057] The three-dimensional coordinates of the light spot are determined according to the light spot depth information and the normalized coordinates of the light spot.

[0058] For example, the two-dimensional coordinates of the light spot (glint_x, glint_y) collected from the eye image are converted to the normalized coordinates of the light spot (x', y'). Specifically, the coordinate conversion is performed based on the camera intrinsic parameter data corresponding to the image acquisition device. The camera intrinsic parameter data is the inherent parameters of the image acquisition device, for example, the intrinsic parameter matrix K:

[0059] Among them, f x Indicates the focal length of the camera in the x direction, f y Indicates the focal length of the camera in the y direction, (c x , c y ) represents the principal point coordinates of the image, and the normalized coordinates of the spot (x', y') can be calculated using the following formula:

[0060] Exemplarily, the position of the light spot relative to the camera optical center is determined by converting the light spot two-dimensional coordinates (glint_x, glint_y) into the light spot normalized coordinates (x', y'), and the depth information of the position is ignored to facilitate subsequent calculations.

[0061] It can be understood that the distance between the position of the light spot in space and the position of the corneal center in space is equal to the size of the user's corneal radius R. The position of the corneal center in space, i.e., the three-dimensional coordinates of the corneal center, can be determined based on the solved two-dimensional coordinates of the corneal center (cornea_x, cornea_y) and preset corneal center depth information. The corneal center depth information and the corneal radius R can be pre-set based on actual measurement data or conventional values, or can be extracted by other methods, which are not limited here.

[0062] For example, assuming that the three-dimensional coordinates of the corneal center C = (C x ,C y ,C z ), the three-dimensional coordinates of the light spot G = (X, Y, Z), and the distance between the three-dimensional coordinates of the corneal center and the three-dimensional coordinates of the light spot is the corneal radius R, which can be obtained from the following equation:

[0063] For example, according to the characteristics of normalized coordinates, the three-dimensional coordinates of the light spot and the normalized coordinates of the light spot have the following relationship: X=Z·x′, Y=Z·y′, and substituting into the above formula yields:

[0064] Eliminating the square root gives:

[0065] (Z·x′-C x) 2 +(Z·y′-C y ) 2 +(ZC z ) 2 =R 2

[0066] The solution to the unknown variable Z in the above quadratic equation is obtained. Since we are only interested in physically meaningful solutions in practical applications, we choose the solution that makes Z positive here, and calculate the three-dimensional coordinates of the light spot G = (X, Y, Z) based on the relationship X = Z·x′ and Y = Z·y′.

[0067] For example, since the positional relationship error between the light source, the light spot, and the corneal center is relatively small, the light source position is calibrated based on the positional relationship between the light source, the light spot, and the corneal center, thereby improving the accuracy of the light source position.

[0068] Step S103: determining the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

[0069] Exemplarily, the preset coordinate parameters include the three-dimensional coordinates of the camera optical center corresponding to the image acquisition device and the three-dimensional coordinates of the cornea center corresponding to the user's eye. The three-dimensional coordinates of the camera optical center are inherent parameters of the image acquisition device and can be determined based on the factory information of the image acquisition device; the three-dimensional coordinates of the cornea center can be determined based on the two-dimensional coordinates of the cornea center (cornea_x, cornea_y) calculated in step S102 and preset cornea center depth information, but are not limited to these parameters and are not defined herein.

[0070] Please refer to FIG. 3 , which is a flowchart of sub-steps of step S103 provided in an embodiment of the present application.

[0071] As shown in FIG3 , in some embodiments, step S103 determines the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot, and preset coordinate parameters, including:

[0072] Step S1031, determining light source depth information of the light source based on a coplanar relationship among the preset three-dimensional coordinates of the corneal center, the three-dimensional coordinates of the light spot, the preset three-dimensional coordinates of the camera optical center, and the three-dimensional coordinates of the light source;

[0073] Step S1032: Determine the three-dimensional coordinates of the light source according to the light source depth information and the two-dimensional coordinates of the light source.

[0074] For example, after the two-dimensional coordinates of the light source are calculated in step S102 , the light source depth information needs to be solved, so as to determine the three-dimensional coordinates of the light source according to the two-dimensional coordinates of the light source and the light source depth information.

[0075] For example, assuming that the three-dimensional coordinates of the light source are L=(X l ,Y l ,Z l ), according to the geometric coplanarity, the three-dimensional coordinates of the light source L, the three-dimensional coordinates of the light spot G, the three-dimensional coordinates of the cornea center C, and the coordinates of the camera optical center O are coplanar, and the following equation can be obtained:

[0076] (LO)×(GO)·(CO)=0

[0077] The values ​​of the three-dimensional coordinates of the light spot G, the three-dimensional coordinates of the corneal center C, and the coordinates of the optical center of the camera O are known information. The two-dimensional coordinates of the light source (x', y') are also obtained by calculation in step S102. The only unknown number in the above formula is the light source depth information, that is, the Z coordinate in the three-dimensional coordinates of the light source. The light source depth information is obtained by solving the above equation. The position of the light source of the smart wearable device in space can be determined according to the two-dimensional coordinates of the light source (x', y') and the light source depth information, and the light source position is calibrated. The process of solving the equation will not be repeated here.

[0078] In some embodiments, the smart wearable device includes a first visual area and a second visual area, the first visual area and the second visual area are respectively provided with a first image acquisition device and a second image acquisition device, and using the image acquisition device to acquire the user's eye image includes:

[0079] Acquire a first eye image by a first image acquisition device, wherein the light spot in the first eye image is formed by a first light source disposed in a first visual area;

[0080] A second eye image is acquired by a second image acquisition device, wherein the light spot in the second eye image is formed by a second light source arranged in a second visual area.

[0081] For example, taking AR glasses as an example, the first visual area and the second visual area represent the left and right lenses respectively. In order to accurately obtain the respective sight directions of the left and right eyes, light sources and image acquisition devices are respectively set at the left and right lenses to obtain the three-dimensional coordinates of the light sources corresponding to the left and right eyes, so as to calculate the respective gaze directions of the left and right eyes.

[0082] Of course, this is not limited to this. Under normal circumstances, there is only a small difference in the gaze direction of the left and right eyes of humans. It is also possible to set a light source and image acquisition device on only one side of the lens, and then infer the gaze of the other eye based on the gaze of one eye. There is no limitation here.

[0083] The method for calibrating the light source position provided in the above embodiment uses the image acquisition device to capture the user's eye image; extracts the two-dimensional coordinates and three-dimensional coordinates of the light spot corresponding to the light spot formed by the light source from the eye image, and determines the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot; determines the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot, and preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source. The position of the light source device is calculated by the light spot formed by the light source in the corneal area in the eye image, thereby reducing the complexity of the light source position calibration, and ensuring the accuracy of the light source calibration through the relationship between the light source, the light spot, and the corneal center, thereby improving the user experience of using the smart wearable device.

[0084] Please refer to Figure 4, which is a schematic diagram of a light source position calibration device provided in an embodiment of the present application. The light source position calibration device can be configured in a server or terminal to execute the aforementioned light source position calibration method.

[0085] As shown in FIG4 , the light source position calibration device includes: an image acquisition module 110 , a light spot extraction module 120 , and a light source calibration module 130 .

[0086] An image acquisition module 110 is configured to acquire an eye image of a user using the image acquisition device;

[0087] a light spot extraction module 120 for extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot;

[0088] The light source calibration module 130 is used to determine the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

[0089] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0090] The methods and apparatus of the present application can be used in a wide variety of general or specialized computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multi-processor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0091] Exemplarily, the above method and apparatus may be implemented in the form of a computer program, which may be run on a smart wearable device as shown in FIG5 .

[0092] Please refer to FIG5 , which is a schematic block diagram of the structure of a smart wearable device provided in an embodiment of the present application.

[0093] As shown in FIG5 , the smart wearable device includes a processor, a memory, and a network interface connected via a system bus, wherein the memory may include a storage medium and an internal memory.

[0094] The storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any one of the light source position calibration methods.

[0095] The processor is used to provide computing and control capabilities to support the operation of the entire smart wearable device.

[0096] The internal memory provides an environment for the operation of the computer program in the storage medium. When the computer program is executed by the processor, the processor can execute any method for calibrating the position of the light source.

[0097] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art will understand that the structure shown in FIG5 is only a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the smart wearable device to which the solution of the present application is applied. A specific smart wearable device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0098] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc.

[0099] In one embodiment, the processor is configured to execute a computer program stored in the memory to implement the following steps:

[0100] Using the image acquisition device to acquire an eye image of the user;

[0101] Extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot;

[0102] The three-dimensional coordinates of the light source corresponding to the light source are determined according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

[0103] It should be noted that, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the calibration of the light source position described above can refer to the corresponding process in the aforementioned light source position calibration control method embodiment, and will not be repeated here.

[0104] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the various embodiments of the light source position calibration method of the present application.

[0105] The computer-readable storage medium may be an internal storage unit of the smart wearable device described in the aforementioned embodiment, such as a hard disk or memory of the smart wearable device. The computer-readable storage medium may also be an external storage device of the smart wearable device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc., equipped on the smart wearable device.

[0106] It should be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0107] It should also be understood that the term "and / or" used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system that includes a series of elements includes not only those elements, but also other elements that are not explicitly listed, or also includes elements that are inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system that includes the element.

[0108] The serial numbers of the embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments. The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for calibrating a light source position, characterized in that: The method is applied to a smart wearable device, wherein the smart wearable device is provided with a light source and an image acquisition device, and the method comprises: Using the image acquisition device to acquire an eye image of the user; Extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot; The three-dimensional coordinates of the light source corresponding to the light source are determined according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

2. The method for calibrating the light source position according to claim 1, wherein: The step of extracting the two-dimensional coordinates and the three-dimensional coordinates of the light spot corresponding to the light spot formed by the light source from the eye image includes: Identifying the light spot in the eye image to obtain the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source; Based on the two-dimensional coordinates of the light spot, the three-dimensional coordinates of the light spot are determined according to preset corneal radius information and the three-dimensional coordinates of the corneal center.

3. The method for calibrating the light source position according to claim 2, wherein: The step of determining the three-dimensional coordinates of the light spot based on the two-dimensional coordinates of the light spot and according to preset corneal radius information and the three-dimensional coordinates of the corneal center includes: Based on preset camera intrinsic parameter data, the two-dimensional coordinates of the light spot are converted into normalized coordinates of the light spot; Based on the normalized light spot coordinates, determining the light spot depth information of the three-dimensional light spot coordinates according to the equivalent relationship between the distance between the three-dimensional light spot coordinates and the three-dimensional coordinates of the corneal center and the corneal radius information; The three-dimensional coordinates of the light spot are determined according to the light spot depth information and the normalized coordinates of the light spot.

4. The method for calibrating the light source position according to claim 2, wherein: The identifying the light spot in the eye image to obtain the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source includes: Identifying the light spot in the eye image to obtain the light spot pixel coordinates of the light spot in the eye image; The pixel coordinates of the light spot are converted to obtain the two-dimensional coordinates of the light spot in the image space.

5. The method for calibrating the light source position according to claim 1, wherein: The determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot includes: The two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source are determined according to the collinear relationship among the two-dimensional coordinates of the light spot, the two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source.

6. The method for calibrating the light source position according to claim 1, wherein: The determining of the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and preset coordinate parameters includes: Determining light source depth information of the light source based on a coplanar relationship among preset three-dimensional coordinates of the corneal center, the three-dimensional coordinates of the light spot, the preset three-dimensional coordinates of the camera optical center, and the three-dimensional coordinates of the light source; The three-dimensional coordinates of the light source are determined according to the light source depth information and the two-dimensional coordinates of the light source.

7. The method for calibrating the light source position according to claim 1, wherein: The step of collecting the user's eye image by using the image collection device includes: identifying the number of light spots in the eye image; Determining the number of target images corresponding to the number of light spots according to the corresponding relationship between the number of light spots and the number of images; The image acquisition device is used to acquire eye images corresponding to the target number of images.

8. The method for calibrating the light source position according to any one of claims 1 to 7, characterized in that: The smart wearable device includes a first visual area and a second visual area, wherein the first visual area and the second visual area are respectively provided with a first image acquisition device and a second image acquisition device, and the use of the image acquisition devices to acquire the user's eye image includes: Acquire a first eye image by a first image acquisition device, wherein the light spot in the first eye image is formed by a first light source disposed in a first visual area; A second eye image is acquired by a second image acquisition device, wherein the light spot in the second eye image is formed by a second light source arranged in a second visual area.

9. A smart wearable device, characterized in that: The smart wearable device includes a processor, a memory, and a computer program stored in the memory and executable by the processor. The smart wearable device is further provided with a light source and an image acquisition device, wherein when the computer program is executed by the processor, the following is achieved: Using the image acquisition device to acquire an eye image of the user; Extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot; The three-dimensional coordinates of the light source corresponding to the light source are determined according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

10. The smart wearable device according to claim 9, characterized in that: The processor, in the process of extracting the two-dimensional coordinates and the three-dimensional coordinates of the light spot corresponding to the light spot formed by the light source from the eye image, is used to implement: Identifying the light spot in the eye image to obtain the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source; Based on the two-dimensional coordinates of the light spot, the three-dimensional coordinates of the light spot are determined according to preset corneal radius information and the three-dimensional coordinates of the corneal center.

11. The smart wearable device according to claim 10, wherein: The processor, in the process of determining the three-dimensional coordinates of the light spot based on the two-dimensional coordinates of the light spot and according to preset corneal radius information and the three-dimensional coordinates of the corneal center, is used to implement: Based on preset camera intrinsic parameter data, the two-dimensional coordinates of the light spot are converted into normalized coordinates of the light spot; Based on the normalized light spot coordinates, determining the light spot depth information of the three-dimensional light spot coordinates according to the equivalent relationship between the distance between the three-dimensional light spot coordinates and the three-dimensional coordinates of the corneal center and the corneal radius information; The three-dimensional coordinates of the light spot are determined according to the light spot depth information and the normalized coordinates of the light spot.

12. The smart wearable device according to claim 10, characterized in that: The processor, in the process of realizing the recognition of the light spot in the eye image and obtaining the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source, is used to realize: Identifying the light spot in the eye image to obtain the light spot pixel coordinates of the light spot in the eye image; The pixel coordinates of the light spot are converted to obtain the two-dimensional coordinates of the light spot in the image space.

13. The smart wearable device according to claim 9, characterized in that: The processor, in the process of determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot, is configured to: The two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source are determined according to the collinear relationship among the two-dimensional coordinates of the light spot, the two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source.

14. The smart wearable device according to claim 9, characterized in that: The processor, in the process of determining the three-dimensional coordinates of the light source corresponding to the light source according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, is used to implement: Determining light source depth information of the light source based on a coplanar relationship among preset three-dimensional coordinates of the corneal center, the three-dimensional coordinates of the light spot, the preset three-dimensional coordinates of the camera optical center, and the three-dimensional coordinates of the light source; The three-dimensional coordinates of the light source are determined according to the light source depth information and the two-dimensional coordinates of the light source.

15. A computer-readable storage medium, characterized in that The computer readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the computer program implements: Capturing an eye image of a user using an image acquisition device; Extracting the two-dimensional coordinates and three-dimensional coordinates of the light spot formed by the light source from the eye image, and determining the two-dimensional coordinates of the light source corresponding to the light source based on the two-dimensional coordinates of the light spot; The three-dimensional coordinates of the light source corresponding to the light source are determined according to the two-dimensional coordinates of the light source, the three-dimensional coordinates of the light spot and the preset coordinate parameters, so as to calibrate the light source position of the light source in the smart wearable device according to the three-dimensional coordinates of the light source.

16. The computer-readable storage medium according to claim 15, wherein: When the computer program is executed by a processor, it realizes: Identifying the light spot in the eye image to obtain the two-dimensional coordinates of the light spot corresponding to the light spot formed by the light source; Based on the two-dimensional coordinates of the light spot, the three-dimensional coordinates of the light spot are determined according to preset corneal radius information and the three-dimensional coordinates of the corneal center.

17. The computer-readable storage medium according to claim 16, wherein: When the computer program is executed by a processor, it realizes: Based on preset camera intrinsic parameter data, the two-dimensional coordinates of the light spot are converted into normalized coordinates of the light spot; Based on the normalized light spot coordinates, determining the light spot depth information of the three-dimensional light spot coordinates according to the equivalent relationship between the distance between the three-dimensional light spot coordinates and the three-dimensional coordinates of the corneal center and the corneal radius information; The three-dimensional coordinates of the light spot are determined according to the light spot depth information and the normalized coordinates of the light spot.

18. The computer-readable storage medium according to claim 16, wherein: When the computer program is executed by a processor, it realizes: Identifying the light spot in the eye image to obtain the light spot pixel coordinates of the light spot in the eye image; The pixel coordinates of the light spot are converted to obtain the two-dimensional coordinates of the light spot in the image space.

19. The computer-readable storage medium according to claim 15, wherein: When the computer program is executed by a processor, it realizes: The two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source are determined according to the collinear relationship among the two-dimensional coordinates of the light spot, the two-dimensional coordinates of the cornea center and the two-dimensional coordinates of the light source.

20. The computer-readable storage medium according to claim 15, wherein When the computer program is executed by a processor, it realizes: Determining light source depth information of the light source based on a coplanar relationship among preset three-dimensional coordinates of the corneal center, the three-dimensional coordinates of the light spot, the preset three-dimensional coordinates of the camera optical center, and the three-dimensional coordinates of the light source; The three-dimensional coordinates of the light source are determined according to the light source depth information and the two-dimensional coordinates of the light source.

Images