Lens barrel adjusting method, lens barrel adjusting device, wearable equipment and storage medium

Abstract

The embodiment of the invention provides a lens cone adjusting method and device, wearable equipment and a storage medium, the lens cone adjusting method is applied to the wearable equipment, the wearable equipment comprises a lens cone, a plurality of light sources and an image acquisition device, the image acquisition device is used for acquiring a human eye image, and the method comprises the following steps: identifying a plurality of light sources in the human eye image, a first image coordinate of a light spot formed by the plurality of light sources in an image coordinate system of the human eye image and a second image coordinate of a pupil of the human eye in the image coordinate system; according to the plurality of first image coordinates, the second image coordinates and light source coordinates of the light source in a camera coordinate system of the image acquisition device, calculating a sight line vector of the human eyes; determining a lens cone movement parameter according to the sight line vector and the center vector of the lens cone; and moving the corresponding lens cone according to the lens cone movement parameter corresponding to each lens cone, so that the center vector of the lens cone is aligned with the sight line vector of the corresponding human eye, thereby enabling the wearable device to accurately complete interpupillary distance adjustment.

Description

Technical Field

[0001] Embodiments of the present application relate to the field of virtual reality technology, and in particular to a lens barrel adjustment method, a lens barrel adjustment device, a wearable device, and a computer-readable storage medium. Background Art

[0002] When a user watches a wearable device, such as a VR helmet, the user's visual experience is optimal only when the center of the screen, the center of the lens, and the center of the human eye are in a straight line, otherwise there will be adverse feelings such as strabismus. Therefore, when manufacturing a wearable device, it is necessary to ensure that the screen and the center of the lens are aligned through the lens barrel structure design, and in order to adapt to the use of people with different pupil distances, the two lens barrels can be moved to the left and right directions respectively. The current automatic pupil distance adjustment method mainly uses two image acquisition devices to respectively shoot the user's left eye and right eye, and combines the external parameters of the two image acquisition devices to solve the distance of the three-dimensional coordinates of the user's two pupils, that is, the pupil distance. However, for two image acquisition devices (such as cameras) that respectively shoot two human eyes, due to the lack of a common viewing area, the difficulty of external parameter calibration is relatively large, which makes it difficult for the wearable device to accurately obtain the user's pupil distance, thereby affecting the pupil distance adjustment effect of the wearable device. Summary of the invention

[0003] The embodiments of the present application provide a lens barrel adjustment method, a lens barrel adjustment device, a wearable device, and a computer-readable storage medium, which can accurately determine the distance moved by the two lens barrels without using external parameters of an image acquisition device, thereby ensuring the pupil distance adjustment effect of the wearable device.

[0004] In a first aspect, an embodiment of the present application provides a lens barrel adjustment method, which is applied to a wearable device, wherein the wearable device includes a lens barrel, multiple light sources and an image acquisition device, wherein the image acquisition device is used to acquire a human eye image, and the method includes: identifying a first image coordinate of a light spot formed by multiple light sources in the human eye image in an image coordinate system of the human eye image, and a second image coordinate of a pupil of the human eye in the image coordinate system; calculating a sight line vector of the human eye according to the multiple first image coordinates, the second image coordinates, and the light source coordinates of the multiple light sources in a camera coordinate system of the image acquisition device; determining a lens barrel movement parameter according to the sight line vector and a center vector of the lens barrel; and moving the corresponding lens barrel according to the lens barrel movement parameter corresponding to each lens barrel so that the center vector of the lens barrel is aligned with the sight line vector of the corresponding human eye.

[0005] In a second aspect, an embodiment of the present application provides a lens barrel adjustment device, which is applied to a wearable device, wherein the wearable device includes a lens barrel, a plurality of light sources, and an image acquisition device, wherein the image acquisition device is used to acquire a human eye image, and the device includes an identification module, a first calculation module, a first determination module, and a movement module. The identification module is used to identify the first image coordinates of the light spots formed by the plurality of light sources in the human eye image in the image coordinate system of the human eye image, and the second image coordinates of the pupil of the human eye in the image coordinate system. The first calculation module is used to calculate the sight line vector of the human eye according to the plurality of the first image coordinates, the second image coordinates, and the light source coordinates of the light source in the camera coordinate system of the image acquisition device. The first determination module is used to determine the lens barrel movement parameter according to the sight line vector and the center vector of the lens barrel. The movement module is used to move the corresponding lens barrel according to the lens barrel movement parameter corresponding to each lens barrel, so that the center vector of the lens barrel is aligned with the sight line vector of the corresponding human eye.

[0006] In a third aspect, an embodiment of the present application provides a wearable device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method in the first aspect when executing the computer program.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising a computer program, wherein when the computer program is executed by a processor, the method in the first aspect is implemented.

[0008] In summary, in the embodiment of the present application, the image acquisition devices corresponding to the left and right eyes are respectively controlled to obtain the human eye images of the two eyes of the user, and the first image coordinates corresponding to the light spots formed by the multiple light sources in the human eye images, and the second image coordinates corresponding to the pupils of the human eyes are identified to complete the preliminary positioning of the light spots and the pupils. Then, according to the multiple first image coordinates, the second image coordinates and the light source coordinates of the camera coordinate system of the light source in the image acquisition device, the position of the light spot on the cornea is determined, and the position of the pupil on the cornea is determined, so as to determine the sight center and sight direction of the human eye according to the positions of the light spot and the pupil on the cornea, thereby calculating the sight vector of the human eye. Then, the relative position and distance between the sight vector and the center vector of the lens barrel can be determined according to the sight vector and the center vector of the lens barrel, so as to determine the lens barrel movement parameters, and it can be understood that the lens barrel movement parameters corresponding to the two lens barrels can be obtained at this time. After moving the corresponding lens barrels according to the moving distances of the two lens barrels, the center vectors of the two lens barrels can be aligned with the corresponding sight vectors of the human eyes. In this way, after adjusting the lens barrels of the left eye and the right eye respectively, the distance between the center vectors of the two lens barrels can match the user's pupil distance, so that the wearable device can accurately complete the pupil distance adjustment without using the external parameters of the image acquisition device to calculate the user's pupil distance, thereby ensuring the pupil distance adjustment effect of the wearable device.

[0009] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0012] Figure 2 A schematic diagram of the structure of a wearable device provided in an embodiment of the present application;

[0013] Figure 3 A schematic diagram of the structure of a wearable device provided in an embodiment of the present application;

[0014] Figure 4 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0015] Figure 5A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0016] Figure 6 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0017] Figure 7 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0018] Figure 8 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0019] Fig. 9 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0020] Fig.10 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0021] Fig.11 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0022] Fig.12 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0023] Fig.13 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application;

[0024] Fig.14 A schematic diagram of a module of a lens barrel adjustment device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0027] As carriers for the integration of the real world and the virtual world, augmented reality (AR), virtual reality (VR), and mixed reality (MR) can combine real scenes with virtual scenes through computers to create a virtual environment for human-computer interaction. Through wearable devices (such as VR helmets), virtual content and real scenes can be integrated, and there is a sense of immersion with seamless conversion between the virtual world and the real world. Therefore, AR, VR, and MR are becoming hot topics in the field of consumer electronics.

[0028] When users watch content in wearable devices, such as VR helmets, the user's visual experience is optimal only when the center of the screen, the center of the lens, and the center of the human eye are in a straight line. Otherwise, they will experience strabismus and other unpleasant feelings. Therefore, when manufacturing wearable devices, the first thing to do is to ensure that the screen and the center of the lens in the lens barrel are aligned through the lens barrel structure design, and in order to adapt to the use of people with different pupil distances, the two lens barrels can be moved to the left and right directions respectively.

[0029] At present, the methods for adjusting the pupil distance are generally manual and automatic. The current automatic method for measuring the pupil distance is to install an image acquisition device for photographing the human eye on each lens barrel, and a number of light sources (such as LED lights) that can illuminate the human eye. When the image acquisition device captures the light spot reflected by the pupil and the light source in the human eye, the three-dimensional coordinates of each pupil in the coordinate system of each image acquisition device can be solved by a related algorithm using the internal parameters of the image acquisition device and the three-dimensional coordinates of the light source relative to the image acquisition device, and then the distance between the three-dimensional coordinates of the two pupils, i.e., the pupil distance, is solved by the external parameters of the two image acquisition devices. However, for two image acquisition devices that respectively photograph two human eyes, since there is no common viewing area, it is difficult to calibrate the external parameters, which makes it unsuitable for mass production, and makes it difficult for the wearable device to accurately obtain the user's pupil distance, thereby affecting the pupil distance adjustment effect of the wearable device.

[0030] In response to the technical problems mentioned in the background technology, an embodiment of the present application provides a lens barrel adjustment method.

[0031] The technical solution of this application will be described in detail below:

[0032] See also Figures 1 to 3 , Figure 1 A flowchart of a lens barrel adjustment method provided in an embodiment of the present application, the method is applied to a wearable device 100, the wearable device 100 includes a lens barrel 20, a light source 30 and an image acquisition device 40, the image acquisition device 40 is used to acquire human eye images, and the method may include the following steps:

[0033] Step 01: Identify the first image coordinates of the light spot formed by the light source 30 in the human eye image in the image coordinate system of the human eye image, and the second image coordinates of the pupil of the human eye in the image coordinate system.

[0034] Specifically, the lens barrel 20 includes a first lens barrel 21 and a second lens barrel 22, wherein the first lens barrel 21 corresponds to the left eye and the second lens barrel 22 corresponds to the right eye. Each lens barrel 20 corresponds to at least a plurality of light sources 30 and an image acquisition device 40, so that the two eyes of the user can be illuminated by a plurality of different light sources 30 and photographed by different image acquisition devices 40.

[0035] When the user wears the wearable device 100, the wearable device 100 can play a voice prompt to remind the user to look straight ahead, so that the user's line of sight remains basically parallel and forward, so that the two line of sight vectors are in the horizontal direction (i.e. Figure 2 The distance in the Y direction) is equal to the pupil distance. The processor 50 of the wearable device 100 can control the multiple light sources 30 to turn on, and control the two image acquisition devices 40 to shoot the two eyes of the user to obtain the corresponding human eye image. It can be understood that there will be multiple light spots and the pupil of the human eye in the human eye image formed by the illumination of multiple light sources 30. At this time, the processor 50 can identify the first image coordinates of the multiple light spots in the human eye image in the image coordinate system, and the second image coordinates of the pupil of the human eye in the image coordinate system, thereby completing the preliminary positioning of the light spots and the pupil.

[0036] Step 02: Calculate the sight vector of the human eye according to the plurality of first image coordinates, the second image coordinates, and the light source coordinates P1 of the plurality of light sources 30 in the camera coordinate system of the image acquisition device 40;

[0037] Specifically, when producing a wearable device, the relative positions of the multiple light sources 30 and the image acquisition device 40 can be set respectively, so that the processor 50 can obtain the light source coordinates P1 of the multiple light sources 30 in the camera coordinate system of the image acquisition device 40. Figure 4 , Figure 4It includes a light path diagram corresponding to the pupil and a light path diagram corresponding to a certain light spot in the image acquisition device 40. Figure 4 It can be seen that the line connecting the corneal sphere center (the position of P5 in the figure) and the pupil center (the position of P8 in the figure) of the human eye can represent the line of sight center and line of sight direction of the human eye, so the processor 50 can determine the line of sight vector of the human eye according to the corneal sphere center and the pupil center. After obtaining the first image coordinates and the second image coordinates, the processor 50 can convert the first image coordinates into the three-dimensional coordinates of the light spot in the camera coordinate system (i.e., the light spot coordinates P2) and convert the second image coordinates into the three-dimensional coordinates of the pupil center in the camera coordinate system (i.e., the first pupil coordinates P3) according to the internal parameters of the corresponding image acquisition device 40. Then, in combination with the light source coordinates P1, the positional relationship between the light spot coordinates P2 and the light source coordinates P1 is determined to determine the corresponding position of the light spot in the cornea, and the positional relationship between the first pupil coordinates P3 and the light source coordinates P1 is determined to determine the corresponding position of the pupil in the cornea. It can be understood that at this time, the processor 50 can obtain the corresponding positions of multiple light spots in the cornea. Next, the processor 50 can determine the position of the corneal center of the human eye based on the corresponding positions of the multiple light spots in the cornea, and then combined with the corresponding position of the pupil in the cornea, the processor 50 can determine the line of sight center and line of sight direction of the human eye, thereby obtaining the line of sight vector of the human eye.

[0038] Step 03: Determine the lens barrel movement parameters according to the sight line vector and the center vector of the lens barrel 20;

[0039] Specifically, the lens barrel movement parameters include a movement direction and a corresponding movement distance. The processor 50 can determine the center vector of the lens barrel 20 according to the specific design of the lens barrel 20. The lens barrel 20 can move bidirectionally in a preset direction. After acquiring the sight line vector and the center vector of the lens barrel 20, the processor 50 can determine the relative position and relative distance between the sight line vector and the center vector to determine the movement direction and movement distance of the lens barrel 20, thereby determining the lens barrel movement parameters.

[0040] The lens barrel 20 may have one or two preset directions.

[0041] In some embodiments, the lens barrel 20 has one preset direction, which is a horizontal direction. The lens barrel 20 can move along the horizontal direction to ensure that the distance between the center vectors of the two lens barrels 20 matches the pupil distance of the user.

[0042] In other embodiments, the preset directions of the lens barrel 20 are 2, and the lens barrel 20 can move in a preset first direction and a second direction, the first direction and the second direction are perpendicular, and the first direction and the second direction are perpendicular to the direction corresponding to the center vector. In the case of determining the moving direction, the processor 50 can decompose the moving direction into the first direction and the second direction, and respectively determine the direction in which the lens barrel 20 needs to move in the first direction and the second direction. For example, the first direction is the horizontal direction, and the second direction is the vertical direction. The processor 50 can determine whether the lens barrel 20 needs to move to the left, or to the right, or not to move in the horizontal direction, and whether it moves upward, downward, or not to move in the vertical direction. Then, the processor 50 can determine the first moving distance according to the relative distance between the sight line vector and the center vector in the first direction, and determine the second moving distance according to the relative distance between the sight line vector and the center vector in the second direction. In this way, the processor 50 can doubly ensure the alignment effect of the sight line vector and the center vector of the lens barrel 20 in two directions by determining the first moving distance and the second moving distance, thereby achieving that the distance between the center vectors of the two lens barrels 20 matches the pupil distance of the user, and the height of the center vectors of the two lens barrels 20 in the vertical direction matches the height of the user's pupil center in the vertical direction.

[0043] Step 04: Move the corresponding lens barrel 20 according to the lens barrel movement parameters corresponding to each lens barrel 20 so that the center vector of the lens barrel 20 is aligned with the corresponding sight line vector of the human eye.

[0044] Specifically, the processor 50 obtains eye images corresponding to the left and right eyes of the user respectively, and determines the lens barrel movement parameters of the first lens barrel 21 according to the left eye image, and determines the lens barrel movement parameters of the second lens barrel 22 according to the right eye image.

[0045] Then, the processor 50 may move the corresponding lens barrel 20 according to the lens barrel movement parameters corresponding to each lens barrel 20 so that the sight vector of the left eye is aligned with the center vector of the corresponding lens barrel 20 , and the sight vector of the right eye is aligned with the center vector of the corresponding lens barrel 20 .

[0046] The alignment of the center vector of the lens barrel 20 with the corresponding sight line vector of the human eye includes that the distance between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye in the moving direction of the lens barrel is less than a preset distance threshold.

[0047] The distance between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye in the moving direction of the lens barrel can be determined according to the relative positions of the intersection of the center vector of the lens barrel 20 and the preset plane and the intersection of the sight vector of the human eye and the preset plane on the preset plane. Figure 2, assuming that the moving directions are the X direction and the Y direction, the preset plane is the xoy plane, and z=0 for any point on the xoy plane. The processor 50 can calculate that the intersection of the center vector of the lens barrel 20 in the xoy plane is (x1, y1, 0), and the intersection of the sight line vector of the human eye in the xoy plane is (x2, y2, 0). Then, the processor 50 can use x1 and x2 to calculate the distance between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye in the X direction of the lens barrel 20, and use y1 and y2 to calculate the distance between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye in the Y direction of the lens barrel 20.

[0048] The preset distance threshold is the upper limit of the reasonable range of the distance between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye while ensuring the user experience. Once the distance between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye is greater than the preset distance threshold, it means that the distance between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye is still large, and the user experience cannot be ensured. Alternatively, when the sight vector coincides with the center vector of the lens barrel 20, it can also be regarded that the sight vector and the center vector of the lens barrel 20 are aligned.

[0049] For example, the center vector includes a first center vector of the first lens barrel 21 and a second center vector of the second lens barrel 22, and the sight vector includes a first sight vector of the left eye and a second sight vector of the right eye. Then, the processor 50 may move the first lens barrel 21 according to the corresponding moving direction and moving distance of the first lens barrel 21 so that the first center vector and the first sight vector coincide, and move the second lens barrel 22 according to the moving direction and moving distance of the second lens barrel 22 so that the second center vector and the second sight vector coincide.

[0050] In this way, after the center vectors of the two lens barrels 20 are aligned with the sight vectors of the corresponding human eyes, the distance between the center vectors of the two lens barrels 20 can match the pupil distance of the human eyes. Obviously, the wearable device 100 of the present application does not need to use external parameters to calculate the pupil distance of the human eyes, but realizes accurate pupil distance adjustment by aligning the center vectors of the two lens barrels 20 with the corresponding human eyes respectively. The current automatic pupil distance adjustment method mainly uses two image acquisition devices to shoot the left eye and right eye of the user respectively, and combines the external parameters of the two image acquisition devices to solve the user's pupil distance. However, for two image acquisition devices (such as cameras) that shoot two human eyes respectively, due to the lack of a common viewing area, the difficulty of external parameter calibration is relatively large, which will also cause the current wearable device production process to be more complicated, and the current wearable device is not suitable for mass production. However, the lens barrel adjustment method of the present application does not require the use of external parameters to calculate the pupil distance. In the process of producing the wearable device 100 using the lens barrel adjustment method of the present application, the external parameter calibration step can be avoided, so that the complexity of the production process of the wearable device 100 using the lens barrel adjustment method of the present application is greatly reduced, thereby making the wearable device 100 using the lens barrel adjustment method of the present application more suitable for mass production than the wearable device using the current automatic pupil distance adjustment method.

[0051] The lens barrel adjustment method provided in this embodiment controls the image acquisition device 40 corresponding to the left and right eyes to obtain the human eye images of the two eyes of the user, and identifies the first image coordinates corresponding to the light spots formed by the multiple light sources 30 in the human eye images, and the second image coordinates corresponding to the pupil of the human eye, so as to complete the preliminary positioning of the light spots and the pupil. Then, according to the multiple first image coordinates, the second image coordinates and the light source coordinates P1 of the camera coordinate system of the light source 30 in the image acquisition device 40, the position of the light spot on the cornea is determined, and the position of the pupil on the cornea is determined, so as to determine the sight center and sight direction of the human eye according to the position of the light spot and the pupil on the cornea, so as to calculate the sight vector of the human eye. Then, the relative position and distance between the sight vector and the center vector of the lens barrel 20 can be determined according to the sight vector and the center vector of the lens barrel 20, so as to determine the lens barrel movement parameters. It can be understood that the lens barrel movement parameters corresponding to the two lens barrels 20 can be obtained at this time. After the corresponding lens barrels 20 are moved according to the moving distances of the two lens barrels 20, the center vectors of the two lens barrels 20 can be aligned with the corresponding sight vector of the human eye. In this way, after adjusting the lens barrels 20 of the left eye and the right eye respectively, the distance between the center vectors of the two lens barrels 20 can match the pupil distance of the user, so that the wearable device 100 can accurately complete the pupil distance adjustment without using the external parameters of the image acquisition device 40 to calculate the pupil distance of the user, thereby ensuring the pupil distance adjustment effect of the wearable device 100.

[0052] See also Figures 3 to 5In some embodiments, step 02: calculating the sight vector of the human eye according to the plurality of first image coordinates, the second image coordinates, and the light source coordinates P1 of the camera coordinate system of the image acquisition device 40 of the plurality of light sources 30, comprises:

[0053] Step 021: acquiring the spot coordinates P2 of the first image coordinates in the camera coordinate system and the first pupil coordinates P3 of the second image coordinates in the camera coordinate system according to the calibration parameters preset by the image acquisition device;

[0054] Step 022: Calculate the sight line vector according to the light source coordinate P1, the light spot coordinate P2 and the first pupil coordinate P3.

[0055] Specifically, when producing the wearable device 100, the processor 50 can calibrate the preset calibration parameters of the image acquisition device 40, that is, the internal parameters of the image acquisition device 40, to achieve two-dimensional coordinate and three-dimensional coordinate mapping of camera imaging.

[0056] After acquiring the first image coordinates and the second image coordinates, the processor 50 may convert the first image coordinates into image plane spot coordinates and the second image coordinates into image plane pupil coordinates based on the calibration parameters preset by the image acquisition device 40. The image plane here may be a normalized plane z=1, and the z direction is as follows: Figure 4 As shown, when generating coordinate points on the camera coordinate system, the processor 50 converts all spatial point coordinates to a plane with a unit distance (e.g., 1 meter) from the origin of the camera coordinate system. The processor 50 can calculate the plane coordinate points of the first image coordinates and the second image coordinates on the normalized plane through the calibration parameters preset by the image acquisition device 40, thereby obtaining the spot coordinates P2 and the first pupil coordinates P3.

[0057] Then, the processor 50 determines the coordinates of the light spot in the image plane in the camera coordinate system (i.e., the light spot coordinates P2) based on the calibration parameters and the image plane light spot coordinates. The processor 50 calibrates the parameters and the image plane pupil coordinates to determine the coordinates of the pupil in the image plane in the camera coordinate system (i.e., the optical first pupil coordinates P3).

[0058] Second pupil coordinates P8 Then, the processor 50 can determine the positions of the multiple light spots in the cornea according to the geometric positions between the multiple light source coordinates P1, the multiple light spot coordinates P2 and the optical center coordinates P4 of the image acquisition device 40 in the camera coordinate system, so as to determine the corneal spherical center coordinates P5 in the camera coordinate system. The processor 50 determines the position of the pupil in the cornea according to the geometric positions between the light source coordinates P1, the first pupil coordinates P3 and the optical center coordinates P4, so as to determine the second pupil coordinates P8 of the pupil in the camera coordinate system. It can be understood that the line connecting the corneal spherical center coordinates P5 and the second pupil coordinates P8 can represent the user's line of sight center and line of sight direction, so the processor 50 can calculate the line of sight vector according to the corneal spherical center coordinates P5 and the second pupil coordinates P8.

[0059] In this way, the processor 50 can first obtain the calibration parameters preset by the image acquisition device 40, and convert the first image coordinates into the spot coordinates P2 and the second image coordinates into the first pupil coordinates P3 according to the calibration parameters, thereby unifying the coordinate systems corresponding to the various coordinates obtained by the processor 50, so as to facilitate the subsequent calculation of the line of sight vector according to the spot coordinates P2, the first pupil coordinates P3, and the light source coordinates P1 in the camera coordinate system.

[0060] See also Figure 3 , Figure 4 and Figure 6 In some embodiments, step 022: calculating the sight line vector according to the light source coordinate P1, the light spot coordinate P2 and the first pupil coordinate P3, includes:

[0061] Step 0221: Calculate the corneal spherical center coordinates P5 in the camera coordinate system according to the light source coordinates P1, the light spot coordinates P2, and the optical center coordinates P4 of the optical center of the image acquisition device 40 in the camera coordinate system;

[0062] Step 0222: Calculate the pupil reflection point coordinates P7 of the pupil on the cornea in the camera coordinate system according to the corneal spherical center coordinates P5, the first pupil coordinates P3 and the preset corneal radius;

[0063] Step 0223: Calculate the second pupil coordinate P8 of the pupil in the camera coordinate system according to the corneal spherical center coordinate P5, the pupil reflection point coordinate P7, and the optical center coordinate P4;

[0064] Step 0224: Calculate the sight line vector based on the second pupil coordinate P8 and the corneal center coordinate P5.

[0065] Specifically, the light spot is formed by the light source 30 irradiating the eye, and when the image acquisition device 40 takes a picture of the human eye, the light emitted by the light source 30 is reflected by the eye to form a reflected light that passes through the optical center of the image acquisition device 40, then the processor 50 can determine the light source reflection point coordinates P6 of the light source 30 on the cornea according to the light source coordinates P1, the light spot coordinates P2, and the optical center coordinates P4 of the optical center of the image acquisition device 40 in the camera coordinate system. At this time, the processor 50 can regard the corneal surface as a spherical mirror with a preset fixed radius, and in combination with the light source reflection point coordinates P6 of the light source 30 on the cornea, the processor 50 can calculate the corneal spherical center coordinates P5 in the camera coordinate system.

[0066] It can be assumed that the distance between the corneal center and the pupil reflection point of the pupil on the cornea is equal to the preset corneal radius, and the first pupil coordinate P3 and the corneal center coordinate P5 are known. Therefore, the processor 50 can obtain the pupil reflection point coordinate P7 of the pupil on the cornea in the camera coordinate system based on the first pupil coordinate P3, the corneal center coordinate P5 and the preset corneal radius.

[0067] The distance between the pupil center and the cornea center of the human eye is generally within a certain range, and the processor 50 can determine the preset distance between the pupil center and the cornea center of the human eye according to the range. At this time, the processor 50 can calculate the second pupil coordinate P8 of the pupil in the camera coordinate system by combining the geometric relationship and corresponding coordinates of the corneal spherical center, pupil reflection point and optical center, and the preset distance between the pupil center and the cornea center of the human eye.

[0068] Finally, the processor 50 can calculate the sight vector according to the second pupil coordinates P8 and the corneal spherical center coordinates P5 in the camera coordinate system to determine the user's sight center and sight direction. In this way, the processor 50 can determine the second pupil coordinates P8 and the corneal spherical center coordinates P5 in the camera coordinate system through the light source coordinates P1, the light spot coordinates P2, the optical center coordinates P4 of the image acquisition device 40, the first pupil coordinates P3 and the preset corneal radius, as well as the geometric relationship between them, thereby accurately completing the determination of the sight vector.

[0069] See also Figure 3 , Figure 4 and Figure 7 In some embodiments, step 0221: calculating the corneal spherical center coordinates P5 in the camera coordinate system according to the light source coordinates P1, the light spot coordinates P2, and the optical center coordinates P4 of the optical center of the image acquisition device 40 in the camera coordinate system, comprises:

[0070] Step 02211: Based on the geometric relationship between the light source 30, the first reflection position of the light spot on the cornea, the optical center and the position of the light spot on the image plane, a first functional relationship between the light spot coordinate P2, the optical center coordinate P4, and the light source reflection point coordinate P6 of the light source 30 on the cornea is established, and a first functional relationship group is determined according to the first functional relationships corresponding to the light sources 20 at multiple different positions;

[0071] Step 02212: based on the distance relationship between the first reflection position and the spherical center of the cornea, determine the second functional relationship between the light source reflection point coordinate P6, the corneal spherical center coordinate P5 and the corneal radius, and determine the second functional relationship group according to the second functional relationships corresponding to the light sources 20 at multiple different positions;

[0072] Step 02213: Based on the coplanar relationship between the incident light, the outgoing light and the normal in the law of reflection, determine the third functional relationship among the light source coordinate P1, the light source reflection point, the corneal spherical center coordinate P5 and the optical center coordinate P4, and determine the third functional relationship group according to the third functional relationships corresponding to the light sources 20 at multiple different positions;

[0073] Step 02214: based on the relationship between the incident angle and the exit angle, determine the fourth functional relationship among the light source coordinates P1, the light source reflection point coordinates P6, the corneal spherical center coordinates P5 and the optical center coordinates P4, and determine a fourth functional relationship group according to the fourth functional relationships corresponding to the light sources 20 at multiple different positions;

[0074] Step 02215: Determine a fifth functional relationship group by combining the first functional relationship group and the third functional relationship group;

[0075] Step 02216: Calculate the corneal spherical center coordinates P5 according to the first functional relationship group, the second functional relationship group, the fourth functional relationship group and the fifth functional relationship group.

[0076] Specifically, the light source 30, the optical center, and the light spot are in the same plane at the first reflection position of the cornea, and the light spot in the human eye image is formed by the reflected light of the light source 30 reflected by the cornea, so the optical center coordinate P4, the light source reflection point coordinate P6 of the light source 30 on the cornea, and the light spot coordinate P2 are located on the same straight line. In addition, the relationship between the distance between the optical center coordinate P4 and the light source reflection point coordinate P6 and the distance between the optical center coordinate P4 and the pixel light spot coordinate is a fixed preset ratio. It can be understood that after the processor 50 obtains the optical center coordinate P4 and the light spot coordinate P2, it can determine the direction between the optical center coordinate P4 and the light source reflection point coordinate P6, and then combine the distance between the optical center coordinate P4 and the light source reflection point coordinate P6 with the preset ratio between the distance between the optical center coordinate P4 and the light source reflection point coordinate P6 and the distance between the optical center coordinate P4 and the pixel light spot coordinate, so as to determine the coordinates of the light source reflection point.

[0077] Therefore, the processor 50 can establish a first functional relationship between the light spot coordinates P2, the optical center coordinates P4, and the light source reflection point coordinates P6 of the light source 30 on the cornea based on the geometric relationship between the light source 30, the first reflection position of the light spot on the cornea, the optical center, and the position of the light spot on the phase plane, such as the optical center coordinates P4, the light source reflection point coordinates P6 of the light source 30 on the cornea, and the light spot coordinates P2 are located on the same straight line, and the preset ratio between the distance between the optical center coordinates P4 and the light source reflection point coordinates P6 and the distance between the optical center coordinates P4 and the pixel light spot coordinates, so as to determine the light source reflection point coordinates P6 of the light source 30 on the cornea using the first functional relationship. For example, the first functional relationship can be expressed as:

[0078] P6=P4+k 1 (P4-P2)for some k 1 (1)

[0079] Among them, k 1 is the preset ratio between the distance between the optical center coordinate P4 and the light source reflection point coordinate P6 and the distance between the optical center coordinate P4 and the pixel spot coordinate. It can be understood that there are multiple light sources, and k corresponding to each light source 1 may be different. When calculating the first functional relationship corresponding to each light source 30, the k corresponding to the light source 30 is brought into consideration. 1 .

[0080] At this time, the processor 50 can obtain the first functional relationship corresponding to each light source 20 according to the light source reflection point coordinates P6 and the light spot coordinates P2 corresponding to the multiple light sources 20, and determine the first functional relationship group according to the first functional relationships corresponding to the light sources 20 at multiple different positions.

[0081] The processor 50 determines the second functional relationship between the light source reflection point coordinate P6, the corneal spherical center coordinate P5 and the corneal radius based on the distance relationship between the first reflection position and the spherical center of the cornea, that is, the distance between the first reflection position and the spherical center of the cornea is a preset corneal radius, so as to use the corneal radius to jointly establish the light source reflection point coordinate P6 and the corneal spherical center coordinate P5. For example, the second functional relationship can be expressed as:

[0082] ‖P6 - P5‖=R (2)

[0083] Where R is the corneal radius.

[0084] At this time, the processor 50 can obtain the second functional relationship corresponding to each light source 20 according to the light source reflection point coordinates P6 corresponding to the multiple light sources 20, and determine the second functional relationship group according to the second functional relationships corresponding to the light sources 20 at multiple different positions.

[0085] The incident light can be represented by the light between the light source 30 and the light source reflection point, the outgoing light can be represented by the light between the light source reflection point and the optical center, and the normal can be represented by the light between the corneal spherical center and the light source reflection point. Based on the coplanar relationship between the incident light, the outgoing light and the normal in the law of reflection, it can be determined that the light source 30, the light source reflection point, the optical center, and the corneal spherical center are located on the same plane. At this time, the processor 50 can take out two vectors from the plane formed by the light source 30, the light source reflection point, the optical center, and the corneal spherical center for cross multiplication based on the principle that the light source 30, the light source reflection point, the optical center, and the corneal spherical center are coplanar, the cross product of two vectors obtains a vector perpendicular to both vectors, and the dot product of mutually perpendicular vectors is 0, and obtains the normal vector of the plane, and then establishes the third functional relationship between the light source coordinate P1, the light source reflection point coordinate P6, the corneal spherical center coordinate P5, and the optical center coordinate P4 based on the principle that the normal vector is perpendicular to any vector in the plane. For example, the third functional relationship can be expressed as:

[0086] (P1 - P4)×(P6 - P4)·(P5 - P4)=0 (3)

[0087] At this time, the processor 50 can obtain the third functional relationship corresponding to each light source 20 according to the light source coordinates P1 and the light source reflection point coordinates P6 corresponding to the multiple light sources 20, and determine the third functional relationship group according to the third functional relationships corresponding to the light sources 20 at multiple different positions.

[0088] According to the law of reflection, it can be determined that the incident angle is equal to the exit angle, and the vector inner product satisfies the relationship a·b=‖a‖‖b‖sinθ, where ‖a‖ is the modulus of vector a, ‖b‖ is the modulus of vector b, and sinθ is the angle between vector a and vector b. The processor 50 can determine the corresponding vector from the plane composed of the light source 30, the light source reflection point, the optical center, and the corneal spherical center based on the relationship between the incident angle and the exit angle and the principle of the vector inner product, and determine the fourth functional relationship of the light source coordinates P1, the light source reflection point coordinates P6, the corneal spherical center coordinates P5, and the optical center coordinates P4. For example, the fourth functional relationship can be expressed as:

[0089] (P1 - P6)·(P6 - P5)·‖P4-P6‖=(P4 - P6)·(P6 - P5)·‖P1-P6‖ (4)

[0090] At this time, the processor 50 can obtain the fourth functional relationship corresponding to each light source 20 according to the light source coordinates P1 and the light source reflection point coordinates P6 corresponding to the multiple light sources 20, and determine the fourth functional relationship group according to the fourth functional relationships corresponding to the light sources 20 at multiple different positions.

[0091] The processor 50 can calculate the corneal radius through the relevant calibration algorithm. At this time, the first functional relationship in the first functional relationship group is brought into the third functional relationship of the corresponding light source in the third functional relationship group. The processor 50 can obtain the fifth functional relationship of the light source coordinates P1, the optical center coordinates P4, the spot coordinates P2 and the corneal spherical center coordinates P5 corresponding to each light source. For example, for light source A, the first functional relationship and the third functional relationship corresponding to light source A can be combined to obtain the fifth functional relationship, where the fifth functional relationship can be expressed as:

[0092] (P1 - P4)×(P2 - P4)·(P5 - P4)=0 (5)

[0093] Then, the processor 50 may determine a fifth functional relationship group according to the fifth functional relationships corresponding to the light sources 20 at multiple different positions. For example, the fifth functional relationship group may be expressed as:

[0094]

[0095] The subscripts of P1 and P2 are the serial numbers of the corresponding light sources 20 .

[0096] All planes composed of all light sources 30, optical centers and spot coordinates P2 corresponding to different light sources 30 intersect at a straight line composed of the corneal spherical center and the optical center (i.e., P5-P4). Therefore, the rank of the fifth functional relationship group is 2, and the number of linearly independent vectors of the basic solution system is 1. The processor 50 can obtain the expression of the solution of the fifth functional relationship group based on this principle. For example, the solution of the fifth functional relationship group is expressed by using the normal vector and corresponding coefficient of the plane where two linearly independent row vectors in the fifth functional relationship group are located. For example, the expression of the solution (P5-P4) of the fifth functional relationship group is (P5-P4)=k 2 *b, where k 2 is the corresponding coefficient, and the coefficient is an arbitrary constant, and b is the normal vector of the plane where the two linearly independent row vectors in the functional relationship group are located.

[0097] Then, by combining the expressions of the solutions according to the first functional relationship group, the second functional relationship group, the fourth functional relationship group, and the fifth functional relationship group, the processor 50 can form an equation group, and there are multiple unknowns in the equation group.

[0098] The variables involved in the first functional relationship include the three-dimensional vector light source reflection point coordinate P6. The processor 50 can list three equations according to the light source reflection point coordinate P6. The variables of the second functional relationship and the fourth functional relationship are scalars, and can only list one equation respectively. The number of variables in the first functional relationship, the second functional relationship, and the fourth functional relationship is related to the number of light sources 30. Therefore, the equation group can list 5*N equations at this time. The variables in the solution of the functional relationship group include three-dimensional variables, and can list three equations. The number of solutions of the functional relationship group is independent of the number of light sources 30. At this time, the equation group can list three equations. Therefore, the total number of equation groups is 5*N+3.

[0099] In the solutions of the first function relationship group, the second function relationship group, the fourth function relationship group and the fifth function relationship group, the variables related to the light source 30 are the preset ratios (i.e., k) between the distance between the light source reflection point coordinate P6 and the optical center coordinate P4 and the light source reflection point coordinate P6 and the distance between the optical center coordinate P4 and the pixel spot coordinate. 1 ), where the light source reflection point coordinate P6 is a three-dimensional variable, and the preset ratio between the distance between the optical center and the light source reflection point and the distance between the optical center and the pixel spot coordinate is a one-dimensional variable, so the number of unknowns is 4*N. The variables unrelated to the light source 30 are the corneal spherical center coordinate P5, and the coefficients in the solution of the fifth functional relationship group (i.e., k 2 ), where the corneal sphere center coordinate P5 is a three-dimensional variable, and the coefficients in the solution of the functional relationship group are one-dimensional variables, so the number of unknowns is 4. In other words, the number of unknowns in the equation group is 4*N+4.

[0100] After solving the equation group, the processor 50 can accurately calculate the corneal spherical center coordinates P5.

[0101] In this way, the processor 50 generates corresponding functional relationships and functional relationship groups in sequence based on the principle that the light source 30, the optical center, and the first reflection position of the light spot on the cornea are coplanar, as well as the geometric relationship between each other, and combines the law of reflection to form a group of equations representing the relationship between the light source coordinates P1, the optical center coordinates P4, the light source reflection point coordinates P6, the light spot coordinates P2 and the corneal spherical center coordinates P5, thereby accurately obtaining the corneal spherical center coordinates P5.

[0102] See also Figure 3 , Figure 4 and Figure 8 In some embodiments, step 0222: calculating the pupil reflection point coordinates P7 of the pupil on the cornea in the camera coordinate system according to the corneal spherical center coordinates P5, the first pupil coordinates P3 and the preset corneal radius, comprises:

[0103] Step 02221: Determine a sixth functional relationship among the optical center coordinate P4, the first pupil coordinate P3 and the pupil reflection point coordinate P7 based on the geometric relationship among the optical center, the position of the pupil on the image plane and the second reflection position of the pupil on the cornea;

[0104] Step 02222: Based on the distance relationship between the second reflection position and the spherical center of the cornea, determine the seventh functional relationship between the pupil reflection point coordinate P7, the corneal spherical center coordinate P5 and the corneal radius;

[0105] Step 02223: Calculate the pupil reflection point coordinates P7 based on the corneal spherical center coordinates P5, the sixth functional relationship and the seventh functional relationship.

[0106] Specifically, after the pupil reflects the light, the reflected light will be refracted once at the second reflection position of the cornea, and the refracted light will be received by the image acquisition device 40 and form the pupil in the human eye image, so the optical center coordinate P4, the pupil reflection point coordinate P7, and the first pupil coordinate P3 are located on the same straight line. In addition, the relationship between the distance between the optical center coordinate P4 and the pupil reflection point coordinate P7 and the distance between the optical center coordinate P4 and the first pupil coordinate P3 is a fixed preset ratio. It can be understood that after the processor 50 obtains the optical center coordinate P4 and the first pupil coordinate P3, it can determine the direction between the optical center coordinate P4 and the first pupil coordinate P3, thereby determining the direction between the optical center coordinate P4 and the pupil reflection point coordinate P7. Combined with the preset ratio between the distance between the optical center coordinate P4 and the pupil reflection point coordinate P7 and the distance between the optical center coordinate P4 and the pixel second pupil coordinate P8, the distance between the optical center coordinate P4 and the pupil reflection point coordinate P7 can be determined, thereby determining the coordinates of the pupil reflection point.

[0107] Therefore, the processor 50 establishes a sixth functional relationship between the optical center coordinate P4, the first pupil coordinate P3, and the pupil reflection point coordinate P7 based on the geometric relationship between the optical center, the position of the pupil in the image plane, and the second reflection position of the pupil in the cornea, such as the relationship that the optical center, the position of the pupil in the image plane, and the second reflection position of the pupil in the cornea are located on the same straight line, and the preset ratio between the distance between the optical center coordinate P4 and the pupil reflection point coordinate P7 and the distance between the optical center coordinate P4 and the pixel second pupil coordinate P8, so as to determine the light source reflection point coordinate P6 of the light source 30 in the cornea by using the sixth functional relationship. For example, the sixth functional relationship can be expressed as:

[0108] P7=P4+k 3 (P4-P3)for some k 3 (7)

[0109] Among them, k 3It is a preset ratio between the distance between the optical center coordinate P4 and the pupil reflection point coordinate P7 and the distance between the optical center coordinate P4 and the pixel second pupil coordinate P8.

[0110] The processor 50 determines the seventh functional relationship between the pupil reflection point coordinate P7, the corneal spherical center coordinate P5 and the corneal radius based on the distance relationship between the second reflection position and the spherical center of the cornea, that is, the distance between the second reflection position and the spherical center of the cornea is the relationship of the corneal radius, so as to use the corneal radius to jointly establish the pupil reflection point coordinate P7 and the corneal spherical center coordinate P5. For example, the sixth functional relationship can be expressed as:

[0111] ‖P7-P5‖=R (8)

[0112] Then, the processor 50 can form an equation group according to the sixth functional relationship and the seventh functional relationship, wherein the unknowns of the equation group are the pupil reflection point coordinate P7 (three-dimensional variable) and the distance between the optical center coordinate P4 and the pupil reflection point coordinate P7, and the preset ratio between the distance between the optical center coordinate P4 and the pixel second pupil coordinate P8 (one-dimensional variable), so the equation group is four equations containing four unknowns. After combining the corneal spherical center coordinate P5, the corneal radius, the optical center coordinate P4, the first pupil coordinate P3 and the equation group, the processor 50 can obtain the pupil reflection point coordinate P7.

[0113] In this way, the processor 50 generates corresponding functional relationships in sequence and forms a set of equations representing the relationship between the optical center coordinate P4, the first pupil coordinate P3 and the pupil reflection point coordinate P7 according to the principle that the optical center and the second reflection position of the pupil on the cornea are on the same straight line, as well as the geometric relationship between them, so as to accurately obtain the pupil reflection point coordinate P7.

[0114] See also Figure 3 , Figure 4 and Fig. 9 In some embodiments, step 0223: calculating the second pupil coordinate P8 of the pupil in the camera coordinate system according to the corneal spherical center coordinate P5, the pupil reflection point coordinate P7, and the optical center coordinate P4, includes:

[0115] Step 02231: Based on the coplanar relationship between the incident light, the outgoing light and the normal line in the law of refraction, determine the eighth functional relationship among the optical center coordinate P4, the corneal spherical center coordinate P5, the pupil reflection point coordinate P7 and the second pupil coordinate P8;

[0116] Step 02232: Based on the relationship between the incident angle and the exit angle, determine a ninth functional relationship among the optical center coordinate P4, the corneal spherical center coordinate P5, the pupil reflection point coordinate P7 and the second pupil coordinate P8;

[0117] Step 02233: Based on the distance relationship between the pupil and the cornea, determine the tenth functional relationship between the second pupil coordinate P8, the corneal spherical center coordinate P5, and the preset distance between the corneal center and the pupil center;

[0118] Step 02234: Calculate the second pupil coordinate P8 according to the corneal center coordinate P5, the pupil reflection point coordinate P7, the eighth functional relationship, the ninth functional relationship and the tenth functional relationship.

[0119] Specifically, according to the law of refraction, it can be determined that the incident light, the outgoing light and the normal are coplanar. The incident light can be represented by the light between the pupil center and the pupil reflection point, the outgoing light can be represented by the light between the pupil reflection point and the optical center, and the normal can be represented by the light between the corneal spherical center and the pupil reflection point. Therefore, the processor 50 can determine that the pupil center, the pupil reflection point, the optical center, and the corneal spherical center are located on the same plane. At this time, the processor 50 can combine the principle that the cross product of two vectors is used to obtain a vector that is perpendicular to both vectors, and the dot product of mutually perpendicular vectors is 0, and take out two vectors from the plane composed of the pupil center, the pupil reflection point, the optical center, and the corneal spherical center for cross product to obtain the normal vector of the plane, and then establish the eighth functional relationship of the optical center coordinate P4, the corneal spherical center coordinate P5, the pupil reflection point coordinate P7, and the second pupil coordinate P8 based on the principle that the normal vector is perpendicular to any vector in the plane. For example, the eighth functional relationship can be expressed as:

[0120] (P7-P4)×(P5-P4)·(P8-P4)=0 (9)

[0121] According to Snell's law, the incident angle θ can be determined 1 and the exit angle θ 2 Meet n 1 sinθ 1 =n 2 sinθ 2 The relationship between n 1 is the corneal refractive index, n 2 is the refractive index of air. And the two vectors satisfy the relationship ‖a×b‖=‖a‖‖b‖sinθ. Therefore, the processor 50 can determine the corresponding vector from the plane composed of the pupil center, the pupil reflection point, the optical center, and the corneal spherical center according to these two relationships, and determine the ninth functional relationship between the optical center coordinate P4, the corneal spherical center coordinate P5, the pupil reflection point coordinate P7 and the second pupil coordinate P8.

[0122] n 1 ·‖(P7-P5)×(P8-P7)‖·‖P4-P7‖=n 2 ·‖(P7-P5)×(P4-P7)‖·‖P8-P7‖ (10)

[0123] The processor 50 then determines a tenth functional relationship between the second pupil coordinate P8, the corneal spherical center coordinate P5, and the preset distance between the corneal center and the pupil center based on the distance relationship between the pupil and the cornea, so as to jointly determine the second pupil coordinate P8 and the corneal spherical center coordinate P5. For example, the tenth functional relationship can be expressed as:

[0124] ‖P8-P5‖=D (11)

[0125] Wherein, D is the preset distance between the center of the cornea and the center of the pupil.

[0126] Then, the processor 50 can form an equation group according to the eighth functional relationship, the ninth functional relationship and the tenth functional relationship, wherein the unknown number of the equation group is the second pupil coordinate P8 (three-dimensional variable), so the equation group is 3 equations containing 3 unknown numbers. The corneal refractive index of the human eye and the distance between the pupil center and the corneal center can be solved by the relevant calibration algorithm, and the air refractive index is approximately equal to 1. Therefore, by combining the corneal spherical center coordinate P5, the pupil reflection point coordinate P7, the optical center coordinate P4 and the equation group, and calculating, the processor 50 can obtain the second pupil coordinate P8.

[0127] In this way, the processor 50 generates corresponding functional relationships in sequence according to the principle that the pupil center, pupil reflection point, optical center, and corneal spherical center are coplanar, as well as the geometric relationship between them, and forms a set of equations representing the relationship between the corneal spherical center coordinates P5, the pupil reflection point coordinates P7, the optical center coordinates P4, and the second pupil coordinates P8, thereby accurately obtaining the second pupil coordinates P8.

[0128] See also Figure 3 , Figure 4 and Fig.10 In some embodiments, the lens barrel movement parameters include a movement direction and a corresponding movement distance. Step 03: Determine the lens barrel movement parameters according to the sight line vector and the center vector of the lens barrel 20, including:

[0129] Step 031: Convert the sight line vector and the center vector to the housing coordinate system of the wearable device 100 to determine the lens barrel movement parameters according to the sight line vector and the center vector in the housing coordinate system.

[0130] Specifically, the parameters required for the conversion between the camera coordinate system and the housing coordinate system of the wearable device 100 can be given by the wearable device 100 when the mechanism is designed. Therefore, the processor 50 can convert the sight vector and the center vector in the camera coordinate system to the sight vector and the center vector in the coordinate system of the wearable device 100 to determine the relative position of the sight vector and the center vector in the housing coordinate system, so that the processor 50 can accurately determine the lens barrel movement parameters according to the sight vector and the center vector in the housing coordinate system. In this way, the processor 50 can accurately move the lens barrel 20 to ensure that the sight vector and the center vector after the lens barrel 20 moves are accurately aligned, thereby improving the user experience.

[0131] See also Figure 3 , Figure 4 and Fig.11 In some embodiments, the lens barrel adjustment method further includes:

[0132] Step 05: Calculate the angle between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye;

[0133] When the included angle is less than the preset threshold, the step of moving the corresponding lens barrel 20 according to the lens barrel movement parameters corresponding to each lens barrel 20 is performed so that the center vector of the lens barrel 20 is aligned with the corresponding sight line vector of the human eye;

[0134] When the angle is greater than the preset threshold, the step of identifying the first image coordinates of the light spots formed by the multiple light sources 30 in the human eye image in the image coordinate system and the second image coordinates of the pupil of the human eye in the image coordinate system is entered again.

[0135] Specifically, when the user watches the content in the wearable device 100, the user's visual experience is optimal only when the center of the screen, the center of the lens, and the center of the human eye are in a straight line. If the angle between the human eye's line of sight and the central vector of the lens barrel 20 is large, strabismus and other undesirable feelings will occur. Therefore, when the user wears the wearable device 100, the wearable device 100 can play a voice prompt to the user to look straight ahead. After the user looks straight ahead, the user's line of sight is basically parallel to the direction of the central vector of the lens barrel 20. At the same time, the processor 50 can preset a threshold value based on the maximum value of the angle between the central vector of the lens barrel 20 and the corresponding human eye's line of sight vector under the premise of ensuring the user's visual experience. Once the angle between the central vector of the lens barrel 20 and the corresponding human eye's line of sight vector exceeds the preset threshold, it means that the user is at risk of strabismus at this time.

[0136] After calculating the sight line vector and obtaining the lens barrel movement parameter, the processor 50 may calculate the angle between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye, for example, by using the above-mentioned vector inner product formula to calculate the angle between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye. Then, it is determined whether the calculated angle is less than a preset threshold.

[0137] When the angle is less than the preset threshold, it can be confirmed that the user's line of sight and the center vector of the lens barrel 20 are roughly in a straight line at this time, and after the lens barrel 20 is moved according to the lens barrel movement parameter corresponding to the line of sight vector, a better visual experience can be guaranteed. Then, at this time, the processor 50 will enter the step of moving the corresponding lens barrel 20 according to the lens barrel movement parameter corresponding to each lens barrel 20, so that the center vector of the lens barrel 20 is aligned with the corresponding line of sight vector of the human eye, so as to complete the pupil distance adjustment while ensuring the user's visual experience.

[0138] When the angle is greater than the preset threshold, it can be confirmed that the angle between the user's line of sight and the center vector of the lens barrel 20 is large, and the user may not look straight when the image acquisition device 40 is shooting, but looks in other directions. Then after the lens barrel 20 is moved according to the lens barrel movement parameter corresponding to the line of sight vector, the user's visual experience will be poor. Therefore, the processor 50 can re-enter the recognition of the first image coordinates of the light spots formed by the multiple light sources 30 in the image coordinate system of the human eye image, and the second image coordinates of the pupil of the human eye in the image coordinate system. That is, the processor 50 re-controls the image acquisition device 40 to collect the user's human eye image and re-determines the lens barrel movement parameter. Of course, when it is confirmed that the angle is greater than the preset threshold, the processor 50 can issue a prompt message, such as playing a voice of "adjustment failed, please look straight at the screen", to remind the user to look straight at the screen.

[0139] In other embodiments, the calculation of the angle between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye can be performed after the lens barrel 20 is moved. After the corresponding lens barrel 20 is moved according to the lens barrel movement parameters corresponding to each lens barrel 20, the processor 50 can calculate the angle between the center vector of the lens barrel 20 and the corresponding sight line vector of the human eye. Then it is determined whether the calculated angle is less than the preset threshold. In the case where the angle is less than the preset threshold, it can be confirmed that the user's sight line and the center vector of the lens barrel 20 are roughly in a straight line at this time, and the user's visual experience is better at this time, and the adjustment can be confirmed to be completed. In the case where the angle is greater than the preset threshold, it can be confirmed that the angle between the user's sight line and the center vector of the lens barrel 20 is large, and the user will have the risk of strabismus when watching the content in the wearable device 100. Therefore, the processor 50 can confirm that the adjustment fails, and re-enter the recognition of the human eye image, the first image coordinates of the light spots formed by the multiple light sources 30 in the image coordinate system of the human eye image, and the second image coordinates of the pupil of the human eye in the image coordinate system. That is, the image acquisition device 40 is re-controlled to acquire the user's eye image, and the lens barrel movement parameters are re-determined to re-adjust the lens barrel 20. Of course, after the processor 50 confirms that the adjustment fails, the processor 50 can issue a prompt message, such as playing a voice message of "adjustment failed, please look up at the screen", to remind the user to look up at the screen.

[0140] In this way, the processor 50 can ensure that the angle between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye is not too large after the adjustment is completed by calculating the angle between the center vector of the lens barrel 20 and the corresponding sight vector of the human eye, thereby ensuring the user's visual perception and usage experience.

[0141] See also Figure 3 , Figure 4 and Fig.12 In some embodiments, the lens barrel adjustment method further includes:

[0142] Step 06: Calculate the center coordinates of the lens barrel 20 in the camera coordinate system according to the first preset positional relationship between the center of the lens barrel 20 and the light source 30 and the light source coordinates P1;

[0143] Step 07: Determine the center vector based on the center coordinates of the barrel.

[0144] Specifically, when the wearable device 100 is produced, the relative position of the center of the lens barrel 20 and the light source 30 can be determined, that is, the first preset position relationship between the center of the lens barrel 20 and the light source 30 can be determined. The processor 50 can locate the coordinates of the light source 30 through an external binocular, and then convert the determined coordinates into the camera coordinate system, thereby obtaining the light source coordinates P1 of the light source 30 in the camera coordinate system. Then, the processor 50 can determine the geometric relationship between the center of the lens barrel 20 and the light source 30 according to the first preset position relationship, and then calculate the lens barrel center coordinates of the center of the lens barrel 20 in the camera coordinate system in combination with the light source coordinates P1 of the light source 30 in the camera coordinate system. After obtaining the lens barrel center coordinates, the processor 50 can determine the center vector according to the lens barrel center coordinates. In this way, the processor 50 can accurately determine the center vector according to the known and accurate first preset position relationship, thereby improving the calculation accuracy of subsequent calculations based on the center vector.

[0145] See also Figure 3 , Figure 4 and Fig.13 In some embodiments, the lens barrel adjustment method further includes:

[0146] Step 08: Acquire a second preset position relationship between the light source 30 and the image acquisition device 40;

[0147] Step 09: According to the second preset position relationship, the coordinates of the preset position of the light source 30 in the camera coordinate system are determined as the light source coordinates P1.

[0148] Specifically, when the wearable device 100 is produced, the relative position of the light source 30 and the image acquisition device 40 can be determined, that is, the second preset position relationship between the light source 30 and the image acquisition device 40 can be determined. The processor 50 can determine the relative position of the preset position of the light source 30 and the origin in the camera coordinate system based on the second preset position relationship and the origin in the camera coordinate system to determine the coordinates of the preset position of the light source 30 in the camera coordinate system. It can be understood that the coordinates can be used as the light source coordinates P1. In this way, the processor 50 can accurately determine the light source coordinates P1 based on the known and accurate second preset position relationship, thereby improving the calculation accuracy of subsequent calculations based on the light source coordinates P1.

[0149] See also Figure 3 , Figure 4 and Fig.14The lens barrel adjustment method of the present application can be implemented by a lens barrel adjustment device 10, which includes an identification module 11, a first calculation module 12, a first determination module 13 and a moving module 14. The identification module 11 is used to identify the first image coordinates of the light spot formed by the light source 30 in the image coordinate system of the human eye image, and the second image coordinates of the pupil of the human eye in the image coordinate system. The first calculation module 12 is used to calculate the sight line vector of the human eye according to the first image coordinates, the second image coordinates, and the light source coordinates P1 of the light source 30 in the camera coordinate system of the image acquisition device 40. The first determination module 13 is used to determine the lens barrel movement parameters according to the sight line vector and the center vector of the lens barrel 20. The moving module 14 is used to move the corresponding lens barrel 20 according to the lens barrel movement parameters corresponding to each lens barrel 20, so that the center vector of the lens barrel 20 is aligned with the corresponding sight line vector of the human eye.

[0150] The first calculation module 12 is specifically used to obtain the spot coordinates P2 of the first image coordinates in the camera coordinate system and the first pupil coordinates P3 of the second image coordinates in the camera coordinate system according to the calibration parameters preset by the image acquisition device 40; and calculate the line of sight vector according to the light source coordinates P1, the spot coordinates P2 and the first pupil coordinates P3.

[0151] The first calculation module 12 is specifically used to calculate the corneal spherical center coordinates P5 in the camera coordinate system according to the light source coordinates P1, the light spot coordinates P2, and the optical center coordinates P4 of the optical center of the image acquisition device 40 in the camera coordinate system; calculate the pupil reflection point coordinates P7 of the pupil on the cornea in the camera coordinate system according to the corneal spherical center coordinates P5, the first pupil coordinates P3 and the preset corneal radius; calculate the second pupil coordinates P8 of the pupil in the camera coordinate system according to the corneal spherical center coordinates P5, the pupil reflection point coordinates P7 and the optical center coordinates P4; calculate the line of sight vector according to the second pupil coordinates P8 and the corneal spherical center coordinates P5.

[0152] The first calculation module 12 is specifically used to establish a first functional relationship between the light spot coordinate P2, the optical center coordinate P4, and the light source reflection point coordinate P6 of the light source 30 on the cornea based on the geometric relationship between the light source 30, the first reflection position of the light spot on the cornea, the optical center and the position of the light spot on the image plane, and determine a first functional relationship group according to the first functional relationships corresponding to the light sources 20 at multiple different positions;

[0153] Based on the distance relationship between the first reflection position and the center of the cornea, the second functional relationship between the light source reflection point coordinate P6, the cornea center coordinate P5 and the cornea radius is determined, and the second functional relationship group is determined according to the second functional relationship corresponding to the light sources 20 at multiple different positions; based on the coplanar relationship between the incident light, the outgoing light and the normal in the reflection law, the third functional relationship between the light source coordinate P1, the light source reflection point, the cornea center coordinate P5 and the optical center coordinate P4 is determined, and the third functional relationship group is determined according to the third functional relationship corresponding to the light sources 20 at multiple different positions; based on the relationship between the incident angle and the outgoing angle , determine the fourth functional relationship among the light source coordinates P1, the light source reflection point coordinates P6, the corneal spherical center coordinates P5 and the optical center coordinates P4, and determine the fourth functional relationship group according to the fourth functional relationship corresponding to the light sources 20 at multiple different positions; determine the fifth functional relationship group by combining the first functional relationship group and the third functional relationship group; calculate the corneal spherical center coordinates P5 according to the first functional relationship group, the second functional relationship group, the fourth functional relationship group and the fifth functional relationship group; calculate the corneal spherical center coordinates P5 according to the first functional relationship group, the second functional relationship group, the fourth functional relationship group and the fifth functional relationship group.

[0154] The first calculation module 12 is specifically used to determine the sixth functional relationship between the optical center coordinates P4, the first pupil coordinates P3 and the pupil reflection point coordinates P7 based on the geometric relationship between the optical center, the position of the pupil in the image plane and the second reflection position of the pupil on the cornea; determine the seventh functional relationship between the pupil reflection point coordinates P7, the corneal spherical center coordinates P5 and the corneal radius based on the distance relationship between the second reflection position and the spherical center of the cornea; calculate the pupil reflection point coordinates P7 according to the corneal spherical center coordinates P5, the sixth functional relationship and the seventh functional relationship.

[0155] The first calculation module 12 is specifically used to determine the eighth functional relationship among the optical center coordinates P4, the corneal spherical center coordinates P5, the pupil reflection point coordinates P7 and the second pupil coordinates P8 based on the coplanar relationship among the incident light, the outgoing light and the normal in the law of refraction; determine the ninth functional relationship among the optical center coordinates P4, the corneal spherical center coordinates P5, the pupil reflection point coordinates P7 and the second pupil coordinates P8 based on the relationship between the incident angle and the outgoing angle; determine the tenth functional relationship among the second pupil coordinates P8, the corneal spherical center coordinates P5 and the preset distance between the corneal center and the pupil center based on the distance relationship between the pupil and the cornea; calculate the second pupil coordinates P8 according to the corneal spherical center coordinates P5, the pupil reflection point coordinates P7, the eighth functional relationship, the ninth functional relationship and the tenth functional relationship.

[0156] The first determining module 13 is specifically used to determine the moving direction and moving distance according to the sight line vector and the center vector.

[0157] The first determining module 13 is specifically configured to decompose the moving direction into a first direction and a second direction, and determine a first moving distance in the first direction and a second moving distance in the second direction.

[0158] The first determining module 13 is specifically configured to decompose the moving direction into a first direction and a second direction, and determine a first moving distance in the first direction and a second moving distance in the second direction.

[0159] The moving module 14 is specifically used to move the first lens barrel 21 according to the moving direction corresponding to the first lens barrel 21 so that the first center vector coincides with the first sight vector; and move the second lens barrel 22 according to the moving direction corresponding to the second lens barrel 22 so that the second center vector coincides with the second sight vector.

[0160] The lens barrel adjustment device 10 also includes a judgment module 15, which is used to calculate the angle between the center vector of the lens barrel 20 and the corresponding line of sight vector of the human eye; when the angle is less than a preset threshold, the step of moving the corresponding lens barrel 20 according to the lens barrel movement parameters corresponding to each lens barrel 20 so that the center vector of the lens barrel 20 is aligned with the corresponding line of sight vector of the human eye is entered; when the angle is greater than the preset threshold, the step of identifying the first image coordinates of the light spots formed by the multiple light sources 30 in the human eye image in the image coordinate system of the human eye image and the second image coordinates of the pupil of the human eye in the image coordinate system is entered again.

[0161] The lens barrel adjustment device 10 further includes a second calculation module 16 and a second determination module 17. The second calculation module 16 is used to calculate the lens barrel center coordinates of the center of the lens barrel 20 in the camera coordinate system according to the first preset position relationship between the center of the lens barrel 20 and the light source 30, and the light source coordinates P1. The second determination module 17 is used to determine the center vector according to the lens barrel center coordinates.

[0162] The lens barrel adjustment device 10 further includes an acquisition module 18 and a third determination module 19. The acquisition module 18 is used to acquire a second preset position relationship between the light source 30 and the image acquisition device 40. The third determination module 19 is used to determine the coordinates of the preset position of the light source 30 in the camera coordinate system according to the second preset position relationship, as the light source coordinates P1.

[0163] In the above text, the lens barrel adjustment device 10 is described from the perspective of functional modules in conjunction with the accompanying drawings. The functional modules can be implemented in hardware form, can be implemented in software form, or can be implemented in combination with hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware coding processor to execute, or can be executed by a combination of hardware and software modules in the coding processor. Optionally, the software module can be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0164] Figure 2 is a schematic block diagram of a wearable device 100 provided in an embodiment of the present application. Figure 2 As shown, the wearable device 100 may include: a memory 60 and a processor 50, wherein the memory 60 is used to store a computer program and transmit the program code to the processor 50. In other words, the processor 50 can call and run the computer program from the memory 60 to implement the method in any embodiment of the present application.

[0165] In some embodiments of the present application, the processor 50 may include but is not limited to:

[0166] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0167] In some embodiments of the present application, the memory 60 includes but is not limited to:

[0168] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0169] In some embodiments of the present application, the computer program may be divided into one or more modules, which are stored in the memory 60 and executed by the processor 50 to complete the method provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the wearable device 100.

[0170] The present application also provides a computer storage medium on which a computer program is stored, and when the computer program is executed by a processor, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0171] When software is used for implementation, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integration. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0172] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0173] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0174] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0175] The above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for adjusting a lens barrel, characterized in that: Applied to a wearable device, the wearable device includes a lens barrel, multiple light sources and an image acquisition device, the image acquisition device is used to acquire human eye images, and the method includes: Identify, in the human eye image, first image coordinates of light spots formed by the plurality of light sources in the image coordinate system of the human eye image, and second image coordinates of the pupil of the human eye in the image coordinate system; Calculate the sight vector of the human eye according to the first image coordinates, the second image coordinates, and the light source coordinates of the light sources in the camera coordinate system of the image acquisition device; Determining a lens barrel movement parameter according to the sight line vector and the center vector of the lens barrel; The corresponding lens barrel is moved according to the lens barrel movement parameter corresponding to each lens barrel, so that the center vector of the lens barrel is aligned with the sight line vector of the corresponding human eye.

2. The lens barrel adjustment method according to claim 1, characterized in that: The calculating the sight line vector of the human eye according to the first image coordinates, the second image coordinates, and the light source coordinates of the light source in the camera coordinate system includes: According to the calibration parameters preset by the image acquisition device, the spot coordinates of the first image coordinates in the camera coordinate system and the first pupil coordinates of the second image coordinates in the camera coordinate system are acquired; The sight line vector is calculated according to the light source coordinates, the light spot coordinates and the first pupil coordinates.

3. The lens barrel adjustment method according to claim 2, characterized in that: The calculating the sight line vector according to the light source coordinates, the light spot coordinates and the first pupil coordinates includes: Calculate the coordinates of the corneal spherical center in the camera coordinate system according to the light source coordinates, the light spot coordinates, and the optical center coordinates of the optical center of the image acquisition device in the camera coordinate system; Calculating the coordinates of the pupil reflection point of the pupil on the cornea in the camera coordinate system according to the corneal spherical center coordinates, the first pupil coordinates and a preset corneal radius; Calculate the second pupil coordinates of the pupil in the camera coordinate system according to the corneal spherical center coordinates, the pupil reflection point coordinates, and the optical center coordinates; The sight line vector is calculated according to the second pupil coordinates and the corneal spherical center coordinates.

4. The lens barrel adjustment method according to claim 3, characterized in that: The step of calculating the corneal spherical center coordinates in the camera coordinate system according to the light source coordinates, the light spot coordinates, and the optical center coordinates of the optical center of the image acquisition device in the camera coordinate system comprises: Based on the geometric relationship between the light source, the first reflection position of the light spot on the cornea, the optical center and the position of the light spot on the image plane, a first functional relationship between the light spot coordinates, the optical center coordinates, and the light source reflection point coordinates of the light source on the cornea is established, and a first functional relationship group is determined according to the first functional relationships corresponding to the light sources at multiple different positions; Based on the distance relationship between the first reflection position and the center of the cornea, determine a second functional relationship between the coordinates of the light source reflection point, the coordinates of the cornea center and the cornea radius, and determine a second functional relationship group according to the second functional relationships corresponding to the light sources at multiple different positions; Based on the coplanar relationship between the incident light, the outgoing light and the normal in the law of reflection, determine the third functional relationship among the light source coordinates, the light source reflection point, the corneal spherical center coordinates and the optical center coordinates, and determine the third functional relationship group according to the third functional relationships corresponding to the light sources at multiple different positions; Based on the relationship between the incident angle and the exit angle, determine a fourth functional relationship among the light source coordinates, the light source reflection point coordinates, the corneal spherical center coordinates and the optical center coordinates, and determine a fourth functional relationship group according to the fourth functional relationships corresponding to the light sources at multiple different positions; Determine a fifth functional relationship group by combining the first functional relationship group and the third functional relationship group; The corneal spherical center coordinates are calculated according to the first functional relationship group, the second functional relationship group, the fourth functional relationship group and the fifth functional relationship group.

5. The lens barrel adjustment method according to claim 3, characterized in that: The step of calculating the coordinates of the pupil reflection point of the pupil on the cornea in the camera coordinate system according to the corneal spherical center coordinates, the first pupil coordinates and a preset corneal radius includes: Determine a sixth functional relationship among the optical center coordinates, the first pupil coordinates, and the pupil reflection point coordinates based on a geometric relationship among the optical center, the position of the pupil on the image plane, and the second reflection position of the pupil on the cornea; Determine a seventh functional relationship among the pupil reflection point coordinates, the corneal spherical center coordinates and the corneal radius based on the distance relationship between the second reflection position and the corneal spherical center; The pupil reflection point coordinates are calculated according to the corneal spherical center coordinates, the sixth functional relationship and the seventh functional relationship.

6. The lens barrel adjustment method according to any one of claims 3 to 5, characterized in that: The calculating, according to the corneal spherical center coordinates, the pupil reflection point coordinates, and the optical center coordinates, the second pupil coordinates of the pupil in the camera coordinate system comprises: Based on the coplanar relationship between the incident light, the outgoing light and the normal line in the law of refraction, determining an eighth functional relationship among the optical center coordinates, the corneal spherical center coordinates, the pupil reflection point coordinates and the second pupil coordinates; Based on the relationship between the incident angle and the exit angle, determining a ninth functional relationship among the optical center coordinates, the corneal spherical center coordinates, the pupil reflection point coordinates and the second pupil coordinates; Based on the distance relationship between the pupil and the cornea, determining a tenth functional relationship among the second pupil coordinates, the cornea spherical center coordinates, and a preset distance between the cornea center and the pupil center; The second pupil coordinates are calculated according to the corneal spherical center coordinates, the pupil reflection point coordinates, the eighth functional relationship, the ninth functional relationship and the tenth functional relationship.

7. The lens barrel adjustment method according to claim 1, characterized in that: The lens barrel movement parameters include a movement direction and a corresponding movement distance, and the lens barrel movement parameters are determined according to the sight line vector and the center vector of the lens barrel, including: The moving direction and the moving distance are determined according to the sight line vector and the center vector.

8. The lens barrel adjustment method according to claim 7, characterized in that: The lens barrel can move in a preset first direction and a second direction, the first direction is perpendicular to the second direction, and the first direction and the second direction are both perpendicular to the direction corresponding to the center vector, and determining the moving direction and the moving distance according to the sight line vector and the center vector includes: The moving direction is decomposed into the first direction and the second direction, and a first moving distance in the first direction and a second moving distance in the second direction are determined.

9. The lens barrel adjustment method according to claim 1, characterized in that: The step of determining the lens barrel movement parameter according to the sight line vector and the center vector of the lens barrel comprises: The sight line vector and the center vector are converted into a housing coordinate system of the wearable device, so as to determine the lens barrel movement parameter according to the sight line vector and the center vector in the housing coordinate system.

10. The lens barrel adjustment method according to any one of claims 7 to 9, characterized in that: The lens barrel comprises a first lens barrel and a second lens barrel, the first lens barrel corresponds to the left eye, the second lens barrel corresponds to the right eye, the center vector comprises a first center vector of the first lens barrel and a second center vector of the second lens barrel, and the sight line vector comprises a first sight line vector of the left eye and a second sight line vector of the right eye; The step of moving the corresponding lens barrel according to the lens barrel movement parameters corresponding to each lens barrel so that the center vector of the lens barrel is aligned with the sight line vector of the corresponding human eye comprises: Moving the first lens barrel according to the moving direction corresponding to the first lens barrel so that the first center vector coincides with the first sight line vector; The second lens barrel is moved according to the moving direction corresponding to the second lens barrel, so that the second center vector coincides with the second sight line vector.

11. The lens barrel adjustment method according to claim 1, characterized in that: The alignment of the center vector of the lens barrel with the sight line vector of the corresponding human eye comprises: a distance between the center vector of the lens barrel and the sight line vector of the corresponding human eye in the moving direction of the lens barrel is less than a preset distance threshold.

12. The lens barrel adjustment method according to claim 1, characterized in that: Also includes: Calculating the angle between the center vector of the lens barrel and the corresponding sight line vector of the human eye; When the included angle is less than a preset threshold, the step of moving the corresponding lens barrel according to the lens barrel movement parameters corresponding to each lens barrel is performed so that the center vector of the lens barrel is aligned with the sight line vector of the corresponding human eye; When the angle is greater than a preset threshold, the step of identifying the first image coordinates of the light spots formed by the multiple light sources in the human eye image in the image coordinate system and the second image coordinates of the pupil of the human eye in the image coordinate system is entered again.

13. The lens barrel adjustment method according to claim 1, characterized in that: Also includes: Calculating the center coordinates of the lens barrel in the camera coordinate system according to the first preset positional relationship between the center of the lens barrel and the light source and the light source coordinates; The center vector is determined according to the center coordinates of the lens barrel.

14. The lens barrel adjustment method according to claim 1, characterized in that: Also includes: Acquiring a second preset position relationship between the light source and the image acquisition device; According to the second preset position relationship, the coordinates of the preset position of the light source in the camera coordinate system are determined as the light source coordinates.

15. A lens barrel adjustment device, characterized in that: Applied to a wearable device, the wearable device comprises a lens barrel, a plurality of light sources and an image acquisition device, the image acquisition device is used to acquire human eye images, and the device comprises: an identification module, used for identifying, in the human eye image, first image coordinates of light spots formed by the plurality of light sources in the image coordinate system of the human eye image, and second image coordinates of the pupil of the human eye in the image coordinate system; A first calculation module, used for calculating the sight line vector of the human eye according to the first image coordinates, the second image coordinates, and the light source coordinates of the light sources in the camera coordinate system of the image acquisition device; A first determining module, used for determining a lens barrel movement parameter according to the sight line vector and the center vector of the lens barrel; The moving module is used to move the corresponding lens barrel according to the lens barrel moving parameters corresponding to each lens barrel, so that the center vector of the lens barrel is aligned with the sight line vector of the corresponding human eye.

16. A wearable device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the lens barrel adjustment method according to any one of claims 1 to 14 is implemented. 17 . A computer-readable storage medium comprising a computer program, wherein when the computer program is executed by a processor, the processor executes the lens barrel adjustment method according to claim 1 .

Images