Method for determining position of target object, and related device

Abstract

Provided in the present disclosure are a method for determining the position of a target object, and a related device. The method comprises: acquiring a plurality of camera parameters of a camera, wherein the plurality of camera parameters comprise a plurality of spatial linear equations having different starting points; determining at least two target distances; on the basis of the at least two target distances and the plurality of camera parameters, selecting at least two groups of target parameters; on the basis of the at least two groups of target parameters, determining the central position of the camera; acquiring a target image collected by the camera, wherein the target image comprises a target object; and on the basis of the central position of the camera and the target image, determining the position of the target object.

Description

Method for determining the position of a target object and related device

[0001] This application claims priority to the Chinese invention patent application entitled “Method and related equipment for determining the position of a target object” filed on December 1, 2023, with application number 202311638669.9, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present disclosure relates to the field of image processing technology, and in particular to a method for determining the position of a target object and related equipment. Background Art

[0003] The central camera model usually refers to a camera model in which the imaging center converges to one point.

[0004] However, the inventors of the present disclosure have discovered that in some scenarios, when the camera is placed inside the lens barrel, it is difficult to simulate the imaging process using a central camera model, thereby making it difficult to locate objects in the physical world based on the captured images. Summary of the Invention

[0005] The present disclosure proposes a method for determining the position of a target object and related devices to solve or partially solve the above problems.

[0006] In a first aspect, the present disclosure provides a method for determining a position of a target object, comprising: obtaining multiple camera parameters of a camera, the multiple camera parameters including multiple spatial straight line equations with different starting points; determining at least two target distances; selecting at least two groups of target parameters based on the multiple camera parameters according to the at least two target distances; determining a center position of the camera according to the at least two groups of target parameters; obtaining a target image captured by the camera, the target image including the target object; and determining the position of the target object based on the center position of the camera and the target image.

[0007] According to a second aspect of the present disclosure, there is provided an apparatus for determining a position of a target object, comprising: a first acquisition module configured to acquire multiple camera parameters of a camera, wherein the multiple camera parameters include multiple spatial straight line equations with different starting points; a first determination module configured to determine at least two target distances; based on the at least two target distances, select at least two groups of target parameters based on the multiple camera parameters; and determine a center position of the camera based on the at least two groups of target parameters; a second acquisition module configured to acquire a target image captured by the camera, wherein the target image includes the target object; and a second determination module configured to determine the position of the target object based on the center position of the camera and the target image.

[0008] In a third aspect of the present disclosure, a computer device is provided, comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to the first aspect.

[0009] According to a fourth aspect of the present disclosure, a non-volatile computer-readable storage medium containing a computer program is provided. When the computer program is executed by one or more processors, the processors are caused to execute the method described in the first aspect.

[0010] In a fifth aspect of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed on a computer, cause the computer to execute the method described in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0012] FIG1A shows a schematic diagram of an exemplary system provided by an embodiment of the present disclosure.

[0013] FIG1B shows a schematic diagram of an exemplary head-mounted wearable device.

[0014] 1C and 1D are schematic diagrams showing exemplary human eye images.

[0015] FIG2A shows a schematic diagram of an exemplary wearable device provided by an embodiment of the present disclosure.

[0016] FIG2B shows a schematic diagram of another exemplary wearable device provided by an embodiment of the present disclosure.

[0017] FIG2C shows a schematic diagram of another exemplary wearable device provided by an embodiment of the present disclosure.

[0018] FIG3 shows a schematic diagram of exemplary camera parameters according to an embodiment of the present disclosure.

[0019] FIG4A shows a flowchart of an exemplary method provided by an embodiment of the present disclosure.

[0020] FIG4B shows a flowchart of an exemplary method for determining target parameters according to an embodiment of the present disclosure.

[0021] FIG4C is a schematic diagram showing an exemplary central camera model fitting result according to an embodiment of the present disclosure.

[0022] FIG4D is a flowchart illustrating an exemplary method for determining the center position of the cornea according to an embodiment of the present disclosure.

[0023] FIG4E shows a schematic diagram of a camera model according to an embodiment of the present disclosure.

[0024] FIG5 shows a schematic diagram of the hardware structure of an exemplary computer device provided by an embodiment of the present disclosure.

[0025] FIG6 shows a schematic diagram of an exemplary device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.

[0027] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative position relationships. When the absolute position of the described object changes, the relative position relationship may also change accordingly.

[0028] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0029] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0030] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0031] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0032] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0033] FIG1A shows a schematic diagram of an exemplary augmented reality system 100 provided by an embodiment of the present disclosure.

[0034] Extended Reality (XR) refers to the use of computers to combine the real and virtual worlds to create a virtual environment where humans and machines can interact. XR technology can further encompass augmented reality (AR), virtual reality (VR), and mixed reality (MR), leveraging hardware devices combined with various technologies to fuse virtual content with real-world scenes.

[0035] As shown in Figure 1A, the system 100 may include various types of wearable devices, such as head-mounted wearable devices (e.g., VR / AR glasses or head-mounted displays (HMD)) 104, operating handles 108, etc. In some scenarios, a camera / camera 110 for taking photos of the operator (user) 102 may also be provided. In some embodiments, when the aforementioned device does not have a processing function, the system 100 may also include an external control device 112 for providing a processing function. The control device 112, for example, may be a computer device such as a mobile phone or a computer. In some embodiments, when any of the aforementioned devices serves as a control device or a main control device, it may communicate with other devices in the system 100 through wired or wireless communication to achieve information interaction.

[0036] In system 100, user 102 can interact with augmented reality system 100 using head-mounted wearable device 104 and operating handle 108. In some scenarios, system 100 can use images captured by camera 110 to identify user 102's posture and gestures, and then interact with user 102 based on the identified posture and gestures. In some embodiments, user 130 can also implement gesture input using bare hands. Head-mounted wearable device 104 can capture front-facing images in real time using a camera or other device located in front of head-mounted wearable device 104, and recognize user 130's gestures by recognizing these images.

[0037] In some embodiments, as shown in FIG1A , the system 100 may further communicate with a server 114 and obtain data, such as images, audio, and video, from the server 114 and output the data through the head-mounted wearable device 104 , for example, by displaying images or videos on a display screen of the head-mounted wearable device 104 , playing audio and video audio using a speaker of the head-mounted wearable device 104 , etc. In some embodiments, as shown in FIG1A , the server 114 may retrieve the required data, such as images, audio, and video, from a database server 116 for storing data.

[0038] In some embodiments, a collection unit for collecting information may be provided on the head-mounted wearable device 104. The collection unit may be of various types.

[0039] In some embodiments, the acquisition unit may further include an environment acquisition unit and a positioning tracking unit, wherein the environment acquisition unit may be used to acquire environmental information around (for example, in front of) the wearable device 104, and the positioning tracking unit may be used to track the wearable device 104. Optionally, the environment acquisition unit may include but is not limited to a three-color camera (for example, an RGB camera), a depth camera, a binocular camera, a laser or other photosensitive element, and the positioning tracking unit may include but is not limited to visual simultaneous localization and mapping (visual SLAM), an inertial measurement unit (IMU), a global positioning system (GPS), ultra-wideband wireless communication technology (UWB), a laser or other modules.

[0040] In some embodiments, the head-mounted wearable device 104 may also be provided with a speed sensor, an acceleration sensor, an angular velocity sensor (e.g., a gyroscope), etc. for collecting speed information or acceleration information of the head-mounted wearable device 104. For another example, the operating handle 108 may also be provided with a speed sensor, an acceleration sensor, an angular velocity sensor (e.g., a gyroscope), etc. for collecting speed information or acceleration information of the wearing gloves 106. It should be noted that, in addition to being provided on the head-mounted wearable device 104 and the operating handle 108, the aforementioned collection unit may also be provided directly on a body part of the interactive user 102 by attachment without relying on a hardware device, thereby collecting relevant information of the body part, such as speed, acceleration, or angular velocity information, or information collected by other sensors or collection units.

[0041] In some embodiments, the head-mounted wearable device 104 may also be provided with a camera or a camera for taking photos of the operator (user) 102 (eg, photos of hands or feet) and images of the environment.

[0042] In some embodiments, the system 100 can identify the posture and gestures of the user 102 through the collected information, and then can perform corresponding interactions based on the identified user posture and gestures.

[0043] FIG. 1B shows a schematic diagram of an exemplary head-mounted wearable device 104 .

[0044] As shown in FIG1B , the head-mounted wearable device 104 may include a lens barrel 1042, inside which a display screen 1044 for displaying an image and an optical component 1046 for processing the light path may be provided. Optionally, the optical component 1046 may further include a plurality of lenses (e.g., lenses 1046A and 1046B). The combination of the plurality of lenses may project light emitted by the display screen 1044 into the human eye 1022, so that the human eye 1022 may view the image displayed on the display screen 1044. It will be understood that FIG1B only exemplarily shows a single-sided structure of the head-mounted wearable device 104. In order to achieve binocular display, the head-mounted wearable device 104 may include two lens barrel structures arranged side by side.

[0045] In some embodiments, as shown in FIG1B , the head-mounted wearable device 104 may also be provided with a camera 1048 for capturing images of the human eye. The camera 1048 may be a charge coupled device (CCD) image sensor, a complementary metal oxide semiconductor (CMOS) image sensor, or the like.

[0046] Optionally, the camera 1048 may be an eye tracking (ET) camera, and the human eye images collected by the camera may be used to implement functions such as pupil distance estimation and eye tracking.

[0047] As shown in FIG1B , in the related art, camera 1048 is usually set outside the lens barrel and usually only one camera is set for each lens barrel. In addition, in order to better capture the complete image of the human eye and not affect the human eye's observation of the screen 1044, the common deployment position of camera 1048 is generally at the outer corner of the eye, the nose wing, or directly below the eye. Referring to FIG1B , if camera 1048 is close to the outside of the device, the camera deployment position shown in FIG1B is the outer corner of the eye. If camera 1048 is close to the inside of the device, the camera deployment position shown in FIG1B is the nose wing.

[0048] However, the inventors of the present disclosure discovered that the camera mounting method used in the related art easily causes camera 1048 to be mounted at a large angle relative to eye 1022. This results in a large angle α between the orientation of camera 1048 and the normal viewing direction of eye 1022 (i.e., camera 1048 is tilted at a high degree). This makes it difficult for the captured eye image to reflect the normal viewing angle of the eye, as shown in Figures 1C and 1D. Furthermore, within the range of eye movement, a single camera 1048 is unable to capture the complete range of eye movement.

[0049] In some cases, the user may need to wear glasses before using the head-mounted wearable device 104. However, since the camera 1048 is set outside the lens barrel 1042 (especially when the camera 1048 is set at the nose wing position), the camera 1048 is higher than the lens barrel 1042, which makes it easy to collide with or squeeze the glasses, affecting the wearing comfort of the head-mounted wearable device 104 and possibly scratching the user's glasses. At the same time, the imaging of the camera 1048 is easily affected by the edge of the glasses. The refraction of light passing through the edge of the glasses reduces the image clarity of the camera 1048 and forms many refracted light spots in the image, which reduces the clarity of the image captured by the camera and produces large distortion, thereby affecting the accuracy of the subsequent pupil distance estimation algorithm and the line of sight estimation algorithm. Alternatively, when the glasses have a frame, the camera is easily blocked by the edge of the glasses and cannot capture a complete eye image, resulting in the failure of the pupil distance estimation function and the line of sight estimation function. In particular, when there is only one camera corresponding to the lens barrel, this problem will be further aggravated.

[0050] In view of this, an embodiment of the present disclosure provides a wearable device in which a camera is disposed inside a lens barrel, which can solve or partially solve the above-mentioned problems to a certain extent.

[0051] FIG2A shows a schematic diagram of an exemplary wearable device 200 provided by an embodiment of the present disclosure.

[0052] As shown in FIG2A , similarly, the wearable device 200 may also include a lens barrel 202, a display screen 204 disposed inside the lens barrel 202, and an optical assembly 206. The optical assembly 206 may further include multiple lenses (e.g., lenses 206A and 206B). The combination of the multiple lenses may project light emitted by the display screen 204 into the human eye 1022, so that the human eye 1022 can view the image displayed on the display screen 204.

[0053] Unlike the wearable device 104 shown in FIG1B , the camera 208 of the wearable device 200 is arranged inside the lens barrel 202 and toward the light-emitting side of the lens barrel. Since the camera 208 is placed inside the lens barrel 202, it will not affect the wearing of glasses, thereby improving the comfort of the wearable device 200. At the same time, as shown in FIG2A , since the camera 208 is arranged inside the lens barrel 202, the distance between the camera 208 and the human eye 1022 is extended, so that the installation angle of the camera 208 relative to the human eye 1022 becomes smaller, and then the angle β between the direction of the camera 208 and the normal view direction of the human eye 1022 is reduced, so that the camera 208 has a better observation angle, and the collected human eye image can better reflect the image of the human eye normal view angle, and the imaging quality is better. In addition, since the camera 208 is placed inside the lens barrel 202, the glasses will not interfere with the imaging of the camera 208, further improving the imaging quality. The improvement in imaging quality also improves the accuracy of algorithms such as pupil distance estimation or line of sight tracking.

[0054] FIG2B shows a schematic diagram of another exemplary wearable device 200 provided by an embodiment of the present disclosure.

[0055] As shown in FIG2B , in some embodiments, unlike FIG2A , camera 208 faces display screen 204 , and wearable device 200 may further include a reflective structure 210 , which may be a reflective film, a reflective mirror, or other reflective structure. By reflecting light from reflective structure 210 , camera 208 can still capture images of the human eye. Furthermore, because an additional reflection process is added to the optical path, the observation angle γ is further reduced, enabling camera 208 to better capture images.

[0056] FIG2C shows a schematic diagram of another exemplary wearable device 200 provided by an embodiment of the present disclosure.

[0057] As shown in FIG2C , in some embodiments, unlike FIG2B , reflective structure 210 can be a semi-transparent, semi-reflective film that transmits half of the light and reflects the other half. Thus, reflective structure 210 can reflect light from the eye outside the lens barrel into camera 208 and transmit light from display screen 204. As can be seen from FIG2B , the use of reflective structure 210 in FIG2C avoids the problem of light blocking caused by the opacity of the reflective structure. Furthermore, the location of reflective structure 210 does not need to be specifically designed based on the location of camera 208; it can simply be placed on the light-emitting side of display screen 204.

[0058] In some embodiments, as shown in Figures 2A to 2C, the wearable device 200 may further include a point light source 212 disposed outside the lens 202. The light emitted by the point light source 212 can form reflected light on the cornea of ​​the human eye 1022, thereby being collected by the camera 208, and then forming a light spot in the image obtained by the camera 208. According to the position of the light spot in the image, it can be used to assist in locating the center of the cornea and the position of the pupil. In some embodiments, in order to make the positioning more accurate, the number of point light sources 212 can be two or more. Optionally, the point light source 212 can be a light emitting diode (LED). In some embodiments, the light emitted by the point light source 212 may not be visible light (for example, it may be infrared light) or may be visible light with relatively weak brightness, thereby improving the user's comfort.

[0059] It can be understood that FIG. 2A to FIG. 2C only exemplarily illustrate a single-side structure of the wearable device 200. In order to achieve binocular display, the wearable device 200 may include two lens barrel structures arranged side by side.

[0060] It can be seen from the above embodiments that the embodiment of the present disclosure not only avoids the aforementioned adverse effects by setting the camera 208 inside the wearable device 200, but also increases the distance between the camera and the eye, thereby having a better camera observation angle and obtaining better imaging quality (i.e., the camera can capture higher quality eye images), which is beneficial to improving the accuracy of the pupil distance estimation algorithm and the line of sight estimation algorithm, and at the same time can also improve the wearing comfort of the wearable device 200.

[0061] Furthermore, the inventors of the present disclosure discovered that, compared with placing the camera outside the lens barrel, when the camera is placed inside the lens barrel, the imaging distortion is affected not only by the camera module itself but also by the optical components in the lens barrel when the camera is imaging.

[0062] As shown in Figures 2A to 2C , compared to the camera 1048 in Figure 1B , the optical component 206 or a portion of the optical component 206 within the lens barrel is also present in the optical path from the camera 208 to the human eye 1022 (the position may vary depending on the relative position of the camera 208 and the lenses in the optical component 206; for example, in addition to the position shown in Figures 2A to 2C , the camera 208 may also be positioned between lens 206A and lens 206B). Therefore, when the camera 208 images the human eye 1022, the optical component 206 or a portion of the optical component 206 affects or changes the optical path from the camera 208 to the human eye 1022, resulting in inconsistent refractive indices at various locations within the lens barrel. This causes the optical axis of the camera 208 to not completely coincide with the optical axis of the optical component 206 within the lens barrel, resulting in potentially asymmetrical distortion of the image captured by the camera 208 (i.e., the distortion produced at various locations in the image captured by the camera is different).

[0063] Furthermore, since the optical path from the camera 208 to the human eye 1022 is changed, the camera 208 does not have a unified projection center, making it impossible to use a central camera model to fit the projection process of the camera in the lens barrel to process the distortion, making it difficult to calibrate the camera parameters.

[0064] In light of this, some embodiments of the present disclosure may first calibrate the parameters of the camera within the lens barrel. The calibrated camera parameters can be used to characterize the pixels in the image captured by the camera and the projection directions corresponding to these pixels. The projection direction (also referred to as the projection direction) can be the spatial line corresponding to a pixel in the image coordinate system (two-dimensional) in the camera coordinate system (three-dimensional). Based on this projection direction, a correspondence between the pixel in the image and a position in three-dimensional space can be established.

[0065] When the camera parameters are calibrated, image acquisition can be performed.

[0066] For example, a motion camera is used to continuously shoot the calibration plate to obtain observation data in multiple postures, and a series of images are generated as input.

[0067] Then, a correspondence between the pixel points in the image and a certain position in the three-dimensional space is established based on the captured image.

[0068] Specifically, three images can be selected from the images collected previously, and a homography transformation matrix between the pixel points of each image and the calibration plate in space can be constructed, and then a one-to-one correspondence between each pixel point and the corresponding coordinate of the pixel point in the calibration plate coordinate system can be established.

[0069] After establishing the correspondence between pixel points and spatial coordinates, all three images are converted to the reference coordinate system. The extrinsic parameters between the three images are calculated by using the property that the spatial points corresponding to the same pixel position in the three images are located on the same straight line.

[0070] In this way, the correspondence between at least some of the pixels and a certain position in the three-dimensional space can be obtained.

[0071] Next, using the remaining images in the captured image, continue to establish the correspondence between the pixel points and a certain position in the three-dimensional space according to the above method until all the pixels of the image correspond to a spatial straight line, that is, the camera parameter calibration is completed.

[0072] Since each pixel corresponds to a spatial line, when storing camera parameters, the same number of camera parameters as the number of pixels needs to be stored. Therefore, the number of camera parameters increases as the image size increases. To reduce the number of camera parameters, in some embodiments, a smooth spline surface can be used to fit the direction, and then the spline control points can be optimized using the camera's observation data. After the optimization is completed, multiple control points of the fitted spline surface can be obtained as camera parameters, thereby reducing the number of camera parameters to be stored. When the camera parameters are needed later, quadratic spline interpolation is performed to restore the spatial line equations for all pixels.

[0073] FIG3 is a schematic diagram showing exemplary camera parameters according to an embodiment of the present disclosure.

[0074] As shown in Figure 3, the camera parameters obtained by calibration are the spatial straight line equations corresponding to some pixel points (that is, each pixel point in the camera imaging corresponds to a direction vector and a starting point position in the three-dimensional space). In some embodiments, in order to save the amount of calculation, only some key pixel points (control points) are sampled and their spatial straight line equations and their starting points are saved as camera parameters. The camera parameters corresponding to other pixel points can be obtained by quadratic spline interpolation. As shown in Figure 3, the spatial straight lines corresponding to each pixel point do not strictly intersect at one point (for example, point A and point B are the intersection points obtained by the convergence of different spatial straight lines), but are distributed in a range interval, indicating that the camera is non-central, that is, there is no unique projection center.

[0075] Therefore, in order to ensure the accuracy of the algorithm, when using a non-parametric camera calibration model to calculate the pupil position and line of sight direction, for the pixel points in the image captured by the camera, since the camera parameters only retain the spatial straight line equations corresponding to the control points, it is necessary to perform quadratic spline interpolation on the starting point position and direction vector in the camera parameters to restore the spatial straight line equations corresponding to all pixel points. The calculation is large and complex.

[0076] In order to simplify the algorithm and save computational effort, in some embodiments, a central camera approximation can be performed under the full camera field of view. By forcibly fitting the non-central camera model with the central camera model, a camera center is obtained, thereby eliminating the need for quadratic spline interpolation of the starting point position, thereby saving computational effort.

[0077] However, the inventors of the present disclosure have found that such processing may result in a large error, and such accuracy is unacceptable for the subsequent interpupillary distance (IPD) estimation algorithm and the eye tracking (ET) algorithm.

[0078] In view of this, the embodiments of the present disclosure provide a method and related equipment for determining the position of a target object. By fitting the camera parameters using a central camera model within a certain distance range of the recommended distance (at least two target distances) to obtain the center position of the camera, the quadratic spline interpolation of the starting point position can be omitted, thereby improving the calculation speed. In addition, the target object is positioned based on the camera center position. At the same time, because at least two target distances are close to the recommended distance, the algorithm accuracy can be guaranteed.

[0079] FIG4A is a flow chart illustrating an exemplary method 500 provided in an embodiment of the present disclosure. The method 500 can be applied to the head-mounted wearable device 104 of FIG1A , the wearable device 200 of FIG2A and FIG2B , and the external device 112 of FIG1A . The method 500 can be used to determine the position of a target object in a three-dimensional space (world coordinate system). As shown in FIG4A , the method 500 can further include the following steps.

[0080] In step 502, multiple camera parameters of the camera are obtained. Optionally, the multiple camera parameters include multiple spatial line equations with different starting points. For example, the multiple camera parameters may be camera parameters obtained by calibrating a non-central camera model using the method provided in the aforementioned embodiment. As shown in FIG4E , each camera parameter may be used to indicate the spatial line equation corresponding to a single pixel point, where the spatial line equation passes through the pixel point and has a starting point (the starting point may be obtained by fitting all camera parameters).

[0081] As mentioned above, since the camera parameters are a spatial straight line equation with only a starting point, when the specific position of the target object in the three-dimensional space (world coordinate system) is uncertain, if the position of the target object is to be determined based on the image captured by the camera and combined with the camera parameters, in order to ensure the accuracy of the algorithm, a non-parametric camera calibration model can be used to calculate the position of the target object. For the pixel points in the image captured by the camera, it is necessary to perform quadratic spline interpolation on the starting point position and direction vector in its camera parameters respectively, which is computationally intensive and complex.

[0082] To simplify the algorithm and reduce computational effort, in some embodiments, a central camera approximation can be performed within the full camera field of view. By forcibly fitting the non-central camera model with the central camera model, a camera center is obtained. This eliminates the need for quadratic spline interpolation of the starting point, thus reducing computational effort. However, this approach can introduce significant errors.

[0083] Therefore, the inventors of the present disclosure discovered that if the approximate position of the target object can be determined in advance, and then the camera parameters within a certain range of the approximate position are used to fit the camera center, it can save computational effort and ensure algorithm accuracy.

[0084] Thus, at step 504, at least two target distances may be determined.

[0085] The at least two target distances can be determined based on a recommended distance, which can be a known or estimated optimal imaging distance between the target object and the vertex of the light-emitting surface of the optical component 206. The at least two target distances can be two distance values ​​within a certain range of the recommended distance. For example, assuming that the recommended distance is 50 mm and good imaging quality can be achieved within a range of 10 mm of the recommended distance, at least two target distances can be selected within the range of 50 ± 10 mm.

[0086] As an optional embodiment, to enhance the robustness of the camera center obtained by subsequent fitting, the at least two target distances include at least a first distance (e.g., 60 mm) and a second distance (e.g., 40 mm) within a range of distances. It is understood that to achieve a better fitting effect and ensure accuracy, at least one distance value (e.g., 50 mm) can be selected within the range of 40 mm to 60 mm to fit the camera center together with the farthest and closest distances.

[0087] In some embodiments, the target object includes a cornea, and the method 500 can be used to determine the center position of the cornea, thereby determining the position of the human eye and implementing certain algorithms related to the position of the human eye. Optionally, after determining the center position of the corneas of both eyes, the method 500 can determine the pupillary distance based on this, thereby implementing pupillary distance measurement.

[0088] In some embodiments, the at least two target distances may include at least two target exit pupil distances. Exit pupil distance refers to the distance between the eye and the last lens of the eyepiece when the entire field of view is clearly visible. In optics, it refers to the distance from the vertex of the last surface of the optical system to the intersection of the exit pupil plane and the optical axis. Thus, selecting the target distance based on the exit pupil distance can better achieve pupil distance measurement.

[0089] In some embodiments, step 504 of determining at least two target distances may further include: determining the at least two target exit pupil distances according to a recommended exit pupil distance of the wearable device 200 .

[0090] The wearable device 200 may be a head-mounted display (HMD), and the recommended pupil distance may be the optimal pupil distance of the HMD. Depending on the model of the HMD, the optimal pupil distance may be different, and accordingly, the selected target pupil distance may also be different. For example, if the optimal pupil distance of the HMD is 20 mm, at least two target pupil distances between 15 mm and 25 mm (e.g., a first distance of 25 mm and a second distance of 15 mm) may be used, and a center camera model may be fitted based on the at least two target pupil distances to obtain the center position of the camera.

[0091] In this way, by fitting the central camera model to the camera parameters within a certain distance range of the exit pupil distance, the average error can be effectively controlled within the accuracy requirements of the IPD / ET algorithm.

[0092] In step 506 , at least two sets of target parameters are selected based on the plurality of camera parameters according to the at least two target distances.

[0093] As mentioned above, the camera parameters include multiple spatial straight line equations corresponding to multiple pixel points of the image captured by the camera and the starting points corresponding to the spatial straight line equations. However, these spatial straight line equations have no end points. If the center camera approximation is performed under the full camera field of view, it will lead to large errors. Therefore, in some embodiments, the target spatial point (target parameter) on the spatial straight line corresponding to each spatial straight line equation can be determined based on at least two selected target distances, and then the center camera approximation is performed based on these target spatial points, which can control the algorithm error within an acceptable range. It can be understood that the target spatial point can be a far end point on the spatial straight line corresponding to the spatial straight line equation, and its specific position can vary depending on the algorithm design, and no specific restrictions are made here.

[0094] Therefore, in some embodiments, as shown in FIG4B , step 506 of selecting at least two sets of target parameters based on the plurality of camera parameters according to the at least two target distances may further include the following steps:

[0095] In step 5062, at least two target space points corresponding to the at least two target distances of each of the spatial straight line equations can be determined based on the at least two target distances and each of the spatial straight line equations and its starting point.

[0096] In this step, because each spatial line equation and its starting point are known, after determining the target distance, the target spatial point of the spatial line equation corresponding to the target distance can be calculated. As an optional embodiment, the target spatial point of the spatial line equation can be calculated based on the spatial line equation and its starting point, with the target distance as the length.

[0097] In this step, for each target distance, each spatial line equation can calculate a target space point.

[0098] In step 5064, at least two sets corresponding to the at least two target distances may be constructed, each set being used to store a set of target parameters.

[0099] In this step, the number of sets is consistent with the number of target distances. For example, when the at least two target distances include a first distance and a second distance, the at least two sets may include a first set corresponding to the first distance and a second set corresponding to the second distance, and these two sets may be used to store target spatial points obtained based on the first distance and the second distance, respectively.

[0100] In step 5066, the target space point corresponding to each target distance may be stored in a set corresponding to the target distance as the target parameter.

[0101] At this point, at least two sets of target parameters are obtained.

[0102] In step 508, the center position of the camera is determined based on the at least two sets of target parameters.

[0103] In this step, since the at least two sets of target parameters are selected according to the at least two target distances and based on the multiple camera parameters, and the target distance is related to the recommended distance (for example, the recommended exit pupil distance), the center position of the camera determined based on the at least two sets of target parameters can control the error within an acceptable range.

[0104] In some embodiments, determining the center position of the camera according to the at least two sets of target parameters includes: using a central camera model to fit the center position of the camera according to the at least two sets of target parameters.

[0105] Figure 4C shows an exemplary central camera model fitting result according to an embodiment of the present disclosure. As shown in Figure 4C , points O1 and O2 correspond to the camera centers of the left and right cameras, respectively. Because the camera centers are located within a certain range of the exit pupil distance, the central camera model fitting of the camera parameters effectively controls the calibration average error within the accuracy requirements of the IPD / ET algorithm.

[0106] After obtaining the fitted camera center, the target object can be located based on the camera center.

[0107] Then, in step 510, a target image captured by the camera may be obtained, the target image including the target object. Optionally, the target image may be an image captured by the camera 208 of the wearable device 200, the target image may include an eye, and the target object may be a cornea.

[0108] In some embodiments, the method 500 can be applied to a wearable device, as shown in Figures 2A to 2C, where the wearable device 200 includes a lens barrel 202 and a camera 208 disposed in the lens barrel 202, at least two point light sources 212 are disposed on the outside of the lens barrel 202, and the target image also includes at least two light spots formed by reflection of the at least two point light sources 212 on the cornea of ​​the target image, so that auxiliary positioning can be performed in combination with the position of the light spots in the image, which helps to improve the accuracy of the algorithm.

[0109] In step 512 , the position of the target object is determined based on the center position of the camera and the target image.

[0110] In this step, once the center position of the camera is known and the target image is acquired, the target object can be located based on the center camera model.

[0111] In some embodiments, as shown in FIG4D , step 512 of determining the position of the target object based on the center position of the camera and the target image may further include the following steps:

[0112] In step 5122, an optimization function is constructed based on the center position of the camera and the positions of the at least two light spots on the target image in combination with a center camera model.

[0113] FIG4E shows a schematic diagram of a camera model according to an embodiment of the present disclosure.

[0114] As shown in Figure 4E, in the IPD / ET algorithm based on the central camera model, the position of the center of the cornea in the three-dimensional space is defined as c, the position of the center of the pupil in the three-dimensional space is defined as p, the position of the camera center in the three-dimensional space obtained by fitting is o, the distance of oc is kc, and the projection of the center of the cornea c in the target image is u o , the corresponding vector is v o , the radius of the cornea is R. Among them, the vector v o It can be obtained based on the camera parameters (space line equation) corresponding to the pixel point corresponding to the center position of the target image.

[0115] Therefore, the relationship between the cornea center c and the camera center o is expressed as:

[0116] c=o+kc·v o

[0117] For each point light source l j (e.g., l1, l2), which forms a light spot q on the cornea j (e.g., q1, q2), spot q j The projection in the image is u j (e.g., u1, u2), o to q j The distance is k qj , the corresponding vector is v qj , where the vector v qj According to the light spot q j The projection (i.e. pixel point) u of the target image j The corresponding camera parameters (space line equation) are obtained.

[0118] Thus, a light spot is formed on the cornea. j The relationship expression with the camera center o is:

[0119] q j =o+k qj ·v qj

[0120] According to the geometric relationship, we can know that:

[0121] ||q j -c||=R

[0122] (ol j )×(u j -l j )·(cl j )=0

[0123] Then we can get the cornea center c and the light spot q formed on the cornea j The relational expression is:

[0124] Among them, c j Based on point light source l j The light spot q j The center position of the cornea is calculated by the relevant parameters, R is a constant, vl qj l j to q j vector, v lqj With vq j It is symmetric about the normal.

[0125] Next, construct the optimization function as follows:

[0126] f=min||c j -c mean ||

[0127] In step 5124, the optimal solution of the optimization function is obtained.

[0128] As shown in the previous optimization function, by finding a cornea center position C mean , so that all the obtained c j With the c mean The error is minimal.

[0129] As an optional embodiment, in combination with the above formulas, the least squares method can be used to solve the optimization function to obtain the optimal solution C mean .

[0130] In step 5126, the center position of the cornea is determined based on the optimal solution.

[0131] As mentioned above, the optimal solution C mean , which is the center position of the cornea.

[0132] The center position of the cornea is now located.

[0133] In some embodiments, the target image includes a first target image corresponding to a first eye (eg, a left eye) and a second target image corresponding to a second eye (eg, a right eye).

[0134] Determining the position of the target object according to the center position of the camera and the target image includes: determining the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye according to the center position of the camera and the target image.

[0135] The method 500 further includes determining the pupillary distance according to the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye.

[0136] In this way, after calculating the corneal center positions of the left and right eyes respectively, the distance between the two positions can be calculated to obtain the user's pupil distance.

[0137] In some embodiments, the wearable device 200 further includes a binocular display module (for example, two display screens 204, corresponding to the first eye and the second eye, respectively). After determining the pupil distance based on the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye, the method further includes: adjusting the binocular display module based on the pupil distance, thereby achieving adaptive adjustment of the wearable device 200 based on the pupil distance, without the need for user manual operation, thereby improving the user experience.

[0138] Optionally, adjusting the binocular display module may be adjusting the image displayed by the binocular display module so that the final imaging effect meets the requirement of the pupil distance.

[0139] In some embodiments, the method 500 further includes: obtaining the pupil position, for example, calculating the pupil position based on the relative relationship between the center position of the target image and the position of the pupil image in combination with the center position of the cornea.

[0140] Then, the sight direction is determined according to the pupil position and the center position of the cornea.

[0141] The relationship between the sight direction g, the pupil position p, and the center position c of the cornea is expressed as follows:

[0142] g=pc

[0143] In this way, by knowing the pupil position and the corneal center position, the user's line of sight direction can be obtained, thereby achieving user line of sight tracking.

[0144] In a more specific embodiment, the method 500 provided in the embodiment of the present disclosure may include the steps of camera parameter calibration of a non-camera center model, fitting a center camera model to obtain a camera center, target image acquisition, determining a target object, and an IPD / ET algorithm, which can achieve better algorithm accuracy.

[0145] It can be seen from the above embodiments that the embodiments of the present disclosure propose a central camera approximate pupil distance estimation and gaze tracking solution. This solution uses a central camera model to fit a non-central camera model, unifies the image starting point position, and subsequently only requires quadratic spline interpolation of the direction vector. While ensuring the accuracy of the pupil distance estimation algorithm and the gaze estimation algorithm, the amount of calculation is reduced by half, greatly improving the efficiency of the algorithm and shortening the calculation time.

[0146] It should be noted that the method of the embodiments of the present disclosure can be performed by a single device, such as a computer or server. The method of the embodiments of the present disclosure can also be applied in a distributed scenario, where multiple devices cooperate to perform the method. In such a distributed scenario, one of the multiple devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the multiple devices will interact with each other to complete the method.

[0147] It should be noted that the above description is limited to some embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the above embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0148] The present disclosure also provides a computer device for implementing the above-described method 500. FIG5 shows a schematic diagram of the hardware structure of an exemplary computer device 600 provided in an embodiment of the present disclosure. The computer device 600 can be used to implement the head-mounted wearable device 104 of FIG1A , the wearable device 200 of FIG2A to FIG2C , the external device 112 of FIG1A , and the server 114 of FIG1A . In some scenarios, the computer device 600 can also be used to implement the database server 116 of FIG1A .

[0149] 5 , a computer device 600 may include a processor 602, a memory 604, a network module 606, a peripheral interface 608, and a bus 610. The processor 602, the memory 604, the network module 606, and the peripheral interface 608 are connected to each other in communication within the computer device 600 via the bus 610.

[0150] The processor 602 may be a central processing unit (CPU), an image processor, a neural network processor (NPU), a microcontroller (MCU), a programmable logic device (PLD), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or one or more integrated circuits. The processor 602 may be used to perform functions related to the technology described in this disclosure. In some embodiments, the processor 602 may also include multiple processors integrated into a single logical component. For example, as shown in FIG5 , the processor 602 may include multiple processors 602a, 602b, and 602c.

[0151] The memory 604 can be configured to store data (e.g., instructions, computer codes, etc.). As shown in Figure 5, the data stored in the memory 604 can include program instructions (e.g., program instructions for implementing the method 500 of the embodiment of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 602 can also access the program instructions and data stored in the memory 604, and execute the program instructions to operate on the data to be processed. The memory 604 can include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 604 can include a random access memory (RAM), a read-only memory (ROM), an optical disc, a magnetic disk, a hard disk, a solid-state drive (SSD), a flash memory, a memory stick, etc.

[0152] The network interface 606 can be configured to provide the computer device 600 with communication with other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC)), a cellular network, the Internet, or a combination thereof. It will be understood that the type of network is not limited to the specific examples above.

[0153] The peripheral interface 608 can be configured to connect the computer device 600 to one or more peripheral devices to enable information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touchpad, a touch screen, a microphone, and various sensors, and output devices such as a display, a speaker, a vibrator, and an indicator light.

[0154] The bus 610 may be configured to transmit information between various components of the computer device 600 (e.g., the processor 602, the memory 604, the network interface 606, and the peripheral interface 608), such as an internal bus (e.g., a processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0155] It should be noted that although the architecture of the computer device 600 shown above only includes the processor 602, memory 604, network interface 606, peripheral interface 608, and bus 610, in a specific implementation, the architecture of the computer device 600 may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the architecture of the computer device 600 may only include the components necessary to implement the embodiments of the present disclosure, and does not necessarily include all the components shown in the figure.

[0156] The present disclosure also provides a camera parameter calibration device. FIG6 shows a schematic diagram of an exemplary device 700 provided by the present disclosure. As shown in FIG6 , the device 700 can be used to implement the method 500 and can further include the following modules.

[0157] The first acquisition module 702 is configured to acquire a plurality of camera parameters of a camera, where the plurality of camera parameters include a plurality of spatial straight line equations with different starting points.

[0158] The first determination module 704 is configured to: determine at least two target distances; select at least two groups of target parameters based on the multiple camera parameters according to the at least two target distances; and determine the center position of the camera according to the at least two groups of target parameters.

[0159] The second acquisition module 706 is configured to acquire a target image captured by the camera, where the target image includes the target object.

[0160] The second determining module 708 is configured to determine the position of the target object according to the center position of the camera and the target image.

[0161] In some embodiments, the camera parameters include multiple spatial straight line equations corresponding to multiple pixel points of the image captured by the camera and the starting points corresponding to the spatial straight line equations; the first determination module 704 is configured to: determine at least two target space points corresponding to the at least two target distances of each spatial straight line equation according to the at least two target distances and each of the spatial straight line equations and its starting point; the target parameters correspond to the target space points according to the target distances.

[0162] In some embodiments, the first determining module 704 is configured to: use a central camera model to fit the center position of the camera according to the at least two sets of target parameters.

[0163] In some embodiments, the target object includes a cornea, and the at least two target distances include at least two target exit pupil distances.

[0164] In some embodiments, the method is applied to a wearable device, which includes a lens barrel and the camera arranged in the lens barrel, at least two point light sources are arranged on the outside of the lens barrel, and the target image also includes at least two light spots formed by reflection of the at least two point light sources on the cornea of ​​the target image.

[0165] In some embodiments, the second determination module 708 is configured to: construct an optimization function based on the center position of the camera and the positions of the at least two light spots on the target image in combination with a central camera model; and determine the center position of the cornea based on the optimization function.

[0166] In some embodiments, the target image includes a first target image corresponding to the first eye and a second target image corresponding to the second eye; the second determination module 708 is configured to: determine the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye based on the center position of the camera and the target image; the second determination module 708 is further configured to: determine the pupil distance based on the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye.

[0167] In some embodiments, the wearable device further includes a binocular display module. After determining the pupil distance based on the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye, the device further includes an adjustment module configured to adjust the binocular display module based on the pupil distance.

[0168] In some embodiments, the second determination module 708 is configured to: obtain a pupil position; and determine a sight direction according to the pupil position and the center position of the cornea.

[0169] In some embodiments, the first determining module 704 is configured to determine the at least two target exit pupil distances according to the recommended exit pupil distance of the wearable device.

[0170] For the convenience of description, the above devices are described as being functionally divided into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0171] The apparatus of the above embodiment is used to implement the corresponding method 500 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0172] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present disclosure also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute method 500 as described in any of the above embodiments.

[0173] The computer-readable media of this embodiment include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0174] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method 500 described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0175] Based on the same inventive concept, corresponding to any of the above-described embodiments of method 500, the present disclosure further provides a computer program product comprising computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processor to perform method 500. For each step in each embodiment of method 500, the processor executing the step can be a member of the corresponding execution entity.

[0176] The computer program product of the above embodiment is used to enable the computer and / or the processor to execute the method 500 described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0177] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Within the scope of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of simplicity.

[0178] In addition, to simplify the description and discussion, and so as not to obscure the embodiments of the present disclosure, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present disclosure, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the purview of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0179] Although the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0180] The embodiments of the present disclosure are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for determining a location of a target object, comprising: Acquire a plurality of camera parameters of a camera, wherein the plurality of camera parameters include a plurality of spatial straight line equations with different starting points; determining at least two target distances; According to the at least two target distances, selecting at least two groups of target parameters based on the plurality of camera parameters; Determining a center position of the camera according to the at least two sets of target parameters; Acquire a target image captured by the camera, wherein the target image includes the target object; The position of the target object is determined according to the center position of the camera and the target image.

2. The method of claim 1, wherein: The camera parameters include a plurality of spatial straight line equations corresponding to a plurality of pixel points of the image captured by the camera and a starting point corresponding to the spatial straight line equation; According to the at least two target distances, selecting at least two groups of target parameters based on the multiple camera parameters includes: Determine at least two target space points of each of the space line equations corresponding to the at least two target distances, respectively, according to the at least two target distances and each of the space line equations and its starting point; The target parameters correspond to the target space points according to the target distance.

3. The method of claim 2, wherein: Determining the center position of the camera according to the at least two sets of target parameters includes: According to the at least two groups of target parameters, a central camera model is adopted to obtain the central position of the camera through fitting.

4. The method of claim 1, wherein: The target object includes a cornea, and the at least two target distances include at least two target exit pupil distances.

5. The method of claim 4, wherein: The method is applied to a wearable device, which includes a lens barrel and the camera arranged in the lens barrel, at least two point light sources are arranged on the outside of the lens barrel, and the target image also includes at least two light spots formed by reflection of the at least two point light sources on the cornea of ​​the target image.

6. The method of claim 5, wherein: Determining the position of the target object according to the center position of the camera and the target image includes: According to the center position of the camera and the positions of the at least two light spots on the target image, combined with a center camera model, an optimization function is constructed; Based on the optimization function, the center position of the cornea is determined.

7. The method of claim 6, wherein: The target image includes a first target image corresponding to the first eye and a second target image corresponding to the second eye; Determining the position of the target object according to the center position of the camera and the target image, including: determining the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye according to the center position of the camera and the target image; The method further includes determining the pupil distance according to the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye.

8. The method of claim 7, wherein: The wearable device further includes a binocular display module. After determining the pupil distance according to the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye, the method further includes: The binocular display module is adjusted according to the pupil distance.

9. The method of claim 6, wherein: The method further comprises: Get the pupil position; The sight direction is determined according to the pupil position and the center position of the cornea.

10. The method of claim 5, wherein: Determine distances to at least two targets, including: The at least two target exit pupil distances are determined according to the recommended exit pupil distance of the wearable device.

11. A device for determining a position of a target object, comprising: A first acquisition module is configured to: acquire a plurality of camera parameters of a camera, wherein the plurality of camera parameters include a plurality of spatial straight line equations with different starting points; A first determination module is configured to: determine at least two target distances; According to the at least two target distances, selecting at least two groups of target parameters based on the multiple camera parameters; determining the center position of the camera according to the at least two groups of target parameters; A second acquisition module is configured to: acquire a target image captured by the camera, wherein the target image includes the target object; The second determination module is configured to determine the position of the target object according to the center position of the camera and the target image.

12. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to any one of claims 1 to 10.

13. The computer device of claim 12, wherein: The computer device includes a wearable device, which includes a display module, a lens barrel arranged on the light-emitting side of the display module, and a camera arranged in the lens barrel, and at least two point light sources are arranged on the outer side of the lens barrel.

14. A non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to any one of claims 1 to 10.

15. A computer program product, comprising computer program instructions, which, when executed on a computer, enable the computer to execute the method according to any one of claims 1 to 10.

Images