Method for determining the position of a target object and related device

Abstract

The present disclosure provides a method for determining the position of a target object and related equipment. The method comprises: obtaining a plurality of camera parameters of a camera, the plurality of camera parameters comprising a plurality of spatial straight line equations with different starting points; determining at least two target distances; selecting at least two groups of target parameters based on the plurality of camera parameters according to the at least two target distances; determining the center position of the camera according to the at least two groups of target parameters; obtaining a target image collected by the camera, the target image comprising the target object; and determining the position of the target object according to the center position of the camera and the target image.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and in particular, to a method for determining a position of a target object and related equipment. BACKGROUND

[0002] A central camera model generally refers to a camera model in which an imaging center converges to a point.

[0003] However, the present inventors have found that in some scenarios, when a camera is placed in a lens barrel, it is difficult to simulate the imaging process by using a central camera model, and thus it is difficult to locate an object in a physical world based on an acquired image. SUMMARY

[0004] The present disclosure provides a method for determining a position of a target object and related equipment to solve or partially solve the above problems.

[0005] In a first aspect, the present disclosure provides a method for determining a position of a target object, comprising:

[0006] obtaining a plurality of camera parameters of a camera, the plurality of camera parameters comprising a plurality of spatial straight line equations with different starting points;

[0007] determining at least two target distances;

[0008] selecting at least two groups of target parameters based on the plurality of camera parameters according to the at least two target distances;

[0009] determining a central position of the camera according to the at least two groups of target parameters;

[0010] obtaining a target image acquired by the camera, the target image comprising the target object;

[0011] determining the position of the target object according to the central position of the camera and the target image.

[0012] In a second aspect, the present disclosure provides an apparatus for determining a position of a target object, comprising:

[0013] a first obtaining module configured to obtain a plurality of camera parameters of a camera, the plurality of camera parameters comprising a plurality of spatial straight line equations with different starting points;

[0014] a first determining module configured to determine at least two target distances, select at least two groups of target parameters based on the plurality of camera parameters according to the at least two target distances, and determine a central position of the camera according to the at least two groups of target parameters;

[0015] A second obtaining module, configured to obtain a target image collected by the camera, the target image comprising the target object.

[0016] A second determining module, configured to determine the position of the target object according to the center position of the camera and the target image.

[0017] In a third aspect, the present disclosure provides a computer device, comprising one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs comprise instructions for performing the method according to the first aspect.

[0018] In a fourth aspect, the present disclosure provides a non-transitory computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method according to the first aspect.

[0019] In a fifth aspect, the present disclosure provides a computer program product, comprising computer program instructions, which, when executed on a computer, cause the computer to perform the method according to the first aspect.

[0020] The method for determining the position of the target object and the related device provided by the embodiments of the present disclosure can obtain the center position of the camera by fitting the camera parameters with the center camera model according to at least two target distances, so that the quadratic spline interpolation of the starting point position can be omitted, thereby improving the calculation speed; and the target object can be positioned based on the center position of the camera, so that the algorithm accuracy can be ensured. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present disclosure or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0022] Figure 1A A schematic diagram of an exemplary system provided by the embodiments of the present disclosure is shown.

[0023] Figure 1B A schematic diagram of an exemplary head-mounted wearable device is shown.

[0024] Figure 1C And Figure 1D A schematic diagram of an exemplary human eye image is shown.

[0025] Figure 2AA schematic diagram of an example wearable device provided by an embodiment of the present disclosure is shown.

[0026] Figure 2B A schematic diagram of another example wearable device provided by an embodiment of the present disclosure is shown.

[0027] Figure 2C A schematic diagram of yet another example wearable device provided by an embodiment of the present disclosure is shown.

[0028] Figure 3 A schematic diagram of example camera parameters according to an embodiment of the present disclosure is shown.

[0029] Figure 4A A flowchart of an example method provided by an embodiment of the present disclosure is shown.

[0030] Figure 4B A flowchart of an example method of determining a target parameter according to an embodiment of the present disclosure is shown.

[0031] Figure 4C A schematic diagram of an example center camera model fitting result according to an embodiment of the present disclosure is shown.

[0032] Figure 4D A flowchart of an example method of determining a corneal center position according to an embodiment of the present disclosure is shown.

[0033] Figure 4E A schematic diagram of a camera model according to an embodiment of the present disclosure is shown.

[0034] Figure 5 A hardware structure schematic diagram of an example computer device provided by an embodiment of the present disclosure is shown.

[0035] Figure 6 A schematic diagram of an example apparatus provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0036] In order to make the purposes, technical solutions and advantages of the present disclosure clearer, the present disclosure is further described in detail below with reference to specific embodiments and the accompanying drawings.

[0037] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the embodiments of the present disclosure shall have the common meaning understood by one of ordinary skill in the art to which the embodiments of the present disclosure belong. The terms "first", "second", and similar terms used in the embodiments of the present disclosure do not denote any order, quantity, or importance, but are used to distinguish different components. The terms "include", "contain", and similar terms mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" or similar terms do not mean only physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are used only to represent relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0038] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type, use range, use scenario, and the like of personal information involved in the present disclosure should be informed to the user and the authorization of the user should be obtained in a proper manner according to relevant laws and regulations.

[0039] For example, in response to receiving an active request of a user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using personal information of the user. Thus, the user can voluntarily choose whether to provide personal information to the software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operation of the technical solutions of the present disclosure according to the prompt information.

[0040] As an optional but non-limiting implementation manner, in response to receiving an active request of a user, the manner of sending prompt information to the user may, for example, be a pop-up window manner, in which the prompt information can be presented in the form of text. In addition, the pop-up window can also carry selection controls for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0041] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation manners of the present disclosure, and other manners meeting relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0042] It can be understood that the data involved in the technical solutions (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws and regulations and relevant provisions.

[0043] Figure 1A A schematic diagram of an example extended reality system 100 is shown.

[0044] Extended Reality (XR) refers to combining reality and virtuality through a computer to create a virtual environment that can be interacted with by a human. XR technology can further include Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR), which utilizes hardware devices in combination with various technical means to fuse virtual content and real scenes.

[0045] As shown in Figure 1A , the system 100 can include various types of wearable devices, such as a head-mounted wearable device (e.g., VR / AR glasses or a head-mounted display (HMD)) 104, an operation handle 108, and the like. In some scenarios, a camera / camera 110 for taking a photo of the operator (user) 102 can also be provided. In some embodiments, when the aforementioned devices do not have processing functions, the system 100 can further include an external control device 112 for providing processing functions. The control device 112, for example, can be a mobile phone, a computer, or the like. In some embodiments, when any of the aforementioned devices functions as a control device or a master control device, it can interact with other devices in the system 100 through wired or wireless communication.

[0046] In the system 100, the user 102 can use the head-mounted wearable device 104 and the operation handle 108 to interact with the extended reality system 100. In some scenarios, the system 100 can use the pictures captured by the camera / camera 110 to recognize the gestures and postures of the user 102, and then complete the interaction with the user 102 based on the recognized gestures and postures. In some embodiments, the user 130 can also input gestures through bare hands, and the head-mounted wearable device 104 can capture real-time images in front of the head-mounted wearable device 104 through a camera or a camera disposed in front of the head-mounted wearable device 104, and recognize the gestures of the user 130 by recognizing the images.

[0047] In some embodiments, as shown in Figure 1A , the system 100 can also communicate with a server 114 and obtain data such as pictures, audio, video, and the like from the server 114, and can output these data through the head-mounted wearable device 104, such as displaying pictures or videos on the display screen of the head-mounted wearable device 104, playing audio and audio carried by video through the speaker of the head-mounted wearable device 104, and the like. In some embodiments, as shown in Figure 1A , the server 114 can retrieve the required data such as pictures, audio, video, and the like from a database server 116 for storing data.

[0048] In some embodiments, the head-mounted wearable device 104 can be provided with a collection unit for collecting information. The type of the collection unit can be various.

[0049] In some embodiments, the collection unit can further include an environment acquisition unit for collecting environment information around (e.g., in front of) the wearable device 104, and a positioning tracking unit for positioning tracking of the wearable device 104. Optionally, the environment acquisition unit can include, but is not limited to, a three-color camera (e.g., an RGB camera), a depth camera, a binocular camera, a laser, etc. The positioning tracking unit can include, but is not limited to, a visual simultaneous localization and mapping (visual SLAM), an inertial measurement unit (IMU), a global positioning system (GPS), an ultra-wideband wireless communication technology (UWB), a laser, etc.

[0050] In some embodiments, the head-mounted wearable device 104 can also be provided with a speed sensor, an acceleration sensor, an angular velocity sensor (e.g., a gyroscope), etc. for collecting speed information or acceleration information of the head-mounted wearable device 104. For example, the operation handle 108 can also be provided with a speed sensor, an acceleration sensor, an angular velocity sensor (e.g., a gyroscope), etc. for collecting speed information or acceleration information of the wearable glove 106. It should be noted that the aforementioned collection units can be provided on the head-mounted wearable device 104 and the operation handle 108, or can be directly attached to the body parts of the interactive user 102 without relying on hardware devices, so as to collect relevant information of the body parts, such as speed or acceleration or angular velocity information, or other information collected by sensors or collection units.

[0051] In some embodiments, the head-mounted wearable device 104 can also be provided with a camera or a video camera for taking photos of the operator (user) 102 (e.g., photos of hands or feet) and environment images.

[0052] In some embodiments, the system 100 can identify the posture, gestures, etc. of the user 102 through the collected information, and then can perform corresponding interactions according to the identified user posture and gestures.

[0053] Figure 1B A schematic diagram of an exemplary head-mounted wearable device 104 is shown.

[0054] As Figure 1BAs shown, the head-mounted wearable device 104 can include a lens barrel 1042, inside which a display screen 1044 for displaying images and an optical assembly 1046 for processing light paths can be arranged. Optionally, the optical assembly 1046 can further include a plurality of lenses (e.g., lenses 1046A and 1046B), which can project light emitted by the display screen 1044 into the human eye 1022 so that the human eye 1022 can view the picture displayed by the display screen 1044. It can be understood that, Figure 1B Only a single-side structure of the head-mounted wearable device 104 is exemplarily shown in the figures, and to realize binocular display, two lens barrel structures arranged side by side can be included in the head-mounted wearable device 104.

[0055] In some embodiments, as Figure 1B shown, the head-mounted wearable device 104 can further be provided with a camera 1048 for capturing human eye images, which can be a charge-coupled device (CCD) image sensor, a complementary metal-oxide-semiconductor (CMOS) image sensor, or the like.

[0056] Optionally, the camera 1048 can be an eye tracking (ET) camera, and the human eye images captured thereby can be used to realize functions such as interpupillary distance estimation and eye tracking.

[0057] As Figure 1B shown, in the related art, the camera 1048 is usually arranged outside the lens barrel and is usually arranged in correspondence with only one lens barrel. Moreover, to better capture complete human eye images without affecting the human eye's viewing of the picture displayed by the display screen 1044, the common deployment position of the camera 1048 is usually at the outer canthus, the nasal ala, or directly below the eye. Referring to Figure 1B shown, if the camera 1048 is close to the outer side of the device, the camera deployment position is the outer canthus. Figure 1B shown, if the camera 1048 is close to the inner side of the device, the camera deployment position is the nasal ala. Figure 1B

[0058] However, the inventors of the present disclosure have found that the way of installing the camera in the related art tends to result in a large installation inclination angle of the camera 1048 relative to the human eye 1022, leading to a large included angle a between the orientation of the camera 1048 and the direct viewing direction of the human eye 1022 (i.e., a high degree of inclination of the camera 1048), so that the captured human eye images are difficult to reflect the images of the direct viewing angle of the human eye, as Figure 1C and Figure 1D shown. Moreover, within the range of eye movement, a single camera 1048 is difficult to capture the complete eye movement state.

[0059] ​In some cases, the user can need to wear glasses first and then use the head-mounted wearable device 104. However, since the camera 1048 is disposed outside the barrel 1042 (in particular, the camera 1048 is disposed at the nasal bridge position), the camera 1048 is higher than the barrel 1042, which is easy to collide or press the glasses, affects the wearing comfort of the head-mounted wearable device 104, and can scratch the user's glasses. At the same time, the imaging of the camera 1048 is easily affected by the edge of the glasses, and the light passing through the edge of the glasses is refracted, which reduces the image clarity of the camera 1048 and forms many refracted spots in the image, so that the clarity of the image collected by the camera is reduced and a large distortion is generated, which affects the accuracy of the subsequent pupil distance estimation algorithm and gaze estimation algorithm. Or, when the glasses have a frame, the camera is easily blocked by the edge of the glasses and cannot collect a complete eye image, resulting in the failure of the pupil distance estimation function and the gaze estimation function. In particular, when the camera corresponding to the barrel is only one, such problems will be further aggravated.

[0060] Therefore, the embodiments of the present disclosure provide a wearable device, which sets the camera inside the barrel and can solve or partially solve the above problems to some extent.

[0061] Figure 2A A schematic diagram of an exemplary wearable device 200 provided by the embodiments of the present disclosure is shown.

[0062] As shown in Figure 2A Similarly, the wearable device 200 can also include a barrel 202, and a display screen 204 and an optical assembly 206 disposed inside the barrel 202. The optical assembly 206 can further include a plurality of lenses (for example, lenses 206A and 206B), and the combination of the plurality of lenses can project the light emitted by the display screen 204 into the human eye 1022, so that the human eye 1022 can watch the picture displayed by the display screen 204.

[0063] Unlike the wearable device 104 shown in Figure 1B The camera 208 of the wearable device 200 is disposed inside the barrel 202 and faces the light-emitting side of the barrel, and since the camera 208 is placed inside the barrel 202, it will not affect the wearing of glasses, thereby improving the comfort of the wearable device 200. At the same time, as Figure 2AAs shown, because the camera 208 is located inside the lens barrel 202, the distance between the camera 208 and the human eye 1022 is increased, resulting in a smaller mounting angle of the camera 208 relative to the human eye 1022. Consequently, the angle β between the orientation of the camera 208 and the frontal viewing direction of the human eye 1022 is reduced, giving the camera 208 a better viewing angle. The acquired image of the human eye better reflects the frontal viewing angle, resulting in better image quality. Furthermore, since the camera 208 is placed inside the lens barrel 202, the glasses do not interfere with the imaging of the camera 208, further improving image quality. This improved image quality also enhances the accuracy of algorithms such as interpupillary distance estimation or gaze tracking.

[0064] Figure 2B A schematic diagram of another exemplary wearable device 200 provided in this disclosure embodiment is shown.

[0065] like Figure 2B As shown, in some embodiments, with Figure 2A The difference is that the camera 208 faces the display screen 204, and the wearable device 200 may also include a reflective structure 210, which can be a reflective film or a mirror, or other structures with reflective surfaces. Through the reflection of light by the reflective structure 210, the camera 208 can still capture images from the human eye. Furthermore, because an additional reflection process is added to the optical path, the observation angle γ is further reduced, allowing the camera 208 to achieve better imaging.

[0066] Figure 2C A schematic diagram of yet another exemplary wearable device 200 provided in this disclosure embodiment is shown.

[0067] like Figure 2C As shown, in some embodiments, with Figure 2B The difference is that the reflective structure 210 can be a semi-transparent, semi-reflective film. This reflective structure 210 can transmit half of the light and reflect the other half. In this way, the reflective structure 210 can reflect light from the eye outside the lens barrel into the camera 208, and can transmit light emitted from the display screen 204. (Comparison) Figure 2B It can be seen that the adoption Figure 2C The reflective structure 210 can avoid the problem of light blocking caused by the reflective structure being opaque. Furthermore, the position of the reflective structure 210 does not need to be specially designed according to the position of the camera 208; it only needs to be placed on the light-emitting side of the display screen 204.

[0068] In some embodiments, such as Figures 2A to 2CAs shown, the wearable device 200 can further include a point light source 212 disposed outside the lens 202, and the light emitted by the point light source 212 can form reflected light on the cornea of the human eye 1022 to be captured by the camera 208, and then form a light spot in the image formed by the camera 208. According to the position of the light spot in the image, the position of the center of the cornea and the pupil can be determined. In some embodiments, in order to make the positioning more accurate, the number of point light sources 212 can be two or more. Optionally, the point light source 212 can be a light-emitting diode (LED). In some embodiments, the light emitted by the point light source 212 can not be visible light (for example, it can be infrared light) or visible light with relatively weak brightness, so as to improve the comfort of the user.

[0069] It can be understood that, Figures 2A to 2C Only the single-side structure of the wearable device 200 is exemplarily shown in the figure, and in order to realize binocular display, two lens barrel structures can be arranged side by side in the wearable device 200.

[0070] As can be seen from the above embodiments, by arranging the camera 208 inside the wearable device 200, not only the above-mentioned adverse effects can be avoided, but also the distance between the camera and the eye can be increased, so as to have a better camera observation angle and obtain better imaging quality (i.e., the camera can shoot higher-quality eye images), which is beneficial to improve the accuracy of the interpupillary distance estimation algorithm and the line-of-sight estimation algorithm, and at the same time, the wearing comfort of the wearable device 200 can be improved.

[0071] Further, the inventors of the present disclosure find that, compared with placing the camera outside the lens barrel, when the camera is placed inside the lens barrel, the imaging distortion is not only affected by the camera module itself, but also affected by the optical components in the lens barrel.

[0072] As Figures 2A to 2C shown, compared with the camera 1048 of Figure 1B , in the light path of the camera 208 to the human eye 1022, there is also the optical component 206 or part of the optical component 206 (different according to the relative position relationship between the camera 208 and each lens in the optical component 206, for example, in addition to Figures 2A to 2C ​As shown in the position, the camera 208 can also be arranged between the lens 206A and the lens 206B, so that when the camera 208 images the human eye 1022, the optical assembly 206 or part of the optical assembly 206 affects or changes the light path of the camera 208 to the human eye 1022, resulting in inconsistent refractive index at various positions in the lens barrel, so that the optical axis of the camera 208 cannot be completely coincident with the optical axis of the optical assembly 206 in the lens barrel, resulting in that the distortion of the image imaged by the camera 208 can no longer be symmetrical (that is, the distortion of the image taken by the camera at various positions is different).

[0073] And because the light path of the camera 208 to the human eye 1022 is changed, the camera 208 does not have a uniform projection center, resulting in that the distortion cannot be processed using the central camera model to fit the projection process of the camera in the lens barrel, so that the camera parameters are difficult to calibrate.

[0074] Therefore, some embodiments of the present disclosure can first calibrate the parameters of the camera in the lens barrel. The camera parameters obtained by calibration can be used to represent the pixel points in the image taken by the camera and the projection directions corresponding to the pixel points. The projection direction (which can also be referred to as the projection direction) can be a spatial straight line corresponding to a pixel point in the image coordinate system (two-dimensional) in the camera coordinate system (three-dimensional), so that the corresponding relationship between the pixel points in the image and a certain position in the three-dimensional space can be established according to the projection direction.

[0075] When calibrating the camera parameters, image acquisition can be performed.

[0076] For example, a series of images are generated as input by continuously taking the calibration board by the motion camera to obtain observation data in multiple postures.

[0077] Then, the corresponding relationship between the pixel points in the image and a certain position in the three-dimensional space is established according to the images taken.

[0078] Specifically, three images can be selected from the images collected in the front, a homography transformation matrix between the pixel points of each image and the calibration board in the space is constructed for each image, and then a one-to-one correspondence between the pixel points and the coordinates corresponding to the pixel points in the calibration board coordinate system is established for each pixel point.

[0079] After the corresponding relationship between the pixel points and the space coordinates is established, the three images are all converted to a reference coordinate system, and the extrinsic parameters between the three images are calculated by using the property that the space points corresponding to the same pixel position on the three images are located on the same straight line.

[0080] In this way, the corresponding relationship between at least part of the pixel points and a certain position in the three-dimensional space can be obtained.

[0081] Then, the correspondence between the pixel points and the positions in the three-dimensional space is continuously established according to the above method by using the remaining images in the images obtained by shooting until all the pixel points of the images correspond to a space straight line, that is, the camera parameter calibration is completed.

[0082] Since each pixel point corresponds to a space straight line, the camera parameters same as the number of pixel points need to be stored when the camera parameters are stored, and therefore the camera parameters increase with the increase of the image size. In order to reduce the number of camera parameters, in some embodiments, the direction can be fitted by using a smooth spline surface, and then the spline control points are optimized by using the observation data of the camera. After the optimization is completed, the control points of the fitted spline surface are obtained as the camera parameters, thereby reducing the storage number of the camera parameters. Subsequently, when the camera parameters are needed to be used, the space straight line equations of all the pixel points are restored by performing secondary spline interpolation.

[0083] Figure 3 A schematic diagram of an example camera parameter according to an embodiment of the present disclosure is shown.

[0084] As shown in Figure 3 , the camera parameters calibrated are the space straight line equations corresponding to some pixel points (that is, each pixel point of the camera imaging corresponds to a direction vector and a starting point position in the three-dimensional space). In some embodiments, in order to save the calculation amount, only some key pixel points (control points) are sampled and their space straight line equations and starting points are saved as the camera parameters, and the camera parameters corresponding to other pixel points can be obtained by using the secondary spline interpolation. Figure 3 As shown in , the space straight lines corresponding to the respective pixel points do not strictly converge at a point (for example, points A and B are the intersection points obtained by converging different space straight lines, respectively), but are distributed in a range interval, which indicates that the camera is non-central, that is, there is no unique projection center.

[0085] Therefore, in order to ensure the accuracy of the algorithm, when the non-parametric camera calibration model is used to calculate the pupil position and the line of sight direction, for the pixel points in the image collected by the camera, since the camera parameters only retain the space straight line equations corresponding to the control points, the starting point positions and the direction vectors in the camera parameters need to be respectively subjected to secondary spline interpolation to restore the space straight line equations corresponding to all the pixel points, which is large in calculation amount and relatively complex in calculation.

[0086] In order to simplify the algorithm to save the calculation amount, in some embodiments, the central camera approximation can be performed under the full camera field of view, a camera center is obtained by forcibly fitting the non-central camera model by using the central camera model, thereby the secondary spline interpolation of the starting point positions can be omitted, and thereby the calculation amount can be saved.

[0087] However, the inventors of the present disclosure find that such processing can bring a large error, which is unacceptable for subsequent inter-pupillary distance (IPD) estimation algorithm and eye tracking (ET) algorithm.

[0088] In view of this, the embodiments of the present disclosure provide a method for determining the position of a target object. The center position of the camera is obtained by fitting the camera parameters with a center camera model within a certain distance range (at least two target distances) of the recommended distance, which can save the quadratic spline interpolation of the starting point position, thereby improving the calculation speed. At the same time, because the at least two target distances are close to the recommended distance, the algorithm accuracy can be ensured.

[0089] Figure 4A A flowchart of an exemplary method 500 provided by the embodiments of the present disclosure is shown. The method 500 can be applied to Figure 1A a head-mounted wearable device 104 of Figure 2A and Figure 2B a wearable device 200 of Figure 1A an external device 112 of The method 500 can be used to determine the position of a target object in a three-dimensional space (world coordinate system), as shown in Figure 4A The method 500 can further include the following steps.

[0090] In step 502, a plurality of camera parameters of a camera are obtained. Optionally, the plurality of camera parameters include a plurality of spatial straight line equations with different starting points. For example, the plurality of camera parameters can be camera parameters obtained by calibrating with a non-center camera model using the method provided in the foregoing embodiments, as shown in Figure 4E Each camera parameter can be used to indicate a spatial straight line equation corresponding to a single pixel point, which passes through the pixel point and has a starting point (which can be obtained by fitting all camera parameters).

[0091] As described previously, since the camera parameter is a spatial straight line equation with only a starting point, when the specific position of a target object in a three-dimensional space (world coordinate system) is not determined, if the position of the target object is to be determined according to the image collected by the camera and in combination with the camera parameter, in order to ensure the accuracy of the algorithm, a non-parametric camera calibration model can be used to calculate the position of the target object. For the pixel points in the image collected by the camera, the starting point position and the direction vector in the camera parameter thereof need to be respectively subjected to quadratic spline interpolation, which is large in calculation amount and relatively complex in calculation.

[0092] In order to simplify the algorithm to save the calculation amount, in some embodiments, the center camera approximation can be performed under the full camera field of view, a camera center can be obtained by forcibly fitting the non-center camera model with the center camera model, so that the quadratic spline interpolation on the starting point position can be omitted, thereby saving the calculation amount. However, such processing can bring a larger error.

[0093] Therefore, the inventors of the present disclosure find that if the approximate position of the target object can be determined in advance, and then the camera parameters within a certain range of the approximate position are used to fit the camera center, both the calculation amount and the algorithm accuracy can be saved.

[0094] Therefore, in step 504, at least two target distances can be determined.

[0095] The at least two target distances can be determined according to a recommended distance, which can be a known or estimated optimal imaging distance of the target object distance from the light-emitting surface vertex of the optical assembly 206. The at least two target distances can be two distance values within a certain range of the recommended distance. For example, assuming that the recommended distance is 50 mm and the imaging quality is good within a range of 10 mm from the recommended distance, at least two target distances can be selected within the range of 50±10 mm.

[0096] As an optional embodiment, in order to make the camera center obtained by subsequent fitting more robust, the at least two target distances at least include a first distance (for example, 60 mm) farthest in the distance range and a second distance (for example, 40 mm) closest in the distance range. It can be understood that in order to achieve better fitting effect to ensure accuracy, at least one distance value (for example, 50 mm) within the interval of 40 mm to 60 mm can also be selected to perform camera center fitting together with the farthest distance and the closest distance.

[0097] In some embodiments, the target object includes the cornea of the eye, and the method 500 can be used to determine the center position of the cornea of the eye, thereby determining the position of the human eye to implement some algorithms related to the position of the human eye. Alternatively, after determining the center positions of the corneas of the two eyes, the method 500 can determine the interpupillary distance based thereon, thereby implementing the interpupillary distance measurement.

[0098] In some embodiments, the at least two target distances can include at least two target pupillary distances. The pupillary distance refers to the distance between the eye and the last piece of lens of the eyepiece when the entire field of view can be seen, and in optics, it refers to the distance from the last vertex of the optical system to the intersection point of the exit pupil plane and the optical axis. In this way, the target distance can be selected based on the pupillary distance, and the interpupillary distance measurement can be better implemented.

[0099] In some embodiments, the step 504 of determining the at least two target distances can further comprise: determining the at least two target pupil distances according to a recommended pupil distance of the wearable device 200.

[0100] The wearable device 200 can be a head-mounted display (HMD), and the recommended pupil distance can be an optimal pupil distance of the head-mounted display, which can be different according to different models of the head-mounted display, and accordingly, the selected target pupil distance can also be different. For example, in the case where the optimal pupil distance of the head-mounted display is 20 mm, at least two target pupil distances (e.g., a first distance of 25 mm and a second distance of 15 mm) between 15 mm and 25 mm can be adopted, and the center camera model fitting can be performed based on the at least two target pupil distances to obtain the center position of the camera.

[0101] In this way, by performing the center camera model fitting on the camera parameters within a certain range of pupil distances, the average error can be effectively controlled within the accuracy requirement of the IPD / ET algorithm.

[0102] At step 506, at least two groups of target parameters are selected based on the plurality of camera parameters according to the at least two target distances.

[0103] As mentioned above, the camera parameters include a plurality of spatial line equations corresponding to a plurality of pixel points of the image captured by the camera and a starting point corresponding to the spatial line equations, but the spatial line equations have no ending point, and if the center camera approximation is performed under the full camera field of view, a large error can be caused. Therefore, in some embodiments, a target spatial point (target parameter) on the spatial line corresponding to each spatial line equation can be determined according to the selected at least two target distances, and then the center camera approximation can be performed based on the target spatial points, so that the algorithm error can be controlled within an acceptable range. It can be understood that the target spatial point can be a distal point on the spatial line corresponding to the spatial line equation, and the specific position thereof can vary according to different algorithm designs, which is not specifically limited herein.

[0104] Therefore, in some embodiments, as shown in Figure 4B The step 506 of selecting at least two groups of target parameters based on the plurality of camera parameters according to the at least two target distances can further comprise the following steps:

[0105] At step 5062, at least two target spatial points corresponding to the at least two target distances, respectively, of each spatial line equation can be determined according to the at least two target distances and the starting point of each spatial line equation.

[0106] In this step, because each of the spatial straight line equations and its starting point is known, after the target distance is determined, the target space point of the spatial straight line equation corresponding to the target distance can be calculated. As an optional embodiment, the target space point of the spatial straight line equation can be calculated according to the spatial straight line equation and its starting point with the target distance as the length.

[0107] In this step, for each target distance, a target space point can be calculated for each spatial straight line equation.

[0108] In step 5064, at least two sets corresponding to the at least two target distances can be constructed, and each set is used to store a group of target parameters.

[0109] In this step, the number of sets is consistent with the number of target distances. For example, when the at least two target distances include a first distance and a second distance, the at least two sets can include a first set corresponding to the first distance and a second set corresponding to the second distance, and the two sets can be used to store target space points obtained based on the first distance and the second distance, respectively.

[0110] In step 5066, the target space point corresponding to each of the target distances can be stored in the set corresponding to the target distance as the target parameter.

[0111] At this point, at least two groups of target parameters are obtained.

[0112] In step 508, the center position of the camera is determined according to the at least two groups of target parameters.

[0113] In this step, because the at least two groups of target parameters are obtained according to the at least two target distances and based on the plurality of camera parameters, and the target distance is related to the recommended distance (for example, the recommended pupillary distance), the center position of the camera determined according to the at least two groups of target parameters can control the error within an acceptable range.

[0114] In some embodiments, determining the center position of the camera according to the at least two groups of target parameters includes fitting the center position of the camera according to the at least two groups of target parameters by using a center camera model.

[0115] Figure 4C A schematic diagram of an example center camera model fitting result according to an embodiment of the present disclosure is shown. As shown in FIG. 6, the center camera model fitting result is obtained according to the at least two groups of target parameters. Figure 4CAs shown, the point O1 and the point O2 respectively correspond to the camera centers of the left and right camera barrels. Since the positions of the camera centers are within a certain distance range of the exit pupil distance, the camera center model fitting can be performed on the camera parameters, so as to effectively control the average error of the calibration within the accuracy requirement of the IPD / ET algorithm.

[0116] After the fitted camera center is obtained, the positioning of the target object can be further performed based on the camera center.

[0117] Therefore, in step 510, the target image captured by the camera can be obtained, and the target image includes the target object. Optionally, the target image can be an image captured by the camera 208 of the wearable device 200, and the target object can be the cornea of the eye.

[0118] In some embodiments, the method 500 can be applied to a wearable device, such as Figures 2A to 2C As shown, the wearable device 200 includes a camera barrel 202 and the camera 208 arranged in the camera barrel 202, and at least two point light sources 212 are arranged outside the camera barrel 202. The target image further includes at least two light spots formed by reflection of the at least two point light sources 212 on the cornea of the eye in the target image, so that the auxiliary positioning can be performed in combination with the positions of the light spots in the image, and the accuracy of the algorithm can be improved.

[0119] In step 512, the position of the target object is determined according to the center position of the camera and the target image.

[0120] In this step, the center position of the camera is known and the target image is obtained, so that the target object can be positioned based on the center camera model.

[0121] In some embodiments, as shown in Figure 4D As shown, the step 512 of determining the position of the target object according to the center position of the camera and the target image can further include the following steps:

[0122] In step 5122, an optimization function is constructed in combination with the center camera model according to the center position of the camera and the positions of the at least two light spots on the target image.

[0123] Figure 4E A schematic diagram of a camera model according to an embodiment of the present disclosure is shown.

[0124] In combination with Figure 4EAs shown in the IPD / ET algorithm based on the center camera model, the position of the corneal center in the three-dimensional space is defined as c, the position of the pupil center in the three-dimensional space is defined as p, the position of the fitted camera center in the three-dimensional space is defined as o, the distance of oc is defined as kc, and the projection of the corneal center c in the target image is defined as u o , and the corresponding vector is v o , and the radius of the cornea is R. The vector v o may be obtained according to the camera parameters (spatial straight line equation) corresponding to the pixel point corresponding to the center position of the target image.

[0125] Therefore, the relationship expression of the corneal center c and the camera center o is:

[0126] c = o + kc·v o

[0127] For each point light source l j (for example, l1, l2), a light spot q j (for example, q1, q2) is formed on the cornea, the projection of the light spot q j in the image is u j (for example, u1, u2), the distance from o to q j is k qj , and the corresponding vector is v qj , wherein the vector v qj may be obtained according to the camera parameters (spatial straight line equation) corresponding to the projection (i.e., pixel point) u j of the light spot q j in the target image.

[0128] Therefore, the relationship expression of the light spot q j formed on the cornea and the camera center o is:

[0129] q j = o + k qj ·v qj

[0130] According to the geometric relationship, it can be known that:

[0131] ||q j -c|| = R

[0132] (o-l j )×(u j -l j )·(c-l j ) = 0

[0133] Further, the relationship expression of the corneal center c and the light spot q j formed on the cornea is:

[0134]

[0135] wherein c j is the center position of the cornea of the eye calculated based on the correlation parameters of the light spot q j generated by the point light source l j , R is a constant, v lqj is the vector from l j to q j , v lqj and v qj are symmetric about the normal.

[0136] Then, an optimization function is constructed as follows:

[0137] f = min || c j - c mean ||

[0138] In step 5124, the optimal solution of the optimization function is solved.

[0139] As shown in the foregoing optimization function, by finding a center position c mean of the cornea of the eye, the error of all the calculated c j and the c mean is minimized.

[0140] As an optional embodiment, in combination with the foregoing formulas, the least square method can be used to solve the optimization function, so as to obtain the optimal solution c mean .

[0141] In step 5126, the center position of the cornea of the eye is determined according to the optimal solution.

[0142] As described above, the optimal solution c mean obtained is the center position of the cornea of the eye.

[0143] Thus, the center position of the cornea of the eye is located.

[0144] In some embodiments, the target image includes a first target image corresponding to a first eye (for example, a left eye) and a second target image corresponding to a second eye (for example, a right eye);

[0145] Determining the position of the target object according to the center position of the camera and the target image includes: determining the center position of the cornea of the eye of the first eye and the center position of the cornea of the eye of the second eye according to the center position of the camera and the target image.

[0146] The method 500 further includes: determining the interpupillary distance according to the center position of the cornea of the eye of the first eye and the center position of the cornea of the eye of the second eye.

[0147] Thus, after the corneal center positions of the left and right eyes are calculated respectively, the distance between the two positions can be obtained to obtain the interpupillary distance of the user.

[0148] In some embodiments, the wearable device 200 further comprises a binocular display module (for example, two display screens 204 corresponding to the first eye and the second eye respectively), and after the interpupillary distance is determined according to the center position of the cornea of the first eye and the center position of the cornea of the second eye, the method further comprises: adjusting the binocular display module according to the interpupillary distance, so as to realize the adaptive adjustment of the wearable device 200 based on the interpupillary distance, without the need for manual operation by the user, thereby improving the user experience.

[0149] Optionally, the adjustment of the binocular display module can be an adjustment of the image displayed by the binocular display module, so that the final imaging effect meets the requirements of the interpupillary distance.

[0150] In some embodiments, the method 500 further comprises:

[0151] The pupil position is obtained, for example, according to the relative relationship between the center position of the target image and the pupil image position, in combination with the center position of the cornea, to calculate the pupil position;

[0152] Then, the line-of-sight direction is determined according to the pupil position and the center position of the cornea.

[0153] The relationship expression of the line-of-sight direction g and the pupil position p and the center position c of the cornea is:

[0154] g=p-c

[0155] Thus, after the pupil position and the corneal center position are known, the line-of-sight direction of the user can be obtained, thereby realizing the line-of-sight tracking of the user.

[0156] In a more specific embodiment, the method 500 provided by the embodiments of the present disclosure can comprise the steps of camera parameter calibration of a non-center camera model, camera center fitting by a center camera model, target image acquisition, determination of a target object, and IPD / ET algorithm, which can realize better algorithm accuracy.

[0157] As can be seen from the above embodiments, the embodiments of the present disclosure propose a center camera approximate interpupillary distance estimation and line-of-sight tracking scheme, which adopts a center camera model to fit a non-center camera model, unifies the image starting point position, and only needs to perform secondary spline interpolation on the direction vector subsequently, thereby reducing the calculation amount by half under the condition of ensuring the accuracy of the interpupillary distance estimation algorithm and the line-of-sight estimation algorithm, greatly improving the efficiency of the algorithm and shortening the calculation time.

[0158] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, for example, a computer or a server, etc. The method of the embodiments can also be applied to a distributed scenario, and be completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only execute one or more steps in the method of the embodiments of the present disclosure, and the multiple devices can interact with each other to complete the method.

[0159] It should be noted that some embodiments of the present disclosure are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown or sequential order in order to achieve the desired results. In some implementations, multitasking and parallel processing can be advantageous.

[0160] The embodiments of the present disclosure also provide a computer device for implementing the method 500 described above. Figure 5 A hardware structure schematic diagram of an exemplary computer device 600 provided by the embodiments of the present disclosure is shown. The computer device 600 can be used to implement the head-mounted wearable device 104 of the Figure 1A the wearable device 200 of the Figures 2A to 2C the external device 112 of the Figure 1A the server 114 of the Figure 1A In some scenarios, the computer device 600 can also be used to implement the database server 116 of the Figure 1A

[0161] As shown in Figure 5 The computer device 600 can include a processor 602, a memory 604, a network module 606, a peripheral interface 608 and a bus 610. The processor 602, the memory 604, the network module 606 and the peripheral interface 608 are connected to each other through the bus 610 for internal communication connection in the computer device 600.

[0162] ​The processor 602 can be a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. The processor 602 can be configured to perform functions related to the techniques described in the present disclosure. In some embodiments, the processor 602 can also include multiple processors integrated as a single logical component. For example, as shown in FIG. 6, the processor 602 can include multiple processors 602a, 602b, and 602c. Figure 5

[0163] The memory 604 can be configured to store data (e.g., instructions, computer code, etc.). As shown in FIG. 6, the data stored by the memory 604 can include program instructions (e.g., program instructions for implementing the method 500 of embodiments of the present disclosure) and data to be processed (e.g., the memory can store configuration files of other modules, etc.). The processor 602 can also access the program instructions and data stored by the memory 604 and execute the program instructions to operate on the data to be processed. The memory 604 can include volatile storage or non-volatile storage. In some embodiments, the memory 604 can include random access memory (RAM), read only memory (ROM), optical disk, magnetic disk, hard disk, solid state disk (SSD), flash memory, memory stick, etc. Figure 5

[0164] The network interface 606 can be configured to provide communication between the computer device 600 and other external devices via a network. The network can be any wired or wireless network capable of transmitting and receiving data. For example, the network can be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC), etc.), a cellular network, the Internet, or a combination thereof. It can be understood that the type of network is not limited to the specific examples described above.

[0165] The peripheral interface 608 can be configured to connect the computer device 600 with one or more peripheral devices to enable information input and output. For example, the peripheral devices can include input devices such as a keyboard, a mouse, a touchpad, a touch screen, a microphone, various sensors, etc., and output devices such as a display, a speaker, a vibrator, an indicator light, etc.

[0166] ​​The bus 610 can be configured to transmit information between various components (e.g., the processor 602, the memory 604, the network interface 606, and the peripheral interface 608) of the computer device 600, such as an internal bus (e.g., a processor-memory bus), an external bus (a USB port, a PCI-E bus), and the like.

[0167] It should be noted that although the architecture of the computer device 600 described above only shows the processor 602, the memory 604, the network interface 606, the peripheral interface 608, and the bus 610, in the specific implementation process, the architecture of the computer device 600 can also include other components necessary for normal operation. In addition, those skilled in the art can understand that the architecture of the computer device 600 described above can also only contain components necessary for implementing the embodiments of the present disclosure, and does not necessarily contain all the components shown in the figure.

[0168] The present disclosure also provides a camera parameter calibration device. Figure 6 A schematic diagram of an exemplary device 700 provided by the embodiments of the present disclosure is shown. As shown, the device 700 can be used to implement the method 500, and can further include the following modules. Figure 6

[0169] The first acquisition module 702 is configured to acquire a plurality of camera parameters of a camera, the plurality of camera parameters including a plurality of spatial straight line equations with different starting points;

[0170] The first determination module 704 is configured to determine at least two target distances; select at least two groups of target parameters based on the plurality of camera parameters according to the at least two target distances; and determine a center position of the camera according to the at least two groups of target parameters.

[0171] The second acquisition module 706 is configured to acquire a target image collected by the camera, the target image including the target object.

[0172] The second determination module 708 is configured to determine a position of the target object according to the center position of the camera and the target image.

[0173] In some embodiments, the camera parameters include a plurality of spatial straight line equations corresponding to a plurality of pixel points of an image collected by the camera and starting points corresponding to the spatial straight line equations.

[0174] The first determination module 704 is configured to:

[0175] ​According to the at least two target distances and each of the spatial straight line equations and the starting point thereof, at least two target spatial points corresponding to the at least two target distances respectively are determined for each of the spatial straight line equations; and the target parameters and the target spatial points correspond to the target distances.

[0176] In some embodiments, the first determining module 704 is configured to: according to the at least two groups of target parameters, adopt a center camera model to fit the center position of the camera.

[0177] In some embodiments, the target object includes an eye cornea, and the at least two target distances include at least two target pupil distances.

[0178] In some embodiments, the method is applied to a wearable device, the wearable device includes a lens barrel and the camera arranged in the lens barrel, at least two point light sources are arranged outside the lens barrel, and the target image further includes at least two light spots formed by reflection of the at least two point light sources on the eye cornea in the target image.

[0179] In some embodiments, the second determining module 708 is configured to:

[0180] According to the center position of the camera and the positions of the at least two light spots on the target image, an optimization function is constructed in combination with a center camera model;

[0181] Based on the optimization function, the center position of the eye cornea is determined.

[0182] In some embodiments, the target image includes a first target image corresponding to a first eye and a second target image corresponding to a second eye.

[0183] The second determining module 708 is configured to: according to the center position of the camera and the target image, determine the center position of the eye cornea of the first eye and the center position of the eye cornea of the second eye.

[0184] The second determining module 708 is further configured to: according to the center position of the eye cornea of the first eye and the center position of the eye cornea of the second eye, determine a interpupillary distance.

[0185] In some embodiments, the wearable device further includes a binocular display module, and after the interpupillary distance is determined according to the center position of the eye cornea of the first eye and the center position of the eye cornea of the second eye, the apparatus further includes an adjusting module configured to: adjust the binocular display module according to the interpupillary distance.

[0186] In some embodiments, the second determining module 708 is configured to: acquire a pupil position; and determine the gaze direction according to the pupil position and a center position of the cornea of the eye.

[0187] In some embodiments, the first determining module 704 is configured to: determine the at least two target pupillary distances according to a recommended pupillary distance of the wearable device.

[0188] For ease of description, the above apparatus is described in various modules in terms of functions. Of course, the functions of the modules can be implemented in one or more software and / or hardware in the implementation of the present disclosure.

[0189] The apparatus of the above embodiments is used to implement the corresponding method 500 in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0190] Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method 500 of any of the above embodiments.

[0191] The computer-readable medium of the present embodiment includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0192] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the method 500 of any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0193] Based on the same inventive concept, the disclosure also provides a computer program product corresponding to the method 500 of any of the above embodiments, which comprises computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the method 500. Corresponding to the execution subject of each step in each embodiment of the method 500, the processor performing the corresponding step can belong to the corresponding execution subject.

[0194] The computer program product of the above embodiments is used to cause the computer and / or the processor to perform the method 500 as described in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which are not repeated here.

[0195] It should be understood by those of ordinary skill in the art that the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope (including claims) of the disclosure is limited to these examples; the above embodiments or technical features between different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the embodiments of the disclosure as described above. For the sake of brevity, they are not provided in detail.

[0196] In addition, in order to simplify the description and discussion, and so as not to make the embodiments of the disclosure difficult to understand, the well-known power / ground connections of integrated circuit (IC) chips and other components can or can not be shown in the provided drawings. In addition, devices can be shown in block diagram form in order to avoid making the embodiments of the disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented to implement the embodiments of the disclosure (i.e., these details should be fully within the understanding of those skilled in the art). Where specific details (e.g., circuitry) are set forth in order to describe an exemplary embodiment of the disclosure, it will be apparent to those skilled in the art that the embodiments of the disclosure can be practiced without these specific details or with variations on these specific details. Therefore, these descriptions should be considered illustrative rather than limiting.

[0197] Although the disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications and variations will be apparent to those skilled in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) can use the embodiments discussed.

[0198] Embodiments of the present disclosure are intended to cover all such alternatives, modifications, and variations as fall within the broad scope of the appended claims. Accordingly, any one or more of the omitted, modified, equivalently replaced, improved, and the like, as long as within the spirit and principle of the embodiments of the present disclosure, should be included in the scope of protection of the present disclosure.

Claims

1. A method for determining the location of a target object, comprising: Obtain multiple camera parameters of the camera, the multiple camera parameters including multiple spatial line equations with different starting points; Determine the distance between at least two targets; Based on the at least two target distances, at least two sets of target parameters are selected based on the plurality of camera parameters; The center position of the camera is determined based on the at least two sets of target parameters; Acquire a target image captured by the camera, the target image including the target object; The position of the target object is determined based on the center position of the camera and the target image.

2. The method as described in claim 1, wherein, The camera parameters include multiple spatial line equations corresponding to multiple pixels of the image captured by the camera, and the starting point corresponding to the spatial line equations; Based on the at least two target distances, at least two sets of target parameters are selected based on the plurality of camera parameters, including: Based on the at least two target distances and each of the spatial line equations and its starting point, determine at least two target spatial points corresponding to each of the at least two target distances for each of the spatial line equations. The target parameters and the target spatial points correspond to each other based on the target distance.

3. The method as described in claim 2, wherein, Determining the center position of the camera based on the at least two sets of target parameters includes: Based on the at least two sets of target parameters, the center position of the camera is obtained by fitting a central camera model.

4. The method of claim 1, wherein, The target object includes the cornea, and the at least two target distances include at least two target exit pupil distances.

5. The method of claim 4, wherein, The method is applied to a wearable device, which includes a lens barrel and a camera disposed within the lens barrel. At least two point light sources are disposed on the outer side of the lens barrel, and the target image also includes at least two light spots formed by reflection of the at least two point light sources on the cornea of ​​the target image.

6. The method of claim 5, wherein, Determining the position of the target object based on the center position of the camera and the target image includes: Based on the center position of the camera and the positions of the at least two light spots on the target image, an optimization function is constructed in conjunction with the central camera model; Based on the optimization function, the center position of the cornea is determined.

7. The method of claim 6, wherein, The target image includes a first target image corresponding to the first eye and a second target image corresponding to the second eye; Determining the position of the target object based on the center position of the camera and the target image includes: determining the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye based on the center position of the camera and the target image; The method further includes: determining the interpupillary distance based on the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye.

8. The method of claim 7, wherein, The wearable device further includes a binocular display module. After determining the interpupillary distance based on the center position of the cornea of ​​the first eye and the center position of the cornea of ​​the second eye, the method further includes: The binocular display module is adjusted according to the interpupillary distance.

9. The method of claim 6, wherein, The method further includes: Obtain the pupil position; The direction of gaze is determined based on the position of the pupil and the center position of the cornea.

10. The method of claim 5, wherein, Determine the distance between at least two targets, including: The at least two target exit pupil distances are determined based on the recommended exit pupil distance of the wearable device.

11. An apparatus for determining the position of a target object, comprising: The first acquisition module is configured to acquire multiple camera parameters of the camera, the multiple camera parameters including multiple spatial line equations with different starting points; The first determining module is configured to: determine the distance between at least two targets; Based on the at least two target distances, at least two sets of target parameters are selected based on the plurality of camera parameters; based on the at least two sets of target parameters, the center position of the camera is determined; The second acquisition module is configured to: acquire a target image captured by the camera, wherein the target image includes the target object; The second determining module is configured to determine the position of the target object based on the center position of the camera and the target image.

12. A computer device comprising one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, the programs comprising instructions for performing the method as claimed in any one of claims 1-10.

13. The computer device as claimed in claim 12, wherein, The computer device includes a wearable device, which includes a display module, a lens barrel disposed on the light-emitting side of the display module, and a camera disposed inside the lens barrel. At least two point light sources are disposed on the outer side of the lens barrel.

14. A non-volatile computer-readable storage medium comprising a computer program, which, when executed by one or more processors, causes the processors to perform the method as described in any one of claims 1-10.

15. A computer program product comprising computer program instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-10.

Images