Methods, apparatuses, devices, and storage media for pupillary distance measurement

Abstract

Embodiments of the present disclosure provide a scheme for pupillary distance measurement, and particularly relate to a method, device, apparatus and storage medium for pupillary distance measurement. According to the embodiments described herein, in the process of pupillary distance measurement, the control unit moves two cameras from respective initial positions to respective end positions, and controls the two cameras to capture left eye images and right eye images respectively during the movement. Further, the control unit determines corresponding calibration parameters for obtaining the pupillary distance, and obtains the pupillary distance of the user based on the corresponding calibration parameters and the captured images. In this way, the accuracy of the pupillary distance measurement is improved, and the hardware cost of the pupillary distance measurement is reduced.

Description

Technical Field

[0001] The exemplary embodiments of this disclosure generally relate to measurement techniques, and more specifically, to methods, apparatus, devices, and computer-readable storage media for measuring pupillary distance. Background Technology

[0002] Interpupillary distance (IPD) refers to the distance between the pupils of both eyes. When people wear wearable devices such as glasses or virtual reality headsets, if the device's IPD does not match the user's, it will cause discomfort and reduce visual quality. Therefore, quickly and accurately obtaining the user's IPD is one of the key technologies for improving the user experience when using wearable devices. Summary of the Invention

[0003] In a first aspect of this disclosure, a method for measuring interpupillary distance (IPD) is provided, comprising: setting a first camera and a second camera at a first initial position and a second initial position, respectively; the first camera being configured to capture an image of a user's left eye and associated with a first set of light sources; the second camera being configured to capture an image of the user's right eye and associated with a second set of light sources; controlling the first camera and the second camera to capture a first set of images and a second set of images of the user's left and right eyes, respectively, while moving the first camera and the second camera to a first end position and a second end position, respectively; acquiring a first set of parameters of the first camera and the first set of light sources corresponding to the first set of images, and a second set of parameters of the second camera and the second set of light sources corresponding to the second set of images; and acquiring the user's IPD at least in part based on the first set of images, the first set of parameters, the second set of images, and the second set of parameters.

[0004] In a second aspect of this disclosure, an apparatus for measuring interpupillary distance (IPD) is provided, comprising: a position control module configured to set a first camera and a second camera at a first initial position and a second initial position, respectively, wherein the first camera is configured to capture an image of a user's left eye and is associated with a first set of light sources, and the second camera is configured to capture an image of the user's right eye and is associated with a second set of light sources; a camera control module configured to control the first camera and the second camera to capture a first set of images and a second set of images of the user's left and right eyes, respectively, during the process of moving the first camera and the second camera to a first end position and a second end position, respectively; a parameter acquisition module configured to acquire a first set of parameters of the first camera and the first set of light sources corresponding to the first set of images, and a second set of parameters of the second camera and the second set of light sources corresponding to the second set of images; and an IPD measurement module configured to acquire the user's IPD at least in part based on the first set of images, the first set of parameters, the second set of images, and the second set of parameters.

[0005] In a third aspect of this disclosure, an electronic device is provided, comprising: a first camera configured to capture an image of a user's left eye; a second camera configured to capture an image of the user's right eye; a first set of light sources associated with the first camera; a second set of light sources associated with the second camera; and a control unit configured to perform the method according to the first aspect described above.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program that can be executed by a processor to perform the method according to a first aspect of this disclosure.

[0007] It should be understood that the content described in this summary section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0008] In the following detailed description, in conjunction with the accompanying drawings, the above and other features, advantages, and aspects of the various implementations of this disclosure will become more apparent. In the accompanying drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0009] Figure 1 A block diagram of an electronic device for interpupillary distance measurement according to some embodiments of the present disclosure is shown;

[0010] Figure 2 A flowchart of a method for measuring interpupillary distance according to some embodiments of the present disclosure is shown; Figure 3A schematic block diagram illustrating the adjustment of camera position according to some embodiments of the present disclosure is shown;

[0011] Figure 4 A flowchart is shown showing a method for obtaining a first set of parameters and a second set of parameters according to some embodiments of the present disclosure;

[0012] Figure 5 A block diagram of an apparatus for measuring interpupillary distance according to some embodiments of the present disclosure is shown; and

[0013] Figure 6 A block diagram is shown that can be used to implement some embodiments of the present disclosure. Detailed Implementation

[0014] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0015] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0016] The term "in response to" indicates that a corresponding event has occurred or a condition has been met. It will be understood that the timing of subsequent actions performed in response to this event or condition is not necessarily strongly correlated with the time when the event or condition is met. In some cases, subsequent actions may be performed immediately upon the occurrence of the event or the fulfillment of the condition; in other cases, they may be performed some time after the event or condition has occurred.

[0017] In the description of embodiments of this disclosure, the term "group" refers to a collection of one or more elements. For example, "a set of images" should be understood as one or more images, and "a set of parameters" should be understood as one or more parameters. In other words, the term "group" as used in embodiments of this disclosure is not limited in terms of the number of elements it contains.

[0018] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0019] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0020] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0021] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0023] As discussed above, quickly and accurately obtaining the user's interpupillary distance (IPD) is one of the key technologies for improving the user experience when using wearable devices. Traditional IPD measurement techniques require gaze estimation and eye position estimation, and support the use of a single fixed camera or multiple fixed cameras for IPD measurement.

[0024] When using a single fixed camera for pupillary distance (PD) measurement, gaze estimation and eye position estimation rely on a constructed eye model. However, the eye is a biometric characteristic of the user, and therefore varies from user to user. Consequently, when the physiological characteristics of the user's eye differ significantly from the constructed eye model, the PD measurement results will have a large error. Furthermore, when using a single fixed camera for PD measurement, the camera's shooting direction needs to be perpendicular to the line connecting the two eyes to reduce the impact of depth estimation errors on the PD measurement results. However, adjusting the camera's shooting direction typically requires the addition of optical components such as hot mirrors, which increases the hardware cost and design complexity of the PD measurement device.

[0025] While using multiple fixed cameras to measure pupil distance improves the accuracy of pupil distance measurement compared to using a single fixed camera, the hardware cost is relatively high.

[0026] According to embodiments of this disclosure, a pupillary distance (PD) measurement scheme based on a moving camera is provided. According to various embodiments of this disclosure, during PD measurement, a control unit moves two cameras from their respective initial positions to their respective final positions. During this movement, the control unit controls the two cameras to capture images of the left and right eyes, respectively. The control unit further determines corresponding calibration parameters for obtaining the PD, and acquires the user's PD based on the corresponding calibration parameters and the captured images. In this way, the accuracy of PD measurement is improved, and the hardware cost of PD measurement is reduced. Example embodiments of this disclosure are described below with reference to the accompanying drawings.

[0027] Figure 1 A block diagram of an electronic device 100 for interpupillary distance measurement according to some embodiments of the present disclosure is shown. Figure 1 As shown, the electronic device 100 includes a first camera 120-1 configured to capture images of the user's left eye and a second camera 120-2 configured to capture images of the user's right eye. For ease of discussion, the first camera 120-1 and the second camera 120-2 may be collectively referred to as camera 120.

[0028] In some embodiments, to facilitate capturing left-eye and right-eye images, the camera 120 may be tilted on the electronic device 110.

[0029] In some embodiments, to increase the accuracy of interpupillary distance measurement, multiple cameras can be configured to capture images of the left eye, and correspondingly, multiple cameras can be configured to capture images of the right eye.

[0030] In some embodiments, the electronic device 100 includes a movable component on which a first camera 120-1 and a second camera 120-2 are configured. In this way, unlike fixed cameras in the conventional art, the camera 120 according to various embodiments of the present disclosure can move within the electronic device 110 to capture multiple left-eye and right-eye images.

[0031] In one specific embodiment, the electronic device 100 is a virtual reality device. The first camera 120-1 is mounted on the left eyeglass tube 140-1 of the virtual reality device, while the second camera 120-2 is mounted on the right eyeglass tube 140-2 (for ease of discussion, the left and right eyeglass tubes 140-1 and 140-2 can be collectively referred to as tube 140). Furthermore, the left and right eyeglass tubes 140-1 and 140-2 are arranged on a slide rail. In this manner, the first camera 120-1 and the second camera 120-2 can move (i.e., translate) along the slide rail with the left and right eyeglass tubes 140-1 and 140-2.

[0032] Furthermore, in this specific embodiment, the slide rail can be marked with corresponding scales. When the first camera 120-1 and the second camera 120-2 move on the slide rail along with the left eyeglass tube 140-1 and the right eyeglass tube 140-2, the positions of the left eyeglass tube 140-1 / first camera 120-1 and the right eyeglass tube 140-2 / second camera 120-2 can be identified by the scales of the slide rail.

[0033] In some embodiments, the position information of the left eyeglass tube 140-1 / first camera 120-1 and the right eyeglass tube 140-2 / second camera 120-2 can be obtained by sensors.

[0034] Alternatively, in some embodiments, the position information of the left eyeglass lens 140-1 / first camera 120-1 and the right eyeglass lens 140-2 / second camera 120-2 can be determined based on one or more corresponding parameters. For example, given the initial positions, the position information of the left eyeglass lens 140-1 / first camera 120-1 and the right eyeglass lens 140-2 / second camera 120-2 can be determined based on parameters such as the movement trajectory and / or movement speed of the left eyeglass lens 140-1 / first camera 120-1 and the right eyeglass lens 140-2 / second camera 120-2 on the slide rail.

[0035] It should be understood that the above-described embodiments for identifying location information are merely exemplary. In other embodiments, other methods may be used to identify the location information of the left eyeglass lens 140-1 / first camera 120-1 and the right eyeglass lens 140-2 / second camera 120-2. This disclosure is not limited in this respect.

[0036] like Figure 1 As shown, the electronic device 100 further includes a control unit 110. In some embodiments, the control unit 100 can perform corresponding logic and operations, and can communicate with other components of the electronic device 100 to interact with control messages and data.

[0037] In a specific embodiment where the electronic device 100 is a virtual reality device, the control unit 100 can control the left eyeglass tube 140-1 and the right eyeglass tube 140-2 to move on a slide rail. In this way, the control unit 110 can control the position of the first camera 120-1 by controlling the position of the left eyeglass tube 140-1, and correspondingly control the position of the second camera 120-2 by controlling the position of the right eyeglass tube 140-2.

[0038] In some embodiments, the electronic device 100 includes at least one light source. Further, the at least one light source can be a movable light source or a non-movable light source. In some embodiments, the movable light source is disposed on a movable component of the electronic device 100, while the non-movable light source is disposed on a non-movable component of the electronic device 100.

[0039] In a specific embodiment where the electronic device 100 is a virtual reality device, a movable light source can be arranged on the left eyeglass tube 140-1 and the right eyeglass tube 140-2. Figure 1 In a specific embodiment, a movable light source 130-1 is disposed on the left eyeglass tube 140-1, a movable light source 130-2 is disposed on the right eyeglass tube 140-2, and a non-movable light source is disposed on the back plates of the left eyeglass tube 140-1 and the right eyeglass tube 140-2 (e.g., Figure 1 (Light source 130-3 shown). For ease of discussion, light sources 130-1, 130-2 and 130-3 can be collectively referred to as light source 130.

[0040] In some embodiments, a first group of at least one light source is associated with a first camera 120-1, and a second group of at least one light source is associated with a second camera 120-2.

[0041] In some embodiments, the association between the camera 120 and the light source can be predefined. For example, in the factory settings of the electronic device 100, the first set of light sources is pre-associated to the first camera 120-1, and the second set of light sources is pre-associated to the second camera 120-2.

[0042] Alternatively or additionally, in some embodiments, the association between camera 120 and the light source may be associated with the positional relationship between camera 120 and the light source. As an example embodiment, camera 120 and the light source mounted on the same movable component (such as lens barrel 140) may be associated with each other. In another example embodiment, camera 120 and the light source are associated with each other when the distance between camera 120 and the light source is less than a threshold distance.

[0043] Alternatively or additionally, in some embodiments, the association between camera 120 and the light source may be associated with a specific function. As an example embodiment, when the light source cooperates with first camera 120-1 to capture a user's left-eye image, the light source is associated with first camera 120-1. Correspondingly, when the light source cooperates with second camera 120-2 to capture a user's right-eye image, the light source is associated with second camera 120-2.

[0044] In short, the relationship between camera 120 and the light source can be predefined or configured based on certain rules. This disclosure is not limited in this respect.

[0045] In some embodiments, the number of light sources associated with the first camera 120-1 / second camera 120-2 is one. This optimizes the hardware cost-effectiveness of the device. Alternatively, in other embodiments, the number of light sources associated with the first camera 120-1 / second camera 120-2 is greater than one. This improves the accuracy of the interpupillary distance measurement results.

[0046] In some embodiments, the light source 130 is a light-emitting diode (LED). Furthermore, since the infrared band is less susceptible to external interference compared to other bands, in some embodiments, the camera 120 is an infrared camera, and the light source is an infrared LED. In this way, the accuracy of the interpupillary distance measurement results will be improved. Alternatively, in some other embodiments, the camera 120 and the light source 130 may also be configured to operate in other bands.

[0047] In some embodiments, the parameters of camera 120 and light source 130 can be pre-calibrated. Example calibration parameters include, but are not limited to, the intrinsic parameters of camera 120, the extrinsic parameters of immovable light source 130 (e.g., light source 130-3) relative to the device coordinate system, and the extrinsic parameters of moving light sources 130 (e.g., light sources 130-1 and 130-2) relative to the camera 130 on their respective lens barrels 140. An example of a device coordinate system is an inertial measurement unit (IMU).

[0048] In some embodiments, certain calibration parameters of the camera 120 and the light source 130 are associated with specific positions (hereinafter referred to as "calibration parameters"). As a specific embodiment, the distance between the left lens barrel 140-1 and the right lens barrel 140-2 is adjusted to a first predetermined distance (e.g., the longest distance, also known as the farthest lens barrel position), and the various calibration parameters of the camera 120 and the light source 130, as well as the current position of the lens barrel 140 / camera 120 (e.g., the scale of the slider), are recorded. Further, the distance between the left lens barrel 140-1 and the right lens barrel 140-2 is adjusted to a second predetermined distance (e.g., the shortest distance, also known as the shortest lens barrel position), and the various calibration parameters of the camera 120 and the light source 130, as well as the current position of the lens barrel 140 / camera 120 (e.g., the scale of the slider), are recorded.

[0049] In some embodiments, the calibration parameters of the camera 120 and the light source 130 corresponding to the first predetermined distance and the second predetermined distance can be calibrated and recorded as factory parameters.

[0050] It should be understood that Figure 1 Only an example of the electronic device 100 used for interpupillary distance measurement is shown. In other words, Figure 1The number of components and their relationships shown can be changed depending on the specific application scenario. The embodiments disclosed herein are not limited in this respect.

[0051] Figure 2 A flowchart of a method 200 for measuring interpupillary distance according to some embodiments of the present disclosure is described. For ease of discussion, refer to... Figure 1 The discussion will focus on method 200 using electronic device 100. Method 200 can be derived from... Figure 1 The control unit 110 in the middle performs the operation.

[0052] In frame 210, control unit 110 sets the first camera 120-1 and the second camera 120-2 to a first initial position and a second initial position, respectively.

[0053] In one particular embodiment, the first initial position and the second initial position correspond to the longest distance between the left eyeglass lens 140-1 and the right eyeglass lens 140-2. Alternatively, in yet another particular embodiment, the first initial position and the second initial position correspond to the shortest distance between the left eyeglass lens 140-1 and the right eyeglass lens 140-2. It should be understood that in other particular embodiments, the first initial position and the second initial position may correspond to any distance between the left eyeglass lens 140-1 and the right eyeglass lens 140-2. The embodiments of this disclosure are not limited in this respect.

[0054] In some embodiments, a first camera 120-1 is configured to capture an image of the user's left eye and is associated with a first set of light sources, and a second camera 120-2 is configured to capture an image of the user's right eye and is associated with a second set of light sources.

[0055] In the following description of some implementations, the light source 130-1 arranged on the left eyeglass tube 140-1 is referred to as an example of the first group of light sources, and the light source 130-2 arranged on the right eyeglass tube 140-2 is referred to as an example of the second group of light sources. In this case, the light source 130-1 is also referred to as the first group of light sources 130-1, and the light source 130-2 is also referred to as the second group of light sources 130-2.

[0056] It should be understood that the above-described relationship between the light source 130 and the camera 120 is merely an example. In other embodiments, the relationship between the light source 130 and the camera 120 can be arbitrarily configured according to actual needs. The various embodiments of this disclosure are not limited in this respect.

[0057] In frame 220, control unit 110 controls the first camera 120-1 to move from a first initial position to a first end position, and controls the first camera 120-1 to capture the user's left eye to obtain a first set of images. Correspondingly, control unit 110 controls the second camera 120-2 to move from a second initial position to a second end position, and controls the second camera 120-2 to capture the user's right eye to obtain a second set of images.

[0058] In other words, during the process of controlling the first camera 120-1 and the second camera 120-2 to move from a first initial position and a second initial position to a first ending position and a second ending position, respectively, the control unit 110 controls the first camera 120-1 and the second camera 120-2 to capture a first set of images and a second set of images for the user's left and right eyes, respectively. It should be understood that the first initial position, the second initial position, the first ending position, and the second ending position may be part of or not part of the camera 120 movement process. In other words, this disclosure is not limited in how the initial position and / or ending position of the camera 120 movement process are defined.

[0059] In some embodiments, the first set of images includes at least a first image captured at a first time point and a second image captured at a second time point, and the second set of images includes at least a third image captured at a third time point and a fourth image captured at a fourth time point. In other words, the first / second set of images includes images captured at at least two different time points. In this way, it can be ensured that the number of captured images meets the minimum number of images required to determine the interpupillary distance.

[0060] In some embodiments, the first time point is different from the third time point, and the second time point is different from the fourth time point. In this way, the movement of the first camera 120-1 and the second camera 120-2 can be controlled independently, making the interpupillary distance measurement process more flexible. Alternatively, in other embodiments, the first time point is the same as the third time point, and the second time point is the same as the fourth time point. In this way, the total time required for interpupillary distance measurement will be shortened.

[0061] In one particular embodiment, the first initial position and the second initial position correspond to the longest distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-2, while the first end position and the second end position can correspond to the shortest distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-2.

[0062] Alternatively, when the first initial position and the second initial position correspond to the shortest distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-2, the first ending position and the second ending position may correspond to the longest distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-2.

[0063] It should be understood that the first and second end positions in the specific embodiments described above are merely examples and should not be construed as limiting the scope of this disclosure. In other embodiments, the first and second end positions may be set at any other location as needed. The embodiments of this disclosure are not limited in this respect.

[0064] In some embodiments, the first camera 120-1 and the second camera 120-2 move from a first initial position and a second initial position to a first ending position and a second ending position at a relatively fast speed, for example, within half a second. In this way, the probability of the user blinking and shifting their gaze during the shooting process is reduced.

[0065] In some embodiments, the movement of the first camera 120-1 and the second camera 120-2 can be controlled independently. In other words, the initial and / or final movement times of the first camera 120-1 and the second camera 120-2 can be different. In this way, the interpupillary distance measurement process becomes more flexible.

[0066] Alternatively or additionally, in some embodiments, the movement of the first camera 120-1 and the second camera 120-2 can be controlled collaboratively, that is, the initial and final movement times of camera 120-1 are the same as those of camera 120-2. In this way, the total time required for interpupillary distance measurement will be shortened.

[0067] In some embodiments, the first camera 120-1 and the second camera 120-2 capture left-eye and right-eye images during camera 120 movement at a relatively high predetermined frame rate (e.g., greater than 60 FPS). In this way, the number of captured images is increased, and the accuracy of interpupillary distance measurement is correspondingly improved.

[0068] refer to Figure 3 To better understand the movement of the camera position. Figure 3 A schematic block diagram 300 illustrating the adjustment of camera position according to some embodiments of the present disclosure is shown. Figure 3 In a specific embodiment, the first initial position and the second initial position correspond to the longest distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-1. The first ending position and the second ending position correspond to the shortest distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-1.

[0069] In a specific embodiment where the electronic device is a virtual reality device, after the user puts on the virtual reality device, the device adjusts the left and right eyepieces to their furthest positions (or closest positions). Next, the device adjusts the left eyepiece 140-1 and right eyepiece 140-2 from their furthest positions to their closest positions. During the movement of the left and right eyepieces 140-1 and 140-2, the first camera 120-1 and the second camera 120-2 respectively capture images of the user's left and right eyes at at least two moments to obtain a first set of images and a second set of images.

[0070] In box 250, control unit 110 acquires the relevant parameters needed to calculate pupillary distance. Figure 2 In a specific embodiment, in block 256, the control unit 110 acquires a first set of parameters of the first camera 120-1 and the first light source 130-1 corresponding to the first set of images, and a second set of parameters of the second camera 120-2 and the second light source 130-2 corresponding to the second set of images.

[0071] Next, we will refer to Figure 4 This section describes in detail the process of obtaining the first set of parameters and the second set of parameters. Figure 4 A flowchart of a method 400 for obtaining a first set of parameters and a second set of parameters according to some embodiments of the present disclosure is shown.

[0072] In block 410, control unit 110 determines the first group position of the first camera 120-1 when capturing the first group of images, and the second group position of the second camera 120-2 when capturing the second group of images.

[0073] In one particular embodiment, during the process of moving the first camera 120-1 and the second camera 120-2 from the first initial position and the second initial position to the first end position and the second end position, the control unit 110 controls the first camera 120-1 and the second camera 120-2 to capture images of the left eye and the right eye respectively, while recording the position of the lens barrel 140 at each shooting moment (such as the coordinates of the slide rail, i.e., the position of the camera 120).

[0074] In frame 420, control unit 110 acquires a first set of calibration parameters for a first position and a second set of calibration parameters for a second position for a first camera 120-1 and a first set of light sources 130-1.

[0075] In frame 430, control unit 110 acquires a third set of calibration parameters for the second camera 120-2 and the second set of light sources 130-2 for the third position and a fourth set of calibration parameters for the fourth position.

[0076] In some embodiments, the first position is a first initial position, the second position is a first ending position, the third position is a second initial position, and the fourth position is a second ending position.

[0077] In box 440, control unit 110 calculates the first set of parameters based on the first set of calibration parameters, the second set of calibration parameters, and the first set of positions.

[0078] In box 450, control unit 110 calculates the second set of parameters based on the third set of calibration parameters, the fourth set of calibration parameters, and the second set of positions.

[0079] In some embodiments, calibration parameters include, but are not limited to, intrinsic parameters of camera 120, extrinsic parameters of immovable light source 130 (such as light source 130-3) relative to the device coordinate system, and extrinsic parameters of movable light source 130 (such as light source 130-1 and 130-2) relative to camera 130 on their respective lens barrel 140.

[0080] In some embodiments, the first set of calibration parameters, the second set of calibration parameters, the third set of calibration parameters, and the fourth set of calibration parameters are known calibration parameters (e.g., recorded in the factory settings). As a specific embodiment, the known calibration parameters include the calibration parameters when the distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-2 is a first predetermined distance (such as the longest distance), that is, the first set of calibration parameters and the third set of calibration parameters corresponding to the first initial position of the first camera 120-1 and the second initial position of the second camera 120-2.

[0081] Accordingly, the known calibration parameters also include calibration parameters when the distance between the left eyeglass tube 140-1 and the right eyeglass tube 140-2 is a second predetermined distance (such as the shortest distance), that is, a second set of calibration parameters and a fourth set of calibration parameters corresponding to the first end position of the first camera 120-1 and the second end position of the second camera 120-2.

[0082] As discussed above, during the process of moving the first camera 120-1 and the second camera 120-2 from the first initial position and the second initial position to the first final position and the second final position, the control unit 110 simultaneously records the position of the lens barrel 140 (e.g., the coordinates of the slide rail) at each shooting moment. Since the movement of the lens barrel 140 on the slide rail is a translational motion, the known first set of calibration parameters, the second set of calibration parameters, the third set of calibration parameters, and the fourth set of calibration parameters can be used as the basis for calculation, and the calibration parameters of the camera 120 and the light source 13-0 at other shooting moments can be obtained through interpolation. In this way, the first set of parameters can be calculated based on the first set of calibration parameters, the second set of calibration parameters, and the first set of positions, and the second set of parameters can be calculated based on the third set of calibration parameters, the fourth set of calibration parameters, and the second set of positions.

[0083] Specifically, in some embodiments, for each position of lens barrel 14-0 (i.e., the position of camera 120), the extrinsic parameters of camera 120 are interpolated based on known calibration parameters to obtain the camera extrinsic parameters at that position. For movable light sources 130 (such as light sources 130-1 and 130-2), if their extrinsic parameters are originally calibrated relative to the camera 120 on their respective lens barrel 140, their extrinsic parameters relative to the camera 120 on their respective lens barrel 140 need to be converted to extrinsic parameters relative to the device coordinate system.

[0084] In this way, the control unit 110 can obtain a first set of parameters and a second set of parameters for calculating the interpupillary distance. It should be understood that the process of obtaining the first set of parameters and the second set of parameters described above is merely exemplary and should not be construed as limiting the embodiments of this disclosure. In other implementations, other methods may be used to obtain the first set of parameters and the second set of parameters, including but not limited to real-time measurement.

[0085] Continue to refer to Figure 2 In box 260, control unit 110 acquires the user's interpupillary distance based at least in part on a first set of images, a first set of parameters, a second set of images, and a second set of parameters.

[0086] According to some embodiments of this disclosure, the time it takes for the camera 120 to move from the initial position to the final position is short, thus it can be assumed that the spatial position of the eyeball has not changed. In this case, images captured by the same camera 120 at different times can be considered equivalent to eyeball images captured simultaneously by multiple cameras at different positions.

[0087] In this context, the user's interpupillary distance can be measured using any multi-camera-based algorithm. One example algorithm is the pupillary-corneal reflex method. Another example algorithm is the triangulation algorithm. It should be understood that the above algorithms are merely examples, and the various embodiments of this disclosure can use any existing or future proposed multi-camera-based algorithm to measure the user's interpupillary distance. The various embodiments of this disclosure are not limited in this respect.

[0088] In some embodiments, to improve the accuracy of interpupillary distance measurement, the user may be guided to gaze at a guide point during the measurement. For example... Figure 2 As shown, in block 220, the control unit 110 presents a guide point at a predetermined position on the display device. Further, in block 230, the control unit 110 provides the user with a prompt associated with the guide point, instructing the user to keep their gaze on the guide point. The prompt can be provided to the user in any manner, such as text or voice. This disclosure is not limited in the manner in which the prompt is provided.

[0089] In a specific embodiment where the electronic device 110 is a virtual reality device, a guide point can be displayed in the virtual reality interface, prompting the user to look at the guide point and remain still.

[0090] Furthermore, users may blink or shift their gaze during the measurement process, which will prevent the determination of the user's interpupillary distance based on the captured images. This results in invalid first and second sets of images being generated during the measurement. Therefore, in some embodiments, the validity of the captured images can be detected.

[0091] As shown in Figure 3, in box 252, the control unit 110 detects the validity of the first group of images and the second group of images. In box 254, the control unit 110 determines whether the first group of images and the second group of images are valid. If the first group of images and the second group of images are valid, then box 256 is executed; if the first group of images and the second group of images are valid, then the execution box 210 is returned.

[0092] In some embodiments, validity detection is performed on all images in the first set of images and the second set of images.

[0093] Alternatively, in some embodiments, to improve detection efficiency, a subset of images from the first set of images and a subset of images from the second set of images may be selected to perform validity detection.

[0094] In some embodiments, if at least some images in the first set of images and / or at least some images in the second set of images show that the user blinked or shifted their gaze during the measurement, then the first set of images and the second set of images are determined to be invalid.

[0095] In some embodiments, the control unit 110 detects the user's left and right pupils in at least a portion of the first set of images and at least a portion of the second set of images according to the pupil detection model. If the left pupil is not detected in at least a portion of the first set of images (or the right pupil is not detected in at least a portion of the second set of images), it can be presumed that the user blinked, and the first and second sets of images are determined to be invalid.

[0096] As a specific embodiment, if the left pupil is not detected in any image of the first group of images (or the right pupil is not detected in any image of the second group of images), then the first group of images and the second group of images are determined to be invalid.

[0097] If the left pupil is detected in at least a portion of the first set of images, the relative position change of the left pupil in at least a portion of the first set of images is detected. Correspondingly, if the right pupil is detected in at least a portion of the first set of images, the relative position change of the right pupil in at least a portion of the second set of images is detected. If the positions of the left / right pupils change, it is presumed that the user's gaze has shifted, and the first and second sets of images are determined to be invalid; otherwise, the first and second sets of images are determined to be valid.

[0098] In some embodiments, detecting the relative positional change of the left / right pupil in at least a portion of the first / second set of images includes detecting the positional change of the left / right pupil in each pair of adjacent images in at least a portion of the first / second set of images.

[0099] Alternatively, in some embodiments, detecting the relative positional change of the left / right pupil in at least a portion of the first / second set of images includes detecting the positional change of the left / right pupil in at least a pair of adjacent images in at least a portion of the first / second set of images.

[0100] In some embodiments, for all input images or a selected portion of the input images, a pupil detection model is used to detect the pupil ellipse. If there are two adjacent images whose pupil ellipse center displacement is greater than a first predetermined number of pixels (e.g., 2 pixels), then the first set of images and the second set of images are determined to be invalid. In some embodiments, the first predetermined number of pixels is associated with at least one of the following: the resolution of camera 120 and the distance between camera 120 and the user's left / right eyes.

[0101] According to various embodiments of this disclosure, a method for measuring interpupillary distance (IPD) based on a moving camera is proposed. In the embodiments of this disclosure, the effect of a multi-camera system is achieved by moving the position of a single camera. In this way, the hardware cost required for IPD measurement is reduced, and the accuracy of IPD measurement is improved.

[0102] Figure 5 A block diagram of an apparatus 500 for interpupillary distance measurement according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in a control unit 110. Various modules / components in the apparatus 500 may be implemented by hardware, software, firmware, or any combination thereof.

[0103] As shown in the figure, the device 500 includes a position control module 510, which is configured to set a first camera 120-1 and a second camera 120-2 at a first initial position and a second initial position, respectively. The first camera 120-1 is configured to capture an image of the user's left eye and is associated with a first group of light sources 130-1. The second camera 120-2 is configured to capture an image of the user's right eye and is associated with a second group of light sources 130-2.

[0104] The device 500 further includes a camera control module 520, configured to control the first camera 120-1 and the second camera 120-2 to capture a first set of images and a second set of images of the user's left eye and right eye, respectively, during the process of moving the first camera 120-1 and the second camera 120-2 to a first end position and a second end position, respectively.

[0105] The device 500 also includes a parameter acquisition module 530, configured to acquire a first set of parameters of the first camera 120-1 and the first light source 130-1 corresponding to the first set of images, and a second set of parameters of the second camera 120-2 and the second light source 130-2 corresponding to the second set of images.

[0106] The device 500 also includes an interpupillary distance measurement module 540, configured to acquire the user's interpupillary distance based at least in part on the first set of images, the first set of parameters, the second set of images, and the second set of parameters.

[0107] In some embodiments, the first set of images includes at least a first image captured at a first time point and a second image captured at a second time point, and the second set of images includes at least a third image captured at a third time point and a fourth image captured at a fourth time point.

[0108] In some embodiments, the parameter acquisition module 530 includes: a position determination module configured to determine a first set of positions where the first camera 120-1 is located when capturing the first set of images, and a second set of positions where the second camera 120-2 is located when capturing the second set of images; a first calibration parameter acquisition module configured to acquire a first set of calibration parameters for the first camera 120-1 and the first set of light sources 130-1 for a first position and a second set of calibration parameters for a second position; a second calibration parameter acquisition module configured to acquire a third set of calibration parameters for the second camera 120-2 and the second set of light sources 130-2 for a third position and a fourth set of calibration parameters for a fourth position; a first parameter calculation module configured to calculate the first set of parameters based on the first set of calibration parameters, the second set of calibration parameters, and the first set of positions; and a second parameter calculation module configured to calculate the second set of parameters based on the third set of calibration parameters, the fourth set of calibration parameters, and the second set of positions.

[0109] In some embodiments, the first position is the first initial position, the second position is the first ending position, the third position is the second initial position, and the fourth position is the second ending position.

[0110] In some embodiments, the device 500 further includes: a guide point presentation module configured to present a guide point at a predetermined position on a display device; and a prompting module configured to provide the user with prompting information associated with the guide point, the prompting information instructing the user to keep looking at the guide point.

[0111] In some embodiments, the parameter acquisition module 530 includes: a validity detection module configured to detect the validity of the first group of images and the second group of images; and a parameter acquisition submodule configured to acquire the first group of parameters and the second group of parameters in response to both the first group of images and the second group of images being detected as valid.

[0112] In some embodiments, the validity detection module includes: a pupil detection module configured to detect the user's left eye pupil and right eye pupil respectively in at least a portion of the first set of images and at least a portion of the second set of images according to a pupil detection model; a left eye position change detection module configured to detect a relative position change of the left eye pupil in at least a portion of the first set of images in response to detecting the left eye pupil in at least a portion of the first set of images; and a right eye position change detection module configured to detect a relative position change of the right eye pupil in at least a portion of the second set of images in response to detecting the right eye pupil in at least a portion of the second set of images.

[0113] In some embodiments, the first camera 120-1 is disposed on the left eyeglass tube 140-1 of the virtual reality device, and the second camera 120-2 is disposed on the right eyeglass tube 140-2 of the virtual reality device, wherein the position of the first camera 120-1 is controlled by controlling the position of the left eyeglass tube 140-1, and wherein the position of the second camera 120-2 is controlled by controlling the position of the right eyeglass tube 140-2.

[0114] The units included in device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units may be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 500 may be implemented at least partially by one or more hardware logic components. By way of example and not limitation, exemplary types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-chips (SoCs), complex programmable logic devices (CPLDs), and so on.

[0115] Figure 6 A block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented is shown. Figure 6 The electronic device 600 shown is merely exemplary and does not constitute any limitation on the functionality and scope of the embodiments described herein.

[0116] like Figure 6 As shown, the electronic device / server 600 is in the form of a general-purpose electronic device. Components of the electronic device / server 600 may include, but are not limited to, one or more processors or processing units 610, memory 620, storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 620. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of the electronic device / server 600.

[0117] Electronic device / server 600 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device / server 600, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 620 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 630 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device / server 600.

[0118] The electronic device / server 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 6 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 620 may include computer program product 625 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0119] The communication unit 640 enables communication with other electronic devices via a communication medium. Additionally, the functionality of the components of the electronic device / server 600 can be implemented as a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, the electronic device / server 600 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0120] Input device 650 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 660 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device / server 600 can also communicate with one or more external devices (not shown) via communication unit 640 as needed. These external devices, such as storage devices, display devices, etc., can communicate with one or more devices that enable user interaction with electronic device / server 600, or with any device (e.g., network card, modem, etc.) that enables electronic device / server 600 to communicate with one or more other electronic devices. Such communication can be performed via input / output (I / O) interfaces (not shown).

[0121] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions thereon, wherein one or more computer instructions are executed by a processor to implement the methods described above.

[0122] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0123] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0124] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0126] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A method for measuring interpupillary distance, comprising: A first camera and a second camera are respectively set at a first initial position and a second initial position. The first camera is configured to capture an image of the user's left eye and is associated with a first set of light sources. The second camera is configured to capture an image of the user's right eye and is associated with a second set of light sources. During the process of moving the first camera and the second camera to the first end position and the second end position respectively, the first camera and the second camera are controlled to capture the first set of images and the second set of images of the user's left eye and right eye respectively. Obtain a first set of parameters for the first camera and the first set of light sources corresponding to the first set of images, and a second set of parameters for the second camera and the second set of light sources corresponding to the second set of images; as well as The user's interpupillary distance is obtained at least in part based on the first set of images, the first set of parameters, the second set of images, and the second set of parameters. Obtaining the first set of parameters and the second set of parameters includes: Determine the first group position where the first camera is located when capturing the first group of images, and the second group position where the second camera is located when capturing the second group of images; Obtain a first set of calibration parameters for the first camera and the first set of light sources at the first position, and a second set of calibration parameters for the second position; Obtain the third set of calibration parameters for the second camera and the second set of light sources at the third position, and the fourth set of calibration parameters for the fourth position; Based on the first set of calibration parameters, the second set of calibration parameters, and the first set of positions, calculate the first set of parameters; and The second set of parameters is calculated based on the third set of calibration parameters, the fourth set of calibration parameters, and the second set of positions.

2. The method according to claim 1, wherein the first set of images includes at least a first image captured at a first time point and a second image captured at a second time point, and the second set of images includes at least a third image captured at a third time point and a fourth image captured at a fourth time point.

3. The method according to claim 1, wherein the first position is the first initial position, the second position is the first end position, the third position is the second initial position, and the fourth position is the second end position.

4. The method according to claim 1, further comprising: The guide point is positioned at a predetermined location on the display device; as well as Provide the user with a prompt message associated with the guide point, the prompt message instructing the user to keep looking at the guide point.

5. The method according to claim 1, wherein obtaining the first set of parameters and the second set of parameters comprises: Detect the validity of the first group of images and the second group of images; as well as In response to both the first set of images and the second set of images being detected as valid, the first set of parameters and the second set of parameters are obtained.

6. The method of claim 5, wherein detecting the validity of the first set of images and the second set of images comprises: The user's left and right pupils are detected in at least a portion of the first set of images and at least a portion of the second set of images, respectively, based on the pupil detection model. In response to detecting the left pupil in at least a portion of the first set of images, the relative position change of the left pupil in at least a portion of the first set of images is detected; as well as In response to detecting the right pupil in at least a portion of the second set of images, the relative position change of the right pupil in at least a portion of the second set of images is detected.

7. The method of claim 1, wherein the first camera is disposed on the left eyeglass barrel of the virtual reality device, and the second camera is disposed on the right eyeglass barrel of the virtual reality device. The position of the first camera is controlled by controlling the position of the left eyeglass lens, and The position of the second camera is controlled by controlling the position of the right eyeglass lens.

8. A device for measuring interpupillary distance, comprising: The position control module is configured to set a first camera and a second camera at a first initial position and a second initial position, respectively. The first camera is configured to capture an image of the user's left eye and associate it with a first set of light sources, and the second camera is configured to capture an image of the user's right eye and associate it with a second set of light sources. The camera control module is configured to control the first camera and the second camera to capture a first set of images and a second set of images of the user's left eye and right eye, respectively, while moving the first camera and the second camera to a first end position and a second end position, respectively. The parameter acquisition module is configured to acquire a first set of parameters of the first camera and the first set of light sources corresponding to the first set of images, and a second set of parameters of the second camera and the second set of light sources corresponding to the second set of images; as well as The interpupillary distance measurement module is configured to obtain the user's interpupillary distance based at least in part on the first set of images, the first set of parameters, the second set of images, and the second set of parameters. The parameter acquisition module includes: The position determination module is configured to determine a first set of positions where the first camera is located when capturing the first set of images, and a second set of positions where the second camera is located when capturing the second set of images; The first calibration parameter acquisition module is configured to acquire a first set of calibration parameters for the first camera and the first set of light sources for a first position and a second set of calibration parameters for a second position; The second calibration parameter acquisition module is configured to acquire the third set of calibration parameters of the second camera and the second set of light sources for the third position and the fourth set of calibration parameters for the fourth position; The first parameter calculation module is configured to calculate the first set of parameters based on the first set of calibration parameters, the second set of calibration parameters, and the first set of positions; and The second parameter calculation module is configured to calculate the second set of parameters based on the third set of calibration parameters, the fourth set of calibration parameters, and the second set of positions.

9. The apparatus of claim 8, wherein the first set of images includes at least a first image captured at a first time point and a second image captured at a second time point, and the second set of images includes at least a third image captured at a third time point and a fourth image captured at a fourth time point.

10. The apparatus of claim 8, wherein the first position is the first initial position, the second position is the first end position, the third position is the second initial position, and the fourth position is the second end position.

11. The apparatus of claim 8, further comprising: The guide point presentation module is configured to present the guide point at a predetermined position on the display device; as well as The prompting module is configured to provide the user with a prompt message associated with the guide point, the prompt message instructing the user to keep looking at the guide point.

12. The apparatus according to claim 8, wherein the parameter acquisition module comprises: The validity detection module is configured to detect the validity of the first group of images and the second group of images; as well as The parameter acquisition submodule is configured to acquire the first set of parameters and the second set of parameters in response to both the first set of images and the second set of images being detected as valid.

13. The apparatus of claim 12, wherein the validity detection module comprises: The pupil detection module is configured to detect the user's left pupil and right pupil in at least a portion of the first set of images and at least a portion of the second set of images, respectively, based on a pupil detection model. A left eye position change detection module is configured to detect the relative position change of the left eye pupil in at least a portion of the first set of images in response to detecting the left eye pupil in at least a portion of the first set of images; as well as The right eye position change detection module is configured to detect the relative position change of the right eye pupil in at least a portion of the second set of images in response to detecting the right eye pupil in at least a portion of the second set of images.

14. The apparatus of claim 8, wherein the first camera is disposed on the left eyeglass barrel of the virtual reality device, and the second camera is disposed on the right eyeglass barrel of the virtual reality device. The position of the first camera is controlled by controlling the position of the left eyeglass lens, and The position of the second camera is controlled by controlling the position of the right eyeglass lens.

15. An electronic device comprising: The first camera was configured to capture an image of the user's left eye; A second camera is configured to capture an image of the user's right eye; The first set of light sources is associated with the first camera; The second set of light sources is associated with the second camera; as well as A control unit configured to perform the method according to any one of claims 1 to 7.

16. The electronic device of claim 15, wherein the electronic device is a head-mounted display device.

17. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images