Head-mounted display device calibration method

Abstract

A head-mounted display device calibration method, which belongs to the technical field of extended reality. The head-mounted display device calibration method comprises: controlling an image sensor to collect content displayed by a head-mounted display device, so as to obtain a first image (110); acquiring a displacement vector between a feature point to be tracked in the first image and a center point of the first image, and when the modulus of the displacement vector exceeds a first target threshold value, controlling a simulated eyeball (410) to rotate, so as to adjust the position of the image sensor, and then recapturing the displayed content, so as to refresh the first image; and when the modulus of the displacement vector does not exceed the first target threshold value, controlling the simulated eyeball (410) to stop rotating (120). Further provided are a head-mounted display device calibration apparatus, a head-mounted display device calibration system, a non-transitory computer-readable storage medium, and a computer program product.

Description

Head-mounted display device calibration method

[0001] Cross-references to related applications

[0002] This application is based on and claims priority to Chinese Patent Application No. 2024105151463, filed on April 26, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application relates to the field of extended reality technology, and more specifically, to a head-mounted display device calibration method. Background Technology

[0004] Extended Reality (XR) devices can construct virtual environments and allow users to interact with real-world scenes. In related technologies, eye-tracking testing of XR devices requires an eye-tracking calibration process using a head-mounted display (HMD). This involves randomly displaying target points on the screen and then tracking them with the eyes. However, this method requires pre-determining the coordinates of the tracking points, and different users have varying standards for eye calibration, which can easily lead to errors in the calibration results. Furthermore, eye calibration is time-consuming and inefficient, negatively impacting the subsequent user experience. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a head-mounted display device calibration method that can improve calibration accuracy, reduce errors, and eliminate the need for manual calibration by using a simulation device, thereby improving calibration efficiency.

[0006] In a first aspect, this application provides a head-mounted display (HMD) device calibration method, applied to a HMD device calibration system, the HMD device calibration system including a rotatable simulated eyeball and an image sensor that rotates synchronously with the simulated eyeball; the method includes:

[0007] The image sensor is controlled to acquire the content displayed on the head-mounted display device to obtain a first image;

[0008] The displacement vector between the feature point to be tracked in the first image and the center point of the first image is obtained. When the magnitude of the displacement vector exceeds the first target threshold, the simulated eyeball is controlled to rotate to adjust the position of the image sensor. Then, the display content is recaptured to refresh the first image.

[0009] When the displacement vector magnitude does not exceed the first target threshold, the simulated eyeball is controlled to stop rotating.

[0010] According to the head-mounted display device calibration method of this application, the content displayed on the head-mounted display device is acquired through an image sensor to obtain a first image, effectively determining the position information of the points that need to be calibrated. Based on the distance between the feature point to be tracked in the first image and the center point of the first image, it is effectively determined whether the distance between the two points meets the calibration requirements. If not, the simulated eyeball is controlled to rotate based on the distance between the feature point to be tracked, the center point of the first image, the simulated eyeball relative to the head-mounted display screen, and the angle between the simulated pupil in the simulated eyeball and the image sensor. This method is applicable to various situations, such as the simulated pupil facing the image sensor directly or the simulated pupil having a certain angle relative to the image sensor. The control method is universal, and the distance between the feature point to be tracked and the center point of the first image meets the calibration requirements. Calibration is performed using a simulated human eyeball, and the calibration is judged by a given threshold to determine whether the calibration meets the requirements, effectively determining the calibration effect, improving the accuracy of calibration, reducing errors, and eliminating the need for manual calibration. Calibration is performed through a simulated device, improving calibration efficiency.

[0011] According to the head-mounted display device calibration method of this application, the control of simulated eyeball rotation includes:

[0012] The rotation of the simulated eyeball is controlled based on the displacement vector, the distance between the simulated eyeball and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball and the image sensor.

[0013] According to the head-mounted display device calibration method of this application, controlling the rotation of the simulated eyeball based on the displacement vector, the distance between the simulated eyeball and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball and the image sensor includes:

[0014] Based on the magnitude and direction of the displacement vector, the rotational speed and rotational direction are determined;

[0015] Based on the positional relationship between the simulated pupil and the image sensor, the deflection angle of the simulated eyeball relative to the image sensor is determined;

[0016] The rotation angle is determined based on the magnitude of the displacement vector, the distance, and the deflection angle.

[0017] The simulated eyeball is controlled to rotate based on the rotation speed, the rotation direction, and the rotation angle.

[0018] According to the head-mounted display device calibration method of this application, determining the rotational speed based on the magnitude of the displacement vector includes:

[0019] In the initial control of the simulated eye movement, the rotation speed is determined based on the initial velocity;

[0020] In cases where the simulated eyeball rotation is not controlled for the first time, the rotation speed is determined based on the displacement vector.

[0021] According to the head-mounted display device calibration method of this application, determining the rotational speed based on the magnitude of the displacement vector includes:

[0022] If the model length is not less than the second target threshold, the rotation speed is determined based on the initial speed, the model length, and the initial distance, where the initial distance is the model length in the case of first controlling the rotation of the simulated eyeball;

[0023] If the module length is less than the second target threshold, the rotation speed is determined as the target speed;

[0024] Wherein, the second target threshold is greater than the first target threshold.

[0025] According to the head-mounted display device calibration method of this application, after controlling the simulated eyeball to stop rotating, the method further includes:

[0026] The image sensor is controlled to acquire the content displayed by the head-mounted display device to obtain a second image;

[0027] Perform image recognition on the second image to obtain at least one image feature;

[0028] Control the simulated eye movement to track the target image feature in at least one image feature.

[0029] Secondly, this application provides a head-mounted display device calibration apparatus for use in a head-mounted display device calibration system, the head-mounted display device calibration system including a rotatable simulated eyeball and an image sensor that rotates synchronously with the simulated eyeball, the apparatus comprising:

[0030] The control module is used to control the image sensor to acquire the content displayed by the head-mounted display device to obtain a first image, and to control the movement of the simulated eyeball based on the displacement vector between the feature point to be tracked in the first image and the center point of the first image, the positional relationship of the simulated pupil on the simulated eyeball relative to the image sensor, and the distance between the simulated eyeball and the head-mounted display device.

[0031] According to the head-mounted display calibration device of this application, the content displayed on the head-mounted display device is acquired by an image sensor to obtain a first image, effectively determining the position information of the points that need to be calibrated; based on the distance between the feature point to be tracked in the first image and the center point of the first image, it is effectively determined whether the distance between the two points meets the calibration requirements. If not, the simulated eyeball is controlled to rotate based on the distance between the feature point to be tracked, the center point of the first image, the simulated eyeball relative to the head-mounted display screen, and the angle between the simulated pupil in the simulated eyeball and the image sensor. This method is applicable to various situations, such as the simulated pupil facing the image sensor directly or the simulated pupil having a certain angle relative to the image sensor. The control method is universal, and the distance between the feature point to be tracked and the center point of the first image meets the calibration requirements. Calibration is performed by simulating the human eyeball, and the calibration is judged by a given threshold to determine whether the calibration meets the requirements, effectively determining the calibration effect, improving the accuracy of calibration, reducing errors, and eliminating the need for manual calibration. Calibration is performed by a simulated device, improving calibration efficiency.

[0032] Thirdly, this application provides a head-mounted display device calibration system, comprising:

[0033] Simulates an eyeball;

[0034] An image sensor, located inside the simulated eyeball and capable of rotating synchronously with the simulated eyeball, is used to capture the content displayed by the head-mounted display device through the simulated eyeball to obtain a first image;

[0035] A control device is connected to the simulated eyeball and the image sensor respectively, and receives the first image transmitted by the image sensor. Based on the displacement vector between the feature point to be tracked in the first image and the center point of the first image, the positional relationship of the simulated pupil on the simulated eyeball relative to the image sensor, and the distance between the simulated eyeball and the head-mounted display device, the control device controls the rotation of the simulated eyeball to complete the calibration.

[0036] Fourthly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the head-mounted display device calibration method as described in the first aspect above.

[0037] Fifthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the head-mounted display device calibration method as described in the first aspect above.

[0038] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the head-mounted display device calibration method as described in the first aspect above.

[0039] The above-described one or more technical solutions in the embodiments of this application have at least one of the following technical effects:

[0040] The image sensor captures the content displayed on the head-mounted display to obtain a first image, effectively determining the location information of the points that need calibration. Based on the distance between the feature point to be tracked and the center point of the first image, it effectively determines whether the distance between the two points meets the calibration requirements. If not, the simulated eyeball is controlled to rotate based on the distance between the feature point to be tracked, the center point of the first image, the simulated eyeball relative to the head-mounted display screen, and the angle between the simulated pupil and the image sensor. This method is applicable to various situations, such as the simulated pupil facing the image sensor directly or having a certain angle relative to the image sensor. The control method is universal, and the distance between the feature point to be tracked and the center point of the first image meets the calibration requirements. Calibration is performed using a simulated human eyeball, and the calibration is judged by a given threshold to determine whether the requirements are met, effectively determining the calibration effect, improving calibration accuracy, reducing errors, and eliminating the need for manual calibration. Calibration is performed through a simulated device, improving calibration efficiency.

[0041] Furthermore, by calculating the displacement vector between the feature point to be tracked and the center point of the first image, the direction and magnitude relationship between the feature point to be tracked and the center point of the first image is obtained. Based on the obtained direction and magnitude relationship, the distance between the center of the simulated eyeball and the head-mounted display screen, and the positional relationship between the simulated pupil and the image sensor, the simulated eyeball and the image sensor are controlled to rotate synchronously, so that the distance between the feature point to be tracked and the center point of the first image continuously decreases. This provides data support for calibration, quantifies the calibration process, and improves the scientificity and accuracy of the calibration process.

[0042] Furthermore, by determining whether the controlled simulated eye rotation is the first rotation, different methods of controlling the simulated eye rotation are adopted. In the case of the first controlled simulated eye rotation, the simulated eye rotation is controlled based on the initial velocity; in the case of non-first controlled rotation, the simulated eye rotation is controlled based on the displacement vector. Different rotation speeds are selected based on different situations, thereby improving the flexibility of simulated eye rotation during the calibration process.

[0043] Furthermore, by determining the relationship between the current model length and a given second target threshold, the speed at which the simulated eyeball rotates is determined. Based on different model lengths, different rotation speeds are selected. When the model length is long, the simulated eyeball is controlled to rotate at a faster speed, and when the model length is short, the simulated eyeball is controlled to rotate at a slower speed. This achieves precise control of the simulated eyeball, improves calibration accuracy, and reduces the difficulty of calibrating the simulated eyeball at high speed under various model length conditions.

[0044] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0045] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0046] Figure 1 is one of the flowcharts illustrating the head-mounted display device calibration method provided in this application embodiment;

[0047] Figure 2 is one of the schematic diagrams of the head-mounted display device calibration method provided in the embodiments of this application;

[0048] Figure 3 is a second schematic flowchart of the head-mounted display device calibration method provided in the embodiments of this application;

[0049] Figure 4 is a schematic diagram of the head-mounted display device calibration device provided in an embodiment of this application;

[0050] Figure 5 is a second schematic diagram of the principle of the head-mounted display device calibration method provided in the embodiments of this application;

[0051] Figure 6 is a schematic diagram of the third principle of the head-mounted display device calibration method provided in the embodiments of this application;

[0052] Figure 7 is a schematic diagram of the structure of the electronic device provided in an embodiment of this application. Detailed Implementation

[0053] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0054] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0055] The following description, in conjunction with the accompanying drawings, details the head-mounted display device calibration method, head-mounted display device calibration device, head-mounted display device calibration system, electronic device, and readable storage medium provided in this application through specific embodiments and application scenarios.

[0056] The head-mounted display device calibration method can be applied to a terminal, specifically executed by the hardware or software within the terminal. This terminal includes portable communication devices such as mobile phones or tablets. It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer.

[0057] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0058] The head-mounted display device calibration method provided in this application embodiment can be executed by a head-mounted display device calibration system, or by an electronic device installed on the head-mounted display device calibration system, or by a functional module or functional entity in the electronic device that can implement the head-mounted display device calibration method. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The head-mounted display device calibration method provided in this application embodiment will be described below using an electronic device as the execution subject as an example.

[0059] As shown in Figure 1, the calibration method for this head-mounted display device includes steps 110 and 120.

[0060] As shown in Figure 4, this head-mounted display calibration method is applied to a head-mounted display calibration system.

[0061] The head-mounted display calibration system includes a rotatable simulated eyeball 410 and an image sensor that rotates synchronously with the simulated eyeball 410.

[0062] The head-mounted display device is used to set up the head-mounted display device calibration system.

[0063] Step 110: Control the image sensor to acquire the content displayed on the head-mounted display device to obtain the first image;

[0064] In this step, the image sensor can be a sensor capable of capturing images, such as a camera or webcam.

[0065] The head-mounted display (HMD) is a device that requires calibration. The content displayed on the HMD's screen can be actual objects or points of interest.

[0066] It is understandable that the displayed content may include multiple actual objects or multiple points.

[0067] The first image is the image corresponding to the content displayed on the head-mounted display device.

[0068] In actual operation, when content appears on the screen of the head-mounted display device, the image sensor is controlled to capture the content on the screen to obtain the first image.

[0069] For example, in actual execution, if a calibration point appears on the screen of the head-mounted display, the camera can be used to start capturing the stream and obtain the first image.

[0070] Step 120: Obtain the displacement vector between the feature point to be tracked in the first image and the center point of the first image. When the magnitude of the displacement vector exceeds the first target threshold, control the simulated eyeball 410 to rotate to adjust the position of the image sensor. Then, recapture the display content to refresh the first image. When the magnitude of the displacement vector does not exceed the first target threshold, control the simulated eyeball 410 to stop rotating.

[0071] In this step, the feature points to be tracked are the objects that need to be tracked within the content of the first acquired image. It is understood that if the first image includes multiple actual objects or multiple points, the feature points to be tracked can be any object or point.

[0072] The first image is acquired by an image sensor. Therefore, it can be understood that the center point of the first image is the center point of the image sensor, or in other words, the center point of the first image is on the optical axis of the image sensor. It is also easy to know that when the acquired first image coincides with the corresponding image on the head-mounted display screen, the center point of the first image coincides with the center point of the head-mounted display screen.

[0073] The simulated eyeball 410 and image sensor are used to simulate the human eye. The simulated eyeball 410 is equipped with a simulated pupil.

[0074] The point of intersection between the line of sight of the simulated eyeball 410 and the screen of the head-mounted display device is the fixation point, which is the intersection of the line connecting the center of the simulated eyeball 410 and the simulated pupil with the screen.

[0075] In this application, the image sensor is located in the simulated eyeball 410 and can rotate synchronously with it. It is possible that the image sensor and the simulated pupil are directly opposite each other, that is, the center point of the simulated pupil is located on the optical axis of the image sensor; it is also possible that there is a certain angle or deviation between the image sensor and the simulated pupil, that is, the center point of the simulated pupil is deviated from the optical axis of the image sensor.

[0076] Of course, when the center point of the simulated pupil is located on the optical axis, the light from the head-mounted display can pass through the simulated pupil and enter the image sensor. When there is a deviation between the two, it can be imagined that there is a certain angle or deflection between the simulated pupil and the image sensor. That is, the line connecting the simulated pupil and the center of the simulated eyeball 410 has a certain angle or deflection relative to the optical axis of the image sensor. At this time, after the light from the head-mounted display falls on other corresponding parts of the simulated eyeball 410, these other corresponding parts can guide the content light of the head-mounted display to the image sensor for image formation.

[0077] Obviously, in order to achieve lossless imaging, in practical design, the location of the simulated pupil or other corresponding part should ideally be missing so that the light from the head-mounted display can directly fall onto the incident surface of the image sensor for imaging, as shown in Figure 4. In this case, the angle between the simulated pupil and the image sensor may be as shown in Figure 5.

[0078] It is not difficult to imagine that when the center point of the simulated pupil is located on the optical axis of the image sensor, the optimal design is that the image sensor is located at the exact center of the simulated eyeball 410 (i.e., the center of the simulated eyeball 410). Obviously, when the image sensor is located at the exact center of the simulated eyeball 410, the deflection angle is approximately considered to be 0.

[0079] The first target threshold is a preset value used to determine the distance between the feature point to be tracked and the center point of the first image. The first target threshold can be determined based on the actual calibration accuracy requirements, and this application does not limit it; for example, the first target threshold can be 0.5 mm or 0.6 mm.

[0080] The first image is updated to an image based on the simulated eyeball rotation 410 after rotation, showing the new distance between the feature point to be tracked and the center point of the first image.

[0081] Understandably, when the optical engine of the head-mounted display is lit, by rotating the simulated eyeball 410, the image sensor's shooting surface can clearly and completely capture the image inside the head-mounted display. The image sensor, which rotates with the simulated eyeball 410, can directly capture image information from different locations within the head-mounted display.

[0082] In actual execution, taking the image sensor located at the center of the simulated eyeball 410 as an example, after obtaining the first image, the distance between the center of the feature point to be tracked and the center point of the first image can be calculated. If the distance between the feature point to be tracked and the center point of the first image is greater than the first target threshold, based on the displacement vector between the feature point to be tracked and the center point of the first image and the distance between the simulated eyeball 410 and the display screen, the image sensor is controlled to rotate synchronously together with the simulated eyeball 410. After rotation, the image sensor again collects the content displayed by the head-mounted display device transmitted through the simulated eyeball 410 to obtain an updated first image; the distance between the center of the feature point to be tracked and the center point of the first image in the updated first image will be updated accordingly.

[0083] Taking the example where the image sensor is not located at the exact center of the simulated pupil, after obtaining the first image, the distance between the center of the feature point to be tracked and the center point of the first image can be calculated. If the distance between the feature point to be tracked and the center point of the first image is greater than the first target threshold, based on the displacement vector between the feature point to be tracked and the center point of the first image, the distance between the simulated eyeball 410 and the display screen, and the angle between the simulated pupil and the image sensor, the simulated eyeball 410 and the image sensor are controlled to rotate. After rotation, the image sensor again collects the content displayed by the head-mounted display device transmitted from the simulated eyeball 410, and obtains an updated first image. The distance between the center of the feature point to be tracked and the center point of the first image in the updated first image will be updated accordingly.

[0084] In actual execution, when the image sensor is not installed in the simulated pupil of the simulated eyeball 410, the relative rotation angle (i.e., the deflection angle) between the center of the simulated pupil and the installation position of the image sensor needs to be recorded in advance. The aforementioned tracking scheme remains unchanged. After the tracking is completed, the image sensor is facing the feature point to be tracked. At this time, the pre-recorded relative rotation angle needs to be rotated so that the simulated pupil is facing the feature point to be tracked, and the eye tracking is judged to be completed.

[0085] After the camera completes tracking, it needs to be rotated horizontally by α degrees as shown in Figure 5 and tilted by β degrees as shown in Figure 6 to align the simulated pupil center with the feature point to be tracked.

[0086] Obviously, the angle between the simulated pupil and the image sensor can be determined by the positional relationship between the line formed by the center point of the simulated pupil and the center of the simulated eyeball 410 and the optical axis of the image sensor; the distance between the simulated eyeball 410 and the display screen of the head-mounted display device can be obtained by the distance between the center of the simulated eyeball 410 and the center of the display screen.

[0087] When the displacement vector magnitude does not exceed the first target threshold, the simulated eyeball 410 stops rotating. In actual execution, after obtaining the first image, the distance between the center of the feature point to be tracked and the center point of the first image can be calculated. If the distance between the feature point to be tracked and the center point of the first image is not greater than the first target threshold, the simulated eyeball 410 can stop rotating.

[0088] It is understandable that the distance between the center of the feature point to be tracked and the center point of the first image obtained in the first acquisition may not be greater than the first target threshold. In this case, there is no need to control the simulated eyeball 410 to stop rotating.

[0089] In actual execution, in the first image acquired for the first time, if the distance between the center of the feature point to be tracked and the center point of the first image is greater than the first target threshold, the simulated eyeball 410 is controlled to rotate based on the feature point to be tracked and the center point of the first image. After the simulated eyeball 410 rotates, a new first image is obtained based on the new first image determined by the feature point to be tracked and the center point of the first image. In the updated first image, if the distance between the center of the feature point to be tracked and the center point of the first image is not greater than the first target threshold, the simulated eyeball 410 is controlled to stop rotating.

[0090] In this way, the human eye simulation device can be controlled by acquiring images from the image acquisition module in real time. The movement of the human eye simulation device will change the feedback image, and the calibration of different eye movement points can be completed through continuous real-time control.

[0091] In actual execution, the distance data between the feature point to be tracked in the first image and the center point of the first image, as well as the deflection angle between the simulated pupil and the image sensor, are acquired in real time. By combining the acquired distance data and deflection angle with the distance between the simulated eyeball and the head-mounted display screen, the simulated eyeball 410 is controlled to rotate in real time, so that the distance between the feature point to be tracked in the first image and the center point of the first image continuously decreases, thereby completing the calibration.

[0092] In actual execution, a head-mounted display is worn on the human eye simulation device, and the incident light from the simulated pupil directly enters the camera. The camera uploads the real-time image to the host computer. Then, the eye-tracking calibration stage begins. The host computer controls the motion control parameters of the human eye simulation device based on the relative relationship between the point to be tracked in the camera image and the center point of the first image, the distance between the simulated eyeball 410 and the head-mounted display screen, and the angle between the simulated pupil and the image sensor. This allows the center point of the first image to approach the point to be tracked until they coincide (considered as coincidence if it is not greater than the first target threshold), as shown in Figure 2. The eye-tracking calibration stage is continuously performed until the calibration of all points is completed.

[0093] According to the head-mounted display device calibration method provided in this application embodiment, the content displayed by the head-mounted display device is collected by an image sensor to obtain a first image, effectively determining the position information of the points that need to be calibrated; based on the distance between the feature point to be tracked in the first image and the center point of the first image, it is effectively determined whether the distance between the two points meets the calibration requirements. If not, based on the distance between the feature point to be tracked, the center point of the first image, the simulated eyeball 410 relative to the head-mounted display screen, and the angle between the simulated pupil in the simulated eyeball 410 and the image sensor, the simulated eyeball 410 is controlled to rotate. This method is applicable to various situations, such as the simulated pupil facing the image sensor directly or the simulated pupil having a certain angle relative to the image sensor. The control method is universal, and the distance between the feature point to be tracked and the center point of the first image meets the calibration requirements. Calibration is performed by simulating the human eye with the simulated eyeball 410, and the calibration is judged by a given threshold to determine whether the calibration meets the requirements, effectively determining the calibration effect, improving the accuracy of calibration, reducing errors, and eliminating the need for manual calibration. Calibration is performed by a simulated device, improving calibration efficiency.

[0094] In some embodiments, controlling the rotation of the simulated eyeball 410 may further include:

[0095] The rotation of the simulated eyeball 410 is controlled based on the displacement vector, the distance between the simulated eyeball 410 and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball 410 and the image sensor.

[0096] In this embodiment, the displacement vector is a vector representing the directional and distance relationship between the feature point to be tracked and the center point of the first image.

[0097] Displacement vectors include direction and magnitude.

[0098] The direction refers to the movement direction from the center point of the first image to the feature point to be tracked.

[0099] The size is the distance between the center point of the first image and the feature point to be tracked, i.e., the modulus.

[0100] The distance is the distance between the screen of the head-mounted display device and the center of the simulated eyeball 410.

[0101] The positional relationship between the simulated pupil on the simulated eyeball 410 and the image sensor is such that there is a certain angle or deflection between the image sensor and the simulated pupil.

[0102] In practice, after obtaining the first image, based on the position information of the feature point to be tracked and the center point of the first image, the displacement vector between the feature point to be tracked and the center point of the first image is calculated. The direction of movement from the center point of the first image to the feature point to be tracked and the distance between the center point of the first image and the feature point to be tracked are determined. Based on the direction and magnitude of the obtained displacement vector, the distance between the center of the simulated eyeball 410 and the head-mounted display screen, and the positional relationship between the simulated pupil and the image sensor, the simulated eyeball 410 and the image sensor are controlled to rotate synchronously, thereby performing calibration and continuously reducing the distance between the center point of the first image and the feature point to be tracked.

[0103] According to the head-mounted display device calibration method provided in the embodiments of this application, the displacement vector between the feature point to be tracked and the center point of the first image is calculated to obtain the direction and magnitude relationship between the feature point to be tracked and the center point of the first image. Based on the obtained direction and magnitude relationship, the distance between the center of the simulated eyeball 410 and the head-mounted display screen, and the positional relationship between the simulated pupil and the image sensor, the simulated eyeball 410 and the image sensor are controlled to rotate synchronously, so that the distance between the feature point to be tracked and the center point of the first image continuously decreases, providing data support for calibration, quantifying the calibration process, and improving the scientificity and accuracy of the calibration process.

[0104] In some embodiments, controlling the rotation of the simulated eyeball 410 based on the displacement vector, the distance between the simulated eyeball 410 and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball 410 and the image sensor may further include:

[0105] Based on the magnitude and direction of the displacement vector, the rotational speed and direction of rotation are determined.

[0106] Based on the positional relationship between the simulated pupil and the image sensor, the deflection angle of the simulated eyeball 410 relative to the image sensor is determined;

[0107] The rotation angle is determined based on the magnitude, distance, and deflection angle of the displacement vector;

[0108] The rotation of the simulated eyeball 410 is controlled based on the rotation speed, rotation direction, and rotation angle.

[0109] In this embodiment, the magnitude of the displacement vector is the distance between the center point of the first image and the feature point to be tracked.

[0110] The rotation speed is the speed at which the simulated eyeball 410 rotates to achieve the calibration effect.

[0111] The rotation direction is the same as the direction the simulated eyeball 410 rotates to achieve the calibration effect.

[0112] The rotation angle is the angle that the simulated eyeball rotates to achieve the calibration effect.

[0113] The distance is the distance between the screen of the head-mounted display device and the center of the simulated eyeball 410.

[0114] It is understandable that after calculating the magnitude, distance, and deflection angle of the displacement vector, the rotation angle of the simulated eyeball 410 can be calculated through the relationship between the three, thereby controlling the rotation of the simulated eyeball 410 based on the rotation angle.

[0115] In actual execution, after calculating the displacement vector, the rotation speed of the simulated eyeball 410 is determined by the magnitude of the displacement vector; the rotation direction of the simulated eyeball 410 is determined by the direction of the displacement vector; and the rotation angle of the simulated eyeball 410 is determined by the magnitude of the displacement vector, the distance between the simulated pupil and the screen of the head-mounted display device, and the deflection angle of the simulated eyeball 410 relative to the image sensor.

[0116] Understandably, different speeds can be selected based on the magnitude of the displacement vector. For example, a larger magnitude of the displacement vector can be used to select a larger rotational speed, while a smaller magnitude of the displacement vector can be used to select a smaller rotational speed.

[0117] Of course, in actual execution, the rotation speed of the simulated eyeball 410 can also be determined by other methods.

[0118] According to the head-mounted display device calibration method provided in the embodiments of this application, the rotation speed of the simulated eyeball 410 is determined by the magnitude of the displacement vector, the rotation direction of the simulated eyeball 410 is determined by the direction of the displacement vector, and the rotation angle of the simulated eyeball 410 is determined by the magnitude of the displacement vector, the distance between the simulated eyeball 410 and the screen of the head-mounted display device, and the deflection angle between the simulated pupil and the image sensor. Thus, based on the rotation speed, rotation direction, and rotation angle, the rotation of the simulated eyeball 410 is precisely controlled, improving the calibration accuracy during the calibration process.

[0119] In some embodiments, determining the rotational speed based on the magnitude of the displacement vector may further include:

[0120] In the initial control of the simulated eyeball 410 rotation, the rotation speed is determined based on the initial velocity;

[0121] In cases where the simulated eyeball 410 is rotated for the first time, the rotation speed is determined based on the displacement vector.

[0122] In this embodiment, the first control of the simulated eyeball 410 rotation is based on the acquired first image. The first determination is that the distance between the feature point to be tracked and the center point of the first image is greater than the first target threshold, thereby controlling the rotation of the simulated eyeball 410.

[0123] The initial velocity is the initial velocity at which the simulated eyeball 410 is first controlled to rotate. The initial velocity can be determined based on the actual situation, and this application does not limit it; for example, the initial velocity can be 60° / s or 65° / s.

[0124] The non-first-time control of the simulated eyeball 410 rotation is based on the updated first image, determining that the distance between the feature point to be tracked and the center point of the first image is still greater than the first target threshold, thereby controlling the rotation of the simulated eyeball 410.

[0125] In actual execution, after obtaining the first image, it is determined that the distance between the feature point to be tracked and the center point of the first image is greater than the first target threshold. The simulated eyeball 410 is controlled to rotate for the first time. During the process of controlling the rotation of the simulated eyeball 410, the initial speed is determined as the rotation speed of the simulated eyeball 410. In subsequent calibration, the rotation of the simulated eyeball 410 is controlled based on the initial speed, rotation direction and rotation angle.

[0126] After the simulated eyeball 410 is rotated for the first time, an image of the simulated eyeball 410 after rotation is acquired to obtain an updated first image. Based on the updated first image, if the distance between the feature point to be tracked and the center point of the first image is still greater than the first target threshold, the simulated eyeball 410 is rotated again based on the displacement vector.

[0127] Understandably, when controlling the rotation of the simulated eyeball 410 for the first time, the rotation speed of the simulated eyeball 410 can be determined based on the actual situation. That is, different rotation speeds can be selected based on the specific value of the magnitude of the displacement vector.

[0128] According to the head-mounted display device calibration method provided in the embodiments of this application, by determining whether the control of the simulated eyeball 410 rotation is the first rotation, different methods of controlling the rotation of the simulated eyeball 410 are adopted. In the case of the first control of the simulated eyeball 410 rotation, the rotation of the simulated eyeball 410 is controlled based on the initial speed; in the case of non-first control, the rotation of the simulated eyeball 410 is controlled based on the displacement vector. Different rotation speeds are selected based on different situations, thereby improving the flexibility of the simulated eyeball 410 rotation during the calibration process.

[0129] In some embodiments, determining the rotational speed based on the magnitude of the displacement vector may further include:

[0130] If the mold length is not less than the second target threshold, the rotation speed is determined based on the initial velocity, mold length, and initial distance; if the mold length is less than the second target threshold, the rotation speed is determined as the target velocity.

[0131] The second target threshold is greater than the first target threshold. In this embodiment, the second target threshold is a preset value for determining the rotation speed of the simulated eyeball 410.

[0132] The second target threshold can be determined based on the actual situation, and this application does not limit it; for example, the second target threshold can be 5mm or 6mm.

[0133] The initial distance is the magnitude of the displacement vector between the center point of the first image, determined based on the first acquired first image, and the feature point to be tracked.

[0134] In actual execution, provided that the mold length is not less than the second target threshold, the rotation speed is determined based on the initial velocity, mold length, and initial distance.

[0135] The rotational speed can be determined based on the following formula: Where v0 is the initial velocity, A is the current module length, and A0 is the initial module length.

[0136] For example, when the second target threshold is 5mm and the current model length is 10mm, the rotation speed of the simulated eyeball 410 is determined based on the above formula.

[0137] The target velocity is the simulated eyeball rotation speed when the magnitude between the center point of the first image and the feature point to be tracked is less than the second target threshold. The target velocity can be the rotation speed in a low-frequency mode.

[0138] The target speed can be determined based on the actual situation, and this application does not limit it; for example, the target speed can be 1° / s or 2° / s.

[0139] In actual execution, with the second target threshold being 5mm and the current mold length being 3mm, the rotation speed can be determined to be 1° / s.

[0140] As shown in Figure 3, after obtaining the first image, the line vector X0 connecting the feature point to be tracked and the center point of the first image is obtained. Then, the displacement vector of the line vector X0 is calculated to obtain the initial magnitude A0 of X0. The simulated eyeball 410 is controlled to move in the X0 direction for the first time with an initial velocity of 60° / s.

[0141] After the initial control of the simulated eyeball 410 movement, the image sensor after rotation is controlled to collect the content on the screen of the head-mounted display device and update the first image; if it is determined based on the updated first image that the distance between the feature point to be tracked and the center point of the first image is greater than the first target threshold, the current magnitude A of the line vector X0 connecting the feature point to be tracked and the center point of the first image after rotation is calculated.

[0142] With the current model length not less than the second target threshold (i.e., 5mm), the simulated eyeball 410 is controlled to... It moves at a speed in the X0 direction.

[0143] When the current model length is less than the second target threshold (i.e., 5 mm) but not less than the first target threshold, control the simulated eyeball 410 to move in the X0 direction at a speed of 1° / s.

[0144] The calibration is completed when the current modulus length is less than the first target threshold.

[0145] It is understandable that by controlling the rotation of the simulated eyeball 410 as described above, a calibration process with a shorter distance between the feature point to be tracked and the center point of the first image can be achieved, thereby improving the accuracy of the calibration.

[0146] According to the head-mounted display device calibration method provided in the embodiments of this application, the speed at which the simulated eyeball 410 is controlled to rotate is determined by determining the relationship between the current mold length and a given second target threshold. Based on different mold lengths, different rotation speeds are selected. When the mold length is long, the simulated eyeball 410 is controlled to rotate at a faster speed, and when the mold length is short, the simulated eyeball 410 is controlled to rotate at a slower speed. This achieves precise control of the simulated eyeball 410, improves calibration accuracy, and reduces the difficulty of calibrating the simulated eyeball 410 at high speed under various mold length conditions.

[0147] In some embodiments, after step 130, the method may further include:

[0148] The image sensor is controlled to capture the content displayed on the head-mounted display device to obtain a second image;

[0149] Perform image recognition on the second image to obtain at least one image feature;

[0150] Control the rotation of the simulated eyeball 410 to track at least one target image feature in an image feature.

[0151] In this embodiment, the second image is the image corresponding to the content displayed on the screen of the head-mounted display device after calibration.

[0152] The second image may include multiple features such as flowers, birds, fish, insects, and triangles. The at least one image feature may be a feature of one or more objects included in the second image. The at least one image feature may also be the edge contours of the objects displayed in the second image.

[0153] In actual execution, the second image can be input into the image recognition algorithm to recognize the second image and obtain multiple image features.

[0154] The target image feature is one of at least one image features that has a tracking requirement. For example, the target image feature could be a triangle.

[0155] In actual execution, after calibrating the head-mounted display device, the image sensor can be controlled to collect the content displayed on the head-mounted display device to obtain a second image. Then, the second image is identified by a computer vision recognition algorithm to obtain multiple image features. From the multiple image features, the target image feature to be tracked is determined, and the simulated eyeball 410 is controlled to rotate to track the target image feature.

[0156] For example, in actual tracking, an image recognition algorithm identifies triangles from multiple image features, and these triangles are used as the target to be tracked. The specific tracking method is similar to the calibration process described above, and will not be repeated here.

[0157] According to the head-mounted display device calibration method provided in the embodiments of this application, by setting an image sensor in the simulated eyeball 410, a second image including multiple different features can be acquired; on this basis, an image recognition algorithm is set to identify each feature in the second image, which can realize further specific tracking based on the object to be tracked after calibration, making the function more powerful; and can realize accurate tracking of the tracking target, improve the tracking accuracy, and reduce tracking error.

[0158] The head-mounted display (HMD) calibration method provided in this application can be executed by a HMD calibration device. This application uses an HMD calibration device executing the calibration method as an example to illustrate the HMD calibration device provided in this application.

[0159] This application also provides a head-mounted display device calibration apparatus.

[0160] Head-mounted display (HMD) calibration devices are used in head-mounted display (HMD) calibration systems.

[0161] The head-mounted display calibration system includes a rotatable simulated eyeball 410 and an image sensor that rotates synchronously with the simulated eyeball 410.

[0162] Head-mounted display devices are used to set up the head-mounted display device calibration system.

[0163] The head-mounted display calibration device includes: a control module.

[0164] The control module is used to control the image sensor to acquire the content displayed by the head-mounted display device to obtain a first image, and to control the movement of the simulated eyeball 410 based on the displacement vector between the feature point to be tracked in the first image and the center point of the first image, the positional relationship of the simulated pupil on the simulated eyeball 410 relative to the image sensor, and the distance between the simulated eyeball 410 and the head-mounted display device.

[0165] According to the head-mounted display calibration device provided in this application embodiment, the device acquires the content displayed on the head-mounted display device through an image sensor to obtain a first image, effectively determining the position information of the points that need to be calibrated. Based on the distance between the feature point to be tracked in the first image and the center point of the first image, it effectively determines whether the distance between the two points meets the calibration requirements. If not, based on the distance between the feature point to be tracked, the center point of the first image, the simulated eyeball 410 relative to the head-mounted display screen, and the angle between the simulated pupil in the simulated eyeball 410 and the image sensor, the device controls the rotation of the simulated eyeball 410. This method is applicable to various situations, such as the simulated pupil facing the image sensor directly or there being a certain angle between the simulated pupil and the image sensor. The control method is universal, and the distance between the feature point to be tracked and the center point of the first image meets the calibration requirements. Calibration is performed through the simulated eyeball 410 that simulates the human eye, and the calibration is judged by a given threshold to determine whether the calibration meets the requirements, effectively determining the calibration effect, improving the accuracy of calibration, reducing errors, and eliminating the need for manual calibration. Calibration is performed through a simulated device, improving calibration efficiency.

[0166] In some embodiments, the control module can also be used for:

[0167] The rotation of the simulated eyeball 410 is controlled based on the displacement vector, the distance between the simulated eyeball 410 and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball 410 and the image sensor.

[0168] In some embodiments, the control module of the simulated eyeball 410 can also be used for:

[0169] Based on the magnitude and direction of the displacement vector, the rotational speed and direction of rotation are determined.

[0170] Based on the positional relationship between the simulated pupil and the image sensor, the deflection angle of the simulated eyeball 410 relative to the image sensor is determined;

[0171] The rotation angle is determined based on the magnitude, distance, and deflection angle of the displacement vector;

[0172] The rotation of the simulated eyeball 410 is controlled based on the rotation speed, rotation direction, and rotation angle.

[0173] In some embodiments, the control module of the simulated eyeball 410 can also be used for:

[0174] In the initial control of the simulated eyeball 410 rotation, the rotation speed is determined based on the initial velocity;

[0175] In cases where the simulated eyeball 410 is rotated for the first time, the rotation speed is determined based on the displacement vector.

[0176] Simulated eyeball 410 In some embodiments, the control module may also be used for:

[0177] If the model length is not less than the second target threshold, the rotation speed is determined based on the initial speed, model length and initial distance. The initial distance is the model length when the simulated eyeball 410 is rotated for the first time.

[0178] If the module length is less than the second target threshold, the rotational speed is determined as the target speed;

[0179] The second target threshold is greater than the first target threshold.

[0180] In some embodiments, the device may further include a tracking module for: (simulating an eyeball 410)

[0181] The image sensor is controlled to capture the content displayed on the head-mounted display device to obtain a second image;

[0182] Perform image recognition on the second image to obtain at least one image feature;

[0183] Control the rotation of the simulated eyeball 410 to track at least one target image feature in an image feature.

[0184] The head-mounted display device calibration device in this embodiment of the application can be a head-mounted display device calibration system, or an electronic device connected to the head-mounted display device calibration system, or a component in the head-mounted display device calibration system or electronic device, such as an integrated circuit or chip. The electronic device can be a terminal, or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc., and can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment of the application does not specifically limit the specific device.

[0185] The head-mounted display device calibration device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0186] The head-mounted display device calibration apparatus provided in this application embodiment can realize the various processes implemented in the method embodiments of Figures 1 to 6. To avoid repetition, it will not be described again here.

[0187] As shown in Figure 4, this application embodiment also provides a head-mounted display device calibration system based on the head-mounted display device calibration method described in any of the above embodiments.

[0188] The head-mounted display calibration system includes: a simulated eyeball 410, an image sensor, and a control device.

[0189] In this embodiment,

[0190] The simulated eyeball 410 is positioned behind the head-mounted display device.

[0191] The image sensor is located inside the simulated eyeball 410 and moves accordingly with the simulated eyeball 410.

[0192] An image sensor is used to capture the content displayed through the head-mounted display device 410 to obtain a first image.

[0193] The control device is connected to the simulated eyeball 410 and the image sensor respectively.

[0194] The control device receives the first image transmitted by the image sensor, and controls the rotation of the simulated eyeball 410 based on the displacement vector between the feature point to be tracked in the first image and the center point of the first image, the positional relationship of the simulated pupil on the simulated eyeball 410 relative to the image sensor, and the distance between the simulated eyeball 410 and the head-mounted display device, in order to complete the calibration.

[0195] According to the head-mounted display device calibration system provided in this application embodiment, the content displayed by the head-mounted display device is collected by an image sensor to obtain a first image, effectively determining the position information of the points that need to be calibrated; based on the distance between the feature point to be tracked in the first image and the center point of the first image, it is effectively determined whether the distance between the two points meets the calibration requirements. If not, based on the feature point to be tracked, the center point of the first image, and the angle between the simulated pupil in the simulated eyeball 410 and the image sensor, the simulated eyeball 410 is controlled to rotate. This is applicable to various situations such as the image sensor and the simulated pupil being directly opposite each other and there being a certain angle between the image sensor and the simulated pupil. The control method is universal, and the distance between the feature point to be tracked and the center point of the first image meets the calibration requirements. Calibration is performed by the simulated eyeball 410 that simulates the human eye, and the calibration is judged by a given threshold to determine whether the calibration meets the requirements, effectively determining the calibration effect, improving the accuracy of calibration, reducing errors, and eliminating the need for manual calibration. Calibration is performed by a simulated device, improving calibration efficiency.

[0196] In some embodiments, as shown in FIG7, this application embodiment also provides an electronic device 700, including a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701. When the program is executed by the processor 701, it implements the various processes of the above-described head-mounted display device calibration method embodiment and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0197] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0198] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described head-mounted display device calibration method embodiments and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0199] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0200] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described head-mounted display device calibration method.

[0201] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0202] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described head-mounted display device calibration method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0203] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0204] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0205] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0206] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0207] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0208] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for calibrating a head-mounted display device, characterized in that, A method for use in a head-mounted display calibration system, the head-mounted display calibration system including a rotatable simulated eyeball and an image sensor that rotates synchronously with the simulated eyeball, the method comprising: The image sensor is controlled to acquire the content displayed on the head-mounted display device to obtain a first image; The displacement vector between the feature point to be tracked in the first image and the center point of the first image is obtained. When the magnitude of the displacement vector exceeds the first target threshold, the simulated eyeball is controlled to rotate to adjust the position of the image sensor. Then, the display content is recaptured to refresh the first image. When the displacement vector magnitude does not exceed the first target threshold, the simulated eyeball is controlled to stop rotating.

2. The head-mounted display device calibration method according to claim 1, characterized in that, The control of simulated eye movement includes: The rotation of the simulated eyeball is controlled based on the displacement vector, the distance between the simulated eyeball and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball and the image sensor.

3. The head-mounted display device calibration method according to claim 2, characterized in that, The method of controlling the rotation of the simulated eyeball based on the displacement vector, the distance between the simulated eyeball and the head-mounted display device, and the positional relationship between the simulated pupil on the simulated eyeball and the image sensor includes: Based on the magnitude and direction of the displacement vector, the rotational speed and rotational direction are determined; Based on the positional relationship between the simulated pupil and the image sensor, the deflection angle of the simulated eyeball relative to the image sensor is determined; The rotation angle is determined based on the magnitude of the displacement vector, the distance, and the deflection angle. The simulated eyeball is controlled to rotate based on the rotation speed, the rotation direction, and the rotation angle.

4. The head-mounted display device calibration method according to claim 3, characterized in that, Determining the rotational speed based on the magnitude of the displacement vector includes: In the initial control of the simulated eye movement, the rotation speed is determined based on the initial velocity; In cases where the simulated eyeball rotation is not controlled for the first time, the rotation speed is determined based on the displacement vector.

5. The head-mounted display device calibration method according to claim 4, characterized in that, Determining the rotational speed based on the magnitude of the displacement vector includes: If the model length is not less than the second target threshold, the rotation speed is determined based on the initial speed, the model length, and the initial distance, where the initial distance is the model length in the case of first controlling the rotation of the simulated eyeball; If the module length is less than the second target threshold, the rotation speed is determined as the target speed; Wherein, the second target threshold is greater than the first target threshold.

6. The head-mounted display device calibration method according to any one of claims 1-5, characterized in that, After controlling the simulated eyeball to stop rotating, the method further includes: The image sensor is controlled to acquire the content displayed by the head-mounted display device to obtain a second image; Perform image recognition on the second image to obtain at least one image feature; Control the simulated eye movement to track the target image feature in at least one image feature.

7. A head-mounted display device calibration device, characterized in that, An apparatus for use in a head-mounted display calibration system, the head-mounted display calibration system including a rotatable simulated eyeball and an image sensor that rotates synchronously with the simulated eyeball, the apparatus comprising: The control module is used to control the image sensor to acquire the content displayed by the head-mounted display device to obtain a first image, and to control the movement of the simulated eyeball based on the displacement vector between the feature point to be tracked in the first image and the center point of the first image, the positional relationship of the simulated pupil on the simulated eyeball relative to the image sensor, and the distance between the simulated eyeball and the head-mounted display device.

8. A head-mounted display device calibration system, characterized in that, include: Simulates an eyeball; An image sensor, located inside the simulated eyeball and capable of rotating synchronously with the simulated eyeball, is used to capture the content displayed by the head-mounted display device through the simulated eyeball to obtain a first image; A control device is connected to the simulated eyeball and the image sensor respectively, and receives the first image transmitted by the image sensor. Based on the displacement vector between the feature point to be tracked in the first image and the center point of the first image, the positional relationship of the simulated pupil on the simulated eyeball relative to the image sensor, and the distance between the simulated eyeball and the head-mounted display device, the control device controls the rotation of the simulated eyeball to complete the calibration.

9. The head-mounted display device calibration system according to claim 8, characterized in that, The image sensor is positioned directly opposite the simulated pupil.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the head-mounted display device calibration method as described in any one of claims 1-6.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the head-mounted display device calibration method as described in any one of claims 1-6.

Images