Calibration method and device of near-to-eye display equipment, medium and near-to-eye display equipment

Abstract

The embodiment of the invention provides a calibration method and device of near-to-eye display equipment, a medium and the near-to-eye display equipment. The calibration method of the near-to-eye display equipment comprises the following steps: when an attitude parameter between an optical machine and a camera changes, acquiring a current attitude parameter of the camera; forming a current test point projected by the camera based on the current attitude parameter of the camera; and if the current test point projected by the camera and the actual display point projected by the ray machine are staggered, on-line calibration between the ray machine and the camera is triggered. In the technical scheme of the embodiment of the invention, if the current test point projected by the camera and the actual display point projected by the light machine are staggered, the calibration operation between the light machine and the camera is triggered, so that the calibration between the light machine and the camera is completed based on the adjustment of the current test point projected by the camera and the actual display point projected by the light machine; and the automatic calibration of the equipment after leaving the factory is realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a calibration method, device, medium and near-eye display device for a near-eye display device. Background Art

[0002] Near-eye display devices are gradually being used in people's lives. Smart glasses are also a type of near-eye display devices. Smart glasses are equipped with an optical machine and a camera. When the smart glasses are subjected to external impact, the posture parameters between the optical machine and the camera change, and the posture parameters cannot be calibrated after the equipment leaves the factory. The smart glasses need to be returned to the factory for repair, resulting in high after-sales costs for the smart glasses. Summary of the invention

[0003] The embodiments of the present application provide a near-eye display device calibration method, device, medium and near-eye display device, which to a certain extent realize the autonomous calibration of the device after leaving the factory, eliminating the need for the device to be returned to the factory for repair, thereby reducing the after-sales cost of the equipment.

[0004] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0005] According to one aspect of an embodiment of the present application, a calibration method for a near-eye display device is provided. The calibration method for a near-eye display device is applied to a near-eye display device, wherein the near-eye display device includes an optical machine and a camera, and the camera is disposed on one side of the optical machine. The calibration method for the near-eye display device includes:

[0006] If the posture parameters between the optical machine and the camera change, the current posture parameters of the camera are obtained;

[0007] Forming a current test point projected by the camera based on the current posture parameters of the camera;

[0008] If the current test point projected by the camera is misaligned with the actual display point projected by the optical machine, a calibration operation between the optical machine and the camera is triggered.

[0009] In some embodiments of the present application, if the current test point projected by the camera is misaligned with the actual display point projected by the optical machine, a calibration operation between the optical machine and the camera is triggered, including:

[0010] Get the coordinates of the current test point projected by the camera and the coordinates of the actual display point projected by the optical machine;

[0011] If the coordinates of the current test point projected by the camera are inconsistent with the coordinates of the actual display point projected by the optical machine, the calibration operation between the optical machine and the camera is triggered, wherein the inconsistency between the coordinates of the current test point projected by the camera and the coordinates of the actual display point projected by the optical machine indicates that the current test point projected by the camera and the actual display point projected by the optical machine are misaligned.

[0012] In some embodiments of the present application, the calibration operation between the triggering optical machine and the camera includes:

[0013] If the near-eye display device is in 3DoF mode, obtain the rotation parameters of the near-eye display device;

[0014] The position of the current test point projected by the camera is adjusted based on the rotation parameters of the near-eye display device to move the current test point projected by the camera to the actual display point projected by the optical machine, so that the current test point projected by the camera coincides with the actual display point projected by the optical machine to complete the calibration operation between the optical machine and the camera.

[0015] In some embodiments of the present application, the rotation parameter of the near-eye display device is the angle difference between a current test point projected by a camera and an actual test point projected by an optical machine in the same projection space.

[0016] In some embodiments of the present application, obtaining the rotation parameters of the near-eye display device includes:

[0017] If the near-eye display device is in a rotating state, obtaining the angular velocity parameters of the gyroscope;

[0018] The angular velocity parameters of the gyroscope are integrated to determine the angle difference between the current test point projected by the camera and the actual test point projected by the optical machine.

[0019] In some embodiments of the present application, adjusting the position of the current test point projected by the camera based on the rotation parameter of the near-eye display device includes:

[0020] Determine the rotation angle of each direction axis in the projection space of the optical machine according to the rotation parameters of the near-eye display device;

[0021] The position of the current test point projected by the camera is adjusted based on the rotation angles of each direction axis.

[0022] In some embodiments of the present application, forming a current test point projected by the camera based on the current posture parameter of the camera includes:

[0023] Get the scene picture taken by the camera;

[0024] Perform deep learning on the scene pictures taken by the camera and form a depth map;

[0025] Determine a plurality of feature points based on the depth map;

[0026] Project multiple feature points into the projection space of the optical machine and form the current test point projected by the camera.

[0027] In some embodiments of the present application, when the posture parameters between the optical machine and the camera change, obtaining the current posture parameters of the camera includes:

[0028] Monitor the posture parameters between the optical machine and the camera;

[0029] When the posture parameters between the optical machine and the camera change, the current posture parameters of the camera are obtained, and the changed camera is adapted based on the optical machine.

[0030] According to one aspect of an embodiment of the present application, a device calibration apparatus is provided, including:

[0031] An acquisition module is used to acquire the current attitude parameters of the camera when the attitude parameters between the optical machine and the camera change;

[0032] A test point module, used to form a current test point projected by the camera based on the current posture parameters of the camera;

[0033] The calibration module is used to trigger the calibration operation between the optical machine and the camera if the current test point projected by the camera is misaligned with the actual display point projected by the optical machine.

[0034] According to one aspect of an embodiment of the present application, a computer-readable medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the calibration method of the near-eye display device as described in the above embodiment is implemented.

[0035] According to one aspect of an embodiment of the present application, a near-eye display device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the calibration method of the near-eye display device as described in the above-mentioned embodiments.

[0036] According to one aspect of an embodiment of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the calibration method of the near-eye display device provided in the above embodiment.

[0037] In the technical solutions provided in some embodiments of the present application, if the posture parameters between the optical machine and the camera change, the current posture parameters of the camera are obtained to respond to the posture change of the camera relative to the optical machine, and the current test point projected by the camera is formed based on the current posture parameters of the camera; if the current test point projected by the camera is misaligned with the actual display point projected by the optical machine, the calibration operation between the optical machine and the camera is triggered to complete the calibration between the optical machine and the camera based on the adjustment of the current test point projected by the camera and the actual display point projected by the optical machine, thereby ensuring the consistency between the current test point projected by the camera and the actual display point projected by the optical machine, realizing autonomous calibration of the equipment after leaving the factory, eliminating the need for the equipment to be returned to the factory for repair, and reducing the after-sales cost of the equipment.

[0038] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0040] Figure 1 A schematic diagram showing a flow chart of a calibration method for a near-eye display device according to an embodiment of the present application is shown;

[0041] Figure 2 Shows Figure 1 Schematic diagram of the process of S120;

[0042] Figure 3 Shows Figure 1 Schematic diagram of the process flow of S130;

[0043] Figure 4 A schematic diagram showing a calibration method for a near-eye display device according to an embodiment of the present application is shown;

[0044] Figure 5 A block diagram of a calibration device according to an embodiment of the present application is shown;

[0045] Figure 6 A schematic diagram of the structure of a computer system suitable for implementing a near-eye display device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0046] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this application will be more comprehensive and complete and fully convey the concept of the example embodiments to those skilled in the art.

[0047] In addition, described feature, structure or characteristic can be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present application. However, those skilled in the art will appreciate that the technical scheme of the present application can be put into practice without one or more of the specific details, or other methods, components, devices, steps, etc. can be adopted. In other cases, known methods, devices, realizations or operations are not shown or described in detail to avoid blurring the various aspects of the application.

[0048] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0049] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0050] Figure 1 The flowchart of the calibration method of the near-eye display device according to an embodiment of the present application is shown. The method can be applied to the near-eye display device, which may include but is not limited to AR (Augmented Reality) devices, VR (Virtual Reality) devices, MR (Mixed Reality) devices, etc., wherein the AR device is AR glasses, AR helmets, etc. Please refer to Figure 1 , Figure 2 and Figure 5 The calibration method of the near-eye display device includes at least steps S110 to S130, which are described in detail as follows (the following description is made by taking the application of the method to the near-eye display device as an example):

[0051] In step S110 , if the posture parameters between the optical machine 10 and the camera 20 change, the current posture parameters of the camera 20 are obtained.

[0052] Among them, the calibration method of the near-eye display device is applied to the near-eye display device. In the embodiment of the present disclosure, the near-eye display device is taken as smart glasses as an example for explanation. The near-eye display device includes an optical machine 10, a camera 20, temples 30 and a frame 60. The camera 20 is arranged on one side of the optical machine 10. At this time, the optical machine 10 is built into the temples 30 of the near-eye display device, and the camera 20 is externally placed on the temples 30 of the near-eye display device.

[0053] When the near-eye display device is impacted by external force, the camera 20 is displaced under the action of the external force, and the posture parameters between the optical machine 10 and the camera 20 change. At this time, the current posture parameters of the camera 20 are obtained, and the current posture parameters of the camera 20 are the angle of the camera 20 relative to the optical machine 10.

[0054] Among them, before the near-eye display device leaves the factory, the optical machine 10 and the camera 20 are fixed after debugging, and the posture parameters between the optical machine 10 and the camera 20 are relatively fixed. The posture parameters between the optical machine 10 and the camera 20 are the angle parameters between the optical machine 10 and the camera 20. At this time, the posture parameters between the optical machine 10 and the camera 20 before the near-eye display device leaves the factory are preset posture parameters, which are determined by human experience and can be adjusted manually according to different application scenarios. In addition, when the near-eye display device is in operation, the posture parameters between the optical machine 10 and the camera 20 are obtained to facilitate the determination of the posture changes between the optical machine 10 and the camera 20.

[0055] When the near-eye display device is impacted by an external force, the optical machine 10 and the camera 20 will be offset by an angle due to the external force. At this time, the posture parameters between the optical machine 10 and the camera 20 are inconsistent with the preset posture parameters, and then the current posture parameters of the camera 20 are obtained. Optionally, the preset posture parameters can be determined through past experience and manually adjusted according to different scenarios.

[0056] refer to Figure 2 In step S120 , a current test point 40 projected by the camera 20 is formed based on the current posture parameters of the camera 20 .

[0057] The current posture parameter of the camera 20 is the posture parameter of the camera 20 after the external force acts on it, and the current posture parameter of the camera 20 is the current angle of the camera 20 relative to the optical machine 10 .

[0058] In one embodiment, the forming of the current test point 40 projected by the camera 20 based on the current posture parameter of the camera 20 includes:

[0059] Step S121, obtaining a scene picture taken by the camera 20;

[0060] The camera 20 shoots externally and obtains scene pictures based on the scene pictures shot by the camera 20 , so as to further determine the posture parameters based on the scene pictures shot by the camera 20 at that time.

[0061] Step S122: performing deep learning on the scene image taken by the camera 20 and forming a depth map;

[0062] The scene image taken by the camera 20 is subjected to deep learning, and the scene image is subjected to graphic processing to form a depth map; at this time, the deep learning includes monocular deep learning, binocular deep learning or stereoscopic deep learning. Monocular deep learning is to estimate the depth from a single image using a machine learning method; binocular deep learning is to estimate the depth from two images using a machine learning method; and stereoscopic deep learning is to estimate the depth from multiple images using a machine learning method.

[0063] In monocular deep learning and binocular deep learning, the scene image is processed by the corresponding deep learning network, and a depth map is formed during the learning process to facilitate subsequent feature point processing based on the depth map.

[0064] Step S123, determining a plurality of feature points based on the depth map;

[0065] Among them, subsequent feature point processing is performed based on the depth map, and feature points are selected in the depth map. At this time, multiple feature points are randomly selected from different rows in the depth map, and the multiple feature points are multiple position points of the outer contour of the feature, so as to facilitate the selection of multiple feature points in the depth map.

[0066] Step S124 , inputting a plurality of feature points into the projection space of the optical engine 10 , and forming a current test point 40 projected by the camera 20 in the projection space.

[0067] Among them, based on the projection space formed by the optical machine 10, multiple feature points are input into the projection space of the optical machine 10. At this time, the multiple feature points are presented in the projection space of the optical machine 10, and the multiple feature points are used as the current test points 40 projected by the camera 20, so as to form the current test points 40 projected by the camera 20 in the projection space, so as to perform comparison and calibration based on the current test points 40.

[0068] In this embodiment, a plurality of feature points are obtained through the camera of the near-eye display device itself, and a current test point projected by the camera is formed based on the plurality of feature points, so as to automatically trigger the calibration operation between the optical machine and the camera through the current test point, thereby completing the calibration between the optical machine and the camera of the near-eye display device without the aid of external equipment, thereby improving the efficiency of the calibration between the optical machine and the camera and reducing the cost of the calibration between the optical machine and the camera.

[0069] refer to Figure 3 and Figure 4 In step S130 , if the current test point 40 projected by the camera 20 is misaligned with the actual display point 50 projected by the optical engine 10 , a calibration operation between the optical engine 10 and the camera 20 is triggered.

[0070] When the device is subjected to external force, the camera 20 is offset relative to the optical machine 10, so that the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are misaligned. At this time, the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are not in the same position.

[0071] In one embodiment of the present application, if the current test point 40 projected by the camera 20 is misaligned with the actual display point 50 projected by the optical engine 10, a calibration operation between the optical engine 10 and the camera 20 is triggered, including:

[0072] Step S131, obtaining the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10;

[0073] In which, a coordinate system is established in the projection space of the optical machine 10, the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are both in the projection space of the optical machine 10, and the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10 are obtained to facilitate comparison between the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10.

[0074] Step S132: If the coordinates of the current test point 40 projected by the camera 20 are inconsistent with the coordinates of the actual display point 50 projected by the optical machine 10, the calibration operation between the optical machine 10 and the camera 20 is triggered, wherein the inconsistency between the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10 indicates that the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are misaligned.

[0075] In the comparison between the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10, the coordinates of the current test point 40 are compared with the coordinates of the actual display point 50 in terms of horizontal quantity and vertical quantity; at this time, if the horizontal quantity of the coordinates of the current test point 40 is inconsistent with the horizontal quantity of the coordinates of the actual display point 50, and / or the vertical quantity of the coordinates of the current test point 40 is inconsistent with the vertical quantity of the coordinates of the actual display point 50, then it is determined that the coordinates of the current test point 40 are inconsistent with the coordinates of the actual display point 50, wherein the inconsistency between the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10 indicates that the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are misaligned.

[0076] If the coordinates of the front test point 40 are inconsistent with the coordinates of the actual display point 50, the calibration operation between the optical machine 10 and the camera 20 is triggered. For the calibration operation between the optical machine 10 and the camera 20, if the near-eye display device is in 3DoF mode, the rotation parameters of the near-eye display device are obtained; the position of the current test point 40 projected by the camera 20 is adjusted based on the rotation parameters of the near-eye display device to move the current test point 40 projected by the camera 20 to the actual display point 50 projected by the optical machine 10, so that the horizontal amount of the coordinates of the current test point 40 is consistent with the horizontal amount of the coordinates of the actual display point 50, and the vertical amount of the coordinates of the current test point 40 is consistent with the vertical amount of the coordinates of the actual display point 50, so that the current test point 40 projected by the camera 20 coincides with the actual display point 50 projected by the optical machine 10, so as to complete the calibration operation between the optical machine 10 and the camera 20.

[0077] Among them, 3DOF means three degrees of freedom, with only rotation coordinates but no displacement coordinates. The center point can only be a set virtual head, and the base point of all observations is derived from the perspective of the head. For example, a camera fixed on a telephone pole can be rotated arbitrarily, but cannot move up, down, left, right, forward, or backward away from the pole; the rotation parameter of the head is the rotation angle of the head.

[0078] The user selects the 3DoF mode in the near-eye display device, the near-eye display device enters the 3DoF mode, and performs online calibration between the optical machine 10 and the camera 20 in the 3DoF mode. At this time, the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are presented in the projection space of the optical machine 10, and the rotation parameters of the near-eye display device are captured. The current test point 40 projected by the camera 20 is adjusted in position as the head rotates. Therefore, the position of the current test point 40 projected by the camera 20 is adjusted based on the rotation parameters of the near-eye display device to match the current test point 40 projected by the camera 20 with the actual display point 50 projected by the optical machine 10, and the online calibration between the optical machine 10 and the camera 20 is completed, so as to adapt the changed optical machine 10 based on the camera 20. Among them, the rotation parameter of the near-eye display device is the angle difference between the current test point 40 projected by the camera 20 and the actual test point projected by the optical machine 10 in the same projection space, and the rotation angle of each direction axis in the projection space of the optical machine 10 is determined according to the rotation parameter of the near-eye display device; the position of the current test point 40 projected by the camera 20 is adjusted based on the rotation angle of each direction axis, so as to adjust the position of the current test point 40 projected by the camera 20 according to the rotation parameter of the near-eye display device, thereby achieving position matching between the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10.

[0079] The near-eye display device is worn on the user's head. If the near-eye display device is in a rotating state, the angular velocity parameters of the gyroscope are obtained; the angular velocity parameters of the gyroscope are integrated to determine the angle difference between the current test point 40 projected by the camera 20 and the actual test point projected by the optical machine 10, as follows:

[0080] R = Exp(∫ωdt)

[0081] Wherein, Exp is the conversion of Lie algebra into Lie group, ω is the rotational angular velocity measured by the gyroscope; and R is the change in the posture of the optical machine 10 and the camera 20 due to the change in the environment.

[0082] In the embodiment of the present disclosure, online calibration between the optical machine 10 and the camera 20 is completed based on the adjustment of the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10, thereby ensuring the consistency between the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10. At this time, the position of the current test point 40 projected by the camera 20 is adjusted according to the rotation parameters of the near-eye display device, thereby achieving position matching between the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10, thereby achieving autonomous calibration of the equipment after leaving the factory, without the need to return the equipment to the factory for repair, and reducing the after-sales cost of the equipment.

[0083] The following describes an apparatus embodiment of the present application, which can be used to perform the calibration method of the near-eye display device in the above-mentioned embodiment of the present application. For details not disclosed in the apparatus embodiment of the present application, please refer to the embodiment of the calibration method of the near-eye display device in the above-mentioned embodiment of the present application.

[0084] Figure 5 A block diagram of a calibration apparatus for a device according to an embodiment of the present application is shown.

[0085] Reference Figure 5 As shown, a calibration device for a device according to an embodiment of the present application includes:

[0086] The acquisition module 210 is used to acquire the current attitude parameters of the camera 20 when the attitude parameters between the optical machine 10 and the camera 20 change;

[0087] A test point module 220, for forming a current test point 40 projected by the camera 20 based on the current posture parameters of the camera 20;

[0088] The calibration module 230 is used to trigger a calibration operation between the optical engine 10 and the camera 20 if the current test point 40 projected by the camera 20 is misaligned with the actual display point 50 projected by the optical engine 10 .

[0089] In one embodiment of the present application, the test point module 220 is also used to: obtain a scene picture taken by the camera 20; perform deep learning on the scene picture taken by the camera 20 and form a depth map; determine multiple feature points based on the depth map; project the multiple feature points to the projection space of the optical machine 10, and form the current test point 40 projected by the camera 20.

[0090] In one embodiment of the present application, the calibration module 230 is further used for:

[0091] Obtain the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10;

[0092] If the coordinates of the current test point 40 projected by the camera 20 are inconsistent with the coordinates of the actual display point 50 projected by the optical machine 10, the calibration operation between the optical machine 10 and the camera 20 is triggered, wherein the inconsistency between the coordinates of the current test point 40 projected by the camera 20 and the coordinates of the actual display point 50 projected by the optical machine 10 indicates that the current test point 40 projected by the camera 20 and the actual display point 50 projected by the optical machine 10 are misaligned.

[0093] In one embodiment of the present application, the calibration module 230 is also used to: if the near-eye display device is in 3DoF mode, obtain the rotation parameters of the near-eye display device; adjust the position of the current test point 40 projected by the camera 20 based on the rotation parameters of the near-eye display device, so as to move the current test point 40 projected by the camera 20 to the actual display point 50 projected by the optical machine 10, so that the current test point 40 projected by the camera 20 coincides with the actual display point 50 projected by the optical machine 10, so as to complete the calibration operation between the optical machine and the camera.

[0094] In one embodiment of the present application, the calibration module 230 is also used for: the rotation parameter of the near-eye display device is the angle difference between the current test point 40 projected by the camera 20 and the actual test point projected by the optical machine 10 in the same projection space.

[0095] In one embodiment of the present application, the calibration module 230 is also used to: if the near-eye display device is in a rotating state, obtain the angular velocity parameters of the gyroscope; integrate the angular velocity parameters of the gyroscope to determine the angle difference between the current test point 40 projected by the camera 20 and the actual test point projected by the optical machine 10.

[0096] In one embodiment of the present application, the calibration module 230 is further used to: determine the rotation angle of each direction axis in the projection space of the optical machine 10 according to the rotation parameters of the near-eye display device; adjust the position of the current test point 40 projected by the camera 20 based on the rotation angle of each direction axis;

[0097] In one embodiment of the present application, a near-eye display device is further provided, the near-eye display device comprising:

[0098] one or more processors;

[0099] A storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the calibration method of the near-eye display device as described in the above-mentioned embodiments.

[0100] In one example, Figure 6 A schematic diagram of the structure of a computer system suitable for implementing a near-eye display device according to an embodiment of the present application is shown.

[0101] It should be noted that Figure 6 The computer system of the near-eye display device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0102] like Figure 6As shown, the computer system includes a central processing unit (CPU) 301 (i.e., the processor as described above), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303, such as executing the method described in the above embodiment. It should be understood that RAM 303 and ROM 302 are the storage devices as described above. In RAM 303, various programs and data required for system operation are also stored. CPU 301, ROM 302 and RAM 303 are connected to each other through bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0103] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, etc.; an output section 307 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. A removable medium 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 310 as needed so that a computer program read therefrom is installed into the storage section 308 as needed.

[0104] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 309, and / or installed from a removable medium 311. When the computer program is executed by a central processing unit (CPU) 301, various functions defined in the system of the present application are executed.

[0105] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. Computer-readable storage medium can be, for example, - but not limited to - a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples of computer-readable storage mediums may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0106] In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0107] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0108] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0109] As another aspect, the present application further provides a computer-readable medium, which may be included in the near-eye display device described in the above embodiment; or may exist independently without being assembled into the near-eye display device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a near-eye display device, the near-eye display device implements the method described in the above embodiment.

[0110] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0111] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0112] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0113] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A calibration method for a near-eye display device, characterized in that: The calibration method of the near-eye display device is applied to the near-eye display device, the near-eye display device includes an optical machine and a camera, and the camera is arranged on one side of the optical machine; the calibration method of the near-eye display device includes: If the posture parameters between the optical machine and the camera change, the current posture parameters of the camera are obtained; Forming a current test point projected by the camera based on the current posture parameters of the camera; If the current test point projected by the camera is misaligned with the actual display point projected by the optical machine, a calibration operation between the optical machine and the camera is triggered.

2. The method according to claim 1, characterized in that: If the current test point projected by the camera is misaligned with the actual display point projected by the optical machine, a calibration operation between the optical machine and the camera is triggered, including: Get the coordinates of the current test point projected by the camera and the coordinates of the actual display point projected by the optical machine; If the coordinates of the current test point projected by the camera are inconsistent with the coordinates of the actual display point projected by the optical machine, the calibration operation between the optical machine and the camera is triggered, wherein the inconsistency between the coordinates of the current test point projected by the camera and the coordinates of the actual display point projected by the optical machine indicates that the current test point projected by the camera and the actual display point projected by the optical machine are misaligned.

3. The method according to claim 2, characterized in that The calibration operation between the triggering optical machine and the camera includes: If the near-eye display device is in 3DoF mode, obtain the rotation parameters of the near-eye display device; The position of the current test point projected by the camera is adjusted based on the rotation parameters of the near-eye display device to move the current test point projected by the camera to the actual display point projected by the optical machine, so that the current test point projected by the camera coincides with the actual display point projected by the optical machine to complete the calibration operation between the optical machine and the camera.

4. The method according to claim 3, characterized in that: The rotation parameter of the near-eye display device is the angle difference between the current test point projected by the camera and the actual test point projected by the optical machine in the same projection space.

5. The method according to claim 4, characterized in that The obtaining of the rotation parameters of the near-eye display device comprises: If the near-eye display device is in a rotating state, obtaining the angular velocity parameters of the gyroscope; The angular velocity parameters of the gyroscope are integrated to determine the angle difference between the current test point projected by the camera and the actual test point projected by the optical machine.

6. The method according to claim 3, characterized in that The step of adjusting the position of the current test point projected by the camera based on the rotation parameter of the near-eye display device includes: Determine the rotation angle of each direction axis in the projection space of the optical machine according to the rotation parameters of the near-eye display device; The position of the current test point projected by the camera is adjusted based on the rotation angles of each direction axis.

7. The method according to any one of claims 1 to 6, characterized in that: The current test point projected by the camera is formed based on the current posture parameters of the camera, including: Get the scene picture taken by the camera; Perform deep learning on the scene pictures taken by the camera and form a depth map; Determine a plurality of feature points based on the depth map; Project multiple feature points into the projection space of the optical machine and form the current test point projected by the camera.

8. A device for calibrating equipment, characterized in that: include: An acquisition module is used to obtain the current attitude parameters of the camera when the attitude parameters between the optical machine and the camera change; A test point module, used to form a current test point projected by the camera based on the current posture parameters of the camera; The calibration module is used to trigger the calibration operation between the optical machine and the camera if the current test point projected by the camera is misaligned with the actual display point projected by the optical machine.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the calibration method of the near-eye display device according to any one of claims 1 to 7 is implemented.

10. A near-eye display device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the calibration method of the near-eye display device as described in any one of claims 1 to 7.

Images