Calibration method, device and system of head-mounted display equipment and storage medium

Abstract

The invention relates to a calibration method, device and system for head-mounted display equipment and a storage medium, and the method comprises the steps: carrying out the combined calibration of an observation camera and a tracking camera, and obtaining the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein during joint calibration, the observation camera shoots a calibration object through a spectacle lens of the head-mounted display device; based on a virtual pattern on a display screen of the head-mounted display equipment, determining a three-dimensional feature coordinate of the calibration object, and obtaining a target image obtained by shooting the virtual pattern by the observation camera; and determining an optical perspective calibration result for the head-mounted display device according to the three-dimensional feature coordinates, the target image and the internal reference of the observation camera in combination with the external reference between the observation camera and the tracking camera. The method can consider the influence of the glasses lenses on the virtual-real combination calibration, thereby improving the accuracy of the virtual-real calibration result.

Description

Technical Field

[0001] The present application relates to the field of smart wearables, and particularly to a calibration method, device, system and storage medium for a head-mounted display device. Background Art

[0002] An augmented reality (AR) glasses or a mixed reality (MR) glasses can serve as a core device for human-computer interaction and play an important role in fields such as industrial manufacturing, medical treatment, social entertainment, and education. Calibrating the combination of virtual and real of the AR / MR glasses is to solve the problem of aligning the spatial relationship between the virtual world and the real world presented on the AR / MR glasses, so as to ensure the immersion and interaction accuracy of the user experience.

[0003] Currently, when calibrating the virtual world and the real world, a single-point interactive calibration method or a camera-based calibration method is usually adopted. Among them, the single-point interactive calibration method requires manual collection of the corresponding data between the virtual target and the real target, so as to calibrate the spatial relationship between the virtual world and the real world. The camera-based calibration method is to complete the calibration of the combination of virtual and real by shooting a specific calibration object.

[0004] Since the distortion of the glasses lens itself affects the accuracy of the calibration result of the combination of virtual and real, the above calibration methods still have the problem of low calibration result accuracy waiting to be solved. Summary of the Invention

[0005] In this embodiment, a calibration method, device, system and storage medium for a head-mounted display device are provided to solve the problem of low calibration result accuracy in the related art.

[0006] In a first aspect, in this embodiment, a calibration method for a head-mounted display device is provided, including:

[0007] Performing joint calibration on the observation camera and the tracking camera according to a set of calibration object images obtained by simultaneously shooting a preset calibration object by the observation camera and the tracking camera, to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein, the observation camera shoots the calibration object through the glasses lens of the head-mounted display device; the tracking camera is disposed on the head-mounted display device;

[0008] Drawing a virtual pattern for the calibration object on the display screen of the head-mounted display device;

[0009] Based on the virtual pattern, determining the three-dimensional feature coordinates of the calibration object, and obtaining a target image obtained by the observation camera shooting the virtual pattern;

[0010] Based on the three-dimensional feature coordinates, the target image, and the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera, determine the optical perspective calibration result for the head-mounted display device.

[0011] In some embodiments, determining the three-dimensional feature coordinates of the calibration object based on the virtual pattern includes:

[0012] Extract the two-dimensional feature coordinates of the calibration object from the virtual pattern;

[0013] According to the two-dimensional feature coordinates and the pre-obtained virtual camera internal parameters, determine the three-dimensional feature coordinates of the calibration object in the three-dimensional coordinate system established with the plane where the display screen is located; the virtual camera internal parameters are determined based on the field of view parameters of the head-mounted display device.

[0014] In some embodiments, determining the optical perspective calibration result for the head-mounted display device according to the three-dimensional feature coordinates, the target image, and the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera, includes:

[0015] Extract the calibration features of the calibration object from the target image to obtain the two-dimensional image coordinates of the calibration features in the target image;

[0016] According to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera, determine a first conversion relationship; the first conversion relationship represents the conversion relationship between the observation camera coordinate system and the virtual world coordinate system; the virtual world coordinate system is the two-dimensional coordinate system of the virtual world presented by the display screen;

[0017] According to the first conversion relationship, and in combination with the external parameters between the observation camera and the tracking camera, determine the optical perspective calibration result for the head-mounted display device.

[0018] In some embodiments, determining the first conversion relationship according to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera includes:

[0019] According to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera, perform geometric solution to determine a third conversion relationship; the third conversion relationship is the conversion relationship between the three-dimensional coordinate system established with the plane where the display screen is located and the observation camera coordinate system;

[0020] According to the third conversion relationship, determine the first conversion relationship.

[0021] In some of these embodiments, according to the first conversion relationship and in combination with the external parameters between the observation camera and the tracking camera, determining the optical perspective calibration result for the head-mounted display device includes:

[0022] Determining a second conversion relationship between the physical space coordinate system established with the tracking camera and the observation camera coordinate system according to the external parameters between the observation camera and the tracking camera;

[0023] Determining the optical perspective calibration result of the head-mounted display device according to the first conversion relationship and the second conversion relationship.

[0024] In some of these embodiments, according to the set of calibration object images obtained by simultaneously photographing a preset calibration object with the observation camera and the tracking camera, jointly calibrating the observation camera and the tracking camera to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera, includes:

[0025] According to the set of calibration object images obtained by simultaneously photographing a preset calibration object with the observation camera and the tracking camera;

[0026] Extracting features from each image in the set of calibration object images to determine the calibration features of the calibration object in each image;

[0027] Determining the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera according to the image coordinates corresponding to the calibration features in the set of calibration object images.

[0028] In a second aspect, in the present embodiment, a calibration device for a head-mounted display device is provided, including: a joint calibration module, a pattern display module, an acquisition module, and an optical perspective calibration module; where:

[0029] The joint calibration module is configured to jointly calibrate the observation camera and the tracking camera according to the set of calibration object images obtained by simultaneously photographing a preset calibration object with the observation camera and the tracking camera, to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; where the observation camera photographs the calibration object through the spectacle lenses of the head-mounted display device; the tracking camera is disposed on the head-mounted display device;

[0030] The pattern display module is configured to draw a virtual pattern for the calibration object on the display screen of the head-mounted display device;

[0031] The acquisition module is configured to determine the three-dimensional feature coordinates of the calibration object based on the virtual pattern, and acquire the target image obtained by photographing the virtual pattern with the observation camera;

[0032] The optical perspective calibration module is configured to determine an optical perspective calibration result for the head-mounted display device according to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera.

[0033] In a third aspect, a calibration system for a head-mounted display device is provided in this embodiment, including: an observation camera, a fixing bracket, and a server; the observation camera is communicatively connected to the server;

[0034] The fixing bracket is used to fix the observation camera and the head-mounted display device to be calibrated; the head-mounted display device is disposed between the lens of the observation camera and the calibration object; the observation camera is used to photograph the calibration object through the spectacle lens of the head-mounted display device, and photograph the virtual pattern drawn on the display screen of the head-mounted display device;

[0035] The server is configured to execute the calibration method for the head-mounted display device described in the first aspect above.

[0036] In a fourth aspect, an electronic device is provided in this embodiment, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the calibration method for the head-mounted display device described in the first aspect above is implemented.

[0037] In a fifth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the calibration method for the head-mounted display device described in the first aspect above is implemented.

[0038] Compared with the related art, in this embodiment, a calibration method, device, system and storage medium for a head-mounted display device are provided. In the calibration method for the head-mounted display device, based on a set of calibration object images obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera, the observation camera and the tracking camera are jointly calibrated to obtain the external parameters between the observation camera and the tracking camera, as well as the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the spectacle lenses of the head-mounted display device; the tracking camera is disposed on the head-mounted display device; a virtual pattern for the calibration object is drawn on the display screen of the head-mounted display device; based on the virtual pattern, the three-dimensional feature coordinates of the calibration object are determined, and a target image obtained by the observation camera photographing the virtual pattern is acquired; according to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera, an optical perspective calibration result for the head-mounted display device is determined. By introducing the observation camera and jointly calibrating it with the tracking camera based on the image taken by the observation camera through the spectacle lenses, and then determining the virtual-real calibration result in combination with the calibration object according to the joint calibration result, the influence of the spectacle lenses on the virtual-real combined calibration can be taken into consideration, thereby improving the accuracy of the virtual-real calibration result.

[0039] Details of one or more embodiments of the present application are set forth in the following drawings and description, which are intended to make other features, objects, and advantages of the present application more comprehensible. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The drawings described herein are provided to further understand the present application and form a part of the present application. The illustrative embodiments and descriptions thereof are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0041] Figure 1 is a hardware structure block diagram of a terminal for the calibration method of the head-mounted display device according to an embodiment of the present application;

[0042] Figure 2 is a flowchart of the calibration method of the head-mounted display device according to an embodiment of the present application;

[0043] Figure 3a is an inaccurate virtual-real combination schematic diagram of a head-mounted display device;

[0044] Figure 3b is an accurate virtual-real combination schematic diagram of a head-mounted display device;

[0045] Figure 4a is a schematic diagram of an ideal light propagation path;

[0046] Figure 4b is a schematic diagram of an actual light propagation path;

[0047] Figure 5It is a flowchart of a calibration method for a head-mounted display device according to some embodiments of the present application;

[0048] Figure 6 It is a structural block diagram of a calibration device for a head-mounted display device according to an embodiment of the present application;

[0049] Figure 7 It is a schematic structural diagram of a calibration system for a head-mounted display device according to an embodiment of the present application. Detailed implementation manners

[0050] To understand the purpose, technical solution and advantages of the present application more clearly, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments.

[0051] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meanings understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "one", "a kind", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, system, product or device including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or devices. The "connection", "connection", "coupling" and other similar words involved in the present application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly connected. The "multiple" involved in the present application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in the present application are only used to distinguish similar objects and do not represent a specific order for the objects.

[0052] In the method embodiment provided in this embodiment, it can be executed on a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 It is a hardware structural block diagram of a terminal for the calibration method of the head-mounted display device in this embodiment. As Figure 1 shown, the terminal may include one or more ( Figure 1a processor 102 (only one is shown in the figure) and a memory 104 for storing data. The processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above terminal. For example, the terminal may further include more or fewer components than those Figure 1 shown in the figure, or have a different configuration from that Figure 1 shown in the figure.

[0053] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the calibration method of the head-mounted display device in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the terminal through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0054] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by the communication provider of the terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0055] In this embodiment, a calibration method for a head-mounted display device is provided. Figure 2 is a flowchart of the calibration method for the head-mounted display device in this embodiment, as Figure 2 shown in the figure, and the process includes the following steps:

[0056] Step S210: Based on the calibration object image set obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera, jointly calibrate the observation camera and the tracking camera to obtain the external parameters between the observation camera and the tracking camera, as well as the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the spectacle lenses of the head-mounted display device; the tracking camera is arranged on the head-mounted display device.

[0057] The head-mounted display device is an intelligent wearable device based on the optical see-through (OST) augmented reality display technology, which superimposes virtual information onto the user's real field of view through a transparent optical element. Exemplarily, the head-mounted display device can be an AR glasses or an MR glasses. In order to be able to achieve precise and natural interaction using the head-mounted display device, it is necessary to perform optical perspective calibration on the head-mounted display device. This optical perspective calibration specifically calibrates the spatial relationship between the virtual world presented on the display screen of the head-mounted display device and the actual real physical world. That is, it is necessary to achieve the alignment of the spatial relationship between the virtual world and the real physical world through calibration that combines virtual and real.

[0058] Figure 3a It is an inaccurate schematic diagram of the combination of virtual and real for the head-mounted display device; Figure 3b It is an accurate schematic diagram of the combination of virtual and real for the head-mounted display device. Please refer to Figure 3a , the extension line of the light rays of the virtual image observed by the left camera from the left screen of the head-mounted display device is the Figure 3a left dotted line in Figure 3a , and the extension line of the light rays of the virtual image observed by the right camera from the right screen of the head-mounted display device is the Figure 3a right dotted line in Figure 3a . The X, Y, and Z axes on the left camera in

[0059] represent the camera coordinate system where the left camera is located; the X, Y, and Z axes on the right camera represent the camera coordinate system where the right camera is located. Figure 3b , Figure 3b Next, please refer to Figure 3b , in Figure 3b , the extension line of the light rays of the virtual image observed by the left camera from the left screen of the head-mounted display device is the Figure 3b left dotted line in Figure 3b , and the extension line of the light rays of the virtual image observed by the right camera from the right screen of the head-mounted display device is the Figure 3bA point in the physical space is imaged on the left and right screens of the head-mounted display device and observed by the left and right cameras respectively. This is a correct combination of virtual and real. Figure 3b The X, Y, and Z axes on the left camera in the middle represent the camera coordinate system where the left camera is located; the X, Y, and Z axes on the right camera represent the camera coordinate system where the right camera is located.

[0060] The optical perspective calibration of AR / MR glasses is to calculate the spatial relationship between the virtual world and the real physical space (the real world is used for convenience in the following description). RW It can be understood that T RW Represents the conversion relationship from the coordinates of the real world to the coordinates of the virtual world. The coordinates of the real world are three-dimensional, where a three-dimensional coordinate system can be established on a tracking camera of the head-mounted display device to represent the coordinate system of the real world. For example, the left eye camera of the AR / MR glasses is used to establish the coordinate system of the real world. The coordinates of the virtual world are two-dimensional and are presented on the display screen of the head-mounted display device. Therefore, the coordinates of the virtual world and the coordinates of the real world can have the following relationship:

[0061] ;

[0062] Among them, P W represents the three-dimensional real-world coordinates, p R Represents the two-dimensional virtual world coordinates, K R The virtual world imaging camera is an imaginary camera that does not exist in the physical world and is used to draw virtual object content on the display screen of the head-mounted display device. The calibration method of the head-mounted display device of this embodiment is used to complete the calibration of T RW The solution.

[0063] Among them, in this step, in order to achieve T RW , the observation camera is used to establish this spatial transformation relationship. Specifically, when using the calibration object for calibration, the head-mounted display device can be fixed between the observation camera lens and the calibration object, so that the observation camera can simulate the human eye's perspective on the head-mounted display device and observe the calibration object through the glasses lens of the head-mounted display device. The position setting of the observation camera, the head-mounted display device, and the calibration object in this step is based on the consideration of the influence of the glasses lens of the head-mounted display device on the imaging.

[0064] Figure 4a It is a schematic diagram of an ideal light propagation path; Figure 4b It is a schematic diagram of the actual light propagation path. Figure 4aAs shown, light rays parallel to the horizontal plane enter from the left and converge at a certain point on the optical axis after passing through the optical element. As Figure 4b shown, in actual situations, when light rays propagate, they are also deflected by the spectacle lens ( Figure 4b the meniscus shape in

[0065] ), which ultimately affects the position of the convergence point on the optical axis. Therefore, in this embodiment, considering the actual production of AR / MR glasses, there will be a layer of glass lens assembled on the outer layer of the optical display component, and this layer of glass lens will cause a certain distortion to the light propagation. Figure 4a In the related art, when calibrating the combination of virtual and real for a head-mounted display device, the assumption of the ideal light propagation path in

[0066] is often adopted, and the calibration is completed based on this. Therefore, the calibration result of the related art that ignores the influence of the glass lens is not accurate enough. CW RC RC That is:

[0067] ;

[0068] To solve the above two transformation relationships, it is necessary to first perform joint calibration on the observation camera and the tracking camera to obtain the internal parameters of the observation camera and the external parameters between the observation camera and the tracking camera. In addition, the internal parameters of the tracking camera can also be solved.

[0069] The observation camera can also include a left-eye camera (hereinafter simply referred to as the left-eye observation camera) and a right-eye camera (hereinafter simply referred to as the right-eye observation camera). During joint calibration, the left-eye observation camera is made to observe the calibration board through the left lens of the head-mounted display device, and the right-eye observation camera is made to observe the calibration board through the right lens of the head-mounted display device. Tracking cameras can be set on the glasses of the head-mounted display device. There can be multiple such tracking cameras, which can be set on the sides of the glasses of the head-mounted display device or in the middle of the lenses of the head-mounted display device. For example, a tracking camera is respectively set on the left side of the left lens and the right side of the right lens. The tracking camera on the left side of the left lens serves as the left-eye tracking camera, and the tracking camera on the right side of the right lens serves as the right-eye tracking camera. In some embodiments, in order to improve the stability of data acquisition, a fixed bracket can be used to fix the observation camera and the head-mounted display device, and this fixed bracket can adjust the positions of the observation camera and the head-mounted display device.

[0070] After that, based on a pre-set data automatic acquisition program, in the form of instructions issued by the program, the observation camera can be controlled to move synchronously with the tracking camera to different positions in a manner fixed relative to the head-mounted display device, and the calibration object fixed in the scene can be synchronously photographed, that is, the left-eye observation camera, the right-eye observation camera, the left-eye tracking camera, and the right-eye tracking camera are simultaneously triggered to photograph the calibration object. It can be understood that the observation camera includes two cameras, the left and the right, and the tracking camera also includes two cameras, the left and the right. Therefore, in a set of data obtained in one acquisition, there are four calibration object images respectively from different cameras. Moving to different positions for synchronous photographing can obtain multiple sets of calibration object images, and these multiple sets of calibration object images can form a calibration object image set. Then, this calibration object image set is used to jointly calibrate the observation camera and the tracking camera. Since the observation camera photographs the calibration object through the glasses lenses of the head-mounted display device, the internal parameters of the finally calibrated observation camera actually include the influence of the glasses lenses of the head-mounted display device on imaging.

[0071] Among them, the internal parameters of the observation camera can include parameters such as the focal length, principal point, and distortion coefficient of the observation camera; the external parameters between the observation camera and the tracking camera can include the relative pose and attitude relationship between the observation camera and the tracking camera.

[0072] In the actual calibration process, one of the above-mentioned left-eye tracking cameras or the above-mentioned right-eye tracking cameras can be selected to establish a three-dimensional coordinate system representing the real-world coordinate system. For example, if the left-eye tracking camera is used to establish the three-dimensional coordinate system, the internal parameters of the left-eye observation camera, the internal parameters of the right-eye observation camera, the external parameters between the left-eye observation camera and the left-eye tracking camera, and the external parameters between the right-eye observation camera and the left-eye tracking camera can be obtained during joint calibration. When setting the three-dimensional coordinate system representing the real-world coordinate system, it can be set according to the requirements of the actual application scenario, or the structural design of the head-mounted display device and the product usage preference. One tracking camera can be selected, or multiple tracking cameras can be selected. This embodiment does not make specific limitations on this.

[0073] Step S220, draw a virtual pattern for the calibration object on the display screen of the head-mounted display device.

[0074] The head-mounted display device presents the virtual world on the display screen. The display screen is theoretically a two-dimensional plane. Therefore, there is a homography between the imaging plane of the observation camera and the display screen. Among them, homography is a projective transformation that describes the mapping relationship between two planes. That is, there is a certain transformation relationship between the imaging plane of the observation camera and the two-dimensional plane where the display screen is located. Based on this, in order to solve the conversion relationship between the observation camera coordinate system and the virtual world coordinate system, a pattern of a calibration object with a known scale can be drawn on the display screen of the head-mounted display device first. The calibration object can specifically be a calibration board with a pattern drawn on it, or a combination of a certain fixed plane and several marker points. The pattern on the calibration object can be a checkerboard, dots, a two-dimensional barcode (aprilGrid), or a pattern suitable for camera calibration customized according to the requirements of the actual application scenario. Exemplarily, when the calibration object is a calibration board containing a checkerboard pattern, a checkerboard with a known scale can be drawn on the display screen.

[0075] Step S230, based on the virtual pattern, determine the three-dimensional feature coordinates of the calibration object, and obtain the target image obtained by the observation camera shooting the virtual pattern.

[0076] A three-dimensional coordinate system R can be established with the plane where the display screen is located. Then, the calibration features of the virtual pattern on the display screen are obtained, and the three-dimensional coordinates of the virtual pattern under this three-dimensional coordinate line R are obtained, so as to obtain the three-dimensional feature coordinates {P RiAmong them, the calibration feature can be a feature that can be extracted from the virtual pattern for calibration. For example, for a checkerboard pattern, it can be the checkerboard corner points. i can represent the i-th calibration feature, such as the i-th checkerboard corner point. Additionally, an observation camera can be used to capture the virtual pattern displayed on the display screen, thereby obtaining a two-dimensional image containing the virtual pattern as the target image.

[0077] Step S240: Determine the optical perspective calibration result for the head-mounted display device based on the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and the external parameters between the observation camera and the tracking camera.

[0078] The two-dimensional image coordinates {p of the calibration feature in the image coordinate system of the target image can be extracted from the target image. ci}, and then, using the three-dimensional feature coordinates {P Ri}, the two-dimensional image coordinates {p ci}, and the internal parameters of the observation camera as inputs, in the way of solving the Perspective-n-Point (PnP) problem, finally complete the conversion relationship T RC between the observation camera coordinate system and the virtual world coordinate system. Then, based on the conversion relationship between the observation camera coordinate system and the virtual world coordinate system, and the conversion relationship between the real world coordinate system and the observation camera coordinate system, solve for the conversion relationship between the real world coordinate system and the virtual world coordinate system, thereby completing the optical perspective calibration for the head-mounted display device.

[0079] Compared with the related technology, due to ignoring the influence of the glasses lenses on the virtual and real calibration, the accuracy of the virtual and real calibration of the head-mounted display device is relatively low. Through the above steps S210 to S240, this embodiment provides a more accurate virtual and real calibration method by introducing the consideration of the influence of the glasses lenses. Furthermore, based on the more accurate virtual and real calibration, the authenticity and immersion of the user experience can be improved.

[0080] Therefore, in the above steps S210 to S240, based on the calibration object image set obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera, the observation camera and the tracking camera are jointly calibrated to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the glasses lens of the head-mounted display device; the tracking camera is arranged on the head-mounted display device; a virtual pattern for the calibration object is drawn on the display screen of the head-mounted display device; based on the virtual pattern, the three-dimensional feature coordinates of the calibration object are determined, and a target image obtained by the observation camera photographing the virtual pattern is acquired; according to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera, the optical perspective calibration result for the head-mounted display device is determined. It introduces an observation camera, jointly calibrates with the tracking camera based on the image photographed by the observation camera through the glasses lens, and then determines the virtual-real calibration result in combination with the calibration object according to the joint calibration result, which can take into account the influence of the glasses lens on the virtual-real combined calibration, thereby improving the accuracy of the virtual-real calibration result.

[0081] In one embodiment, based on the above step S230, to determine the three-dimensional feature coordinates of the calibration object based on the virtual pattern, it may specifically include:

[0082] Extract the two-dimensional feature coordinates of the calibration object from the virtual pattern; according to the two-dimensional feature coordinates and the pre-acquired virtual camera internal parameters, determine the three-dimensional feature coordinates of the calibration object in the three-dimensional coordinate system established with the plane where the display screen is located; the virtual camera internal parameters are determined based on the field of view parameters of the head-mounted display device.

[0083] Since a virtual pattern with a known scale is drawn on the display screen, the image coordinates of the calibration features of the virtual pattern can be directly extracted as the two-dimensional feature coordinates {p Ri}. That is, the two-dimensional feature coordinates here are known. When establishing a three-dimensional coordinate system R with the plane where the display screen is located and calculating the three-dimensional coordinates of the calibration features of the virtual pattern in this three-dimensional coordinate system R, the following formula can be referred to for calculation:

[0084] ;

[0085] wherein, K R is the virtual camera internal parameter. This virtual camera is the above-mentioned imaginary virtual world imaging camera used to draw a virtual pattern on the display screen of the head-mounted display device. The virtual camera internal parameter can be obtained based on the factory FOV parameter of the optical components of the head-mounted display device.

[0086] Thus, based on the homography between the imaging plane of the observation camera and the display screen, in a three-dimensional coordinate system R constructed by the display screen, this embodiment can determine the three-dimensional coordinates of the calibration features of the virtual pattern on the display screen, and further refine the solution of the conversion relationship between the observation camera coordinate system and the virtual world coordinate system into a PnP problem of solving the external parameters from the three-dimensional coordinate system R of the display screen to the observation camera. Therefore, it is possible to provide an accurate and reliable basis for subsequent solving of the PnP problem by combining the target image of the virtual pattern captured by the observation camera.

[0087] In addition, in one embodiment, based on the above step S240, according to the three-dimensional feature coordinates, the target image, and the internal parameters of the observation camera, and combining the external parameters between the observation camera and the tracking camera, determining the optical perspective calibration result for the head-mounted display device may specifically include:

[0088] Extract the calibration features of the calibration object from the target image to obtain the two-dimensional image coordinates of the calibration features in the target image; determine the first conversion relationship according to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera; the first conversion relationship represents the conversion relationship between the observation camera coordinate system and the virtual world coordinate system; the virtual world coordinate system is the two-dimensional coordinate system of the virtual world presented by the display screen; determine the optical perspective calibration result for the head-mounted display device according to the first conversion relationship and in combination with the external parameters between the observation camera and the tracking camera.

[0089] First, image processing techniques can be used to extract the calibration features from the target image captured by the observation camera. Then, based on the two-dimensional image coordinates of the calibration features in the target image, the three-dimensional feature coordinates in the above three-dimensional coordinate system R, and the internal parameters of the observation camera, the conversion relationship between the observation camera coordinate system and the virtual world coordinate system is determined. Also, since the conversion relationship between the observation camera coordinate system and the real world coordinate system can be determined according to the external parameters between the observation camera and the tracking camera, combining the first conversion relationship with the external parameters between the observation camera and the tracking camera can determine the conversion relationship between the real world coordinate system and the virtual world coordinate system.

[0090] Therefore, in this embodiment, when introducing the observation camera and combining the internal parameters of the observation camera, the virtual-real calibration considering lens distortion is realized, thereby improving the accuracy of the calibration result.

[0091] Specifically, in one embodiment, determining the first conversion relationship according to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera may include:

[0092] Based on the three-dimensional feature coordinates, two-dimensional image coordinates, and the internal parameters of the observation camera, perform geometric solution to determine the third conversion relationship; the third conversion relationship is the conversion relationship between the three-dimensional coordinate system established with the plane where the display screen is located and the observation camera coordinate system; according to the third conversion relationship, determine the first conversion relationship.

[0093] The PnP problem refers to solving the pose of the camera (i.e., the rotation matrix and translation vector) through known three-dimensional points (in the world coordinate system) and their corresponding two-dimensional projection points (in the image plane). Therefore, in this embodiment, the three-dimensional feature coordinates in the three-dimensional coordinate system R can be used as a known three-dimensional point, and the two-dimensional image coordinates extracted from the target image captured by the observation camera can be used as the two-dimensional projection points corresponding to the above-known three-dimensional points in the image plane. In this way, the external parameters from the three-dimensional coordinate system R to the observation camera can be solved through the PnP problem, thereby obtaining the above-mentioned third conversion relationship T. CR After that, calculate the first conversion relationship T according to the following formula. RC :

[0094] ;

[0095] Therefore, in this embodiment, the solution of the conversion relationship between the observation camera coordinate system and the virtual world coordinate system can be regarded as a PnP problem, and the three-dimensional coordinates in the three-dimensional coordinate system constructed on the plane where the screen is located and the two-dimensional image coordinates in the target image captured by the observation camera using the calibration features of the calibration object are used to finally accurately solve the first conversion relationship.

[0096] Additionally, in one embodiment, according to the first conversion relationship, combined with the external parameters between the observation camera and the tracking camera, determine the optical perspective calibration result for the head-mounted display device, which may specifically include:

[0097] According to the external parameters between the observation camera and the tracking camera, determine the second conversion relationship between the physical space coordinate system established with the tracking camera and the observation camera coordinate system; according to the first conversion relationship and the second conversion relationship, determine the optical perspective calibration result of the head-mounted display device.

[0098] Since the real-world coordinate system is established based on the coordinate system of the tracking camera, according to the external parameters between the observation camera and the tracking camera, the conversion relationship between the real-world coordinate system and the observation camera coordinate system can be determined as the above-mentioned second conversion relationship T. CW The observation camera coordinate system is a three-dimensional coordinate system. After that, based on the first conversion relationship T RC and the second conversion relationship T CW obtain T RW .

[0099] In this embodiment, by introducing an observation camera, the solution of the spatial relationship between the real world and the virtual world is decomposed, and accurate virtual-real calibration can ultimately be achieved.

[0100] In addition, in one embodiment, based on a set of calibration object images obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera, the observation camera and the tracking camera are jointly calibrated to obtain the external parameters between the observation camera and the tracking camera, as well as the internal parameters of the observation camera. Specifically, it may include:

[0101] Based on a set of calibration object images obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera; extracting features from each image in the set of calibration object images to determine the calibration features of the calibration object in each image; and determining the external parameters between the observation camera and the tracking camera, as well as the internal parameters of the observation camera, according to the image coordinates corresponding to each calibration feature in the set of calibration object images.

[0102] Taking the example where the observation camera includes a left-eye observation camera and a right-eye observation camera, the tracking camera includes a left-eye tracking camera and a right-eye tracking camera, and the calibration object is a checkerboard pattern marker board. Based on a pre-set data automatic acquisition program, in the form of program instructions, the left-eye observation camera and the right-eye observation camera are controlled to move synchronously with the left-eye tracking camera and the right-eye tracking camera to different positions in a manner fixed relative to the head-mounted display device, and the marker board fixed in the scene is synchronously photographed. At the same time, the left-eye observation camera, the right-eye observation camera, the left-eye tracking camera, and the right-eye tracking camera are triggered to photograph the marker board. Each time an image is acquired, a set of four calibration object images can be obtained. After multiple acquisitions, multiple sets of calibration object images can be obtained, forming a set of calibration object images. Then, the left-eye observation camera, the right-eye observation camera, the left-eye tracking camera, and the right-eye tracking camera are jointly calibrated using this set of calibration object images. Since both the left-eye observation camera and the right-eye observation camera photograph the calibration object through the glasses lenses of the head-mounted display device, the internal parameters of the left-eye observation camera and the right-eye observation camera finally calibrated actually include the influence of the glasses lenses of the head-mounted display device on imaging. Therefore, based on the joint calibration of this embodiment, the accuracy of subsequent virtual-real calibration of the head-mounted display device can be improved.

[0103] It should also be noted that after the joint calibration is completed, the left-eye observation camera captures the virtual checkerboard drawn on the display screen to obtain the left-eye target image; the right-eye observation camera captures the virtual checkerboard drawn on the display screen to obtain the right-eye target image. After that, the spatial relationship calculation between the virtual world and the real world is performed for the left-eye observation camera and the right-eye observation camera respectively. Based on the left-eye target image, the internal parameters of the left-eye observation camera, and the three-dimensional feature coordinates of the checkerboard corners in the three-dimensional coordinate system R established on the plane where the display screen is located, the spatial relationship calculation between the virtual world and the real world for the left display screen of the head-mounted display device is obtained; based on the right-eye target image, the internal parameters of the right-eye observation camera, and the three-dimensional feature coordinates of the checkerboard corners in the three-dimensional coordinate system R established on the plane where the display screen is located, the spatial relationship calculation between the virtual world and the real world for the right display screen is obtained.

[0104] Figure 5 is a flowchart of a calibration method for a head-mounted display device of some embodiments, as Figure 5 shown, the calibration method of the head-mounted display device includes the following steps:

[0105] Step S501, based on the calibration object, jointly calibrate the observation camera and the tracking camera; among them, the specific process of the joint calibration can refer to the above embodiments and will not be elaborated here.

[0106] Step S502, the observation camera captures the virtual pattern on the display screen of the head-mounted display device to obtain the target image;

[0107] Step S503, calculate the three-dimensional feature coordinates of the virtual pattern in the three-dimensional coordinate system R on the plane where the display screen is located, and the two-dimensional image coordinates of the calibration features in the target image;

[0108] Step S504, according to the data obtained in steps S501 to S503, solve the spatial relationship between the virtual world and the observation camera;

[0109] Step S505, according to the joint calibration result of step S501 and the spatial relationship obtained in step S504, solve the spatial relationship between the virtual world and the real world.

[0110] The above steps S501 to S505 realize an automatic calibration that takes into account the influence of lens distortion on the virtual-real combination calibration. The calibration result is more accurate, and the calibration process can be automatically executed based on program control throughout the process without manual participation, improving the calibration efficiency of the head-mounted display device and saving labor costs.

[0111] In this embodiment, a calibration device for a head-mounted display device is further provided. This device is used to implement the above-mentioned embodiments and preferred implementation manners, and those that have been described will not be repeated here. The following terms such as "module", "unit", "sub-unit", etc. can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0112] Figure 6 is a structural block diagram of the calibration device 60 of the head-mounted display device in this embodiment. As Figure 6 shown, the calibration device 60 of the head-mounted display device includes: a joint calibration module 62, a pattern display module 64, an acquisition module 66, and an optical perspective calibration module 68; where:

[0113] The joint calibration module 62 is used to perform joint calibration on the observation camera and the tracking camera according to a set of calibration object images obtained by simultaneously photographing a preset calibration object by the observation camera and the tracking camera, so as to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the glasses lens of the head-mounted display device; the tracking camera is arranged on the head-mounted display device; the pattern display module 64 is used to draw a virtual pattern for the calibration object on the display screen of the head-mounted display device; the acquisition module 66 is used to determine the three-dimensional feature coordinates of the calibration object based on the virtual pattern, and acquire a target image obtained by the observation camera photographing the virtual pattern; the optical perspective calibration module 68 is used to determine the optical perspective calibration result for the head-mounted display device according to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera.

[0114] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combined form.

[0115] It should be noted that specific examples in this embodiment can refer to the examples described in the above-mentioned embodiments and alternative implementation manners, and will not be repeated here.

[0116] In this embodiment, a calibration system for a head-mounted display device is further provided. Figure 7 is a structural schematic diagram of the calibration system 70 of the head-mounted display device in this embodiment. As Figure 7 shown, the calibration system 70 of the head-mounted display device includes: an observation camera 72, a fixed bracket 74, and a server 76; the observation camera 72 is communicatively connected to the server 76;

[0117] The fixing bracket 74 is used to fix the observation camera 72 and the head-mounted display device to be calibrated; the head-mounted display device is arranged between the lens of the observation camera 72 and the calibration object; the observation camera 72 is used to photograph the calibration object through the spectacle lens of the head-mounted display device, and photograph the virtual pattern drawn on the display screen of the head-mounted display device; the server 76 is used to execute the calibration method of the head-mounted display device provided in any of the above embodiments.

[0118] The observation camera 72 may include a left-eye observation camera 721 and a right-eye observation camera 722; the left-eye observation camera 721 may photograph the calibration object through the left spectacle lens of the head-mounted display device; the right-eye observation camera 722 may photograph the calibration object through the right spectacle lens of the head-mounted display device. The fixing bracket 74 is used to fix the observation camera 72 and the head-mounted display device. Among them, the support height of the observation camera 72 and the head-mounted display device supported by the fixing bracket 74, and the distance between the observation camera 72 and the head-mounted display device can both be adjusted by the telescoping of the fixing bracket 74.

[0119] In this embodiment, a calibration system for optically transparent calibration of a head-mounted display device is provided. Based on the calibration system 70 of the head-mounted display device in this embodiment, a convenient and accurate virtual-real combination calibration considering the influence of lens distortion can be realized. Additionally, based on the calibration system 70 of the head-mounted display device in this embodiment, automatic calibration can be realized without relying on manual participation, so that it can be applied to the factory calibration of the head-mounted display device in a mass production environment.

[0120] In this embodiment, an electronic device is further provided, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments.

[0121] Optionally, the above electronic device may further include a transmission device and an input / output device. Among them, the transmission device is connected to the above processor, and the input / output device is connected to the above processor.

[0122] Optionally, in this embodiment, the above processor may be configured to execute the following steps through a computer program:

[0123] S1, according to the calibration object image set obtained by simultaneously photographing a preset calibration object by an observation camera and a tracking camera, jointly calibrate the observation camera and the tracking camera to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the spectacle lens of the head-mounted display device; the tracking camera is arranged on the head-mounted display device;

[0124] S2. Draw a virtual pattern for the calibration object on the display screen of the head-mounted display device;

[0125] S3. Based on the virtual pattern, determine the three-dimensional feature coordinates of the calibration object, and obtain the target image obtained by the observation camera shooting the virtual pattern;

[0126] S4. According to the three-dimensional feature coordinates, the target image, and the internal parameters of the observation camera, combined with the external parameters between the observation camera and the tracking camera, determine the optical perspective calibration result for the head-mounted display device.

[0127] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation manners, and will not be elaborated in this embodiment.

[0128] In addition, in combination with the calibration method of the head-mounted display device provided in the above embodiments, a storage medium can also be provided to implement it in this embodiment. A computer program is stored on the storage medium; when the computer program is executed by a processor, any one of the calibration methods of the head-mounted display device in the above embodiments is implemented.

[0129] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0130] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties.

[0131] Obviously, the drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations according to these drawings without creative work. In addition, it can be understood that although the work done during this development process may be complex and time-consuming, for those of ordinary skill in the art, some design, manufacturing, or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient disclosure of the present application.

[0132] As used in this application, the term "embodiment" means that the specific features, structures, or characteristics described in connection with an embodiment may be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily mean the same embodiment, nor does it mean independence or alternative to other embodiments that are mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in this application can be combined with other embodiments without conflict.

[0133] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several modifications and improvements can still be made, and these all fall within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A calibration method for a head-mounted display device, characterized in that, Including: Performing joint calibration on the observation camera and the tracking camera based on a set of calibration object images obtained by simultaneously photographing a preset calibration object with the observation camera and the tracking camera, to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the glasses lens of the head-mounted display device; the tracking camera is disposed on the head-mounted display device; Drawing a virtual pattern for the calibration object on the display screen of the head-mounted display device; Based on the virtual pattern, determining the three-dimensional feature coordinates of the calibration object, and obtaining a target image obtained by the observation camera photographing the virtual pattern; According to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera, determining an optical perspective calibration result for the head-mounted display device.

2. The calibration method of the head-mounted display device according to claim 1, wherein Based on the virtual pattern, determining the three-dimensional feature coordinates of the calibration object, including: Extracting two-dimensional feature coordinates of the calibration object from the virtual pattern; According to the two-dimensional feature coordinates and the pre-obtained internal parameters of the virtual camera, determining the three-dimensional feature coordinates of the calibration object in a three-dimensional coordinate system established with the plane where the display screen is located; the internal parameters of the virtual camera are determined based on the field of view parameters of the head-mounted display device.

3. The calibration method of the head-mounted display device according to claim 1, characterized in that According to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, and in combination with the external parameters between the observation camera and the tracking camera, determining an optical perspective calibration result for the head-mounted display device, including: Extracting calibration features of the calibration object from the target image to obtain two-dimensional image coordinates of the calibration features in the target image; According to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera, determining a first conversion relationship; the first conversion relationship represents the conversion relationship between the observation camera coordinate system and the virtual world coordinate system; the virtual world coordinate system is a two-dimensional coordinate system of the virtual world presented by the display screen; According to the first conversion relationship, and in combination with the external parameters between the observation camera and the tracking camera, determining an optical perspective calibration result for the head-mounted display device.

4. The calibration method of the head-mounted display device according to claim 3, characterized in that, According to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera, determining a first conversion relationship, including: Performing geometric solution according to the three-dimensional feature coordinates, the two-dimensional image coordinates, and the internal parameters of the observation camera to determine a third conversion relationship; the third conversion relationship is the conversion relationship between a three-dimensional coordinate system established with the plane where the display screen is located and the observation camera coordinate system; According to the third conversion relationship, determining the first conversion relationship.

5. The calibration method of the head-mounted display device according to claim 3, wherein According to the first conversion relationship, and in combination with the external parameters between the observation camera and the tracking camera, determining an optical perspective calibration result for the head-mounted display device, including: According to the external parameters between the observation camera and the tracking camera, determining a second conversion relationship between the physical space coordinate system established with the tracking camera and the observation camera coordinate system; Determine the optical see-through calibration result of the head-mounted display device according to the first conversion relationship and the second conversion relationship.

6. The calibration method of the head-mounted display device according to any one of claims 1 to 5, characterized in that, Based on a set of calibration object images obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera, perform joint calibration on the observation camera and the tracking camera to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera, including: A set of calibration object images obtained by simultaneously photographing a preset calibration object with an observation camera and a tracking camera; Extract features from each image in the set of calibration object images to determine the calibration features of the calibration object in each image; Based on the image coordinates corresponding to each calibration feature in the set of calibration object images, determine the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera.

7. A calibration device for a head-mounted display device, characterized in that, Including: A joint calibration module, a pattern display module, an acquisition module, and an optical see-through calibration module; wherein: The joint calibration module is configured to perform joint calibration on the observation camera and the tracking camera based on a set of calibration object images obtained by simultaneously photographing a preset calibration object with the observation camera and the tracking camera, to obtain the external parameters between the observation camera and the tracking camera, and the internal parameters of the observation camera; wherein, the observation camera photographs the calibration object through the spectacle lenses of the head-mounted display device; the tracking camera is disposed on the head-mounted display device; The pattern display module is configured to draw a virtual pattern for the calibration object on the display screen of the head-mounted display device; The acquisition module is configured to determine the three-dimensional feature coordinates of the calibration object based on the virtual pattern, and acquire a target image obtained by the observation camera photographing the virtual pattern; The optical see-through calibration module is configured to determine the optical see-through calibration result for the head-mounted display device according to the three-dimensional feature coordinates, the target image, the internal parameters of the observation camera, in combination with the external parameters between the observation camera and the tracking camera.

8. A calibration system for a head-mounted display device, characterized in that, Including: An observation camera, a fixed bracket, and a server; the observation camera is communicatively connected to the server; The fixed bracket is used to fix the observation camera and the head-mounted display device to be calibrated; the head-mounted display device is disposed between the lens of the observation camera and the calibration object; the observation camera is used to photograph the calibration object through the spectacle lenses of the head-mounted display device, and photograph the virtual pattern drawn on the display screen of the head-mounted display device; The server is configured to execute the calibration method of the head-mounted display device according to any one of claims 1 to 6.

9. An electronic device, comprising a memory and a processor, characterized in that, A computer program is stored in the memory, and the processor is configured to run the computer program to execute the calibration method of the head-mounted display device according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the calibration method of the head-mounted display device according to any one of claims 1 to 6 are implemented.

Images