Viewpoint distance test method, head-mounted display device and computer readable medium

Abstract

The embodiment of the invention discloses a viewpoint distance testing method, head-mounted display equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: in response to a detected viewpoint distance test request, performing spatial modeling according to a camera device included in the head-mounted display equipment to obtain a viewpoint space; determining a calibration object; aligning the viewpoint space with the scene space according to the viewpoint space coordinate system and a scene space coordinate system corresponding to the scene space; according to the aligned viewpoint space, viewpoint distance information corresponding to the calibration object is generated, and the viewpoint distance information represents the distance between the calibration object and the target user in the aligned viewpoint space. According to the embodiment, the viewpoint distance of the user can be determined, the distance between the calibration object and the user in the scene space is measured through the viewpoint distance, the difference between the measured position distance and the object depth sensed by the user wearing the device is reduced, and therefore the visual experience of the user is improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technologies, and particularly to a viewpoint distance testing method, a head-mounted display device, and a computer-readable medium. Background Art

[0002] A head-mounted display device (such as an AR glasses) can be used to construct a three-dimensional scene in front of the eyes of a wearing user to provide a strong visual experience. In the constructed three-dimensional scene, the user can interact with virtual objects in the scene to obtain relevant information of the objects. For example, the distance between a virtual object and the user himself can be measured. Currently, when measuring the distance between a virtual object in a three-dimensional scene and a wearing user, the commonly used method is to determine the straight-line distance between two points through the user's position and the virtual object's position in three-dimensional space.

[0003] However, when performing viewpoint distance testing in the above manner, there are often the following technical problems: Due to the parallax between the user's left and right eyes, the distance result directly measured through the user's position and the virtual object's position is different from the actual depth perception under the user's left and right eye perspectives, resulting in a poor visual experience for the user.

[0004] The above information disclosed in this background art section is only used to enhance the understanding of the background of the inventive concept of the present invention. Therefore, it may include information that does not form the prior art known to those of ordinary skill in the art in this country. Summary of the Invention

[0005] This summary of the disclosure is used to introduce concepts in a concise form, and these concepts will be described in detail in the following detailed implementation section. This summary of the disclosure is not intended to identify the key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] Some embodiments of the present disclosure propose a viewpoint distance testing method, a head-mounted display device, and a computer-readable medium based on a head-mounted display device to solve one or more of the technical problems mentioned in the above background art section.

[0007] In a first aspect, some embodiments of the present disclosure provide a method for testing the viewing distance based on a head-mounted display device. The method includes: in response to detecting a viewing distance test request, performing spatial modeling based on a camera device included in the head-mounted display device to obtain a viewing space, where the viewing space corresponds to a viewing space coordinate system, the head-mounted display device includes a camera device corresponding to the left eye of a target user and a camera device corresponding to the right eye of the target user, the axial distance between the two camera devices included in the head-mounted display device corresponds to a target distance range, and the target user is a user wearing the head-mounted display device; determining a calibration object, where the calibration object is a virtual object in the scene space displayed by the head-mounted display device; aligning the viewing space with the scene space according to the viewing space coordinate system and the scene space coordinate system corresponding to the scene space; generating viewing distance information corresponding to the calibration object according to the aligned viewing space, where the viewing distance information represents the distance between the calibration object and the target user in the aligned viewing space.

[0008] Optionally, the camera device included in the head-mounted display device corresponds to camera device parameter information; and the performing spatial modeling based on the camera device included in the head-mounted display device to obtain a viewing space includes: performing three-dimensional spatial modeling through a modeling tool to obtain a three-dimensional space model; for each camera device included in the head-mounted display device, modeling the camera device in the three-dimensional space model to obtain a virtual camera device; and determining the updated three-dimensional space model as the viewing space, where the axial distance between the two virtual camera devices in the updated three-dimensional space model corresponds to the target distance range.

[0009] Optionally, the aligning the viewing space with the scene space according to the viewing space coordinate system and the scene space coordinate system corresponding to the scene space constructed in the head-mounted display device includes: performing coordinate transformation processing on the viewing space coordinate system according to the scene space coordinate system to align the viewing space with the scene space; for each virtual camera device in the aligned viewing space, updating the device parameter information of the virtual camera device through the scene space and the camera device parameter information corresponding to the target camera device, where the target camera device is the camera device corresponding to the virtual camera device among the various camera devices included in the head-mounted display device.

[0010] Optionally, before generating the viewpoint distance information corresponding to the calibrated object according to the aligned viewpoint space, the method further includes: for each virtual camera device model in the aligned viewpoint space, determining the viewing angle range information according to the virtual device parameter information corresponding to the virtual camera device, where the viewing angle range information represents the viewing angle range of the virtual camera device in the aligned viewpoint space; generating target parallax position information according to the determined viewing angle range information, where the target parallax position information represents the position of the target parallax plane under the viewing angle of the target user in the aligned viewpoint space.

[0011] Optionally, the method further includes: displaying the generated viewpoint distance information on the head-mounted display device.

[0012] In a second aspect, some embodiments of the present disclosure provide a head-mounted display device, including: one or more processors; an optical display system including at least one display screen and optical elements for imaging in front of a user's eyes; a camera device system including a camera device corresponding to the user's left eye of the target user and a camera device corresponding to the user's right eye of the target user, where the target user is the user wearing the head-mounted display device; a storage device storing one or more programs thereon, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the first aspect.

[0013] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium storing a computer program thereon, where the program implements the method described in any implementation manner of the first aspect when executed by a processor.

[0014] The above-mentioned various embodiments of the present disclosure have the following beneficial effects: Through the viewpoint distance measurement method based on a head-mounted display device in some embodiments of the present disclosure, the user's viewpoint distance can be determined, and the distance between the calibration object and the user in the scene space can be measured through the viewpoint distance, reducing the difference between the measured position distance and the object depth perceived by the wearing user, thereby improving the user's visual experience. Specifically, the reason for the poor relevant user visual experience is that: due to the parallax between the user's left and right eyes, the distance result directly measured through the user's position and the virtual object position is different from the actual depth perception under the user's left and right eye perspectives, resulting in a poor user visual experience. Based on this, in some embodiments of the present disclosure, the viewpoint distance measurement method based on a head-mounted display device first, in response to detecting a viewpoint distance measurement request, performs spatial modeling according to the camera device included in the head-mounted display device to obtain a viewpoint space. Among them, the above-mentioned viewpoint space corresponds to a viewpoint space coordinate system. The above-mentioned head-mounted display device includes a camera device corresponding to the left eye of the target user and a camera device corresponding to the right eye of the above-mentioned target user. The axial distance between the two camera devices included in the above-mentioned head-mounted display device corresponds to a target distance range. The above-mentioned target user is the user wearing the above-mentioned head-mounted display device. Thus, the camera device of the head-mounted display device can simulate the user's eyes, and a viewpoint space that can represent the user's perspective and is used to determine the user's viewpoint distance can be constructed. Then, a calibration object is determined. Among them, the above-mentioned calibration object is a virtual object in the scene space displayed by the above-mentioned head-mounted display device. Thus, the virtual object selected by the target user in the scene space can be determined as the calibration object, thereby determining the distance measurement target. After that, according to the above-mentioned viewpoint space coordinate system and the scene space coordinate system corresponding to the above-mentioned scene space, the above-mentioned viewpoint space is aligned with the above-mentioned scene space. Thus, the virtual camera device included in the viewpoint space can be used to align the constructed viewpoint space with the scene space including the calibration object, so that the user perspective simulated by the virtual camera device matches the real perspective of the target user. Finally, according to the aligned viewpoint space, viewpoint distance information corresponding to the above-mentioned calibration object is generated. Among them, the above-mentioned viewpoint distance information represents the distance between the above-mentioned calibration object and the above-mentioned target user in the aligned viewpoint space. Thus, through the aligned viewpoint space, the distance between the calibration object and the user's viewpoint can be determined, and viewpoint distance information can be generated. The viewpoint distance information can effectively represent the object distance perceived by the user in the virtual space, thereby reducing the difference between the measured position distance of the calibration object and the object depth perceived by the wearing user, thereby improving the user's visual experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the elements and components are not necessarily drawn to scale.

[0016] Figure 1 is an architectural diagram of an exemplary system to which some embodiments of the present disclosure may be applied;

[0017] Figure 2 is a flowchart of some embodiments of a method for testing the viewing distance based on a head-mounted display device according to the present disclosure;

[0018] Figure 3 is a flowchart of some other embodiments of a method for testing the viewing distance based on a head-mounted display device according to the present disclosure;

[0019] Figure 4 is a schematic diagram of the principle for determining the zero parallax plane of a method for testing the viewing distance based on a head-mounted display device according to the present disclosure;

[0020] Figure 5 is a schematic diagram of the principle for determining the viewing distance of a method for testing the viewing distance based on a head-mounted display device according to the present disclosure;

[0021] Figure 6 is a schematic structural diagram of an electronic device suitable for implementing some embodiments of the present disclosure. Specific Embodiments

[0022] The embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0023] In addition, it should be noted that for the sake of convenience of description, only the parts related to the relevant invention are shown in the drawings. Without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0024] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules, or units, and are not used to limit the order of the functions performed by these devices, modules, or units or their interdependent relationships.

[0025] It should be noted that the modifications of "one" and "multiple" mentioned in this disclosure are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly specified in the context, it should be understood as "one or more".

[0026] The names of the messages or information exchanged between multiple devices in the embodiments of this disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0027] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0028] Figure 1 An exemplary system architecture 100 of a viewpoint distance test method based on a head-mounted display device to which some embodiments of the present disclosure can be applied is shown.

[0029] As Figure 1 shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0030] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various communication client applications may be installed on the terminal devices 101, 102, 103, such as web browser applications, program development class applications, search class applications, instant messaging tools, email clients, social platform software, etc. The terminal device 103 may be a head-mounted display device. Various screen mirroring applications may be installed on the terminal device 103.

[0031] The terminal devices 101, 102 may be hardware or software. When the terminal devices 101, 102 are hardware, they may be various electronic devices with a display screen and supporting information display, including but not limited to smart phones, tablet computers, e-book readers, laptop portable computers, and desktop computers, etc. When the terminal devices 101, 102 are software, they may be installed in the above-listed electronic devices. It may be implemented as, for example, multiple software or software modules for providing distributed services, or may be implemented as a single software or software module. No specific limitation is made here.

[0032] The server 105 may be a server providing various services, such as a background server that provides support for the information displayed on the terminal devices 101, 102, 103. The background server may analyze and process data such as received requests, and feedback the processing results to the terminal devices.

[0033] It should be noted that the method for testing the viewing distance based on the head-mounted display device provided in the embodiments of the present disclosure can be executed by the intelligent terminal included in the terminal device 103.

[0034] It should be noted that the server can be hardware or software. When the server is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When the server is software, it can be implemented as multiple software or software modules for providing distributed services, or as a single software or software module. Specific limitations are not made here.

[0035] Continue to refer to Figure 2 , which shows the process 200 of some embodiments of the method for testing the viewing distance based on the head-mounted display device according to the present disclosure. The method for testing the viewing distance based on the head-mounted display device includes the following steps:

[0036] Step 201, in response to detecting a viewing distance test request, perform spatial modeling according to the imaging device included in the head-mounted display device to obtain a viewing space.

[0037] In some embodiments, the execution subject of the method for testing the viewing distance based on the head-mounted display device (such as Figure 1 the computing device 103 shown) can, in response to detecting a viewing distance test request, perform spatial modeling according to the imaging device included in the head-mounted display device to obtain a viewing space. Among them, the above-mentioned viewing space corresponds to a viewing space coordinate system. The above-mentioned head-mounted display device includes an imaging device corresponding to the left eye of the target user and an imaging device corresponding to the right eye of the above-mentioned target user. The above-mentioned head-mounted display device can be, but is not limited to, one of the following: AR glasses, MR glasses, VR glasses. The above-mentioned imaging device can be, but is not limited to, a 3D camera. The above-mentioned viewing space can be a three-dimensional space for determining the user's zero parallax distance and measuring the distance between the calibration object and the user. The axial distance between the two imaging devices included in the above-mentioned head-mounted display device corresponds to a target distance range. The above-mentioned target user is the user wearing the above-mentioned head-mounted display device. The above-mentioned axial distance can be the horizontal distance between the lenses of the two imaging devices. The axial distance between the two imaging devices included in the above-mentioned head-mounted display device corresponding to the target distance range can mean that the axial distance between the two imaging devices is within the distance interval represented by the above-mentioned target distance range. As an example, the above-mentioned target distance range can be 29 mm to 32 mm. In practice, the above-mentioned execution subject can use the SLAM technology through the imaging device included in the head-mounted display device to perform spatial modeling to obtain a three-dimensional space model as the viewing space. Thus, the interpupillary distance of the wearing user can be simulated through the axial distance between the 3D cameras.

[0038] In some alternative implementations of some embodiments, the above-mentioned execution entity may perform spatial modeling based on the imaging device included in the head-mounted display device through the following steps to obtain a viewpoint space:

[0039] First step, perform three-dimensional spatial modeling through a modeling tool to obtain a three-dimensional space model. In practice, the above-mentioned execution entity may construct a three-dimensional space model through a modeling tool. As an example, the above-mentioned modeling tool may be Maya modeling software.

[0040] Second step, for each imaging device included in the above-mentioned head-mounted display device, in the above-mentioned three-dimensional space model, model the above-mentioned imaging device to obtain a virtual imaging device. In practice, the above-mentioned execution entity may model in the above-mentioned three-dimensional space model according to the imaging devices included in the head-mounted display device through the above-mentioned modeling tool (for example, create a camera object) to obtain two virtual imaging devices.

[0041] Third step, determine the updated three-dimensional space model as the viewpoint space. Among them, the axial distance between the two virtual imaging devices in the updated three-dimensional space model corresponds to the above-mentioned target distance range. The axial distance between the two virtual imaging devices corresponding to the above-mentioned target distance range may mean that the axial distance between the two virtual imaging devices is within the distance range characterized by the above-mentioned target distance range. The axial distance between the two virtual imaging devices may also be the same as the axial distance between the two imaging devices.

[0042] Step 202, determine a calibration object.

[0043] In some embodiments, the above-mentioned execution entity may determine a calibration object. Among them, the above-mentioned calibration object is a virtual object in the scene space displayed by the above-mentioned head-mounted display device. The above-mentioned calibration object may be a virtual object selected by the target user in the above-mentioned scene space. The above-mentioned virtual object may be a scene element in the above-mentioned scene space. The above-mentioned scene elements include but are not limited to objects, pictures, videos, texts. The scene object may be a two-dimensional object or a three-dimensional object.

[0044] Step 203, align the viewpoint space with the scene space according to the viewpoint space coordinate system and the scene space coordinate system corresponding to the scene space.

[0045] In some embodiments, the above-mentioned execution entity may align the above-mentioned viewpoint space with the above-mentioned scene space according to the above-mentioned viewpoint space coordinate system and the scene space coordinate system corresponding to the above-mentioned scene space. In practice, the above-mentioned execution entity may align the above-mentioned viewpoint space coordinate system with the scene space coordinate system corresponding to the scene space through translation transformation and rotation transformation.

[0046] In some alternative implementations of some embodiments, the above-mentioned execution entity can align the above-mentioned view point space with the above-mentioned scene space according to the above-mentioned view point space coordinate system and the scene space coordinate system corresponding to the above-mentioned scene space through the following steps:

[0047] First step, according to the above-mentioned scene space coordinate system, perform coordinate transformation processing on the above-mentioned view point space coordinate system to align the above-mentioned view point space with the above-mentioned scene space. In practice, the above-mentioned execution entity can determine the coordinates of the coordinate origin of the above-mentioned view point space coordinate system in the above-mentioned view point space coordinate system as the comparison coordinates. Then, determine the translation matrix and rotation matrix between the two coordinate systems through the origin coordinates of the above-mentioned view point space coordinate system and the determined comparison coordinates. Finally, the coordinate system transformation of the above-mentioned view point space coordinate system can be performed through the determined translation matrix and rotation matrix to align the view point space and the scene space.

[0048] Second step, for each virtual camera device in the aligned view point space, update the device parameter information of the above-mentioned virtual camera device through the above-mentioned scene space and the camera device parameter information corresponding to the target camera device. Wherein, the above-mentioned target camera device is the camera device corresponding to the above-mentioned virtual camera device among the various camera devices included in the above-mentioned head-mounted display device. The above-mentioned camera device parameter information may be the camera parameters of the corresponding camera device. The above-mentioned camera parameters may include, but are not limited to: lens distortion coefficient, camera focal length, view matrix, camera orientation, viewing angle, window aspect ratio, frustum height, frustum width, near clipping plane information, far clipping plane information, interaxial separation distance of the camera, and zero parallax information. The above-mentioned near clipping plane information may be the plane coordinate range corresponding to the near clipping plane in the coordinate system of the aligned view point space. The above-mentioned far clipping plane information may be the plane coordinate ranges respectively corresponding to the far clipping planes in the coordinate system of the aligned view point space. The above-mentioned interaxial separation distance of the camera may be the binocular pupil distance of the target user measured by the target camera device. The above-mentioned interaxial separation distance of the camera can enable the virtual camera device to simulate the binocular pupil distance of the target user in the above-mentioned scene space, thereby simulating the perspective of the target user. The above-mentioned zero parallax information may be the position of the zero parallax plane measured by the above-mentioned target camera device or preset. In practice, the above-mentioned execution entity can update the device parameter information of the virtual camera device to the corresponding camera device parameter information.

[0049] Step 204, generate the view point distance information corresponding to the calibration object according to the aligned view point space.

[0050] In some embodiments, the above-mentioned execution entity may generate viewpoint distance information corresponding to the above-mentioned calibrated object according to the aligned viewpoint space. Wherein, the above-mentioned viewpoint distance information represents the distance between the above-mentioned calibrated object and the above-mentioned target user in the aligned viewpoint space. The above-mentioned viewpoint distance information may be the horizontal distance between the above-mentioned calibrated object and the above-mentioned target user in the aligned viewpoint space. In practice, the above-mentioned execution entity may determine the position coordinates of the virtual camera device and the calibrated object in the above-mentioned aligned viewpoint space. Then, the horizontal distance between the position coordinates of the virtual camera device and the position coordinates of the calibrated object is determined as the viewpoint distance information. For example, if the direction of the user's perspective simulated by the virtual camera device is the same as the positive direction of the x-axis of the coordinate system of the aligned viewpoint space, the above-mentioned execution entity may determine the absolute value of the difference between the abscissa of the position coordinates of the virtual camera device and the abscissa of the position coordinates of the calibrated object as the viewpoint distance information.

[0051] Optionally, the above-mentioned execution entity may also display the generated viewpoint distance information in the above-mentioned head-mounted display device. In practice, the above-mentioned execution entity may directly display the generated viewpoint distance information in the above-mentioned scene space in the above-mentioned head-mounted display device. The above-mentioned execution entity may also display the generated viewpoint distance information on the surface of the above-mentioned calibrated object or at a preset distance.

[0052] The above embodiments of the present disclosure have the following beneficial effects: Through the viewpoint distance testing method based on a head-mounted display device according to some embodiments of the present disclosure, the user's viewpoint distance can be determined, and the distance between the calibration object and the user in the scene space can be measured through the viewpoint distance, reducing the difference between the measured position distance and the object depth perceived by the wearing user, thereby improving the user's visual experience. Specifically, the reason for the relatively poor user visual experience is that: due to the parallax between the user's left and right eyes, the distance results directly measured through the user's position and the virtual object position are different from the actual depth perception under the user's left and right eye perspectives, resulting in a relatively poor user visual experience. Based on this, in the viewpoint distance testing method based on a head-mounted display device according to some embodiments of the present disclosure, first, in response to detecting a viewpoint distance testing request, spatial modeling is performed according to the imaging device included in the head-mounted display device to obtain a viewpoint space. Among them, the above-mentioned viewpoint space corresponds to a viewpoint space coordinate system. The above-mentioned head-mounted display device includes an imaging device corresponding to the left eye of the target user and an imaging device corresponding to the right eye of the above-mentioned target user. The axial distance between the two imaging devices included in the above-mentioned head-mounted display device corresponds to a target distance range. The above-mentioned target user is the user wearing the above-mentioned head-mounted display device. Thus, the imaging device of the head-mounted display device can simulate the user's eyes, and a viewpoint space that can represent the user's perspective and is used to determine the user's viewpoint distance can be constructed. Then, a calibration object is determined. Among them, the above-mentioned calibration object is a virtual object in the scene space displayed by the above-mentioned head-mounted display device. Thus, the virtual object selected by the target user in the scene space can be determined as the calibration object, thereby determining the distance testing target. After that, according to the viewpoint space coordinate system and the scene space coordinate system corresponding to the above-mentioned scene space, the above-mentioned viewpoint space and the above-mentioned scene space are aligned. Thus, the constructed viewpoint space can be aligned with the scene space including the calibration object through the virtual imaging device included in the viewpoint space, so that the user perspective simulated by the virtual imaging device matches the real perspective of the target user. Finally, according to the aligned viewpoint space, viewpoint distance information corresponding to the above-mentioned calibration object is generated. Among them, the above-mentioned viewpoint distance information represents the distance between the calibration object and the above-mentioned target user in the aligned viewpoint space. Thus, the distance between the calibration object and the user's viewpoint can be determined through the aligned viewpoint space, and viewpoint distance information can be generated. The viewpoint distance information can effectively represent the object distance perceived by the user in the virtual space, thereby reducing the difference between the measured position distance of the calibration object and the object depth perceived by the wearing user, thereby improving the user's visual experience.

[0053] Further referring to Figure 3 , a flowchart 300 of some other embodiments of the viewpoint distance testing method based on a head-mounted display device according to the present disclosure is shown. The viewpoint distance testing method based on a head-mounted display device includes the following steps:

[0054] Step 301, in response to detecting a viewpoint distance test request, perform spatial modeling based on the camera device included in the head-mounted display device to obtain a viewpoint space.

[0055] Step 302, determine a calibration object.

[0056] Step 303, align the viewpoint space and the scene space according to the scene space coordinate system corresponding to the viewpoint space coordinate system and the scene space.

[0057] In some embodiments, the specific implementation and the technical effects brought by steps 301-303 can refer to Figure 2 Steps 201-203 in the corresponding embodiments, which will not be elaborated here.

[0058] Step 304, for each virtual camera device model in the aligned viewpoint space, determine the viewing angle range information according to the virtual device parameter information corresponding to the virtual camera device.

[0059] In some embodiments, the above-mentioned execution entity can determine the viewing angle range information for each virtual camera device model in the aligned viewpoint space according to the virtual device parameter information corresponding to the virtual camera device. Among them, the above-mentioned viewing angle range information represents the viewing angle range of the virtual camera device in the aligned viewpoint space. In practice, the above-mentioned execution entity can determine the viewing angle, aspect ratio of the viewport, height of the viewing frustum, width of the viewing frustum, near clipping plane information, and far clipping plane information included in the virtual device parameter information corresponding to the virtual camera device model as the viewing angle range information. The above-mentioned viewing angle range information can represent the viewing frustum (i.e., the viewing angle range) under the user's perspective simulated by the virtual camera device.

[0060] Step 305, generate target parallax position information according to the determined viewing angle range information.

[0061] In some embodiments, the above-mentioned execution entity can generate target parallax position information according to the determined viewing angle range information. Among them, the above-mentioned target parallax position information represents the position of the target parallax plane under the viewing angle of the target user in the aligned viewpoint space. The above-mentioned target parallax plane can be a zero parallax plane. In practice, first, the above-mentioned execution entity can simulate the viewing frustum of the virtual camera device corresponding to the left eye of the target user and the viewing frustum of the virtual camera device corresponding to the right eye of the target user in the aligned viewpoint space through the determined two viewing angle range information respectively. Then, the above-mentioned execution entity can determine the position coordinates of the intersection of the two simulated viewing frustums as the target parallax position information.

[0062] In the top-down view angle, the principle schematic of determining the target parallax position information is as Figure 4As shown Figure 4 It includes a virtual camera device 401 corresponding to the left eye of the target user, a virtual camera device 402 corresponding to the right eye of the target user, a simulated frustum 403 of the virtual camera device 401, a simulated frustum 404 of the virtual camera device 402, and a schematic zero parallax plane 405 located at the intersection of the frustum 403 and the frustum 404 and parallel to the virtual camera devices 401 and 402.

[0063] It should be added that parallax is an optical phenomenon, which refers to the relative movement of the position of the same object in the field of view due to the change of the observer's perspective. When an object moves relative to a distant and stable object in the background, the observer may see the object displaced visually relative to the background. Zero parallax refers to the state where there is no parallax phenomenon between the object and the background when observing the object. This usually occurs when the observer is at a specific position, such that the change in the perspective of the object is very small or negligible. In this case, the approximate position of the object does not change with the change of the observer's perspective, and the visual position of the object is consistent with the real position. The zero parallax plane is the plane where the zero parallax points exist from the user's perspective.

[0064] In the process of adopting the above technical solution to solve the problems mentioned in the background technology, the following problems often occur: in the process of determining the user's viewing point distance, there are often coordinate misalignments or binocular viewing point misalignments in the virtual reality / augmented reality environment, resulting in a large error between the perceived distance and the actual distance of the calibration object by the wearing user.

[0065] Facing the above technical problems, the inventor decides to adopt the following solution:

[0066] Step 306, establish a viewing point coordinate system in the aligned viewing point space according to the target parallax position information.

[0067] In some embodiments, the above execution subject can establish a viewing point coordinate system in the aligned viewing point space according to the above target parallax position information. In practice, the above execution subject can use the above target parallax position information as the origin position, take the positive direction of the Z-axis of the viewing point space coordinate system corresponding to the aligned viewing point space as the positive direction of the vertical axis, take the negative direction of the X-axis of the viewing point space coordinate system corresponding to the aligned viewing point space as the positive direction of the horizontal axis, and take the negative direction of the Y-axis of the viewing point space coordinate system corresponding to the aligned viewing point space as the positive direction of the vertical axis to establish a viewing point coordinate system.

[0068] Step 307, determine the position information of the calibration object.

[0069] In some embodiments, the above-mentioned execution entity may determine the calibrated object position information. Wherein, the above-mentioned calibrated object position information represents the position of the above-mentioned calibrated object in the constructed viewpoint coordinate system. In practice, the above-mentioned execution entity may determine the coordinate position of the calibrated object in the established viewpoint space coordinate system as the calibrated object position information.

[0070] Step 308, determine the user position information according to the virtual camera device model.

[0071] In some embodiments, the above-mentioned execution entity may determine the user position information according to the virtual camera device model. Wherein, the above-mentioned user position information represents the position of the above-mentioned target user in the constructed viewpoint coordinate system. In practice, the above-mentioned execution entity may determine the position coordinates of the above-mentioned virtual camera device model in the above-mentioned viewpoint coordinate system as the user position information.

[0072] Step 309, generate the viewpoint distance information according to the calibrated object position information and the user position information.

[0073] In some embodiments, the above-mentioned execution entity may generate the viewpoint distance information according to the above-mentioned calibrated object position information and the above-mentioned user position information. In practice, the above-mentioned execution entity may determine the horizontal distance between the above-mentioned calibrated object position information and the origin of the viewpoint space coordinate system as the first horizontal distance, and determine the horizontal distance between the above-mentioned user position information and the origin of the viewpoint space coordinate system as the second horizontal distance. Then, the difference between the above-mentioned second horizontal distance and the first horizontal distance may be determined as the viewpoint distance information to generate the viewpoint distance information.

[0074] The principle of determining the viewpoint distance is schematically shown as Figure 5 shown, Figure 5 including the virtual camera device 501 corresponding to the left eye of the target user, the virtual camera device 502 corresponding to the left eye of the target user, the schematic zero parallax plane 503, the calibrated object 504, and the aligned viewpoint space (i.e., the scene space) 505. Among them, the zero parallax plane 503 is parallel to the virtual camera devices 501 and 502, and the zero parallax plane 503 is perpendicular to the symmetry axis between the virtual camera devices 501 and 502.

[0075] The above steps 306-307 are an inventive point of the embodiments of the present disclosure, which solve the technical problem that there is a large error between the perceived distance and the actual distance of the calibration object by the wearing user. The factors that often cause a large error between the perceived distance and the actual distance of the calibration object by the wearing user are as follows: In the process of determining the user's viewing point distance, there are often coordinate system misalignments or binocular viewing point misalignments in the virtual reality / augmented reality environment, resulting in a large error between the perceived distance and the actual distance of the calibration object by the wearing user. If the above factors are solved, the error between the perceived distance and the actual distance of the calibration object by the wearing user can be reduced, and the accuracy of the determined viewing point distance can be improved. To achieve this effect, in this application, first, a user-centered coordinate system is constructed based on the target parallax plane. Thus, the spatial misalignment between the global coordinate system of the scene and the actual viewing point of the user can be eliminated. Then, the virtual object is mapped into the viewing point coordinate system, and its spatial position is accurately calibrated through three-dimensional coordinate transformation. Thus, the positioning deviation caused by the mismatch between the absolute position of the calibration object in the scene coordinate system and the user's perspective can be avoided. Finally, the viewing point distance is determined based on the position of the wearing user in the viewing point coordinate system. Thus, by aligning the coordinate system and determining the position within the user coordinate system, coordinate system misalignment or binocular viewing point misalignment can be avoided, thereby reducing the error between the perceived distance and the actual distance of the calibration object by the wearing user and improving the accuracy of viewing point distance determination.

[0076] Reference is made below to Figure 6 , which shows a schematic structural diagram of a head-mounted display device 600 (such as Figure 1 the terminal device 103 in Figure 6 ) suitable for use in implementing some embodiments of the present disclosure.

[0077] As Figure 6 shown, the head-mounted display device 600 may include a processing device 601 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the head-mounted display device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0078] Typically, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, at least one display screen, a speaker, a vibrator, etc.; and a communication device 609. At least one display screen can combine with optical elements to image the display content in front of the user's eyes. The communication device 609 can allow the head-mounted display device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 a head-mounted display device 600 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. More or fewer devices can be alternatively implemented or had. Figure 6 Each block shown in can represent one device or multiple devices as needed.

[0079] The above-mentioned head-mounted display device may further include an optical display system. The optical display system includes at least one display screen and optical elements for imaging in front of the user's eyes. The above-mentioned head-mounted display device may further include a camera device system. The camera device system includes a camera device corresponding to the user's left eye of the target user and a camera device corresponding to the user's right eye of the target user, where the target user is the user wearing the head-mounted display device.

[0080] Optionally, the above-mentioned head-mounted display device may include a head-mounted display device body and a smart terminal, and the smart terminal is communicatively connected to the head-mounted display device body.

[0081] In particular, according to some embodiments of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such some embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of some embodiments of the present disclosure are executed.

[0082] It should be noted that the computer-readable media described in some embodiments of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0083] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0084] The above computer-readable medium may be included in the above head-mounted display device; or it may exist independently and not be assembled into the head-mounted display device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the head-mounted display device, the head-mounted display device is caused to: in response to detecting a viewpoint distance test request, perform spatial modeling based on a camera device included in the head-mounted display device to obtain a viewpoint space, wherein the above viewpoint space corresponds to a viewpoint space coordinate system, the head-mounted display device includes a camera device corresponding to the left eye of a target user and a camera device corresponding to the right eye of the above target user, the axial distance between the two camera devices included in the head-mounted display device corresponds to a target distance range, and the above target user is a user wearing the head-mounted display device; determine a calibration object, wherein the above calibration object is a virtual object in the scene space displayed by the head-mounted display device; align the above viewpoint space with the above scene space according to the above viewpoint space coordinate system and the scene space coordinate system corresponding to the above scene space; generate viewpoint distance information corresponding to the above calibration object according to the aligned viewpoint space, wherein the above viewpoint distance information represents the distance between the above calibration object and the above target user in the aligned viewpoint space.

[0085] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++; and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or may be connected to an external computer (e.g., by connecting through the Internet using an Internet service provider).

[0086] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that, in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0087] The functions described above can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that may be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0088] The above description is only some preferred embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, technical solutions formed by mutually replacing the above features with technical features having similar functions (but not limited to) disclosed in the embodiments of the present disclosure.

Claims

1. A method for testing the viewing distance based on a head-mounted display device, comprising: In response to detecting a viewing distance test request, perform spatial modeling based on the camera devices included in the head-mounted display device to obtain a viewing point space, wherein the viewing point space corresponds to a viewing point space coordinate system, the head-mounted display device includes a camera device corresponding to the left eye of the target user and a camera device corresponding to the right eye of the target user, and the axial distance between the two camera devices included in the head-mounted display device corresponds to a target distance range, and the target user is a user wearing the head-mounted display device; Determine a calibration object, wherein the calibration object is a virtual object in the scene space displayed by the head-mounted display device; Align the viewing point space with the scene space according to the viewing point space coordinate system and the scene space coordinate system corresponding to the scene space; Generate viewing distance information corresponding to the calibration object according to the aligned viewing point space, wherein the viewing distance information represents the distance between the calibration object and the target user in the aligned viewing point space.

2. The method according to claim 1, wherein The camera devices included in the head-mounted display device correspond to camera device parameter information; and performing spatial modeling based on the camera devices included in the head-mounted display device to obtain a viewing point space includes: Perform three-dimensional spatial modeling through a modeling tool to obtain a three-dimensional space model; For each camera device included in the head-mounted display device, model the camera device in the three-dimensional space model to obtain a virtual camera device; Determine the updated three-dimensional space model as the viewing point space, wherein the axial distance between the two virtual camera devices in the updated three-dimensional space model corresponds to the target distance range.

3. The method according to claim 1, wherein, The aligning the viewing point space with the scene space according to the viewing point space coordinate system and the scene space coordinate system corresponding to the scene space includes: Perform coordinate transformation processing on the viewing point space coordinate system according to the scene space coordinate system to align the viewing point space with the scene space; For each virtual camera device in the aligned viewing point space, update the device parameter information of the virtual camera device through the scene space and the camera device parameter information corresponding to the target camera device, wherein the target camera device is the camera device corresponding to the virtual camera device among the various camera devices included in the head-mounted display device.

4. The method according to claim 1, wherein Before generating the viewing distance information corresponding to the calibration object according to the aligned viewing point space, the method further includes: For each virtual camera device model in the aligned viewing point space, determine the viewing angle range information according to the virtual device parameter information corresponding to the virtual camera device, wherein the viewing angle range information represents the viewing angle range of the virtual camera device in the aligned viewing point space; Generate target parallax position information according to the determined viewing angle range information, wherein the target parallax position information represents the position of the target parallax plane under the viewing angle of the target user in the aligned viewing point space.

5. The method according to claim 1, wherein The method further includes: Display the generated viewing distance information on the head-mounted display device.

6. A head-mounted display device, comprising: One or more processors; An optical display system, including at least one display screen and optical elements for imaging in front of the user's eyes; A camera device system, including a camera device corresponding to the user's left eye of the target user and a camera device corresponding to the user's right eye of the target user, where the target user is the user wearing the head-mounted display device; A storage device, on which one or more programs are stored. When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-5.

7. The head-mounted display device according to claim 6, wherein, The head-mounted display device includes a head-mounted display device body and a smart terminal, and the smart terminal is communicatively connected to the head-mounted display device body.

8. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the method according to any one of claims 1-5.

Images