Security boundary positioning method and head-mounted display device

Abstract

The invention relates to the technical field of computer vision, and provides a security boundary positioning method and a head-mounted display device, which are used for improving the accuracy and stability of security boundary positioning. When the head-mounted display device is started, if the anchor point is not scanned in the surrounding environment, at least one anchor point associated with the 3D data of the security boundary is created at different orientations to ensure that the security boundary can be positioned based on the anchor point in immersive experience, so that when a user is in a dangerous situation, the security boundary can be positioned in the immersive experience. And according to the 3D data of the security boundary associated with the target anchor point, drawing and displaying the corresponding security boundary to guarantee user security. Due to the fact that the multiple anchor points are located in different directions of the surrounding environment, the success rate of anchor point scanning is increased, the problem that the safety boundary cannot be positioned due to anchor point scanning failure is solved, the robustness of safety boundary positioning is improved, the multiple anchor points can better capture changes of the surrounding environment, and therefore the accuracy of safety boundary positioning is improved.

Description

Background Art

[0002] With the development of computer vision, the application scenarios of Extended Reality (XR) products are becoming more and more extensive, including head-mounted display devices such as Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR). When users use head-mounted display devices for immersive experience, the safety boundary based on 6 degrees of freedom (DoF) positioning is an essential function to ensure user safety.

[0003] The process of locating the safety boundary mainly consists of two important parts: setting the safety boundary and determining whether the safety boundary is exceeded and the display of the safety boundary.

[0004] At present, the positioning method of the safety boundary is mainly based on Simultaneous Localization and Mapping (SLAM), and the safety boundary of SLAM positioning has the following risks:

[0005] (1) Environmental complexity and dynamic changes: SLAM technology needs to operate effectively in complex and dynamically changing environments. Factors such as object movement, vehicle speed changes, and light fluctuations in dynamic environments will affect the data collection of sensors in head-mounted display devices, thereby affecting map construction and positioning accuracy, resulting in the inability to accurately locate safety boundaries.

[0006] (2) Sensor noise and data quality: SLAM relies on multiple sensors to obtain environmental information, but sensor noise, data quality, and accuracy differences may affect the performance of the SLAM system, thereby affecting the positioning results of the safety boundary.

[0007] For example, the measurement accuracy of lidar may be disturbed in bad weather, and visual sensors may not perform as expected in low-light environments, resulting in failure in safety boundary positioning.

[0008] (3) Cumulative error problem: SLAM may not be able to achieve positioning under the global map. Relying solely on the visual odometer that matches frames between frames, cumulative errors will occur over a long period of time, resulting in inaccurate safety boundary positioning.

[0009] (4) In large scenes, loop detection is a challenge that SLAM systems need to face. A small error at the beginning may not be detected. When the loop goes around once, the error accumulates and the loop cannot be closed.

[0010] (5) Data registration error: The positioning result of SLAM is achieved based on feature matching between images. However, in some scenes with sparse textures (such as white walls on all sides, smooth desktops, etc.), sometimes unrealistic features are extracted, and feature matching faces the risk of registration errors, resulting in low positioning accuracy or positioning failure of the safety boundary.

[0011] Therefore, providing a high-precision and robust safety boundary positioning method to ensure user experience safety is an urgent problem to be solved in head-mounted display devices. Summary of the Invention

[0012] Embodiments of the present application provide a safety boundary positioning method and a head-mounted display device for improving the accuracy and robustness of safety boundary positioning.

[0013] In a first aspect, embodiments of the present application provide a safety boundary positioning method applied to a head-mounted display device, including:

[0014] When the head-mounted display device is started, scan the surrounding environment where the user is located;

[0015] Determine whether at least one anchor point is scanned in the surrounding environment within a preset time;

[0016] When no anchor point is scanned, create at least one anchor point in the surrounding environment and associate a 3D data of a safety boundary with each anchor point; wherein, different anchor points are located in different orientations of the surrounding environment;

[0017] When at least one anchor point is scanned, according to the comparison result between the number of anchor points and a preset number threshold, select a target anchor point from the at least one anchor point, and when the user is in a dangerous situation, draw and display a safety boundary according to the 3D data associated with the target anchor point.

[0018] The beneficial effects of the above technical solution are as follows: When the head-mounted display device is started, the surrounding environment where the user is located is scanned. If no anchor point is scanned within the preset time, at least one anchor point is created in the surrounding environment where the user is located to ensure that there are anchor points associated with 3D data of the safety boundary in the surrounding environment where the user is located. In this way, when the user is in a dangerous situation, a corresponding safety boundary is drawn and displayed according to the 3D data of the safety boundary associated with the target anchor point among the at least one scanned anchor points to ensure user safety. Since multiple anchor points are located in different orientations of the surrounding environment, the success rate of anchor point scanning is increased, thereby reducing the problem that the safety boundary cannot be located due to anchor point scanning failure, improving the robustness of safety boundary positioning, and moreover, multiple anchor points can better capture changes in the surrounding environment, thereby improving the accuracy of safety boundary positioning.

[0019] Optionally, selecting a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold includes:

[0020] When the number of anchor points is greater than 1 and does not exceed the preset number threshold, taking the first scanned anchor point as the target anchor point; the preset number threshold is an integer greater than 2;

[0021] When the number of anchor points is greater than 1 and exceeds the preset number threshold, obtaining 3D data of the multiple scanned anchor points, and selecting a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points;

[0022] When the number of anchor points is equal to 1, directly taking the scanned anchor point as the target anchor point.

[0023] The beneficial effects of the above technical solution are as follows: Since the number of scanned anchor points can reflect the stability of the surrounding environment, when only one anchor point is scanned, it indicates that the surrounding environment changes greatly. Therefore, directly taking this anchor point as the target anchor point to ensure user safety. When multiple anchor points are scanned, if the number of anchor points does not exceed the preset number threshold, it indicates that there are certain changes in the surrounding environment, and the first scanned anchor point is the fastest scanned anchor point. Therefore, taking the first anchor point as the target anchor point to ensure user safety. Otherwise, it indicates that the surrounding environment changes little, and an optimal target anchor point can be selected from the multiple anchor points to ensure user safety. In this way, by using different methods to determine the target anchor point in different environmental situations, the stability of the anchor points in the surrounding environment is ensured, thereby improving the stability of the safety boundary positioning.

[0024] Optionally, each 3D data at least includes the 6Dof pose of a safety boundary in the anchor coordinate system of the corresponding anchor point;

[0025] Then, obtaining 3D data of the multiple scanned anchor points, and selecting a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points includes:

[0026] Obtaining the 6Dof poses in the 3D data associated with the multiple scanned anchor points;

[0027] Calibrating the 6Dof poses of the multiple anchor points to obtain a target pose;

[0028] Calculating the errors between the 6Dof poses of each obtained anchor point and the target pose, and taking the anchor point corresponding to the 6Dof pose with the smallest error as the target anchor point.

[0029] The beneficial effects of the above technical solution are as follows: In response to the situation where more than a preset number threshold is scanned, by calibrating the 6Dof pose of the safety boundaries associated with multiple anchor points, an anchor point with the smallest error from the calibrated target pose is selected as the target anchor point, thereby improving the positioning accuracy of the anchor point and further improving the accuracy of safety boundary positioning.

[0030] Optionally, the head-mounted display device is equipped with at least one camera. The scanning of the surrounding environment where the user is located and the determination of whether at least one anchor point is scanned in the surrounding environment within a preset time include:

[0031] Invoking a first process to control the camera to scan the surrounding environment where the user is located; the first process is a background process;

[0032] Within a preset time, determining whether the first process receives at least one anchor point scanned by the camera;

[0033] The creating of at least one anchor point in the surrounding environment and associating 3D data of a safety boundary with each anchor point includes:

[0034] Invoking a second process to create at least one anchor point in the surrounding environment and associate 3D data of a safety boundary with each anchor point; the second process and the first process belong to the same application;

[0035] The selecting of a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and the preset number threshold, and when the user is in a dangerous situation, drawing and displaying a safety boundary according to the 3D data associated with the target anchor point includes:

[0036] Invoking the first process to select a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and the preset number threshold, and when the user is in a dangerous situation, invoking the second process to draw and display a safety boundary according to the 3D data associated with the target anchor point.

[0037] The beneficial effects of the above technical solution are as follows: Since the first process is a background process, when calling the first process to obtain at least one anchor point in the user's surrounding environment and during the process of determining the acquisition result, the setting interface of the safety boundary will not be displayed in the foreground, thus avoiding interrupting the user's immersive experience and improving the quality of the experience. When the first process fails to obtain an anchor point, through inter-process communication, the second process creates at least one anchor point associated with the 3D data of the safety boundary in the surrounding environment, so that when the user is in a dangerous situation, the second process is called to draw and display the corresponding safety boundary based on the 3D data of the safety boundary associated with the target anchor point to ensure the safety of the user. Since the second process for creating and drawing the safety boundary and the first process for scanning and determining the safety boundary belong to the same application, the conversion process of the anchor point coordinates during the generation and use of the safety boundary is reduced, and the positioning efficiency of the safety boundary is improved.

[0038] Optionally, the step of calling the second process to create at least one anchor point in the surrounding environment and associate a 3D data of a safety boundary with each anchor point includes:

[0039] Controlling the second process to send a scanning instruction to the first process, so that the first process controls the camera to scan the surrounding environment where the user is located according to the scanning instruction, and obtains environmental images at at least one azimuth in the surrounding environment;

[0040] Receiving at least one environmental image sent by the first process through the second process;

[0041] For each environmental image, calling the second process to perform the following operations:

[0042] Extracting feature points in the environmental image and determining an anchor point in the surrounding environment according to the feature points;

[0043] Obtaining a 3D model of a safety boundary created for the anchor point and determining the 6DOf pose of the 3D model in the anchor coordinate system corresponding to the anchor point;

[0044] Associating the 3D model and the 6DOf pose as the 3D data of a safety boundary with the anchor point.

[0045] The beneficial effects of the above technical solution are as follows: When the second process creates anchor points at different azimuths in the surrounding environment, a 3D model of a safety boundary and the 6Dof pose of the 3D model are associated with each anchor point. In this way, when an anchor point is scanned subsequently, the safety boundary can be drawn by obtaining the 3D data associated with the anchor point, thus realizing the positioning process of the safety boundary based on the anchor point and ensuring the safety of the user.

[0046] Optionally, the first process and the second process are started in the following manner:

[0047] Create a custom tracking space, where the tracking space is used to define the initial pose of the head-mounted display device. The tracking space serves as a virtual origin, and the tracking data of the head-mounted display device is transformed and applied relative to the virtual origin;

[0048] Set the tracking space as an object of the XR session; where the XR session is used to ensure that all virtual objects are under the same root node;

[0049] Add a component for managing the pose of the virtual object, and set at least one anchor point as a virtual object in the tracking space;

[0050] Start the first process and the second process in the same application, so that the first process and the second process share the tracking space.

[0051] The beneficial effects of the above technical solution are as follows: By starting the first process and the second process in the same application, the first process and the second process share the same tracking space for defining the initial pose of the head-mounted display device. In this way, by setting at least one anchor point as a virtual object in the tracking space and correctly aligning it with other virtual objects in the XR space, during the safety boundary positioning process, the transformation of the anchor point coordinate system when the second process generates anchor points and the first process uses anchor points can be reduced, thereby improving the safety boundary positioning efficiency based on anchor points and then timely ensuring user safety.

[0052] Optionally, when the user is in a dangerous situation, calling the second process to draw and display a safety boundary according to the 3D data associated with the target anchor point includes:

[0053] Call the first process to obtain the 6Dof pose of the head-mounted display device;

[0054] Call the first process to determine whether the user is in a dangerous situation according to the 6Dof pose of the head-mounted display device;

[0055] When it is determined that the user is in a dangerous situation, control the first process to send a safety instruction to the second process;

[0056] Call the second process to load the 3D data of the safety boundary associated with the target anchor point according to the safety instruction, and draw and display the safety boundary according to the obtained 3D data.

[0057] The beneficial effects of the above technical solution are as follows: By determining whether the user enters or exits the safety boundary through the first process for dangerous situation judgment, since the first process is a background process, the dangerous situation judgment process will not affect the user's immersive experience, thus improving the quality of the immersive experience. Moreover, since the second process and the first process belong to the same application, the conversion process of the anchor coordinate system of the 3D data is reduced, the display efficiency of the safety boundary is improved, and the user's safety is guaranteed in a timely manner.

[0058] In a second aspect, an embodiment of the present application provides a head-mounted display device, including a processor, a memory, a display screen, and a communication interface. The communication interface, the display screen, the memory, and the processor are connected through a bus;

[0059] The communication interface is used for receiving and sending data;

[0060] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0061] When the head-mounted display device is started, scan the surrounding environment where the user is located;

[0062] Determine whether at least one anchor point is scanned in the surrounding environment within a preset time;

[0063] When no anchor point is scanned, create at least one anchor point in the surrounding environment and associate 3D data of a safety boundary with each anchor point; wherein, different anchor points are located in different orientations of the surrounding environment;

[0064] When at least one anchor point is scanned, according to the comparison result between the number of anchor points and a preset number threshold, select a target anchor point from the at least one anchor point, and when the user is in a dangerous situation, draw a safety boundary according to the 3D data associated with the target anchor point and display it through the display screen.

[0065] Optionally, the processor selects a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold. The specific operation is as follows:

[0066] When the number of anchor points is greater than 1 and does not exceed the preset number threshold, take the first scanned anchor point as the target anchor point; the preset number threshold is an integer greater than 2;

[0067] When the number of anchor points is greater than 1 and exceeds the preset number threshold, obtain the 3D data of the scanned multiple anchor points, and select a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points;

[0068] When the number of anchor points is equal to 1, directly take the scanned anchor point as the target anchor point.

[0069] Optionally, each 3D data includes at least a 6Dof pose of a safety boundary in the anchor coordinate system of the corresponding anchor point;

[0070] Then, the processor selects a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points. The specific operation is as follows:

[0071] Obtain the 6Dof poses in the 3D data associated with the multiple scanned anchor points;

[0072] Calibrate the 6Dof poses of the multiple anchor points to obtain a target pose;

[0073] Calculate the error between the 6Dof pose of each obtained anchor point and the target pose, and use the anchor point corresponding to the 6Dof pose with the smallest error as the target anchor point.

[0074] Optionally, the head-mounted display device is equipped with at least one camera. The processor scans the surrounding environment where the user is located and determines whether at least one anchor point is scanned in the surrounding environment within a preset time. The specific operation is as follows:

[0075] Call the first process to control the camera to scan the surrounding environment where the user is located; the first process is a background process;

[0076] Within the preset time, determine whether the first process receives at least one anchor point scanned by the camera;

[0077] The processor creates at least one anchor point in the surrounding environment and associates 3D data of a safety boundary with each anchor point. The specific operation is as follows:

[0078] Call the second process to create at least one anchor point in the surrounding environment and associate 3D data of a safety boundary with each anchor point; the second process and the first process belong to the same application;

[0079] The processor selects a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold, and when the user is in a dangerous situation, draws and displays a safety boundary according to the 3D data associated with the target anchor point. The specific operation is as follows:

[0080] Call the first process to select a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and the preset number threshold, and when the user is in a dangerous situation, call the second process to draw and display a safety boundary according to the 3D data associated with the target anchor point.

[0081] Optionally, the processor invokes a second process to create at least one anchor point in the surrounding environment and associates 3D data of a safety boundary with each anchor point. The specific operations are as follows:

[0082] Control the second process to send a scanning instruction to the first process, so that the first process controls the camera to scan the surrounding environment where the user is located according to the scanning instruction, and obtain environmental images at at least one azimuth in the surrounding environment;

[0083] Receive at least one environmental image sent by the first process through the second process;

[0084] For each environmental image, invoke the second process to perform the following operations:

[0085] Extract feature points in the environmental image and determine an anchor point in the surrounding environment according to the feature points;

[0086] Obtain a 3D model of a safety boundary created for the anchor point and determine the 6DOf pose of the 3D model in the anchor coordinate system corresponding to the anchor point;

[0087] Associate the 3D model and the 6DOf pose as 3D data of a safety boundary with the anchor point.

[0088] Optionally, the processor starts the first process and the second process in the following manner:

[0089] Create a custom tracking space, where the tracking space is used to define the initial pose of the head-mounted display device. The tracking space serves as a virtual origin, and the tracking data of the head-mounted display device is transformed and applied relative to the virtual origin;

[0090] Set the tracking space as an object of the XR session; where the XR session is used to ensure that all virtual objects are under the same root node;

[0091] Add a component for managing the pose of the virtual object and set the at least one anchor point as a virtual object in the tracking space;

[0092] Start the first process and the second process in the same application, so that the first process and the second process share the tracking space.

[0093] Optionally, when the user is in a dangerous situation, the processor invokes the second process to draw and display a safety boundary according to the 3D data associated with the target anchor point. The specific operations are as follows:

[0094] Invoke the first process to obtain the 6Dof pose of the head-mounted display device;

[0095] Invoke the first process to determine whether the user is in a dangerous situation according to the 6Dof pose of the head-mounted display device;

[0096] When it is determined that the user is in a dangerous situation, control the first process to send a safety instruction to the second process;

[0097] Invoke the second process to load the 3D data of the safety boundary associated with the target anchor point according to the safety instruction, and draw and display the safety boundary according to the obtained 3D data.

[0098] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the safety boundary positioning methods are implemented.

[0099] The technical effects brought by any one of the implementation manners in the second aspect to the third aspect can refer to the technical effects brought by the corresponding implementation manners in the first aspect, which will not be elaborated here. Description of the Drawings

[0100] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to these drawings without creative efforts.

[0101] Figure 1 It is a flowchart of a safety boundary positioning method provided by an embodiment of the present application;

[0102] Figure 2A It is a schematic diagram of multiple anchor points provided by an embodiment of the present application;

[0103] Figure 2B It is a schematic diagram of no anchor point scanned provided by an embodiment of the present application;

[0104] Figure 2C It is a schematic diagram of creating a safety boundary prompt provided by an embodiment of the present application;

[0105] Figure 3A It is a schematic diagram of a 3D model of a custom safety boundary provided by an embodiment of the present application;

[0106] Figure 3B It is a storage schematic diagram of 3D data of an anchor point and a safety boundary provided by an embodiment of the present application;

[0107] Figure 4A Flowchart of the method for determining a target anchor provided by an embodiment of this application;

[0108] Figure 4B Effect diagram of safety boundary display provided by an embodiment of this application;

[0109] Figure 4C Schematic diagram of the launcher scenario provided by an embodiment of this application;

[0110] Figure 5 Flowchart of a method for positioning a safety boundary provided by an embodiment of this application;

[0111] Figure 6 Flowchart of the method for starting multiple processes provided by an embodiment of this application;

[0112] Figure 7 Flowchart of a method for creating an anchor provided by an embodiment of this application;

[0113] Figure 8 Flowchart of the method for associating an anchor with a safety boundary provided by an embodiment of this application;

[0114] Figure 9 Flowchart of a method for protecting user safety provided by an embodiment of this application;

[0115] Figure 10 Structure diagram of the head-mounted display device provided by an embodiment of this application. Detailed implementation manners

[0116] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this application. Apparently, the described embodiments are some but not all of the embodiments of the technical solutions of this application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments recorded in this application document without creative efforts belong to the scope of protection of the technical solutions of this application.

[0117] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of this application. In addition, although the disclosed content in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosed contents can also constitute a complete technical solution alone.

[0118] In addition, the terms "comprising" and "having" and any variations thereof are intended to cover inclusion without exclusivity. For example, a product or device comprising a series of components need not be limited to those components clearly listed, but may include other components not clearly listed or inherent to such products or devices.

[0119] As used herein, the term "module" refers to any known or later-developed hardware, software, firmware, artificial intelligence, fuzzy logic, or a combination of hardware or / and software code capable of performing functions related to that element.

[0120] Before introducing the data processing method provided by the embodiments of the present application, some terms involved in the data processing method provided by the embodiments of the present application are explained.

[0121] A head-mounted display device usually needs to set a safety boundary to ensure that when the user moves within the safety boundary, they will not collide with objects outside the safety boundary and cause accidents. However, when using the safety boundary based on SLAM positioning currently, it is easy to encounter the situation of not finding the safety boundary, which is an important safety hazard for maintaining the personal safety of users.

[0122] In view of this, the embodiments of the present application provide a method for positioning a safety boundary to improve the stability and accuracy of the safety boundary. When using a head-mounted display device, at least one anchor point is created in the surrounding environment where the user is located, and 3D data of a safety boundary is associated with each anchor point, thereby increasing the success rate of anchor point scanning, reducing the problem of safety boundary positioning failure caused by anchor point scanning failure, improving the stability of safety boundary positioning, and at least one anchor point is located in different orientations of the surrounding environment, which can better reflect changes in the surrounding environment and further improve the stability and accuracy of safety boundary positioning based on anchor points. In addition, when multiple anchor points are scanned, the positioning accuracy of the safety boundary is effectively improved through the fusion positioning technology of multiple anchor points.

[0123] The following will specifically describe a method for positioning a safety boundary provided by the embodiments of the present application with reference to the accompanying drawings through specific embodiments.

[0124] See Figure 1 , which is a flowchart of a method for positioning a safety boundary provided by the embodiments of the present application. This process is executed by a head-mounted display device and mainly includes the following steps:

[0125] S101: When the head-mounted display device is started, scan the surrounding environment where the user is located.

[0126] In some embodiments, the user drives the head-mounted display device to scan the surrounding environment by turning the head.

[0127] In some embodiments, the startup process of the head-mounted display device includes, but is not limited to: startup when the device is powered on, startup when switching from the screen-off state to the screen-on state, and startup when opening an XR application.

[0128] S102: Determine whether at least one anchor point is scanned in the surrounding environment within a preset time. When no anchor point is scanned, execute S103; when at least one anchor point is scanned, execute S104.

[0129] Within the preset time, if an anchor point is scanned in the surrounding environment, it indicates that an anchor point has been created in this surrounding environment, and the positioning process of the safety boundary can be directly entered based on the scanning result of the anchor point. If no anchor point is scanned in the surrounding environment, it indicates that no anchor point has been created in this surrounding environment, and an anchor point needs to be created first before entering the positioning process of the safety boundary.

[0130] In some embodiments, each anchor point can be represented by some feature points in the surrounding environment. In this way, it can be determined whether an anchor point is scanned through feature point matching. Specifically, when the number of matching point pairs is greater than the preset matching threshold, it is determined that an anchor point is scanned; otherwise, it is determined that no anchor point is matched.

[0131] S103: Create at least one anchor point in the surrounding environment and associate 3D data of a safety boundary with each anchor point.

[0132] Among them, different anchor points are located in different orientations of the surrounding environment, and each anchor point corresponds to a unique identifier.

[0133] In some embodiments, the position information of each anchor point is the coordinate mean of the corresponding feature points in 3D space. Among them, the determination process of the 3D coordinates of the feature points belongs to the prior art and is not the focus of this application, so it will not be elaborated here.

[0134] In some embodiments, multiple anchor points can be set at different positions and directions with the user himself as a reference. During the setting process, it is ensured that the multiple anchor points are not coplanar and are distributed as evenly as possible at different heights and positions around the user to reduce geometric dilution error.

[0135] For example, as Figure 2A shown, an anchor point is created in each of the four directions of the front upper, back lower, left upper, and right lower of the user's position. Among them, the distances from these four anchor points to the user can be the same or different.

[0136] It should be noted that Figure 2A this is only an example, and the embodiments of this application do not impose restrictive requirements on the number, position, and direction of the anchor points. In practical applications, they can be flexibly adjusted according to requirements.

[0137] In some embodiments, after creating at least one anchor point in the surrounding environment, to achieve anchor-based secure boundary positioning, 3D data of a secure boundary needs to be associated with each anchor point.

[0138] In some embodiments, each 3D data includes at least a 3D model of the secure boundary and the 6Dof pose of the 3D model in the anchor coordinate system corresponding to the respective anchor point.

[0139] When no anchor point is scanned, as Figure 2B shown, a prompt message of "Secure boundary not found" is given through the binocular display screen. When the user selects "Create new secure boundary", it jumps to Figure 2C the "Select secure boundary mode" shown. A 3D data of a secure boundary is associated with each created anchor point. Among them, the "In-situ boundary" mode is to set a secure boundary with a fixed shape, and the "Customize secure boundary" mode is to set a secure boundary with an unfixed shape by drawing a line with a handle.

[0140] Taking the selection of the "Customize secure boundary" mode as an example, as Figure 3A shown, a closed geometric figure is drawn on the ground plane through the handle ray, and a solid model with an upward opening is generated based on this geometric figure. This solid model serves as the 3D model of the secure boundary, which can be represented by some boundary points on the geometric figure.

[0141] As shown in Table 1, the 3D coordinates of the boundary points of a 3D model of a secure boundary in a certain anchor coordinate system are presented.

[0142] Table 1. 3D coordinates of boundary points

[0143] (x, y, z) (-0.0566619336605072，-1.5119432210922242，0.9907272458076477) (-0.008554408326745034，-1.5117210149765015，1.0161421298980713) (0.01004519872367382，-1.5117210149765015，1.0166373252863653) (0.02635982632637024，-1.5116004943847657，1.0199939012527466) (0.053333573043346408，-1.511612057685852，0.9856778303255005) …

[0144] In some embodiments, the 6Dof pose of the 3D model includes displacement and rotation.

[0145] Taking the anchor point shown in Figure 2A as an example, Figure 3A the 6Dof poses of the 3D model shown in

[0146] in the anchor coordinate systems of these four anchor points are shown in Table 2.

[0147]

[0148]

[0149] Among them, rotation is represented by quaternions.

[0150] It should be noted that the process of determining the 6Dof pose of the 3D model is not the inventive point of this application and belongs to the prior art, so the embodiments of this application will not elaborate on it.

[0151] In some embodiments, the 3D data of the multiple created anchor points and the safety boundaries associated with each anchor point can be stored in a specific directory local to the head-mounted display device.

[0152] For example, as Figure 3B shown, it is a schematic diagram of the storage method of the 3D data of the multiple anchor points and the safety boundaries associated with each anchor point. In the / boundaries / VYI directory, the 3D model of the safety boundary is stored in the boundary.obj file, the boundary point coordinates of the safety boundaries associated with the four anchor points are stored in the boundary_VYI_0.txt to boundary_VYI_3.txt files, and the 6Dof poses of the four safety boundaries in the anchor coordinate system of the corresponding anchor points are stored in the objRelativeTransform.txt file.

[0153] In some embodiments, the position information of at least one anchor point created in the surrounding environment and the 3D data of the associated safety boundary can be managed by EnvironmentManagement, and when the SLAM coordinate system of the head-mounted display device changes, the position information of the anchor point is updated.

[0154] S104: Select a target anchor point from at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold.

[0155] In some embodiments, when the scanned number of anchor points is different, the method for determining the target anchor point is different. For the specific determination process, see Figure 4A , which mainly includes the following steps:

[0156] S1041: Determine whether the number of anchor points is greater than 1. If so, execute S1042; otherwise, execute S1045.

[0157] S1042: Determine whether the number of anchor points exceeds the preset number threshold. If so, execute S1043; otherwise, execute S1044.

[0158] Among them, the preset number threshold is an integer greater than 2.

[0159] S1043: Obtain the 3D data of the multiple scanned anchor points, and select a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points.

[0160] When the number of anchor points is greater than 1 and exceeds a preset number threshold, it indicates that the surrounding environment changes little (such as light changes, object movement, etc.), and the scanning environment is good. Therefore, a target anchor point can be determined by fusing multiple scanned anchor points. The specific process is as follows:

[0161] S1043_1: Obtain the 6Dof poses in the 3D data associated with multiple scanned anchor points.

[0162] In some embodiments, when fusing and positioning multiple anchor points, to reduce the computational amount, calibration and comparison can be performed only by obtaining the 6Dof poses in the 3D data of the safety boundaries associated with multiple anchor points.

[0163] S1043_2: Calibrate the 6Dof poses of multiple anchor points to obtain a target pose.

[0164] In some embodiments, the triangulation method or the least squares method can be used to fuse the 6Dof poses of multiple anchor points to obtain the calibrated target pose.

[0165] S1043_3: Calculate the error between the 6Dof pose of each obtained anchor point and the target pose respectively, and take the anchor point corresponding to the 6Dof pose with the smallest error as the target anchor point.

[0166] In some embodiments, taking the calibrated target pose as a reference, calculate the error between the 6Dof pose of the 3D model corresponding to each obtained anchor point and the target pose respectively. By comparing each error, select the anchor point corresponding to the 6Dof pose with the smallest error as the target anchor point.

[0167] For the case where the number of anchor points is greater than 1 and exceeds the preset number threshold, the 6Dof poses of the safety boundaries associated with multiple anchor points are fused by the triangulation method or the least squares method to obtain the calibrated target pose, and an anchor point with the smallest error from the calibrated target pose is selected as the target anchor point, thereby improving the positioning accuracy of the anchor point and further improving the accuracy of the safety boundary positioning.

[0168] S1044: Take the first scanned anchor point as the target anchor point.

[0169] When the number of anchor points is greater than 1 but does not exceed the preset number threshold, it indicates that there are certain changes in the surrounding environment, and the first scanned anchor point is the fastest scanned anchor point. Therefore, the first anchor point is taken as the target anchor point.

[0170] S1045: Directly take the scanned anchor point as the target anchor point.

[0171] When the number of anchor points is equal to 1, it indicates that the surrounding environment changes greatly and the scanning environment is poor. Therefore, directly take the scanned anchor point as the target anchor point.

[0172] Since the number of scanned anchor points can reflect the stability of the surrounding environment, when only one anchor point is scanned, it indicates that the surrounding environment has changed greatly. Therefore, this anchor point is directly used as the target anchor point to ensure user safety. When multiple anchor points are scanned, if the number of anchor points does not exceed the preset number threshold, it indicates that there are certain changes in the surrounding environment, and the first scanned anchor point is the fastest scanned anchor point. Therefore, the first anchor point is used as the target anchor point to ensure user safety. Otherwise, it indicates that the surrounding environment has changed little, and an optimal target anchor point can be selected from multiple anchor points to ensure user safety. In this way, by using different methods to determine the target anchor point in different environmental situations, the stability of the anchor points in the surrounding environment is ensured, thereby improving the stability of the safety boundary positioning.

[0173] S105: When the user is in a dangerous situation, draw and display the safety boundary according to the 3D data associated with the target anchor point.

[0174] After determining the target anchor point, an anchor point coordinate system is established based on the position information of the target anchor point, and the 3D model of the safety boundary associated with the target anchor point is loaded. Thus, within the anchor point coordinate system of the target anchor point, it is judged whether the user is in a dangerous situation. When in a dangerous situation, draw the safety boundary according to the loaded 3D data and display it in a perspective mode. Otherwise, enter the launcher scene.

[0175] In some embodiments, the determination conditions for dangerous situations include but are not limited to: the user is outside the safety boundary, or the user is inside the safety boundary but the distance from the safety boundary is less than the preset safety threshold. In practical applications, users can flexibly adjust according to actual needs.

[0176] As Figure 4B shown, when it is determined that the user is in a dangerous situation, turn on the perspective mode to display the safety boundary, and prompt the user to return inside the safety boundary to ensure their own safety.

[0177] As Figure 4C shown, when it is determined that the user is not in a dangerous situation, directly enter the launcher scene for 3D experience.

[0178] It should be noted that the pose of the user in the anchor point coordinate system can be determined by the preset conversion relationship between the anchor point coordinate system and the SLAM coordinate system. The method of determining the pose of the user through SLAM technology belongs to the prior art and is not the inventive point of this application, so it will not be elaborated here.

[0179] In an embodiment of the present application, when the head-mounted display device is started, the surrounding environment where the user is located is scanned. If no anchor point is scanned within a preset time, at least one anchor point is created in the surrounding environment where the user is located to ensure that there is an anchor point associated with 3D data of a safety boundary in the surrounding environment where the user is located. In this way, when the user is in a dangerous situation, according to the 3D data of the safety boundary associated with the target anchor point among the at least one scanned anchor point, the corresponding safety boundary is drawn and displayed to ensure the safety of the user. Since multiple anchor points are located in different orientations of the surrounding environment, the success rate of anchor point scanning is increased, thereby reducing the problem that the safety boundary cannot be located due to the failure of anchor point scanning, improving the robustness of safety boundary positioning. Moreover, multiple anchor points can better capture changes in the surrounding environment, thereby improving the accuracy of safety boundary positioning.

[0180] In some embodiments, when the head-mounted display device supports multi-process concurrent processing, the positioning efficiency of the safety boundary can be improved by starting multiple processes.

[0181] See Figure 5 , which is another flow of the safety boundary positioning method provided by the embodiment of the present application. This flow is executed by the head-mounted display device and mainly includes the following steps:

[0182] S501: When the head-mounted display device is started, call the first process to control the camera to scan the surrounding environment where the user is located.

[0183] In some embodiments, the head-mounted display device is equipped with at least one camera. When the head-mounted display device is started, call the first process to control the camera to scan the surrounding environment where the user is located.

[0184] S502: Within a preset time, determine whether the first process receives at least one anchor point scanned by the camera in the surrounding environment. When no anchor point is received, execute S503; when at least one anchor point is received, execute S504.

[0185] In some embodiments, the first process is a background process, so that the scanning process of the surrounding environment can be realized without the user's awareness, and the immersive experience of the user will not be affected.

[0186] S503: Call the second process to create at least one anchor point in the surrounding environment and associate 3D data of a safety boundary with each anchor point.

[0187] When the first process does not receive an anchor point, send a message of scanning failure to the second process. The second process pops up a prompt message of "Safety boundary not found" according to this message. As Figure 2B shown, when the user selects "New safety boundary", it jumps to Figure 2CFor the "Select Safety Boundary Mode" shown, the second process creates anchor points according to the corresponding mode and associates 3D data of a safety boundary with each created anchor point.

[0188] In some embodiments, the second process belongs to the same application as the first process, thus sharing the same tracking space, reducing the conversion process of the anchor point coordinate system when creating and using anchor points, and improving the positioning efficiency of the safety boundary.

[0189] In some embodiments, the startup processes of the first process and the second process are as Figure 6 shown and mainly include the following steps:

[0190] S500_1: Create a custom tracking space.

[0191] Among them, the tracking space is used to define the initial pose of the head-mounted display device. The tracking space serves as a virtual origin, and the tracking data of the head-mounted display device is converted and applied relative to the virtual origin. Specifically, when the head-mounted display device is initialized, a point in the real world is selected as the tracking origin, and the tracking data of all tracking entities (such as the head-mounted display device, the handle, etc.) is determined relative to the tracking origin. The role of the virtual origin is to convert the tracking data relative to the tracking origin into the Unity scene, enabling the user to be at the desired position in the virtual environment.

[0192] In the Unity application, the virtual origin is a special GameObject used to convert the tracking data of the head-mounted display device into the world space of the Unity scene. The virtual origin serves as the center of the tracking space, and all tracking entities (such as the head-mounted display device, the handle, etc.) are positioned relative to the virtual origin.

[0193] S500_2: Set the tracking space as an object of the XR session.

[0194] Among them, the XR session is the core component that manages the interaction between the XR space and the head-mounted display device, and is used to ensure that all virtual objects (such as anchor points, virtual cameras simulating the user's eyes, interaction objects, etc.) in the XR space are under the same root node.

[0195] S500_3: Add a component for managing the pose of the virtual object and set at least one anchor point as a virtual object in the tracking space.

[0196] In some embodiments, the component for managing the position and rotation of the virtual object is the Transform component.

[0197] In some embodiments, by adding at least one anchor point as a virtual object in the tracking space, it helps to uniformly manage its position and rotation with other virtual objects, ensuring that all virtual objects are correctly aligned in the XR space.

[0198] S500_4: Start the first process and the second process in the same application so that the first process and the second process share the tracking space.

[0199] Taking the head-mounted display device with the Android system as an example, by starting the first process and the second process in the same Android application, the first process and the second process share the same tracking space, and thus share the same tracking data of the head-mounted display device.

[0200] In the embodiments of the present application, the first process and the second process are started in the same application, so that the first process and the second process share the same tracking space for defining the initial pose of the head-mounted display device. And, by adding at least one anchor point as a virtual object in the tracking space, it is correctly aligned with other virtual objects. In this way, during the safety boundary positioning process, the conversion of the anchor point coordinate system when the second process generates anchor points and the first process uses anchor points can be reduced, thereby improving the efficiency of anchor-based safety boundary positioning and timely ensuring user safety.

[0201] In some embodiments, after the first process and the second process are started, when the first process does not receive an anchor point, it indicates that the camera scanning fails and no anchor point for positioning the safety boundary is scanned in the surrounding environment. Therefore, the user is prompted to look around so that the first process controls the camera to scan the surrounding environment and calls the second process to create at least one anchor point in the surrounding environment.

[0202] Specifically, the creation process of at least one anchor point is as Figure 7 shown, mainly including the following steps:

[0203] S5031: Control the second process to send a scanning instruction to the first process, so that the first process controls the camera to scan the surrounding environment where the user is located according to the scanning instruction, and obtains environmental images at least in one azimuth in the surrounding environment.

[0204] In some embodiments, the user drives the camera to scan the surrounding environment by turning himself, so as to obtain environmental images at least in one azimuth. Among them, each environmental image can be a binocular image, or a depth image and an RGB image.

[0205] Taking four azimuths as an example, an environmental image is obtained at different distances in the four directions of front upper, back lower, left upper, and right lower of the user's position respectively.

[0206] S5032: Receive at least one environmental image sent by the first process through the second process.

[0207] Using inter - process communication, the first process sends at least one environmental image scanned by the camera to the second process, so that the second process creates an anchor point in the corresponding orientation based on the at least one environmental image and associates 3D data of a safety boundary with each anchor point. Since the first process is a background process, the scanning process of the surrounding environment when the device starts will not be perceived by the user, and the user can normally view the 3D picture. At this time, the relevant information of the safety boundary will not interrupt the user's immersive experience.

[0208] S5033: For each environmental image, call the second process to extract the feature points in the environmental image and determine an anchor point in the surrounding environment based on the feature points.

[0209] After receiving at least one environmental image, the second process extracts the feature points in each environmental image and represents an anchor point in the corresponding orientation with the extracted feature points. Each anchor point corresponds to a unique identifier.

[0210] In some embodiments, it can be determined whether an anchor point is scanned through feature point matching subsequently. Specifically, when the number of matching point pairs is greater than a preset matching threshold, it is determined that an anchor point is scanned; otherwise, it is determined that no anchor point is matched.

[0211] In some embodiments, the position information of each anchor point can be the coordinate mean of the feature points in the corresponding environmental image in the 3D space. Among them, the process of determining the 3D coordinates of the feature points belongs to the prior art and is not the focus of this application, so it will not be elaborated here.

[0212] S5034: Call the second process to obtain the 3D model of a safety boundary created for the anchor point and determine the 6DOf pose of the 3D model in the anchor coordinate system corresponding to the anchor point.

[0213] Among them, the relevant descriptions of the 3D model and the 6Dof pose are referred to the foregoing embodiments and will not be elaborated here.

[0214] S5035: Call the second process to associate the 3D model and the 6DOf pose as the 3D data of a safety boundary with the anchor point.

[0215] In some embodiments, after the second process creates at least one anchor point and associates the 3D data of a safety boundary with each anchor point, the anchor points and the associated 3D data of the safety boundary can be stored in a specific directory local to the head - mounted display device.

[0216] See Figure 8 , the complete flowchart for creating an anchor point associated with 3D data mainly includes the following steps:

[0217] S801: When the head - mounted display device starts, start the first process.

[0218] Among them, the first process is a background process for surrounding environment scanning and dangerous situation judgment.

[0219] S802: Call the first process to determine whether there is an anchor in the surrounding environment and whether the 3D data of the safety boundary is associated locally with the anchor. If so, execute S803; otherwise, execute S804.

[0220] S803: The first process reads the 3D data of the safety boundary and enters the dangerous situation determination process.

[0221] S804: Start the second process within the application to which the first process belongs.

[0222] By making the first process and the second process belong to the same application, the first process and the second process share the same tracking space, so that the first process and the second process share the pose of the head-mounted display device and the pose of the anchor, reducing the conversion process of the anchor coordinate system.

[0223] S805: Create a tracking space and set the tracking space as an object of the XR session, and add a component for managing the pose of the virtual object.

[0224] S806: Call the second process to prompt the user to look around, so as to enable the first process to scan the surrounding environment.

[0225] S807: Call the second process to receive the scanning result sent by the first process, and create and save at least one anchor according to the scanning result.

[0226] Among them, the position information of the anchor can be represented by the 3D coordinates of the feature points. The 3D coordinates of the feature points can be determined by the SLAM technology. When the SLAM space coordinate system of the head-mounted display device changes, update the position information of the anchor.

[0227] S808: Call the second process to create and associate a 3D data of a safety boundary for each anchor, and add at least one anchor as a virtual object of the tracking space.

[0228] Among them, each 3D data includes at least a 3D model of a safety boundary and the 6Dof pose of the model in the corresponding anchor coordinate system.

[0229] S809: Call the second process to store at least one anchor and the 3D data of the safety boundary associated with each anchor in the local directory.

[0230] In the embodiments of the present application, by making the first process and the second process belong to the same Android application, the two processes share the same tracking space, and the anchor point is a virtual object in the tracking space. In this way, the first process and the second process share the position information of the anchor point in the tracking space, thereby reducing the coordinate conversion process of the anchor point position between different applications by one step.

[0231] S504: Call the first process to select a target anchor point from at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold.

[0232] When at least one anchor point is scanned, the target anchor point is determined according to the number of scanned anchor points. For the specific determination process, please refer to Figure 4A , which will not be elaborated here.

[0233] S505: When the user is in a dangerous situation, call the second process to draw and display a safety boundary according to the 3D data associated with the target anchor point.

[0234] After determining the target anchor point, the first process determines whether the user is in a dangerous situation based on the tracking data of the target anchor point and the user (i.e., the 6Dof pose of the head-mounted display device), and when in danger, displays the safety boundary through communication with the second process. For the specific process, please refer to Figure 9 , which mainly includes the following steps:

[0235] S5051: Call the first process to obtain the 6Dof pose of the head-mounted display device.

[0236] Since the first process and the second process belong to the same application, and the 6Dof pose of the head-mounted display device is relative to the tracking space, it can be shared by the first process and the second process.

[0237] Among them, the 6Dof pose of the head-mounted display device can be determined by SLAM technology, which belongs to the prior art and is not the inventive point of the present application, so it will not be elaborated here.

[0238] S5052: Call the first process to determine whether the user is in a dangerous situation according to the 6Dof pose of the head-mounted display device. If so, execute S5053; otherwise, execute S5055.

[0239] Based on the 6Dof pose of the head-mounted display device, the first process can obtain the 6Dof pose of the head-mounted display device in the anchor point coordinate system of the target anchor point by using the preset conversion relationship between the anchor point coordinate system and the SLAM coordinate system, and then determine the positional relationship with the safety boundary associated with the target anchor point. Specifically, when it is determined that the head-mounted display device is outside the safety boundary or the distance from the safety boundary is less than the preset safety threshold, the first process determines that the user is in a dangerous situation and needs to display the safety boundary.

[0240] S5053: Control the first process to send a security instruction to the second process.

[0241] When it is determined that the user is in a dangerous situation, the first process sends a security instruction to the second process.

[0242] S5054: Invoke the second process to load the 3D data of the security boundary associated with the target anchor point according to the security instruction, draw the security boundary based on the obtained 3D data, and display it.

[0243] After receiving the security instruction, the second process loads the 3D data of the security boundary associated with the target anchor point from the local directory, renders the 3D model of the security boundary on the corresponding layer of the surfaceView, and passes it to the Runtime. The Runtime renders it on the binocular display screen of the head-mounted display device in the form of an Overlay and displays it through the perspective mode, as Figure 4B shown.

[0244] S5055: Directly enter the launcher scene.

[0245] Among them, the display effect of the Launcher scene is as Figure 4C shown.

[0246] In the embodiments of the present application, the first process determines whether the user enters or exits the security boundary to judge the dangerous situation. Since the first process is a background process, the dangerous situation determination process will not affect the user's immersive experience, thereby improving the quality of the immersive experience. Moreover, since the second process and the first process belong to the same application, the conversion process of the anchor coordinate system of the 3D data is reduced, the display efficiency of the security boundary is improved, and the user's safety is guaranteed in a timely manner.

[0247] Based on the same technical concept, the embodiments of the present application provide a head-mounted display device, which can implement the steps of the above-mentioned security boundary positioning method and achieve the same technical effects.

[0248] See Figure 10 , the head-mounted display device provided by the embodiments of the present application mainly includes a processor 1001, a memory 1002, a display screen 1003, and a communication interface 1004. The communication interface 1004, the display screen 1003, the memory 1002, and the processor 1001 are connected through a bus 1005;

[0249] The communication interface 1004 is used for sending and receiving data;

[0250] The display screen 1003 includes a left-eye screen and a right-eye screen and is used for displaying a user interface;

[0251] The memory 1002 stores a computer program, and the processor 1001 executes the above Figure 1 and Figure 5 steps of the method for positioning the security boundary shown.

[0252] In some embodiments, the memory 1002 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, programs required to run the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc. The memory 1002 may be a volatile memory, such as a random-access memory (RAM); the memory 1002 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD), or a solid-state drive (SSD); or the memory 1002 is any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1002 may be a combination of the above memories.

[0253] The processor 1001 may include one or more central processing units (CPUs), GPUs, or a digital processing unit, etc. When the processor 1001 is used to call the computer program stored in the memory 1002, it implements Figure 1 and Figure 5 steps of any one of the methods for positioning the security boundary in

[0254] It should be noted that Figure 10 is only an example, which gives the necessary hardware for the head-mounted display device to execute any one of the steps of the method for positioning the security boundary provided in the embodiments of the present application. Those not shown, the head-mounted display device may also include hardware of conventional XR devices such as a pickup, a microphone, a handle, a power supply, a button, an IMU, etc.

[0255] In the embodiments of the present application, the specific connection medium between the communication interface 1004, the display screen 1003, the memory 1002, and the processor 1001 is not limited. In the embodiments of the present application, the communication interface 1004 is connected to the bus 1005 between the display screen 1003, the memory 1202, and the processor 1001, which is described in thick lines in Figure 10 The connection manners between other components are only for illustrative purposes and are not to be construed as limiting. The bus 1005 may be divided into an address bus, a data bus, a control bus, etc. For ease of description,Figure 10 It is only described by a thick line, but does not describe that there is only one bus or one type of bus.

[0256] In some embodiments, for the convenience of description, the head-mounted display device can be divided into various modules (or units) according to functions and described separately. Of course, when implementing the present application, the functions of the various modules (or units) can be implemented in the same or multiple software or hardware.

[0257] Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, various aspects of the present application can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuit", "module", or "system" here.

[0258] The embodiment of the present application also provides a computer-readable storage medium for storing some instructions, and when these instructions are executed, the steps of any one of the security boundary positioning methods in the foregoing embodiments can be completed.

[0259] The embodiment of the present application also provides a computer program product for storing a computer program, and the computer program is used to execute the steps of any one of the security boundary positioning methods in the foregoing embodiments.

[0260] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0261] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 a block or multiple blocks.

[0262] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions specified in one or more of the flow Figure 1 charts or multiple flow charts and / or block Figure 1 diagrams or multiple block diagrams.

[0263] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one or more of the flow Figure 1 charts or multiple flow charts and / or block Figure 1 diagrams or multiple block diagrams.

[0264] It is apparent that those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to cover these modifications and variations.

Claims

1. A method for positioning a safety boundary, characterized in that, Applied to a head-mounted display device, the method includes: When the head-mounted display device is started, scanning the surrounding environment where the user is located; Determining whether at least one anchor point is scanned in the surrounding environment within a preset time; When no anchor point is scanned, creating at least one anchor point in the surrounding environment and associating 3D data of a safety boundary with each anchor point; wherein different anchor points are located in different orientations of the surrounding environment; When at least one anchor point is scanned, according to the comparison result of the number of anchor points and a preset number threshold, selecting a target anchor point from the at least one anchor point, and when the user is in a dangerous situation, drawing and displaying a safety boundary according to the 3D data associated with the target anchor point.

2. The method according to claim 1, characterized in that, The step of selecting a target anchor point from the at least one anchor point according to the comparison result of the number of anchor points and a preset number threshold includes: When the number of anchor points is greater than 1 and does not exceed the preset number threshold, taking the first scanned anchor point as the target anchor point; the preset number threshold is an integer greater than 2; When the number of anchor points is greater than 1 and exceeds the preset number threshold, obtaining the 3D data of the multiple scanned anchor points, and selecting a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points; When the number of anchor points is equal to 1, directly taking the scanned anchor point as the target anchor point.

3. The method according to claim 2, characterized in that, Each 3D data includes at least the 6Dof pose of a safety boundary in the anchor coordinate system of the corresponding anchor point; Then the step of obtaining the 3D data of the multiple scanned anchor points and selecting a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points includes: Obtaining the 6Dof poses in the 3D data associated with the multiple scanned anchor points; Calibrating the 6Dof poses of the multiple anchor points to obtain a target pose; Calculating the error between the 6Dof pose of each obtained anchor point and the target pose, and taking the anchor point corresponding to the 6Dof pose with the minimum error as the target anchor point.

4. The method according to any one of claims 1 to 3, characterized in that, The head-mounted display device is equipped with at least one camera, and the steps of scanning the surrounding environment where the user is located and determining whether at least one anchor point is scanned in the surrounding environment within a preset time include: Invoking a first process to control the camera to scan the surrounding environment where the user is located; the first process is a background process; Within a preset time, determining whether the first process receives at least one anchor point scanned by the camera; The step of creating at least one anchor point in the surrounding environment and associating 3D data of a safety boundary with each anchor point includes: Invoking a second process to create at least one anchor point in the surrounding environment and associating 3D data of a safety boundary with each anchor point; the second process and the first process belong to the same application; The step of selecting a target anchor point from the at least one anchor point according to the comparison result of the number of anchor points and a preset number threshold, and when the user is in a dangerous situation, drawing and displaying a safety boundary according to the 3D data associated with the target anchor point includes: Call the first process to select a target anchor from the at least one anchor according to the comparison result between the number of anchors and a preset number threshold, and when the user is in a dangerous situation, call the second process to draw and display a safety boundary according to the 3D data associated with the target anchor.

5. The method according to claim 4, wherein The calling the second process to create at least one anchor in the surrounding environment and associate 3D data of a safety boundary with each anchor includes: Controlling the second process to send a scanning instruction to the first process, so that the first process controls the camera to scan the surrounding environment where the user is located according to the scanning instruction, and obtain environmental images in at least one direction in the surrounding environment; Receiving at least one environmental image sent by the first process through the second process; For each environmental image, calling the second process to perform the following operations: Extracting feature points in the environmental image and determining an anchor in the surrounding environment according to the feature points; Obtaining a 3D model of a safety boundary created for the anchor and determining the 6DOf pose of the 3D model in the anchor coordinate system corresponding to the anchor; Associating the 3D model and the 6DOf pose as 3D data of a safety boundary with the anchor.

6. The method according to claim 5, wherein The first process and the second process are started in the following manner: Create a custom tracking space, where the tracking space is used to define the initial pose of the head-mounted display device, the tracking space serves as a virtual origin, and the tracking data of the head-mounted display device is transformed and applied relative to the virtual origin; Setting the tracking space as an object of the XR session; where the XR session is used to ensure that all virtual objects are under the same root node; Adding a component for managing the pose of the virtual object and setting the at least one anchor as a virtual object in the tracking space; Starting the first process and the second process in the same application, so that the first process and the second process share the tracking space.

7. The method according to claim 4, wherein The calling the second process to draw and display a safety boundary according to the 3D data associated with the target anchor when the user is in a dangerous situation includes: Calling the first process to obtain the 6Dof pose of the head-mounted display device; Calling the first process to determine whether the user is in a dangerous situation according to the 6Dof pose of the head-mounted display device; When it is determined that the user is in a dangerous situation, controlling the first process to send a safety instruction to the second process; Calling the second process to load the 3D data of the safety boundary associated with the target anchor according to the safety instruction, draw a safety boundary according to the obtained 3D data and display it.

8. A head-mounted display device, characterized in that, Including a processor, a memory, a display screen and a communication interface, the communication interface, the display screen, the memory and the processor are connected through a bus; The communication interface is used for sending and receiving data; The memory stores a computer program, and the processor performs the following operations according to the computer program: Scanning the surrounding environment where the user is located when the head-mounted display device is started; Determine whether at least one anchor point is scanned in the surrounding environment within a preset time; When no anchor point is scanned, create at least one anchor point in the surrounding environment and associate 3D data of a safety boundary with each anchor point; wherein, different anchor points are located in different orientations of the surrounding environment; When at least one anchor point is scanned, select a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold, and when the user is in a dangerous situation, draw a safety boundary according to the 3D data associated with the target anchor point and display it through the display screen.

9. The head-mounted display device according to claim 8, wherein The processor selects a target anchor point from the at least one anchor point according to the comparison result between the number of anchor points and a preset number threshold, and the specific operation is as follows: When the number of anchor points is greater than 1 and does not exceed the preset number threshold, take the first scanned anchor point as the target anchor point; the preset number threshold is an integer greater than 2; When the number of anchor points is greater than 1 and exceeds the preset number threshold, obtain the 3D data of the multiple scanned anchor points, and select a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points; When the number of anchor points is equal to 1, directly take the scanned anchor point as the target anchor point.

10. The head-mounted display device according to claim 9, wherein, Each 3D data at least includes the 6Dof pose of a safety boundary in the anchor coordinate system of the corresponding anchor point; Then the processor selects a target anchor point from the multiple anchor points according to the 3D data of the multiple anchor points, and the specific operation is as follows: Obtain the 6Dof poses in the 3D data associated with the multiple scanned anchor points; Calibrate the 6Dof poses of the multiple anchor points to obtain a target pose; Calculate the error between the 6Dof pose of each obtained anchor point and the target pose respectively, and take the anchor point corresponding to the 6Dof pose with the smallest error as the target anchor point.

Images