Method and device for displaying position of object in three-dimensional space in mixed reality system

Abstract

The invention relates to the technical field of MR, and provides a method and equipment for displaying the position of an object in a three-dimensional space in a mixed reality system, which are used for improving the accuracy of sensing a target position point of a virtual object in the three-dimensional space by a user. According to the method, a two-dimensional graphic plane is drawn to mark the ground plane where a user is located, the two-dimensional graphic plane serves as a projection plane of a target position point, and an imaginary ray of the projection point pointing to the target position point is drawn, so that whether the target position point is located above or below the ground plane and the elevation between the target position point and the ground plane are visually marked; the virtual connecting line for visually marking the target position point and the plane position of the user is drawn through the projection point of the target position point and the three-dimensional coordinate of the user, so that the user can accurately sense the specific position of the target position point in the three-dimensional space of the user through the two-dimensional graph plane, the virtual ray and the virtual connecting line, and when multiple target position points exist, the user can accurately sense the specific position of the target position point in the three-dimensional space of the user. The clearness of the display interface is improved by avoiding the intersection of the connecting lines of the multiple target position points.

Description

Technical Field

[0001] The present application relates to the technical field of Mixed Reality (MR), and provides a method and device for displaying the position of an object in a three-dimensional space in a mixed reality system. Background Art

[0002] The MR system can project the image of the real world into the virtual world through video see-through technology, or project the image of the virtual world into the real world through optical see-through technology, so as to achieve the mixed display and interaction of the real world and the virtual world.

[0003] In the MR system, it is necessary to display some objects in the three-dimensional space where the user is located, which are not in the same plane as the ground where the user is located, and are far away from the user and / or blocked. Although the position of the target point in the MR system can render the corresponding object in the form of a virtual model according to its three-dimensional coordinates in the three-dimensional space where the user is located onto the display screen, if only the virtual object is displayed, due to the lack of adjacent reference objects, the user will have an illusion about the position of the virtual object, and the user needs to subjectively imagine the specific position where the virtual object is located, and the accuracy of position perception is relatively low. Summary of the Invention

[0004] Embodiments of the present application provide a method and device for displaying the position of an object in a three-dimensional space in a mixed reality system, which are used to improve the accuracy of position perception of the object in the three-dimensional space.

[0005] On the one hand, embodiments of the present application provide a method for displaying the position of an object in a three-dimensional space in a mixed reality system, including:

[0006] Establish a world coordinate system according to the location where the user is located, and obtain the three-dimensional coordinates of the user and the target position points where at least one virtual object is placed in the world coordinate system respectively, wherein the target position points are not in the same plane as the ground plane where the user is located;

[0007] On the current screen of the display, for each target position point, draw a two-dimensional graphic plane indicating the ground plane where the user is located;

[0008] Project according to the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and the projection point is the center of the two-dimensional graphic plane;

[0009] Draw a virtual ray pointing from the projection point to the target position point, and use the virtual ray to indicate the elevation information of the target position point from the ground plane;

[0010] Draw a virtual connection line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user.

[0011] On the other hand, an embodiment of the present application provides an MR device, including a processor, a memory, and a display, wherein the display, the memory, and the processor are connected via a bus;

[0012] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0013] Establishing a world coordinate system according to the location of the user, and obtaining the three-dimensional coordinates of the user and a target location point where at least one virtual object is placed in the world coordinate system, wherein the target location point is not in the same plane as the ground plane where the user is located;

[0014] On the current screen of the display, for each target location point, drawing a two-dimensional graphic plane indicating the plane where the user is located;

[0015] Projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, where the projection point is the center of the two-dimensional graphic plane;

[0016] Draw a virtual ray pointing from the projection point to the target position point, and use the virtual ray to indicate the elevation information of the target position point from the ground plane;

[0017] According to the projection point of the target position point and the three-dimensional coordinates of the user, a virtual line is drawn to indicate the plane position of the target position point.

[0018] Optionally, the processor uses the virtual ray to mark the elevation information of the target position point from the ground plane, and the specific operation is:

[0019] When the arrow of the virtual ray is perpendicular to the plane of the two-dimensional figure and points upward, the height of the target position point above the ground plane is indicated by the length of the virtual ray, and the height is marked on one side of the virtual ray;

[0020] When the arrow of the virtual ray is perpendicular to the two-dimensional graphic plane and points downward, the length of the virtual ray is used to indicate the depth of the target location point below the ground plane, and the depth is marked on one side of the virtual ray.

[0021] Optionally, when the two-dimensional graphic plane is a non-closed square grid, the processor projects the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and draws a virtual ray from the projection point to the target position point, and uses the virtual ray to indicate the elevation information of the target position point from the ground plane. The specific operations are:

[0022] Draw a first concentric circle marking the target location point, wherein the plane where the first concentric circle is located is parallel to the plane where the non-closed square grid is located, the radius of the first concentric circle is determined according to the number of objects and the current screen size, and the side length of each sub-square grid is determined according to the current screen size;

[0023] The non-closed square grid is used as the projection surface of the target position point, and the three-dimensional coordinates of the target position point are projected onto the non-closed square grid to obtain a projection point, where the projection point is the center of the non-closed square grid;

[0024] A first virtual ray is drawn from the projection point to the center of the first concentric circle, and the first virtual ray is used to indicate the elevation information of the target position point from the ground plane.

[0025] Optionally, the processor draws a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user, and the specific operation is:

[0026] Determine the horizontal distance and the vertical distance between the target position point and the user on a two-dimensional plane according to the projection point of the target position point and the three-dimensional coordinates of the user;

[0027] On the plane where the virtual ray is located, two virtual lines are drawn that are connected at right angles, wherein the plane defined by the two virtual lines is perpendicular to the line between the target position point and the user;

[0028] The horizontal distance between the target location point and the user is marked on the horizontal virtual line of the two virtual lines; the vertical distance between the target location point and the user is marked on the vertical virtual line of the two virtual lines;

[0029] Among them, the horizontal direction refers to the direction in the ground plane where the user is located and perpendicular to the facial orientation of the user, and the vertical direction refers to the direction in the ground plane where the user is located and parallel to the facial orientation of the user. The horizontal virtual line is on the side of the vertical virtual line close to the virtual ray, and the vertical virtual line is below the horizontal virtual line.

[0030] Optionally, the positional relationship between the two virtual lines and the virtual ray includes:

[0031] When the target position point is on the left side of the user, the horizontal virtual line and the vertical virtual line are displayed on the left side of the virtual ray, and the height or depth in the elevation information is displayed on the right side of the virtual ray;

[0032] When the target position point is on the right side of the user, the horizontal virtual line and the vertical virtual line are displayed on the right side of the virtual ray, and the height or depth in the elevation information is displayed on the left side of the virtual ray.

[0033] Optionally, when there are multiple virtual objects, the non-closed square grids corresponding to the multiple target position points are spliced ​​into a non-closed grid with a special-shaped outline.

[0034] Optionally, when the two-dimensional graphic plane is N second concentric circles with equal spacing, the radius of the smallest circle in the second concentric circles is determined according to the number of objects and the current screen size;

[0035] The processor projects the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and draws a virtual ray pointing from the projection point to the target position point, and uses the virtual ray to indicate the elevation information of the target position point from the ground plane. The specific operations are:

[0036] Using the second concentric circle as the projection plane of the target position point, and projecting the three-dimensional coordinates of the target position point onto the second concentric circle to obtain a projection point, where the projection point is the center of the second concentric circle;

[0037] A second virtual ray is drawn with the center of the second concentric circle pointing to the target location point, and the second virtual ray is used to indicate the elevation information of the target location point from the ground plane.

[0038] Optionally, the processor draws a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user, and the specific operation is:

[0039] Determine a plane distance between the target location point and the user according to the projection point of the target location point and the three-dimensional coordinates of the user;

[0040] Draw a virtual line between the center of the second concentric circle and the user, and mark the plane distance in the center of the virtual line in a manner parallel to the virtual line;

[0041] Draw a third concentric circle under the feet of the user to indicate the ground plane where the user is located, and draw a hollow cross on the third concentric circle to indicate the axis of the ground plane, wherein the center of the hollow cross coincides with the center of the third concentric circle;

[0042] A virtual arc line and an angle are drawn between the center of the second concentric circle, the virtual line of the user, and the axis closest to the virtual line, and the virtual arc line and the angle indicate the orientation of the target location point.

[0043] Optionally, when the plane distance between the target location point and the user is less than a preset threshold, the processor further executes:

[0044] The third concentric circle indicating the ground plane, the hollow cross, the virtual line between the second concentric circle and the user, and the virtual arc and angle indicating the plane orientation are stopped from being displayed on the current screen.

[0045] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer device to execute the steps of a method for displaying the position of an object in a three-dimensional space in a mixed reality system provided in an embodiment of the present application.

[0046] The embodiments of the present application provide a method and device for displaying the position of an object in a three-dimensional space in a mixed reality system, and the beneficial effects are as follows:

[0047] For the display of three-dimensional virtual objects in the three-dimensional space where the user is located that are not on the same plane as the ground plane, the three-dimensional coordinates of the target position point of each virtual object in the world coordinate system of the user are obtained, and the target position point is projected onto a two-dimensional graphic plane indicating the plane where the user is located to obtain a virtual ray between the projection point and the target position point. Since the virtual ray is a directed line segment with an arrow, and the projection point at one end of the virtual ray is the center of the two-dimensional graphic plane, the direction of the arrow can intuitively indicate whether the target position point is above or below the ground plane, and the length of the virtual ray can intuitively indicate the distance between the target position point and the ground plane, so that the user can accurately perceive the elevation information between the target position point and the ground plane. At the same time, by drawing a virtual line indicating the plane position of the target position point, the user can accurately perceive the plane information of the target position point and the user in the three-dimensional space, and then accurately perceive the specific position of the target position point relative to the user, thereby improving the user experience.

[0048] At the same time, when drawing virtual lines representing the plane positions of target position points, the intersection of virtual lines of multiple target position points can be avoided, thereby improving the clarity of the display interface and being more user-friendly.

[0049] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A schematic diagram showing a three-dimensional obstruction between a user and a target location provided in an embodiment of the present application;

[0052] Figure 2A and Figure 2B A schematic diagram of an existing MR device displaying the position of an object in three-dimensional space;

[0053] Figure 3 A flow chart of a method for displaying the position of an object in a three-dimensional space in a mixed reality system provided in an embodiment of the present application;

[0054] Figure 4 A flow chart of a method for displaying a target position point in a three-dimensional space provided in an embodiment of the present application;

[0055] Figure 5 A flow chart of a method for displaying the planar position of a target position point provided in an embodiment of the present application;

[0056] Fig. 6A A schematic diagram showing the elevation information and plane position of a target location point provided in an embodiment of the present application;

[0057] Figure 6B A schematic diagram showing the relationship between a target location point and a user's location provided in an embodiment of the present application;

[0058] Fig. 7A and Figure 7B A schematic diagram of marking multiple target location points provided in an embodiment of the present application;

[0059] Figure 8 A flow chart of another method for displaying a target position point in a three-dimensional space provided in an embodiment of the present application;

[0060] Fig. 9 A flow chart of another method for displaying the planar position of a target point provided in an embodiment of the present application;

[0061] Fig. 10A A schematic diagram of the elevation information of the target location point provided in the embodiment of the present application;

[0062] Fig. 10B A schematic diagram of the plane position marking of the target position point provided in the embodiment of the present application;

[0063] Fig. 10C Another schematic diagram of the relationship between a target location point and a user's location provided in an embodiment of the present application;

[0064] Fig.11 Another schematic diagram of marking multiple target location points provided in an embodiment of the present application;

[0065] Fig.12 Another schematic diagram of the relationship between the target location point and the user provided in an embodiment of the present application;

[0066] Fig.13 A structural diagram of an MR device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0067] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0068] For MR devices, virtual objects can be displayed superimposed on real objects through video perspective technology or optical perspective technology. During the superimposed display process, objects that are not on the same plane as the ground plane where the user is located, are far away from the user and / or are blocked may be displayed in the three-dimensional space where the user is located.

[0069] like Figure 1 As shown, it is a schematic diagram of an MR device displaying an object in the three-dimensional space where the user is located. There is a three-dimensional obstruction between the target position point where the virtual object is placed and the user, and the target position point is far away from the user.

[0070] Depend on Figure 1 It can be seen that although the MR system can render the corresponding object in the form of a virtual model according to the target position point and its three-dimensional coordinates in the three-dimensional space where the user is located on the display screen, due to the lack of nearby reference objects, the user will have an illusion about the position of the virtual object, and the user needs to subjectively imagine the specific location of the virtual object, and the accuracy of position perception is low.

[0071] At present, MR devices on the market have made the following improvements to the displayed UI interface in order to improve the perception of objects in the user's three-dimensional space:

[0072] (1) Use a grid to mark the ground plane where the user is located, such as Figure 2AAs shown, the squares representing the ground plane can change with the viewing angle. However, due to the lack of three-dimensional marking of the target location point relative to the ground plane squares, the user still cannot intuitively perceive the specific location of the target location point.

[0073] (2) The distance marking method is a commonly used marking method in traditional 3D modeling software, such as Figure 2B As shown in the figure, the distance lines (dX, dY, dZ) on the XYZ axes are used to indicate the specific position of the target location of the placed object relative to the user. However, when there are multiple objects, there are many distance lines and they intersect, resulting in a cluttered UI interface. Users cannot quickly and accurately perceive the specific position of each virtual object, which is not user-friendly.

[0074] In view of this, an embodiment of the present application provides a method for displaying the position of an object in a three-dimensional space in a mixed reality system. The method uses a drawn two-dimensional graphic plane to mark the ground plane where the user is located for the user and the target position point in the three-dimensional space, and uses the two-dimensional graphic plane as the projection plane of the three-dimensional coordinates of the target position point. Through the virtual ray pointing from the projection point to the target position point, it is intuitively marked whether the target position point is above or below the ground plane, as well as the distance between the target position point and the ground plane. At the same time, through the projection point of the target position point and the three-dimensional coordinates of the user, a drawing is drawn to intuitively mark the plane position of the target position point and the user. Therefore, through the two-dimensional graphic plane, virtual rays and virtual lines, the user can accurately perceive the specific position of the target position point in the three-dimensional space. Moreover, when there are multiple target position points, the intersection of the lines connecting the multiple target position points can be avoided, thereby improving the clarity of the display interface and being more user-friendly.

[0075] See also Figure 3 , is a flow chart of a method for displaying the position of an object in a three-dimensional space in a mixed reality system provided in an embodiment of the present application, and the process mainly includes the following steps:

[0076] S301: Establish a world coordinate system according to the location of the user, and obtain the three-dimensional coordinates of the user and the target location point where at least one virtual object is placed in the world coordinate system.

[0077] In one example, in the MR system, the gravity coordinate system of the user on the ground is used as the world coordinate system (x, y, z), that is, the z-axis of the world coordinate system is perpendicular to the local sea level and upward, and there is no restriction on the direction of the x-axis and y-axis. The center of contact between the user's feet and the ground plane in the three-dimensional space is taken as the user's position, and the three-dimensional coordinates are marked as (x 0 , Y 0 , Z 0 ).

[0078] Assume that at least one virtual object needs to be displayed in the user's field of view, and a target position point where the at least one virtual object is located is not on the same plane as the ground plane where the user is located.

[0079] For example, assuming that the three-dimensional coordinates of the first target position point in the three-dimensional space are (X 1 , Y 1 , Z 1 ), where Z 1 <Z 0 , Z 2 >Z 0 , that is, the first target position point is below the ground plane.

[0080] S302: On the current screen of the display, for each target location point, a two-dimensional graphic plane is drawn to indicate the plane where the user is located.

[0081] Assuming that the width of the current screen is w pixels and the height is h pixels, the size of the current screen is subsequently used to determine the size of the target location point and the ground plane mark.

[0082] It should be noted that, since the ground plane where the user is located is unique, when there are multiple target location points where virtual objects need to be placed, the two-dimensional graphic planes corresponding to the target location points are on the same horizontal plane and can be spliced.

[0083] S303: Projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point.

[0084] In one example, in order to more intuitively indicate the positional relationship between the ground plane and the target location point in three-dimensional space, when drawing a two-dimensional graphic plane of the ground plane where the user is located, the projection point of the target location point can be used as the center of the two-dimensional graphic plane.

[0085] S304: Draw a virtual ray from the projection point to the target position point, and use the virtual ray to indicate the elevation information of the target position point from the ground plane.

[0086] In one example, a virtual ray drawn with the projection point as the starting point and the target position point as the end point is a directed line segment with a direction arrow, wherein the positional relationship between the target position point and the ground plane can be obtained according to the direction of the virtual ray arrow.

[0087] Specifically, when the arrow of the virtual ray is perpendicular to the plane of the two-dimensional graphic and points upward, the length of the virtual ray is used to indicate the height of the target position point above the ground plane, and the height is marked on one side of the virtual ray; when the arrow of the virtual ray is perpendicular to the plane of the two-dimensional graphic and points downward, the length of the virtual ray is used to indicate the depth of the target position point below the ground plane, and the depth is marked on one side of the virtual ray.

[0088] S305: Draw a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user.

[0089] In one example, the plane position relationship between the target location point and the user in the three-dimensional space can be indicated by a horizontal distance and a vertical distance, or by a direction angle and a plane distance.

[0090] In the embodiment of the present application, two different display modes of the target position point in three-dimensional space are provided according to different drawing modes of the two-dimensional graphic plane and the virtual connecting line of the plane.

[0091] Method 1

[0092] When the two-dimensional graphic plane is a non-closed square grid, the position display process of the object in the three-dimensional space in the MR system can be seen in Figure 4 , mainly includes the following steps:

[0093] S40 1: Establishing a world coordinate system according to the location of the user, and obtaining the three-dimensional coordinates of the user and a target location point where at least one virtual object is placed in the world coordinate system, wherein the target location point is not in the same plane as the ground plane where the user is located.

[0094] S402: On the current screen of the display, for each target location point, a non-closed square grid is drawn to indicate the plane where the user is located, and the non-closed square grid is used as a projection surface of the target location point.

[0095] In one example, the side length of each sub-square grid in the non-closed square grid is determined according to the current screen size.

[0096] Taking the current screen width as w pixels and height as h pixels as an example, the side length of each sub-square grid satisfies

[0097] The size of the non-closed square grid can be set according to actual conditions, for example, a 6*6 non-closed square grid, and the non-closed square grids corresponding to each target position point can be extended and spliced.

[0098] In one example, in order to more intuitively indicate the positional relationship between the ground plane and the target location point in the three-dimensional space, the projection point of the target location point on the non-closed square grid may be used as the center of the non-closed square grid.

[0099] S403: Draw a first concentric circle marking the target location point, wherein the plane where the first concentric circle is located is parallel to the plane where the non-closed square grid is located.

[0100] In one example, a first concentric circle parallel to the plane where the non-closed square grid is located is drawn with the target position point as the center of the circle, wherein the radius of the first concentric circle is determined according to the number of objects and the current screen size.

[0101] Take the first concentric circle as two circles as an example, assuming that the radius of the inner circle is r 1 , the radius of the outer circle is r 2 , r 2 For r 1 twice, where r 1 Satisfies the following formula:

[0102]

[0103] Wherein, n is the number of virtual objects displayed, that is, the number of target position points.

[0104] It should be noted that the relationship between the number of first concentric circles and the radius can be adjusted according to actual conditions, and the embodiments of the present application do not impose any restrictive requirements.

[0105] S404: Projecting the three-dimensional coordinates of the target position point onto a non-closed square grid to obtain a projection point.

[0106] Assume that the user's three-dimensional coordinates in the three-dimensional space are (X 0 , Y 0 , Z 0 ), the three-dimensional coordinates of the first target position point in three-dimensional space are (X 1 , Y 1 , Z 1 ), then the three-dimensional coordinates of the projection point (i.e., the center of the grid) of the target position point on the non-closed square grid are (X 1 , Y 1 , Z 0 ).

[0107] S405: Draw a first virtual ray from the projection point to the center of the first concentric circle, and use the first virtual ray to indicate the elevation information of the target position point from the ground plane.

[0108] In one example, when the arrow of the first virtual ray is perpendicular to the non-closed square grid and points upward, the length of the first virtual ray is used to indicate the height of the target position point above the ground plane, and the height is marked on one side of the first virtual ray; when the arrow of the first virtual ray is perpendicular to the non-closed square grid and points downward, the length of the first virtual ray is used to indicate the depth of the target position point below the ground plane, and the depth is marked on one side of the first virtual ray.

[0109] S406: Draw a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user.

[0110] Taking the example of marking the plane position relationship between the target position point and the user in three-dimensional space by horizontal distance and vertical distance, there are marking methods such as Figure 5 As shown, it mainly includes the following steps:

[0111] S4061: Determine the horizontal distance and vertical distance between the target position point and the user on the two-dimensional plane according to the projection point of the target position point and the three-dimensional coordinates of the user.

[0112] In one example, the direction in the ground plane where the user is located and perpendicular to the user's facial orientation is taken as the horizontal direction, and the direction in the ground plane where the user is located and parallel to the user's facial orientation is taken as the vertical direction. The three-dimensional coordinates (X 1 , Y 1 , Z 0 ) and the user's three-dimensional coordinates (X 0 , Y 0 , Z 0 ), calculate the horizontal distance and vertical distance respectively.

[0113] S4062: Draw two right-angled and connected virtual lines on the plane where the virtual ray is located.

[0114] In one example, the plane determined by the two virtual lines is perpendicular to the line between the projection point of the target position and the user position (the line is not displayed), wherein the longitudinal virtual line is below the transverse virtual line, and the transverse virtual line is on the side of the longitudinal virtual line close to the virtual ray.

[0115] Optionally, the lengths of the two virtual lines may be equal, or may be proportional to the horizontal distance and the vertical distance.

[0116] S4063: Marking the horizontal distance between the target location point and the user on the horizontal virtual line of the two virtual lines, and marking the vertical distance between the target location point and the user on the vertical virtual line of the two virtual lines.

[0117] In one example, a horizontal dotted line represents the horizontal distance between the target location point and the user on a two-dimensional plane, and a vertical dotted line represents the vertical distance between the target location point and the user on a two-dimensional plane. Therefore, the value of the horizontal distance is marked on the horizontal dotted line, and the value of the vertical distance is marked on the vertical dotted line.

[0118] In one example, when the planar position relationship between the target position point and the user in the three-dimensional space is indicated by the horizontal distance and the vertical distance, when the target position point is on the left side of the user, the horizontal virtual line and the vertical virtual line are displayed on the left side of the virtual ray, and the height or depth in the elevation information is displayed on the right side of the virtual ray; when the target position point is on the right side of the user, the horizontal virtual line and the vertical virtual line are displayed on the right side of the virtual ray, and the height or depth in the elevation information is displayed on the left side of the virtual ray.

[0119] like Fig. 6A As shown in FIG. 1 , it is a schematic diagram of the elevation information and plane position of the first target position point, wherein the non-closed square grid represents the ground plane where the user is located, the first concentric circle marks the target position point, the projection point of the center of the first concentric circle on the non-closed square grid is the center of the non-closed square grid, the first target position point is on the right side of the user, the arrow of the virtual ray indicates downward that the first target position point is below the ground plane and the depth is 0.8m, the horizontal distance between the first target position point and the user in the two-dimensional plane is 1.2m, and the vertical distance is 6m, and the position relationship between the first target position point where the virtual object is placed and the user in the three-dimensional space is indicated as follows: Figure 6B shown.

[0120] In one example, when there are multiple virtual objects to be displayed, the non-closed square grids corresponding to the projection points of multiple target position points can be spliced ​​into a non-closed grid with an irregular contour to cover all non-closed direction grids, and the side length of each sub-square grid remains unchanged.

[0121] Take three target positions as an example, assuming that the first target position is located on the right side of the user and at a depth of 0.8m below the user's plane, with a lateral distance of 1.2m and a longitudinal distance of 6m from the user in the two-dimensional plane; the second target position is located on the left side of the user and at a depth of 1m below the user's plane, with a lateral distance of 1.6m and a longitudinal distance of 5.4m from the user in the two-dimensional plane; the third target position is located on the left side of the user and at a height of 1.3m above the user's plane, with a lateral distance of 0.6m and a longitudinal distance of 5m from the user in the two-dimensional plane. The third target position is closest to the user's plane distance, followed by the second target position, and then the first target position. The non-closed square grid corresponding to the third target position is taken as the first non-closed square grid. Based on the first non-closed square grid, it starts to extend in the direction of the non-closed square grid corresponding to the second target position, which is also located on the left side of the user, until it covers the non-closed square grid corresponding to the second target position, such as Fig. 7A Then, based on the first non-closed square grid, the grid begins to extend in the direction of the non-closed square grid corresponding to the first target position point on the right side of the user until the non-closed square grid corresponding to the first target position point is covered, as shown in FIG. Figure 7BIn the process of splicing, the elevation information corresponding to each target position point remains unchanged, so as to obtain the final non-closed grid.

[0122] It should be noted that when splicing the non-closed square grids, there are no restrictive requirements for the extended squares of the closed square grids, and they can be adjusted according to actual conditions.

[0123] In one example, the principle of minimizing the area of ​​the spliced ​​non-closed grid can be followed. In this way, the occlusion of other objects in the virtual interface by the non-closed grid can be reduced, making it easier for users to observe other objects.

[0124] For example, when splicing the non-closed square grids, the projection points corresponding to the target position points can be obtained, and the center of each projection point can be calculated. Then, starting from the center, the circle is extended to the outer circle to make the total connection line between the projection points as short as possible, until the non-closed square grids corresponding to all the projection points are covered, and the final non-closed grid is obtained.

[0125] Method 2

[0126] When the two-dimensional graphic plane is N equally spaced second concentric circles, the position display process of the object in the three-dimensional space in the MR system can be seen in Figure 8 , mainly includes the following steps:

[0127] S801: Establish a world coordinate system according to the location of the user, and obtain the three-dimensional coordinates of the user and a target location point where at least one virtual object is placed in the world coordinate system, wherein the target location point is not in the same plane as the ground plane where the user is located.

[0128] S802: On the current screen of the display, for each target location point, draw a second concentric circle indicating the plane where the user is located, and use the second concentric circle as a projection plane of the target location point.

[0129] In one example, N equally spaced second concentric circles are drawn to mark the projection surface of the target position point in the three-dimensional space on the plane where the user is located, wherein the center of the second concentric circle is the projection point of the target position point, the radius of the smallest circle in the second concentric circles is determined according to the number of objects and the current screen size, and the radii of the N second concentric circles are multiples, and N is an integer greater than 1. Optionally, the spacing is the radius of the smallest circle.

[0130] The embodiment of the present application does not impose any restrictive requirements on the spacing and number of the second concentric circles. In one example, N is 3.

[0131] Taking the current screen width as w pixels and height as h pixels as an example, the radius of the smallest circle in the second concentric circle satisfies the following formula:

[0132]

[0133]

[0134] Among them, n is the number of target location points, and a is the radius coefficient.

[0135] S803: Project the three-dimensional coordinates of the target position point onto the second concentric circle to obtain a projection point, wherein the projection point is the center of the second concentric circle.

[0136] Assume that the user's three-dimensional coordinates in the three-dimensional space are (X 0 , Y 0 , Z 0 ), the three-dimensional coordinates of the first target position point in three-dimensional space are (X 1 , Y 1 , Z 1 ), then the three-dimensional coordinates of the projection point (i.e., the center of the circle) of the target position point on the second concentric circle are (X 1 , Y 1 , Z 0 ), the projection point can be used as the center of the second concentric circle.

[0137] S804: Draw a second virtual ray with the center of the second concentric circle pointing to the target location point, and use the second virtual ray to indicate the elevation information of the target location point from the ground plane.

[0138] In one example, when the arrow of the second virtual ray is perpendicular to the second concentric circle and pointing upward, the length of the second virtual ray is used to indicate the height of the target position point above the ground plane, and the height is marked on one side of the second virtual ray; when the arrow of the second virtual ray is perpendicular to the second concentric circle and pointing downward, the length of the second virtual ray is used to indicate the depth of the target position point below the ground plane, and the depth is marked on one side of the second virtual ray.

[0139] Take the first target position point Z 1 <Z 0 For example, the elevation information of the first target location point is indicated as follows Fig. 10A As shown, the first target location point is at a depth of 0.8 m below the ground level where the user is located.

[0140] S805: Draw a virtual line indicating the plane position of the target location point according to the projection point of the target location point and the three-dimensional coordinates of the user.

[0141] Taking the plane distance and direction angle to mark the plane position relationship between the target position point and the user in the three-dimensional space as an example, there are marking methods such as Fig. 9 As shown, it mainly includes the following steps:

[0142] S8051: Determine the plane distance between the target position point and the user according to the projection point of the target position point and the three-dimensional coordinates of the user.

[0143] In one example, the projection point of the target position and the user are located in the same plane, so the plane distance between the target position point and the user can be calculated based on the three-dimensional coordinates of the two in the three-dimensional space.

[0144] S8052: Draw a virtual line between the center of the second concentric circle and the user, and mark the plane distance in the center of the virtual line in a manner parallel to the virtual line.

[0145] Since the center of the second concentric circle is the projection point of the target location point on the plane where the user is located, the center of the second concentric circle can be connected to the user's location point, and the connecting line is a dotted line. The length of the dotted line represents the distance between the target location point and the user on the two-dimensional plane, and the distance value and unit are marked in the center of the dotted line in a manner parallel to the dotted line.

[0146] like Fig. 10B As shown, the horizontal distance between the target location point and the user is marked, wherein the plane distance between the first target location point and the user is 6.11m.

[0147] S8053: Draw a third concentric circle under the feet of the user to indicate the ground plane where the user is located, and draw a hollow cross on the third concentric circle to indicate the axis of the ground plane, and the center of the hollow cross coincides with the center of the third concentric circle.

[0148] In one example, in order to indicate the direction information between the target location point and the user, a third concentric circle is drawn under the feet of the user to indicate the ground where the user is located, and the plane where the third concentric circle is located is perpendicular to the gravity axis. At the same time, a hollow cross is drawn on the third concentric circle, and the center of the hollow cross coincides with the center of the third concentric circle and is coplanar with the third concentric circle, which is used to indicate the axis of the plane where the user is located, that is, the X-axis and Y-axis of the world coordinate system where the user is located. The four sides of the hollow cross represent the positive direction of the Y-axis, the positive direction of the X-axis, the negative direction of the Y-axis, and the negative direction of the X-axis of the world coordinate system, respectively. The length of each side is equal to the outer circle radius of the third concentric circle, and the length of the endpoint of each side close to the center from the center is 1.1 times the inner circle radius of the third concentric circle.

[0149] It should be noted that the size of the hollow cross can be adjusted according to actual needs, and this embodiment of the present application does not make any special requirements.

[0150] In one example, the size of the third concentric circle can be determined according to the user's body information. In order to distinguish the third concentric circle from the second concentric circle, the number of circles in the third concentric circle is different from that in the second concentric circle.

[0151] Taking two concentric circles, the third concentric circle, as an example, assume that the radius of the inner circle is 0.5 m, which is approximately half of the average width of an adult, and the radius of the outer circle is twice the radius of the inner circle, i.e., 1 m, which is approximately half of the average width of an adult plus the average arm length.

[0152] S8054: Draw a virtual arc and an angle between the virtual connection line between the center of the second concentric circle and the user and the axis closest to it, and mark the azimuth of the target position point with the virtual arc and the angle.

[0153] In one example, an angle will be formed between the virtual connection line between the user and the center of the second concentric circle and the axis marking line of the axial marking formed by the hollow cross that is closest to the virtual connection line. Take this angle as the azimuth of the target position point. The arc of this angle is marked with a dotted line, and the angle value is marked on the arc.

[0154] Taking the first target position point as an example, as Fig. 10C shown, the first target position point is located in the direction deviating 11.3° from an axis on the XY plane where the user is located.

[0155] When multiple virtual objects need to be displayed, that is, the number of target position points is multiple and the user is unique. Therefore, only one third concentric circle and a hollow cross need to be drawn, and the azimuths of different target position points are marked on the third concentric circle and the hollow cross respectively.

[0156] As Fig.11 shown, it is the rendering effect diagram of multiple target position points. Among them, the first target position point is at a position 0.8 m deep below the user's ground plane, with a plane azimuth of 11.3°, and the plane distance from the user is 6.11 m. The second target position point is at a position 1.33 m high above the user's ground plane, with a plane azimuth of 6.8°, and the plane distance from the user is 5.03 m.

[0157] In one example, when the plane distance between the target position point and the user is less than the preset threshold, to prevent the interface from being chaotic due to superposition, the display of the third concentric circle marking the ground plane, the hollow cross, the virtual connection line between the second concentric circle and the user, and the virtual arc and angle marking the plane azimuth can be stopped.

[0158] Taking the preset threshold as 1.5 m as an example, assume that the three-dimensional coordinates of the third target position point are (X 3 , Y 3 , Z 3 ), that is, As Fig.12 shown, it is the rendering effect of the third target position point.

[0159] In an embodiment of the present application, for a user and a target location point where a virtual object is placed that are not on the same plane in a three-dimensional space, a world coordinate system is established according to the user's location, the three-dimensional coordinates of the user and the target location point are obtained, and a two-dimensional graphic plane indicating the plane where the user is located is drawn for each target location point, and the plane is projected as the projection plane of the target location point, the projection point is the center of the two-dimensional graphic plane, and a virtual ray pointing from the projection point to the target location point is drawn to intuitively indicate whether the target location point is above or below the ground plane and the elevation between the two. At the same time, a virtual line that intuitively indicates the plane position of the target location point and the user is drawn through the projection point of the target location point and the three-dimensional coordinates of the user, so that through the two-dimensional graphic plane, the virtual ray and the virtual line, when there is an obstruction between the user and the target location point or the target location point has no adjacent reference object, the user can still accurately perceive the specific position of the target location point in the user's three-dimensional space, and when there are multiple target location points, the connection line of the elevation information and the plane information marked in this way can avoid the intersection of the connection lines of multiple target location points, improve the clarity of the display interface, and be more user-friendly.

[0160] Based on the same technical concept, an embodiment of the present application provides an MR device, which can implement the steps in the method for displaying the position of an object in a three-dimensional space in the above-mentioned mixed reality system and can achieve the same technical effect, which will not be repeated here.

[0161] See also Fig.13 The MR device includes a processor 1301, a memory 1302 and a display 1303, wherein the display 1303, the memory 1302 and the processor 1301 are connected via a bus 1304;

[0162] The memory 1302 stores a computer program, and the processor 1301 performs the following operations according to the computer program:

[0163] Establishing a world coordinate system according to the location of the user, and obtaining the three-dimensional coordinates of the user and a target location point where at least one virtual object is placed in the world coordinate system, wherein the target location point is not in the same plane as the ground plane where the user is located;

[0164] On the current screen of the display 1303, for each target location point, a two-dimensional graphic plane is drawn indicating the plane where the user is located;

[0165] Projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, where the projection point is the center of the two-dimensional graphic plane;

[0166] Draw a virtual ray pointing from the projection point to the target position point, and use the virtual ray to indicate the elevation information of the target position point from the ground plane;

[0167] According to the projection point of the target position point and the three-dimensional coordinates of the user, a virtual line is drawn to indicate the plane position of the target position point.

[0168] Optionally, the processor 1301 uses the virtual ray to mark the elevation information of the target location point from the ground plane, and the specific operation is:

[0169] When the arrow of the virtual ray is perpendicular to the plane of the two-dimensional figure and points upward, the height of the target position point above the ground plane is indicated by the length of the virtual ray, and the height is marked on one side of the virtual ray;

[0170] When the arrow of the virtual ray is perpendicular to the two-dimensional graphic plane and points downward, the length of the virtual ray is used to indicate the depth of the target location point below the ground plane, and the depth is marked on one side of the virtual ray.

[0171] Optionally, when the two-dimensional graphic plane is a non-closed square grid, the processor 1301 projects the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and draws a virtual ray from the projection point to the target position point, and uses the virtual ray to indicate the elevation information of the target position point from the ground plane. The specific operations are:

[0172] Draw a first concentric circle marking the target location point, wherein the plane where the first concentric circle is located is parallel to the plane where the non-closed square grid is located, the radius of the first concentric circle is determined according to the number of objects and the current screen size, and the side length of each sub-square grid is determined according to the current screen size;

[0173] The non-closed square grid is used as the projection surface of the target position point, and the three-dimensional coordinates of the target position point are projected onto the non-closed square grid to obtain a projection point, where the projection point is the center of the non-closed square grid;

[0174] A first virtual ray is drawn from the projection point to the center of the first concentric circle, and the first virtual ray is used to indicate the elevation information of the target position point from the ground plane.

[0175] Optionally, the processor 1301 draws a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user, and the specific operation is:

[0176] Determine the horizontal distance and the vertical distance between the target position point and the user on a two-dimensional plane according to the projection point of the target position point and the three-dimensional coordinates of the user;

[0177] On the plane where the virtual ray is located, two virtual lines are drawn that are connected at right angles, wherein the plane defined by the two virtual lines is perpendicular to the line between the target position point and the user;

[0178] The horizontal distance between the target location point and the user is marked on the horizontal virtual line of the two virtual lines; the vertical distance between the target location point and the user is marked on the vertical virtual line of the two virtual lines;

[0179] Among them, the horizontal direction refers to the direction in the ground plane where the user is located and perpendicular to the facial orientation of the user, and the vertical direction refers to the direction in the ground plane where the user is located and parallel to the facial orientation of the user. The horizontal virtual line is on the side of the vertical virtual line close to the virtual ray, and the vertical virtual line is below the horizontal virtual line.

[0180] Optionally, the positional relationship between the two virtual lines and the virtual ray includes:

[0181] When the target position point is on the left side of the user, the horizontal virtual line and the vertical virtual line are displayed on the left side of the virtual ray, and the height or depth in the elevation information is displayed on the right side of the virtual ray;

[0182] When the target position point is on the right side of the user, the horizontal virtual line and the vertical virtual line are displayed on the right side of the virtual ray, and the height or depth in the elevation information is displayed on the left side of the virtual ray.

[0183] Optionally, when there are multiple virtual objects, the non-closed square grids corresponding to the multiple target position points are spliced ​​into a non-closed grid with a special-shaped outline.

[0184] Optionally, when the two-dimensional graphic plane is N second concentric circles with equal spacing, the radius of the smallest circle in the second concentric circles is determined according to the number of objects and the current screen size;

[0185] The processor 1301 projects the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and draws a virtual ray pointing from the projection point to the target position point, and uses the virtual ray to indicate the elevation information of the target position point from the ground plane. The specific operations are:

[0186] Using the second concentric circle as the projection plane of the target position point, and projecting the three-dimensional coordinates of the target position point onto the second concentric circle to obtain a projection point, where the projection point is the center of the second concentric circle;

[0187] A second virtual ray is drawn with the center of the second concentric circle pointing to the target location point, and the second virtual ray is used to indicate the elevation information of the target location point from the ground plane.

[0188] Optionally, the processor 1301 draws a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user, and the specific operation is:

[0189] Determine a plane distance between the target location point and the user according to the projection point of the target location point and the three-dimensional coordinates of the user;

[0190] Draw a virtual line between the center of the second concentric circle and the user, and mark the plane distance in the center of the virtual line in a manner parallel to the virtual line;

[0191] Draw a third concentric circle under the feet of the user to indicate the ground plane where the user is located, and draw a hollow cross on the third concentric circle to indicate the axis of the ground plane, wherein the center of the hollow cross coincides with the center of the third concentric circle;

[0192] A virtual arc line and an angle are drawn between the center of the second concentric circle, the virtual line of the user, and the axis closest to the virtual line, and the virtual arc line and the angle indicate the orientation of the target location point.

[0193] Optionally, when the plane distance between the target location point and the user is less than a preset threshold, the processor 1301 further executes:

[0194] The third concentric circle indicating the ground plane, the hollow cross, the virtual line between the second concentric circle and the user, and the virtual arc and angle indicating the plane orientation are stopped from being displayed on the current screen.

[0195] It should be noted that Fig.13 This is just an example, and provides the necessary hardware for the MR device to execute the steps of a method for displaying the position of an object in a three-dimensional space in a mixed reality system provided in an embodiment of the present application. If not shown, the MR device may also include a two-hand handle, an IMU, a speaker, a pickup, a power supply, etc.

[0196] The memory in the MR device of the present application may be a volatile memory, such as a random-access memory (RAM); the memory may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory is any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, and the memory may be a combination of the above memories, but is not limited thereto. The processor may include one or more central processing units (CPUs) or a general-purpose processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof.

[0197] An embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete a method for displaying the position of an object in a three-dimensional space in a mixed reality system in the aforementioned embodiment.

[0198] An embodiment of the present application also provides a computer program product for storing a computer program, wherein the computer program is used to execute a method for displaying the position of an object in a three-dimensional space in a mixed reality system in the aforementioned embodiment.

[0199] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0200] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0201] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0202] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0203] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for displaying the position of an object in a three-dimensional space in a mixed reality system, It is characterized in that include: Establishing a world coordinate system according to the location of the user, and obtaining the three-dimensional coordinates of the user and a target location point where at least one virtual object is placed in the world coordinate system, wherein the target location point is not in the same plane as the ground plane where the user is located; On the current screen of the display, for each target location point, a two-dimensional graphic plane is drawn indicating the plane where the user is located; Projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, where the projection point is the center of the two-dimensional graphic plane; Draw a virtual ray pointing from the projection point to the target position point, and use the virtual ray to indicate the elevation information of the target position point from the ground plane; According to the projection point of the target position point and the three-dimensional coordinates of the user, a virtual line is drawn to indicate the plane position of the target position point.

2. The method according to claim 1, It is characterized in that Using the virtual ray to mark the elevation information of the target position point from the ground plane includes: When the arrow of the virtual ray is perpendicular to the plane of the two-dimensional figure and points upward, the height of the target position point above the ground plane is indicated by the length of the virtual ray, and the height is marked on one side of the virtual ray; When the arrow of the virtual ray is perpendicular to the two-dimensional graphic plane and points downward, the length of the virtual ray is used to indicate the depth of the target location point below the ground plane, and the depth is marked on one side of the virtual ray.

3. The method according to claim 1, It is characterized in that When the two-dimensional graphic plane is a non-closed square grid, projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and drawing a virtual ray from the projection point to the target position point, and using the virtual ray to indicate the elevation information of the target position point from the ground plane, including: Draw a first concentric circle marking the target location point, wherein the plane where the first concentric circle is located is parallel to the plane where the non-closed square grid is located, the radius of the first concentric circle is determined according to the number of objects and the current screen size, and the side length of each sub-square grid is determined according to the current screen size; The non-closed square grid is used as the projection surface of the target position point, and the three-dimensional coordinates of the target position point are projected onto the non-closed square grid to obtain a projection point, where the projection point is the center of the non-closed square grid; A first virtual ray is drawn from the projection point to the center of the first concentric circle, and the elevation information of the target position point from the ground plane is marked with the first virtual ray.

4. The method according to claim 3, It is characterized in that Drawing a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user, including: Determine the horizontal distance and the vertical distance between the target position point and the user on a two-dimensional plane according to the projection point of the target position point and the three-dimensional coordinates of the user; On the plane where the virtual ray is located, two virtual lines are drawn that are connected at right angles, wherein the plane defined by the two virtual lines is perpendicular to the line between the target position point and the user; Marking the lateral distance between the target location point and the user on a lateral lateral lateral line among the two lateral lateral lines; Marking the longitudinal distance between the target location point and the user on the longitudinal imaginary line among the two imaginary lines; Among them, the horizontal direction refers to the direction in the ground plane where the user is located and perpendicular to the facial orientation of the user, and the vertical direction refers to the direction in the ground plane where the user is located and parallel to the facial orientation of the user. The horizontal virtual line is on the side of the vertical virtual line close to the virtual ray, and the vertical virtual line is below the horizontal virtual line.

5. The method according to claim 3, It is characterized in that The positional relationship between the two virtual lines and the virtual ray includes: When the target position point is on the left side of the user, the horizontal virtual line and the vertical virtual line are displayed on the left side of the virtual ray, and the height or depth in the elevation information is displayed on the right side of the virtual ray; When the target position point is on the right side of the user, the horizontal virtual line and the vertical virtual line are displayed on the right side of the virtual ray, and the height or depth in the elevation information is displayed on the left side of the virtual ray.

6. The method according to any one of claims 1 to 5, It is characterized in that When there are multiple virtual objects, the non-closed square grids corresponding to the multiple target position points are spliced ​​into a non-closed grid with a special-shaped outline.

7. The method according to claim 1, It is characterized in that When the two-dimensional graphic plane is N second concentric circles with equal spacing, the radius of the smallest circle in the second concentric circles is determined according to the number of objects and the current screen size; Projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, and drawing a virtual ray pointing from the projection point to the target position point, and using the virtual ray to indicate the elevation information of the target position point from the ground plane, including: Using the second concentric circle as the projection plane of the target position point, and projecting the three-dimensional coordinates of the target position point onto the second concentric circle to obtain a projection point, where the projection point is the center of the second concentric circle; A second virtual ray is drawn with the center of the second concentric circle pointing to the target location point, and the second virtual ray is used to indicate the elevation information of the target location point from the ground plane.

8. The method according to claim 7, It is characterized in that Drawing a virtual line indicating the plane position of the target position point according to the projection point of the target position point and the three-dimensional coordinates of the user, including: Determine a plane distance between the target location point and the user according to the projection point of the target location point and the three-dimensional coordinates of the user; Draw a virtual line between the center of the second concentric circle and the user, and mark the plane distance in the center of the virtual line in a manner parallel to the virtual line; Draw a third concentric circle under the feet of the user to indicate the ground plane where the user is located, and draw a hollow cross on the third concentric circle to indicate the axis of the ground plane, wherein the center of the hollow cross coincides with the center of the third concentric circle; A virtual arc line and an angle are drawn between the center of the second concentric circle, the virtual line of the user, and the axis closest to the virtual line, and the virtual arc line and the angle indicate the orientation of the target location point.

9. The method according to claim 7 or 8, It is characterized in that When the plane distance between the target location point and the user is less than a preset threshold, the method includes: The third concentric circle indicating the ground plane, the hollow cross, the virtual line between the second concentric circle and the user, and the virtual arc and angle indicating the plane orientation are stopped from being displayed on the current screen.

10. An MR device, It is characterized in that It includes a processor, a memory and a display, wherein the display, the memory and the processor are connected via a bus; The memory stores a computer program, and the processor performs the following operations according to the computer program: Establishing a world coordinate system according to the location of the user, and obtaining the three-dimensional coordinates of the user and a target location point where at least one virtual object is placed in the world coordinate system, wherein the target location point is not in the same plane as the ground plane where the user is located; On the current screen of the display, for each target location point, drawing a two-dimensional graphic plane indicating the plane where the user is located; Projecting the three-dimensional coordinates of the target position point onto the two-dimensional graphic plane to obtain a projection point, where the projection point is the center of the two-dimensional graphic plane; Draw a virtual ray pointing from the projection point to the target position point, and use the virtual ray to indicate the elevation information of the target position point from the ground plane; According to the projection point of the target position point and the three-dimensional coordinates of the user, a virtual line is drawn to indicate the plane position of the target position point.

Images