Interface layout method and system based on MR mixed reality technology

Abstract

This invention discloses an interface layout method and system based on MR (Mixed Reality) technology in the field of Mixed Reality. The method includes the following steps: obtaining the user's field of vision; within the user's field of vision, the interface is presented in a 360-degree surround view when the user rotates around themselves as the origin, or in a planar / curved form when the user moves, thereby determining the interface display range; wherein, the interface display range is based on the user's horizon line, ranging from a field of view angle of 10 degrees above the horizon line to a field of view angle of 60 degrees below the horizon line. The interface layout method and system based on MR technology provided by this invention, through a surround-view, tiled interface layout centered on the user's eye point, distributes the interface within a suitable visual distribution range, effectively displaying interface elements and content. Combined with the three-dimensional display characteristics of mixed reality, it optimizes and improves the arrangement of interface display elements and the interface layout method, solving the problems of chaotic interface element arrangement and unreasonable interface layout.

Description

Technical Field

[0001] This invention relates to the field of MR (Mixed Reality) technology, specifically to an interface layout method and system based on MR technology. Background Technology

[0002] In recent years, smart wearable device technology has developed rapidly. Mixed reality technology, with its characteristic of blending the virtual and real worlds, enhances the realism of the user experience. Smart wearable devices, such as smart glasses, are increasingly widely used. Their display interfaces, as an important carrier of interactive content, effectively realize the transmission of information between the real environment, the virtual environment, and the user.

[0003] Currently, various display terminals mainly feature large flat surfaces in their interface layout and display element arrangement. In terms of display format, they still retain the characteristics of flat displays such as web pages. The layout range is mainly tiled, and the displayed content is mainly displayed in the traditional waterfall-style sliding display mode. The limited display content leads to poor visual recognition effect and poor user experience. To address these issues, we propose an interface layout method and system based on MR (Mixed Reality) technology to optimize the interface layout effect of MR technology. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide an interface layout method and system based on MR mixed reality technology, which lays out interface elements within the interface display area, solves the technical problems of limited display content, poor visual recognition effect and poor user experience, and effectively improves the user experience.

[0005] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0006] In a first aspect, the present invention provides an interface layout method based on MR (Mixed Reality) technology, comprising the following steps:

[0007] Obtain the user's field of vision;

[0008] Within the user's line of sight, the interface is presented in a 360-degree three-dimensional surround when the user rotates around himself as the origin, or in a planar / curved form when the user moves, thereby determining the interface display range;

[0009] The interface display range is based on the user's eye level, ranging from a field of view of 10 degrees above the eye level to a field of view of 60 degrees below the eye level.

[0010] Based on the first aspect, further, the interface display range is composed of a first interface layout strip, a second interface layout strip, and a third interface layout strip, wherein the first interface layout strip is composed of a range with a field of view of 10 degrees above the user's eye level, the second interface layout strip is composed of a range with a field of view of 35 degrees below the user's eye level, and the third interface layout strip is composed of a range from 35 degrees to 60 degrees below the user's eye level.

[0011] Furthermore, based on the first aspect, the interface is composed of multiple display panels.

[0012] Based on the first aspect, the interface further includes holographic images and interface display elements, the interface display elements including controls and interface icons.

[0013] Based on the first aspect, further, the holographic image is arranged in the center, and the controls and interface icons are arranged around the holographic image.

[0014] Secondly, the present invention provides an interface layout system based on MR (Mixed Reality) technology, comprising:

[0015] The acquisition module is used to acquire the user's field of vision.

[0016] The presentation module is used to determine the interface display range by presenting the interface in a 360-degree three-dimensional surround when the user rotates around itself as the origin or in a planar / curved form when the user moves, based on the user's eye level. The interface display range is a range from a field of view of 10 degrees above the eye level to a field of view of 60 degrees below the eye level.

[0017] Based on the second aspect, further, the interface display range is composed of a first interface layout strip, a second interface layout strip, and a third interface layout strip, wherein the first interface layout strip is composed of a range with a field of view of 10 degrees above the user's eye level, the second interface layout strip is composed of a range with a field of view of 35 degrees below the user's eye level, and the third interface layout strip is composed of a range from 35 degrees to 60 degrees below the user's eye level.

[0018] Furthermore, based on the second aspect, the interface is composed of multiple display panels.

[0019] Furthermore, based on the second aspect, the interface includes holographic images and interface display elements, the interface display elements including controls and interface icons.

[0020] Furthermore, based on the second aspect, the holographic image is arranged in the center, and the controls and interface icons are arranged around the holographic image.

[0021] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0022] The interface layout method and system based on MR (Mixed Reality) technology provided by this invention distributes the interface within a suitable visual range and arranges interface elements within a comfortable eye range by using a wraparound and tiled interface layout centered on the user's eye point. This effectively displays the content of the interface elements. Combined with the three-dimensional display characteristics of mixed reality, it optimizes and improves the arrangement of interface display elements and the interface layout method, solves the problems of chaotic interface element arrangement and unreasonable interface layout, reduces visual fatigue, and greatly improves user experience and comfort. Attached Figure Description

[0023] Figure 1 This is a diagram showing the interface layout when the user rotates the screen in Embodiment 1 of the present invention.

[0024] Figure 2 This is a diagram showing the interface layout when the user moves in Embodiment 1 of the present invention;

[0025] Figure 3 This is a diagram showing the interface layout effect when the user moves around the holographic image in Embodiment 1 of the present invention;

[0026] Figure 4 This is a schematic diagram of the interface layout in Embodiment 2 of the present invention;

[0027] Figure 5 This is a diagram illustrating the layout of interface elements in Embodiment 2 of the present invention.

[0028] Figure 6 This is a flowchart of the interface layout method in an application example of the present invention.

[0029] In the diagram: 1. First interface layout strip; 2. Second interface layout strip; 3. Third interface layout strip. Detailed Implementation

[0030] In terms of interface layout: the traditional approach is to follow the flat display method of web pages and flat display terminals, and most of them maintain the waterfall scrolling style of web pages, with a limited number of display objects.

[0031] The arrangement of elements displayed on the interface: Since the information displayed on the interface is not limited by space, the arrangement of the displayed content lacks a rule and has a weak sense of order. All the information is arranged in the space, and the use and display of menu navigation are unclear. The interaction is complicated, which can easily cause distraction and increase cognitive load.

[0032] Display elements refer to elements such as holographic images, controls, and icons displayed on the interface.

[0033] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments and specific features in the embodiments are detailed descriptions of the technical solution of the present application, rather than limitations thereof. In the absence of conflict, the embodiments and technical features in the embodiments can be combined with each other.

[0034] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0035] Example 1

[0036] The interface layout method based on MR (Mixed Reality) technology provided in this embodiment includes the following steps:

[0037] Obtain the user's field of vision;

[0038] Within the user's field of vision, the interface is presented in a 360-degree 3D rotation when the user rotates around themselves, or in a flat / curved form when the user moves. This determines the display area of ​​the interface. Figure 1-2 As shown, with the user's eye point as the center, when the user turns around, the interface is arranged on the inner wall of a cylinder and cone that surround the user's eye point in 360°; when the user moves, the interface is laid out in a flat manner, perpendicular to the line of sight, thus forming an interface layout band.

[0039] In this embodiment, the interface display range is based on the user's eye level, extending from a field of view of 10 degrees above the eye level to a field of view of 60 degrees below the eye level. The interface display range consists of a first interface layout strip, a second interface layout strip, and a third interface layout strip. The first interface layout strip covers a field of view of 10 degrees above the user's eye level, the second interface layout strip covers a field of view of 35 degrees below the user's eye level, and the third interface layout strip covers a field of view of 35 to 60 degrees below the user's eye level. Figure 3 As shown, this is the field of view formed by the user surrounding the holographic image. When viewing the holographic image, the observation range is formed as the observer moves around the holographic image, and all interface elements are arranged within this range.

[0040] In this embodiment, the interface is composed of multiple display panels. Essentially, each panel presents interface elements in the form of small planes, thereby solving the problem of large-scale interface elements appearing larger when closer and smaller when farther away, and improving viewing comfort.

[0041] In this embodiment, the interface includes a holographic image and interface display elements. The interface display elements include controls and interface icons. The holographic image is arranged in the center, and the controls and interface icons are arranged around the holographic image.

[0042] Example 2

[0043] like Figure 4-5 As shown, this embodiment provides an interface layout system based on MR (Mixed Reality) technology, including:

[0044] The acquisition module is used to acquire the user's field of vision.

[0045] The presentation module is used to determine the interface display range within the user's field of vision. The interface is presented in a 360-degree surround view when the user rotates around themselves as the origin, or in a planar / curved form when the user moves. The interface display range is based on the user's eye level, extending from a field of view of 10 degrees above the eye level to a field of view of 60 degrees below the eye level. The interface display range consists of a first interface layout strip, a second interface layout strip, and a third interface layout strip. The first interface layout strip is defined by a field of view of 10 degrees above the user's eye level, the second interface layout strip is defined by a field of view of 35 degrees below the user's eye level, and the third interface layout strip is defined by a field of view of 35 to 60 degrees below the user's eye level.

[0046] In the interface layout system provided in this embodiment, the interface is composed of multiple display panels, including holographic images and interface display elements. The interface display elements include controls and interface icons. The holographic image is arranged in the center, and the controls and interface icons are arranged around the holographic image.

[0047] Application examples

[0048] Combination Figure 6 Step 1: Collect the field of view data of the mixed reality glasses to determine the layout range of interface elements; this application example uses HoloLens2 as an example.

[0049] Specifically, the field of view in HoloLens2 is a rectangular visible area of ​​480mm × 303mm, consisting of a horizontal field of view of 43°, a vertical field of view of 28.5°, and a distance of 600mm from the eye point.

[0050] Step 2: Collect the elements displayed on the interface and count their usage frequency;

[0051] Step 3: According to the three interface layout strips in Embodiment 1 of the present invention, the interface elements are now arranged within the rectangular visible area of ​​HoloLens2. At this time, the user can see all the content laid out in the interface without having to turn their head.

[0052] like Figure 4As shown, the display area is divided according to the frequency of use of interface elements and the importance of information. The holographic image is arranged in the very center of the image, and the other interface display elements are arranged around it in a wraparound structure.

[0053] Step 4, according to Embodiment 1 of the present invention, and due to the limitation of the field of view, the interface is defined as several display panels. In essence, the interface elements are presented in the form of small planes on each panel, thereby solving the shortcomings of the previous large-panel interface elements being larger when closer and smaller when farther away, and improving viewing comfort.

[0054] This application example uses two interface layout methods centered on the user's eye point: a wraparound layout and a tiled layout. The interface is mainly distributed within the visual distribution range verified by human-computer interaction. Combined with the characteristics of the HoloLens 2 smart wearable device, the interface elements are arranged within the eye comfort range (boundary constraint range), which can effectively display the content of the interface elements.

[0055] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0056] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0057] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0058] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0059] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An interface layout method based on MR (Mixed Reality) technology, characterized in that, Includes the following steps: Obtain the user's field of vision; Within the user's line of sight, the interface is presented in a 360-degree three-dimensional surround when the user rotates around himself as the origin, or in a planar / curved form when the user moves, thereby determining the interface display range; The interface display range is based on the user's eye level, ranging from a field of view of 10 degrees above the eye level to a field of view of 60 degrees below the eye level. The interface display range consists of a first interface layout strip, a second interface layout strip, and a third interface layout strip. The first interface layout strip is formed within a field of view of 10 degrees above the user's eye level, the second interface layout strip is formed within a field of view of 35 degrees below the user's eye level, and the third interface layout strip is formed within a field of view of 35 to 60 degrees below the user's eye level. The interface is composed of multiple display panels.

2. The interface layout method according to claim 1, characterized in that, The interface includes holographic images and interface display elements, which include controls and interface icons.

3. The interface layout method according to claim 2, characterized in that, The holographic image is arranged in the center, and the controls and interface icons are arranged around the holographic image.

4. An interface layout system based on MR (Mixed Reality) technology, characterized in that, include: The acquisition module is used to acquire the user's field of vision. The presentation module is used to determine the interface display range by presenting the interface in a 360-degree three-dimensional surround when the user rotates around itself as the origin or in a planar / curved form when the user moves, thereby determining the interface display range. The interface display range is based on the user's eye level and ranges from a field of view of 10 degrees above the eye level to a field of view of 60 degrees below the eye level. The interface display range consists of a first interface layout strip, a second interface layout strip, and a third interface layout strip. The first interface layout strip covers a field of view of 10 degrees above the user's eye level, the second interface layout strip covers a field of view of 35 degrees below the user's eye level, and the third interface layout strip covers a field of view of 35 to 60 degrees below the user's eye level. The interface is composed of multiple display panels.

5. The interface layout system as described in claim 4, characterized in that, The interface includes holographic images and interface display elements, which include controls and interface icons.

6. The interface layout system as described in claim 5, characterized in that, The holographic image is arranged in the center, and the controls and interface icons are arranged around the holographic image.

Images