Exhibition interaction method and device based on XR large space

By using space markers and infrared sensors in large spaces combined with an information collector of XR headsets, the target camera is automatically adjusted to display interactive images, solving the problem of cumbersome gesture control in the existing technology, and realizing immersive style without gestures Interactive experience.

CN118915915AInactive Publication Date: 2024-11-08BEI JING XIN ZHI YUAN LANG WANG LUO KE JI YOU XIAN GONG SI
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202411056320.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing XR-based simulated multi-person interaction system needs to be manipulated through gestures in extended real multi-person interaction scenarios, resulting in cumbersome interaction process.

Method used

Use space markers and infrared sensors in large space to obtain virtual exhibition images, collect target adjustment information through the information collector on the XR headset, and then analyze and adjust the target camera to display the corresponding interactive picture to achieve immersive interaction without gestures. .

Benefits of technology

Through the automatic adjustment of the target camera, users can complete immersive interaction with the virtual exhibition screen without gestures, simplifying the multi-person interaction process.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118915915A_ABST
    Figure CN118915915A_ABST
Patent Text Reader

Abstract

The invention discloses an exhibition interaction method and device based on an XR large space, and the method comprises the steps: a server obtains a virtual exhibition picture corresponding to the large space through a space mark in the large space and an infrared sensor corresponding to a road in the large space; collecting target adjustment information according to an information collector installed on the XR head-mounted equipment; analyzing the target adjustment information to obtain a target adjustment parameter for adjusting the target camera; and based on the target adjustment parameter, displaying an interaction picture corresponding to the target adjustment parameter in the virtual exhibition picture through a display device in the XR head-mounted device, so that the direction of the target camera can be adjusted along with the target adjustment information collected by an information collector installed on the XR head-mounted device through the method. Therefore, the display device in the XR head-mounted device displays the interaction picture corresponding to the target adjustment parameter in the virtual exhibition picture, and immersive interaction with the virtual exhibition picture can be completed without a gesture.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an exhibition interaction method based on an XR large space in the technical field, and in particular to an exhibition interaction method based on an XR large space, and also to an exhibition interaction device based on an XR large space applying the method. Background Art

[0002] Extended reality (XR) includes augmented reality (AR), virtual reality (VR), mixed reality (MR) and other forms. Among them, virtual reality (VR) uses equipment to simulate a virtual world, providing users with simulations of vision, hearing and other senses, with a full sense of "immersion" and "presence"; augmented reality (XR) is a new technology that "seamlessly" integrates real-world information and virtual-world information. It simulates and superimposes physical information that is difficult to experience in a certain time and space range in the real world through computers and other scientific technologies, applies virtual information to the real world, and is perceived by human senses, thereby achieving a sensory experience beyond reality; the real environment and virtual objects are superimposed on the same screen or space in real time; mixed reality (MR) is to mix the real world and the virtual world together to produce a new visual environment, which contains both physical entities and virtual information, and must be "real-time"; and extended reality (XR) includes augmented reality (AR), virtual reality (VR), mixed reality (MR) and other forms. The emergence and use of XR also makes interaction in virtual space possible.

[0003] The existing XR-based simulated multi-person interaction system has the following problems when in use: when the reality multi-person interaction scene is extended, it is necessary to control various situations in the scene through gestures, which is rather cumbersome. During the multi-person interaction process, gestures are also required to control; therefore, people urgently need an XR-based simulated multi-person interaction system to solve the above problems. Summary of the invention

[0004] In order to solve the technical problem that in the prior art, when extending the reality multi-person interaction scene, it is necessary to control various situations in the scene through gestures, which makes the interaction process more cumbersome, the present invention provides an exhibition interaction method and device based on XR large space.

[0005] The present invention is implemented by the following technical scheme: an exhibition interaction method based on an XR large space, which includes the following steps: using space markers in the large space and infrared sensors corresponding to roads arranged in the large space to obtain a virtual exhibition screen corresponding to the large space; according to target adjustment information collected by an information collector installed on an XR head-mounted device, the target adjustment information includes information for adjusting a target camera on the XR head-mounted device; analyzing the target adjustment information to obtain target adjustment parameters for adjusting the target camera; based on the target adjustment parameters, displaying an interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen through a display device in the XR head-mounted device.

[0006] As a further improvement of the above scheme, the use of space markers in a large space and infrared sensors corresponding to the roads in the large space includes the following steps: high-definition motion cameras are set around the roads in the large space, wherein the shooting lenses on each of the high-definition motion cameras correspond to the roads in the large space; the movements of pedestrians on the roads in the large space are captured by infrared sensors arranged in a matrix to create a virtual large space picture in the large space; wherein the virtual exhibition picture includes virtual characters and virtual exhibits.

[0007] As a further improvement of the above-mentioned scheme, the use of spatial markers in a large space and infrared sensors corresponding to the roads in the large space also includes the following steps: obtaining photos of exhibition items exhibited in the large space; based on the photos of the exhibition items, constructing virtual item models corresponding to the photos of the exhibition items; based on the actual relative positions of the exhibition items in the large space, placing the virtual item models in the virtual large space screen in XR.

[0008] As a further improvement of the above scheme, the information collector includes an infrared sensor, and the target adjustment information includes a target infrared image carrying a head position; the target adjustment information is analyzed to obtain target adjustment parameters for adjusting the target camera, including: analyzing the head position in the target infrared image to obtain head movement information; and converting the head movement information into the target adjustment parameters for adjusting the target camera.

[0009] As a further improvement of the above scheme, the head position in the target infrared image is analyzed to obtain head movement information, including the following steps: taking the average coordinates of the first number of target position coordinates sampled before as the starting position of the head movement; taking the average coordinates of the second number of target position coordinates sampled last as the ending position of the head movement; determining the movement distance information and movement direction information of the head based on the movement starting position and the movement ending position; and acquiring the head movement information based on the movement distance information and the movement direction information.

[0010] As a further improvement of the above-mentioned scheme, based on the target adjustment parameters, displaying the interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen through the display device in the XR head-mounted device includes the following steps: presetting the mapping between the target adjustment parameters and the target camera adjustment direction; based on the target adjustment parameters and the mapping, obtaining the target adjustment direction of the target camera; based on the target adjustment direction, obtaining the interactive screen corresponding to the target adjustment direction in the virtual exhibition screen; and displaying the interactive screen through the display device in the XR head-mounted device.

[0011] As a further improvement of the above solution, the method also includes the following steps: using a format conversion tool to convert the virtual item models in OBJ format into GLTF format to create a digital resource library of cultural relics; the digital resource library of cultural relics is electrically connected to the cloud server.

[0012] The present invention also provides an exhibition interaction device based on an XR large space, which applies any of the above-mentioned exhibition interaction methods based on an XR large space, and includes: a virtual picture acquisition module, which is used to use space markers in the large space and infrared sensors corresponding to roads arranged in the large space to acquire a virtual exhibition picture corresponding to the large space; an adjustment information acquisition module, which is used to collect target adjustment information according to an information collector installed on an XR head-mounted device, and the target adjustment information includes information for adjusting a target camera on the XR head-mounted device; an adjustment parameter acquisition module, which is used to analyze the target adjustment information to obtain target adjustment parameters for adjusting the target camera; and a picture display module, which is used to display an interactive picture corresponding to the target adjustment parameters in the virtual exhibition picture through a display device in the XR head-mounted device based on the target adjustment parameters.

[0013] Compared with the existing exhibition interaction method based on XR large space, the exhibition interaction method and device based on XR large space of the present invention have the following beneficial effects: the server uses the space markers in the large space and the infrared sensors corresponding to the roads arranged in the large space to obtain the virtual exhibition screen corresponding to the large space; according to the target adjustment information collected by the information collector installed on the XR head-mounted device; the target adjustment information is analyzed to obtain the target adjustment parameters for adjusting the target camera; based on the target adjustment parameters, the interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen is displayed by the display device in the XR head-mounted device. Through the above method, the target camera can adjust the direction according to the target adjustment information collected by the information collector installed on the XR head-mounted device, so that the display device in the XR head-mounted device displays the interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen, and the immersive interaction with the virtual exhibition screen can be completed without gestures. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flowchart of an XR large space-based exhibition interaction method according to an embodiment of the present invention.

[0015] Figure 2 The figure is a flow chart of the adjustment information acquisition step according to an embodiment of the present invention.

[0016] Figure 3 This is a flow chart of the adjustment information acquisition steps according to another embodiment of the present invention.

[0017] Figure 4 The figure is a flow chart of the adjustment parameter acquisition steps according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0019] See also Figure 1 In an embodiment of the present application, an exhibition interaction method based on an XR large space is proposed, and the method includes the following steps:

[0020] S101, using space markers in the large space and infrared sensors corresponding to roads arranged in the large space to obtain a virtual exhibition picture corresponding to the large space;

[0021] The virtual exhibition screen includes virtual characters and virtual exhibits.

[0022] Specifically, the server sets high-definition motion cameras around the roads in the large space, wherein the shooting lenses on each high-definition motion camera correspond to the roads in the large space; the movements of pedestrians on the roads in the large space are captured through infrared sensors set up in a matrix to create a virtual large space picture in the large space.

[0023] S102, collecting target adjustment information according to an information collector installed on the XR head mounted device, where the target adjustment information includes information for adjusting a target camera on the XR head mounted device;

[0024] The information collector includes an infrared sensor, and the target adjustment information includes an infrared image of the target carrying the head position.

[0025] The average coordinates of the target position coordinates of the first number of samples are used as the starting position of the head movement; the average coordinates of the target position coordinates of the last second number of samples are used as the ending position of the head movement; the movement distance information and movement direction information of the head are determined based on the movement starting position and the movement ending position; based on the movement distance information and the movement direction information, the head movement information is obtained.

[0026] S103, analyzing the target adjustment information to obtain target adjustment parameters for adjusting the target camera;

[0027] Specifically, the server analyzes the head position in the target infrared image to obtain head movement information; and converts the head movement information into target adjustment parameters for adjusting the target camera.

[0028] S104. Based on the target adjustment parameters, an interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen is displayed through a display device in the XR head mounted device.

[0029] Specifically, the server presets a mapping between target adjustment parameters and target camera adjustment directions; based on the target adjustment parameters and the mapping, the target adjustment direction of the target camera is obtained; based on the target adjustment direction, an interactive screen corresponding to the target adjustment direction in the virtual exhibition screen is obtained; and the interactive screen is displayed through a display device in the XR head-mounted device.

[0030] In an embodiment of the present application, the server uses spatial markers in a large space and infrared sensors corresponding to roads arranged in the large space to obtain a virtual exhibition screen corresponding to the large space; according to target adjustment information collected by an information collector installed on the XR head-mounted device; the target adjustment information is analyzed to obtain target adjustment parameters for adjusting the target camera; based on the target adjustment parameters, an interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen is displayed by a display device in the XR head-mounted device. Through the above method, the target camera can adjust its direction according to the target adjustment information collected by the information collector installed on the XR head-mounted device, so that the display device in the XR head-mounted device displays the interactive screen corresponding to the target adjustment parameters in the virtual exhibition screen, and immersive interaction with the virtual exhibition screen can be completed without gestures.

[0031] In an application embodiment, please refer to Figure 2 , step S2 comprises:

[0032] S201, setting high-definition motion cameras around the road in the large space, wherein the shooting lens of each high-definition motion camera corresponds to the road in the large space;

[0033] S202, capturing the movements of pedestrians on the road in the large space through infrared sensors arranged in a matrix, and creating a virtual large space picture in the large space; wherein the virtual exhibition picture includes virtual characters and virtual exhibits.

[0034] Among them, Figure 3 As shown, step S202 includes the following steps:

[0035] S301, obtaining photos of exhibit items exhibited in the large space;

[0036] Acquire multiple exhibit items from a large space, use photographing equipment to photograph the multiple exhibit items in different dimensions, and obtain an overview of each exhibit item.

[0037] S302, based on the photos of the exhibited items, constructing virtual item models corresponding to the photos of the exhibited items;

[0038] Use the distance measurement APP to measure the exhibition items in the display cabinet to obtain the length, width, height and other structural data of each exhibition item; based on the overview map of each exhibition item obtained by S301 and the length, width, height and other structural data of each exhibition item, use the modeling software to reconstruct the measured exhibition items in three dimensions, obtain the virtual item model and export it in OBJ format.

[0039] S303: Based on the actual relative positions of the exhibited items in the large space, the virtual item models are placed in the virtual large space screen in XR.

[0040] In the embodiment of the present application, by using photos of the exhibited items, a virtual item model corresponding to the exhibited item photos is constructed, so that the exhibited items can be restored one-to-one, thereby enhancing the interactive experience of the interactive objects.

[0041] In one embodiment of the application, the method also includes using a format conversion tool to convert the virtual item models in OBJ format into GLTF format to create a digital resource library of cultural relics; the digital resource library of cultural relics is connected to the cloud server.

[0042] In order to make the three-dimensional models of cultural relics display normally in the system, the format conversion tool is used to convert the cultural relic models in OBJ format into GLTF format. This can ensure that users can conduct various XR interactions with the three-dimensional models of the exhibition item photos during the entire experience process to ensure the interactive experience; in addition, the exhibition item photos and virtual item models recognized by XR are stored in the cloud server for unified management, which is convenient for subsequent XR functions to call related images and models.

[0043] In one embodiment of the application, Figure 4 As shown, step S3 includes the following steps:

[0044] S401, analyzing the head position in the target infrared image to obtain head movement information;

[0045] Specifically, the server uses the average coordinates of the first number of target position coordinates sampled as the starting position of the head movement; uses the average coordinates of the last second number of target position coordinates sampled as the ending position of the head movement; determines the movement distance information and movement direction information of the head based on the movement starting position and movement ending position; and obtains the head movement information based on the movement distance information and movement direction information.

[0046] S402: Convert the head motion information into target adjustment parameters for adjusting the target camera.

[0047] The target adjustment parameters include the adjustment direction and adjustment angle of the target camera. Specifically, the head movement direction and the head adjustment angle are parsed from the head movement information, and the head movement information is converted into the target adjustment parameters for adjusting the target camera.

[0048] In an embodiment of the present application, the head motion information is converted into a target adjustment parameter for adjusting the target camera, so that the target camera rotates along with the movement of the user's head, and immersive interaction with the virtual exhibition screen can be completed without gestures.

[0049] This embodiment provides an exhibition interaction device based on an XR large space, and an exhibition interaction method based on an XR large space, which includes:

[0050] A virtual picture acquisition module, which is used to acquire a virtual exhibition picture corresponding to the large space by using space markers in the large space and infrared sensors corresponding to roads arranged in the large space;

[0051] An adjustment information acquisition module, which is used to collect target adjustment information based on an information collector installed on the XR head-mounted device, where the target adjustment information includes information for adjusting a target camera on the XR head-mounted device;

[0052] An adjustment parameter acquisition module, which is used to analyze the target adjustment information to obtain target adjustment parameters for adjusting the target camera;

[0053] A screen display module is used to display the interactive screen corresponding to the target adjustment parameter in the virtual exhibition screen through the display device in the XR head-mounted device based on the target adjustment parameter. Compared with the prior art, the advantages of the exhibition interaction device based on XR large space are the same as those of the exhibition interaction method based on XR large space in Example 1, which will not be repeated here.

[0054] In one embodiment of the application, a computer terminal is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The steps of the exhibition interaction method based on XR large space are implemented when the processor executes the program. When the method is applied, it can be applied in the form of software, such as being designed as an independently running program and installed on a computer terminal, which can be a computer, a smart phone, a control system, and other Internet of Things devices. The method can also be designed as an embedded running program and installed on a computer terminal.

[0055] In one embodiment of the application, this embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the exhibition interaction method based on the XR large space are implemented. When the method is applied, it can be applied in the form of software, such as a program designed to be independently run on a computer-readable storage medium.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. An exhibition interaction method based on XR large space, the method is applied to electronic equipment, characterized in that: The method comprises the following steps: Using space markers in the large space and infrared sensors corresponding to roads arranged in the large space, a virtual exhibition picture corresponding to the large space is obtained; According to target adjustment information collected by an information collector installed on the XR head mounted device, the target adjustment information includes information for adjusting a target camera on the XR head mounted device; Analyze the target adjustment information to obtain target adjustment parameters for adjusting the target camera; Based on the target adjustment parameter, an interactive screen corresponding to the target adjustment parameter in the virtual exhibition screen is displayed through a display device in the XR head mounted device.

2. The exhibition interaction method based on XR large space as claimed in claim 1, characterized in that: The method of utilizing the space marker in the large space and the infrared sensor corresponding to the road in the large space includes the following steps: The high-definition motion cameras are arranged around the roads in the large space, wherein the shooting lenses on the high-definition motion cameras correspond to the roads in the large space; The movements of pedestrians on the road in the large space are captured by infrared sensors arranged in a matrix, so as to create a virtual large space picture in the large space; wherein the virtual exhibition picture includes virtual characters and virtual exhibits.

3. The exhibition interaction method based on XR large space as claimed in claim 2, characterized in that: The method of utilizing the space markers in the large space and the infrared sensors corresponding to the roads in the large space also includes the following steps: Acquiring photos of exhibit items exhibited in the large space; Based on the photos of the exhibited items, constructing virtual item models corresponding to the photos of the exhibited items; Based on the actual relative positions of the exhibit objects in the large space, the virtual object models are XR placed in the virtual large space screen.

4. The exhibition interaction method based on XR large space as claimed in claim 1, characterized in that: The information collector includes an infrared sensor, and the target adjustment information includes an infrared image of the target carrying the head position; The analyzing the target adjustment information to obtain the target adjustment parameters for adjusting the target camera includes: analyzing the head position in the target infrared image to obtain head movement information; The head motion information is converted into the target adjustment parameter for adjusting the target camera.

5. The exhibition interaction method based on XR large space as claimed in claim 4, characterized in that: The step of analyzing the head position in the target infrared image to obtain head movement information comprises the following steps: taking the average coordinate of the target position coordinates of the first number of samples as the starting position of the movement of the head; taking the average coordinate of the last second number of the sampled target position coordinates as the end position of the movement of the head; Determine the movement distance information and movement direction information of the head based on the movement starting position and the movement ending position; Based on the movement distance information and the movement direction information, head movement information is acquired.

6. The exhibition interaction method based on XR large space as claimed in claim 1, characterized in that: The method of displaying an interactive screen corresponding to the target adjustment parameter in the virtual exhibition screen through a display device in the XR head mounted device based on the target adjustment parameter comprises the following steps: Presetting a mapping between the target adjustment parameter and the target camera adjustment direction; Based on the target adjustment parameter and the mapping, obtaining a target adjustment direction of the target camera; Based on the target adjustment direction, acquiring an interactive screen corresponding to the target adjustment direction in the virtual exhibition screen; The interactive screen is displayed through a display device in the XR head-mounted device.

7. The exhibition interaction method based on XR large space as claimed in claim 3, characterized in that: The method further comprises the following steps: The virtual item models in OBJ format are converted into GLTF format using a format conversion tool to create a digital resource library of cultural relics; the digital resource library of cultural relics is electrically connected to the cloud server.

8. An exhibition interactive device based on XR large space, the device comprising: A virtual picture acquisition module, which is used to acquire a virtual exhibition picture corresponding to the large space by using space markers in the large space and infrared sensors corresponding to roads arranged in the large space; An adjustment information acquisition module, which is used to collect target adjustment information based on an information collector installed on the XR head-mounted device, where the target adjustment information includes information for adjusting a target camera on the XR head-mounted device; An adjustment parameter acquisition module, which is used to analyze the target adjustment information to obtain target adjustment parameters for adjusting the target camera; A screen display module is used to display an interactive screen corresponding to the target adjustment parameter in the virtual exhibition screen through a display device in the XR head-mounted device based on the target adjustment parameter.

9. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Product display method and system based on augmented reality, medium and electronic equipment

    CN111739169A

  • Augmented reality interaction method and device and augmented reality equipment

    CN117311506A

  • Method, device and equipment for implementing XR augmented reality scene and storage medium

    CN117994477A