Multi-person panoramic virtual space interaction method

Abstract

The present invention relates to a multi-person panoramic virtual space interaction method, comprising the following steps: initializing a basic scene on a server, the basic scene comprising a basic virtual spherical space and a virtual ground; multiple users respectively running clients, and each user's digital human avatar is located on the virtual ground; the administrator plays a VR panoramic resource, and the server synchronizes the VR panoramic resource playback instructions and interactive hotspots to each common client, and the common client plays the VR panoramic resource in a free visit mode by default. The present invention is supported by a server and a client, and only uses panoramic pictures and VR panoramic resources to construct a virtual space. Through the control of multi-user digital human interactive data, the coordinated operation of the entire system is achieved, and the generation of the virtual space no longer depends on the three-dimensional model, thereby reducing the generation cost of the virtual space and improving the generation efficiency of the virtual space.

Description

Technical Field

[0001] The invention relates to a multi-person panoramic virtual space interaction method, belonging to the technical field of immersive playback of interactive content. Background Art

[0002] Virtual reality (abbreviated as VR) is a manipulable space composed of multi-dimensional information, based on the interaction between people and the real world around them. Its most important goal is to provide a real experience and facilitate natural human-computer interaction. Among them, VR panoramic resources based on panoramic pictures and other materials have integrated panoramic pictures and a variety of interactive hotspot materials such as audio, video, pictures, text, and 3D models, and have been widely used in the fields of VR house viewing and VR digital scenic spots.

[0003] Currently, the experience form of VR panoramic resources is in the form of a single-person visitor experience. Although there may be multiple people online at the same time, there is a lack of data interaction and connection between each other, and the data is still displayed in the form of a single-person visitor experience. In particular, there is a lack of multiple people appearing in the VR panoramic resources in the form of virtual digital avatars for free and controlled visits.

[0004] According to the applicant's understanding, there are also some multi-person virtual space collaboration technologies on the market, which can enable multiple people to appear in virtual scenes as digital human avatars, see each other, and take virtual tours together. However, this type of technology is usually based on three-dimensional model virtual space, not virtual space constructed based on panoramic images, VR panoramic resources, etc. The generation cost of three-dimensional model virtual space is high, and the generation time is also long, which is not conducive to the popularization and application of multi-person panoramic virtual space. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a method for quickly realizing a multi-person panoramic virtual space.

[0006] In order to solve the above technical problems, the technical solution proposed by the present invention is: a multi-person panoramic virtual space interaction method, comprising the following steps:

[0007] Step 1: Configure a server and a client, wherein the server is responsible for data synchronization between multiple clients, managing VR panoramic resources and managing playback materials, and the client is used for actual user operations;

[0008] Step 2: Initializing a basic scene on a server, wherein the basic scene includes a basic virtual spherical space and a virtual ground, wherein the virtual ground is a circular plane, the center of the virtual ground overlaps with the center of the basic virtual spherical space, and the virtual ground is located in the basic virtual spherical space;

[0009] Step 3: Multiple users run the client separately. Each user selects a corresponding digital human image as his / her avatar in the virtual space and joins the basic virtual spherical space. Each digital human avatar is located on the virtual ground and obtains a randomly assigned position. One of the users is an administrator who has the authority to manage space, resources and users. The other users are ordinary users. The client run by the administrator is called the management client, and the client run by the ordinary user is called the ordinary client.

[0010] Step 4: After entering the basic virtual spherical space, the administrator selects to play a VR panoramic resource on the management client. The VR panoramic resource is composed of one or more panoramic pictures in a certain jump switching order. Interactive hotspots are added to the panoramic pictures. The interactive hotspots include position coordinates, hotspot icons, scene jump icons, and play materials. The play materials include audio, video, pictures, and / or text.

[0011] The server synchronizes the VR panoramic resource playback instructions and interactive hotspots to each common client, and the common client starts to play the VR panoramic resource;

[0012] When the management client starts playing the VR panoramic resource, the common client plays the VR panoramic resource in the free visit mode by default, and the playing method includes the following steps:

[0013] All clients uniformly play the panoramic images and interactive hotspot playback materials in the VR panoramic resources on the inner surface of the basic virtual spherical space;

[0014] Each digital human avatar walks freely on the virtual ground, and the server synchronizes the position coordinates, movements, postures and frontal orientation of each digital human avatar in real time, so that each digital human avatar can see the position coordinates, movements, postures and frontal orientation of other digital human avatars;

[0015] When each user clicks on the hotspot icon in the VR panoramic resource to trigger the playback of the corresponding interactive hotspot, the playback operation can only be seen by the user himself, and other users cannot see it;

[0016] When each user clicks the scene jump icon in the VR panoramic resource, the click will trigger a jump from the current panoramic image to another panoramic image. When different users click different scene jump icons, different users will see different panoramic images played in the basic virtual spherical space.

[0017] The present invention is supported by a server and a client, and only uses panoramic images and VR panoramic resources to construct a virtual space. Through the control of multi-user digital human interaction data, the coordinated operation of the entire system is achieved. The generation of the virtual space no longer depends on the three-dimensional model, thereby reducing the generation cost of the virtual space and improving the generation efficiency of the virtual space. Through the collaboration of multiple clients and servers, the present invention realizes functions such as roaming interaction in the virtual space, free experience of VR panoramic resources, and controlled experience of VR panoramic resources. It is very conducive to the popularization and application of multi-person panoramic virtual space, enriches the user experience, and has important application value in the fields of concentrated group visits and explanations in VR scenes, concentrated teaching, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The present invention will be further described below in conjunction with the accompanying drawings.

[0019] Figure 1 It is a schematic diagram of the server and client.

[0020] Figure 2 It is a schematic diagram of the basic virtual spherical space.

[0021] Figure 3 It is a schematic diagram of multiple users joining the basic virtual spherical space.

[0022] Figure 4 This is a schematic diagram of playing VR panoramic resources in free visit mode.

[0023] Figure 5 This is a diagram of the coordinate relationship between ordinary users and administrators in the controlled synchronous visiting mode.

[0024] Figure 6 It is a schematic diagram of the multiple relationship between the basic virtual space radius R and the virtual ground radius r in the controlled synchronous viewing mode.

[0025] Figure 7 This is a schematic diagram of the perspective consistency between ordinary users and administrators in the controlled synchronous viewing mode.

[0026] Figure 8 and Fig. 9 In the controlled synchronous visiting mode, when the administrator rotates, the synchronous virtual spherical space corresponding to ordinary users rotates in the opposite direction.

[0027] Fig.10 Schematic diagram for teachers and students to join the basic virtual spherical space.

[0028] Fig.11 Shown is a schematic diagram of playing a panoramic picture when a teacher and students visit a scene together. DETAILED DESCRIPTION Embodiment 1

[0029] This embodiment relates to a multi-person panoramic virtual space interaction method, comprising the following steps:

[0030] Step 1: Configure the server and client, such as Figure 1 As shown, the server is responsible for data synchronization between multiple clients, managing VR panoramic resources and managing playback materials, the playback materials include audio, video, pictures and text, etc. The client is used for actual user operations. Multiple clients can be connected through the network and interact at the same time. The server and the client are also connected through the network.

[0031] Step 2: Initialize a basic scene on the server, the basic scene includes a basic virtual spherical space and a virtual ground, the virtual ground is a circular plane, the center of the virtual ground overlaps with the center of the basic virtual spherical space, and the virtual ground is located in the basic virtual spherical space. The virtual ground is preferably set as a virtual transparent ground, so that users can see 720-degree panoramic images at various angles when visiting VR panoramic resources.

[0032] like Figure 2 As shown, it is assumed that the center position of the basic virtual spherical space is O and the radius is R; the virtual ground is a circular plane by default, whose center position overlaps with the center of the sphere and whose radius is r. In order to ensure that when the VR panoramic resources are played on the sphere, visitors standing on the virtual ground can have a better field of view, and to minimize the degree of deformation of the panoramic image when watching the entire ground, according to the applicant's test data, it is best to set the range of R to: 3r≤R≤10r.

[0033] Step 3: Multiple users run the client separately, such as Figure 3 As shown, each user selects a corresponding digital human image as his or her own avatar in the virtual space and joins it into the basic virtual spherical space. Each digital human avatar is located on the virtual ground and obtains a randomly assigned position to avoid multiple people appearing in the same position at the same time, which would affect the visual effect. One of the users is an administrator who has the authority to manage space, resources and users. The other users are ordinary users. The client run by the administrator is called an administrative client, and the client run by an ordinary user is called an ordinary client.

[0034] After the administrator joins the basic virtual spherical space, a symbol showing the administrator's characteristics will be displayed above the administrator's digital avatar's head, making it easier for other users to identify him or her.

[0035] Fig.10 It is a schematic diagram of teachers and students joining the basic virtual spherical space, in which the teacher is an administrator with a "Teacher 01" mark above his avatar, and the students are ordinary users, and they select different digital human images respectively.

[0036] Step 4: The management client selects to play a VR panoramic resource. The VR panoramic resource consists of one or more panoramic pictures in a certain jump switching order. Interactive hotspots are added to the panoramic pictures. The interactive hotspots include location coordinates, hotspot icons, scene jump icons and playback materials. The playback materials include audio, video, pictures and / or text. The above-mentioned VR panoramic resources, interactive hotspots and other key information are saved and transmitted through resource json files. Fig.11 That is Fig.10 The diagram shown is a schematic diagram of playing a panoramic picture when a teacher and students visit a scene together, with marking symbols such as "Student 01" and "Student 02" on the top.

[0037] The server synchronizes the VR panoramic resource playback instructions and interactive hotspots (saved in the form of resource json files) to each common client, and the common client starts playing the VR panoramic resource;

[0038] When the management client starts playing the VR panoramic resource, the common client plays the VR panoramic resource in the free visit mode by default, and the playing method includes the following steps:

[0039] All clients will play the panoramic images and interactive hotspots in the VR panoramic resources on the inner surface of the basic virtual spherical space. Figure 4 As shown, according to the panoramic image playback logic in the json file, the first panoramic image is first played on the spherical surface, and the two-dimensional coordinates of each hotspot icon (including panoramic image scene jump icons, audio / video / image / text and other interactive hotspot playback materials) described in the json file in the panoramic image are converted into three-dimensional spherical coordinates, and the icon material of each hotspot is displayed at the spherical coordinate position corresponding to the hotspot.

[0040] Each digital human avatar walks freely on the virtual ground, and the server synchronizes the position coordinates, movements, postures and frontal orientation of each digital human avatar in real time, so that each digital human avatar can see the position coordinates, movements, postures and frontal orientation of other digital human avatars;

[0041] When each user clicks on the hotspot icon in the VR panoramic resource to trigger the playback of the corresponding interactive hotspot, a playback window will pop up at the location for playback, which can also be closed at any time. This playback operation can only be seen by the user himself, and other users cannot see it;

[0042] When each user clicks the scene jump icon in the VR panoramic resource, the click will trigger a jump from the current panoramic image to another panoramic image. When different users click different scene jump icons, different users will see different panoramic images played in the basic virtual spherical space.

[0043] In order to make the user experience more realistic, when different users are distributed in multiple panoramic image scenes through their own operations, the server will synchronize the user data between users in the same panoramic image scene at the moment. The user data includes the location coordinates, actions, postures, frontal orientation and other data of each digital person, so that users in the same panoramic scene can perceive that someone is visiting the scene at the same time as themselves; and users who are currently invisible are in other scenes. Figure 4 For example, in the panoramic picture M of the current VR panoramic resource, there are users u1, u2, u5, and u7, and in the panoramic picture N, there are users u3, u4, and u6.

[0044] Fig.11 That is Fig.10 The diagram shown is a schematic diagram of playing a panoramic picture when a teacher and students visit a scene together. There are marking symbols such as "Student 01" and "Student 02" above the students. The VR panoramic resources seen by each user are different, but each user can perceive the position and direction of other users who are visiting the scene at the same time as himself.

[0045] In addition, if the administrator chooses to stop playing the current VR resource and play the next VR resource, all users will regain the relevant data of the new VR resource, and first enter the first panoramic picture of the new VR resource together, and then move freely, play hot spots and jump scenes according to personal needs.

[0046] Fig.10 and Fig.11 A diagram of a teacher and students visiting a scene together. The administrator’s avatar has a “Teacher 01” sign above it, and they are in the VR resource with ordinary users. Embodiment 2

[0047] The difference between this embodiment and the first embodiment is that when the management client in this embodiment starts playing the VR panoramic resources, the common clients also have a controlled synchronous viewing mode to play the VR panoramic resources. At this time, all common clients are under the unified control of the management client, that is, all digital human avatars located in the basic virtual spherical space simultaneously play the same panoramic picture of the VR panoramic resources, watch the same angle, and play the playback materials of the same interactive hotspots.

[0048] Preferably, at any time when playing VR panoramic resources, the administrator can issue instructions to require all users to enter the controlled synchronous viewing mode. In specific implementation, the method for playing VR panoramic resources in the controlled synchronous viewing mode by the common client includes the following steps:

[0049] After the administrator starts the controlled synchronous tour mode, the position coordinates of all users on the virtual ground remain unchanged, and the body orientations of the digital avatars of all ordinary users are uniformly rotated to the same orientation as that of the administrator; at the same time, all users end their current playback content;

[0050] like Figure 5 As shown, the administrator's panoramic playback space is still the basic virtual spherical space (the center of the sphere is O and the radius is R), and the ordinary clients that receive the controlled synchronous visit mode playback instructions all create an independent synchronous virtual spherical space for playing panoramic pictures. The size of the synchronous virtual spherical space is consistent with that of the basic virtual spherical space, and the position of the administrator's digital human avatar in the basic virtual spherical space is the same as the position of the ordinary user's digital human avatar in its corresponding synchronous virtual spherical space.

[0051] The server manages the panoramic picture currently played by the client and synchronizes it to the common client, and each common user plays the panoramic picture in the synchronized virtual spherical space created by each user;

[0052] Only the administrator can click on the interactive hotspots in the panoramic picture, and ordinary customers cannot click on them. However, both the management client and the ordinary client can synchronously play the playback materials of the interactive hotspots. Therefore, when the administrator clicks on the playback materials of the interactive hotspots such as audio, video, picture, text, etc., the server will synchronize the information and the ordinary client will play the same playback materials.

[0053] Due to the panoramic scene-style 720-degree all-round visual display space, in the controlled synchronous viewing mode, in order to allow all users to strictly follow the administrator's perspective to visit and experience the content, a mechanism is set up to allow all ordinary users to synchronize the administrator's perspective. The details are as follows:

[0054] 1. The administrator's perspective is synchronized when the position moves.

[0055] When the digital human avatar of an administrator or ordinary user moves on the virtual ground, the position of the digital human avatar of the ordinary user on the virtual ground is adjusted in real time, so that the position of the digital human avatar of the administrator in the basic virtual spherical space and the position of the digital human avatar of the ordinary user in the corresponding synchronized virtual spherical space always remain the same; when the administrator or ordinary user moves on the virtual ground, the distance between the administrator's ground coordinates and the ground coordinates of other ordinary users will continue to change. In order to ensure that the changes in the panoramic picture viewing area viewed by the administrator when the position moves are synchronized to the ordinary user in real time, it is necessary to adjust the coordinate position of the center of the sphere of the synchronized virtual spherical space of the ordinary user in real time. Figure 6 and Figure 7 As shown, the center of the basic virtual spherical space is O, the position coordinates of the administrator on the virtual ground are A1, the coordinates of an ordinary user on the virtual ground are A2, and the center coordinates of the synchronized virtual spherical space created by the ordinary user are O2 and the radius is R. The relationship between O2 and O is: O2=O+(A2-A1), where O2, O, A2, and A1 are all vector coordinates.

[0056] It should be noted that, due to the initialization, such as Figure 2 As shown, the radius R of the basic virtual spherical space is not less than 3r, and the farthest distance between A2 and A1 is 2r, so there will be no situation where the synchronized virtual spherical space of a certain ordinary user cannot surround the entire virtual ground. In this way, each ordinary user can see all other users (including administrators and other ordinary users) located at different positions on the virtual ground in the basic virtual spherical space, and there will be no situation where an ordinary user cannot see because he / she is beyond the basic virtual spherical space.

[0057] 2. The administrator's perspective is synchronized when the spatial orientation changes.

[0058] Initially, since all users' digital avatars are forced to face the same direction, everyone's perspective is unified and the images they see are also unified. However, when administrators and ordinary users move in space, their body directions may change. In this case, the following methods can be used to keep the perspective of the panoramic image that administrators and ordinary users see consistent, such as Figure 8 and Fig. 9 As shown, when the body of the administrator's digital human avatar rotates, the orientation of the digital humans of all ordinary users remains unchanged, and their corresponding synchronized virtual spherical spaces rotate in the opposite direction by the same angle. When the body orientation of the digital human avatar of an ordinary user rotates, its corresponding synchronized virtual spherical space rotates along with its rotation direction and angle, so that the playback screen area and angle seen by the digital human avatar of the ordinary user and the digital human avatar of the administrator are completely consistent, that is, the rotation of each digital human avatar itself does not affect the panoramic screen viewing area on the playback sphere that it sees.

[0059] Preferably, in the controlled synchronous visit mode, the movement, walking and other posture and position data of the administrator and ordinary users in the virtual ground space can be synchronized through the server, so that each can see the movement and posture of other users.

[0060] This embodiment can also be improved as follows:

[0061] 1) When the ordinary client plays the VR panoramic resource in a controlled synchronous viewing mode, the management client plays some interactive materials while playing the VR panoramic resource. The interactive materials include interactive questions and auxiliary playback materials. The auxiliary playback materials include audio, video, pictures or text.

[0062] Interactive answering: The administrator selects the test questions on the server and can choose to send answering instructions to all ordinary clients at the same time. After receiving the test questions, ordinary users will display the test questions in the virtual window of their ordinary clients and answer the questions. After answering and submitting the questions, the administrator can push the score ranking list to the ordinary clients. Furthermore, the administrator can also select certain ordinary users and send answering instructions.

[0063] Playing of auxiliary materials such as audio, video, pictures, and text: The administrator selects auxiliary materials on the server and issues auxiliary material playing instructions to all ordinary users. After receiving the auxiliary materials, ordinary users play the auxiliary materials in the virtual window of their ordinary clients. Furthermore, the administrator can also select certain specific auxiliary users and issue auxiliary material playing instructions.

[0064] 2) When the common client adopts the controlled synchronous viewing mode, the administrator can select a new VR panoramic resource from the VR panoramic resource library of the server at any time to play it on the management client during the current VR panoramic resource playback process; at this time, all common clients play the new VR panoramic resource synchronously.

[0065] 3) In the controlled synchronous visit mode or the free visit mode, the administrator can issue a command to end the current VR resource playback or other auxiliary material playback, and all clients will exit the visit mode.

Claims

1. A multi-person panoramic virtual space interaction method, comprising the following steps: Step 1: Configure a server and a client, wherein the server is responsible for data synchronization between multiple clients, managing VR panoramic resources and managing playback materials, and the client is used for actual user operations; Step 2: Initializing a basic scene on a server, wherein the basic scene includes a basic virtual spherical space and a virtual ground, wherein the virtual ground is a circular plane, the center of the virtual ground overlaps with the center of the basic virtual spherical space, and the virtual ground is located in the basic virtual spherical space; Step 3: Multiple users run the client separately. Each user selects a corresponding digital human image as his / her avatar in the virtual space and joins the basic virtual spherical space. Each digital human avatar is located on the virtual ground and obtains a randomly assigned position. One of the users is an administrator who has the authority to manage space, resources and users. The other users are ordinary users. The client run by the administrator is called the management client, and the client run by the ordinary user is called the ordinary client. Step 4: After entering the basic virtual spherical space, the administrator selects to play a VR panoramic resource on the management client. The VR panoramic resource is composed of one or more panoramic pictures in a certain jump switching order. Interactive hotspots are added to the panoramic pictures. The interactive hotspots include position coordinates, hotspot icons, scene jump icons, and play materials. The play materials include audio, video, pictures, and / or text. The server synchronizes the VR panoramic resource playback instructions and interactive hotspots to each common client, and the common client starts to play the VR panoramic resource; When the management client starts playing the VR panoramic resource, the common client plays the VR panoramic resource in the free visit mode by default, and the playing method includes the following steps: All clients uniformly play the panoramic images and interactive hotspot playback materials in the VR panoramic resources on the inner surface of the basic virtual spherical space; Each digital human avatar walks freely on the virtual ground, and the server synchronizes the position coordinates, movements, postures and frontal orientation of each digital human avatar in real time, so that each digital human avatar can see the position coordinates, movements, postures and frontal orientation of other digital human avatars; When each user clicks on the hotspot icon in the VR panoramic resource to trigger the playback of the corresponding interactive hotspot, the playback operation can only be seen by the user himself, and other users cannot see it; When each user clicks the scene jump icon in the VR panoramic resource, the click will trigger a jump from the current panoramic image to another panoramic image. When different users click different scene jump icons, different panoramic images will be played in the basic virtual spherical space seen by different users. When the management client starts playing VR panoramic resources, the common clients also play VR panoramic resources in a controlled synchronous viewing mode. At this time, all common clients are under the unified control of the management client, that is, all digital human avatars located in the basic virtual spherical space play the same panoramic picture of VR panoramic resources, watch the same angle, and play the same interactive hotspot playback materials at the same time; The method for playing VR panoramic resources by the common client in a controlled synchronous viewing mode includes the following steps: After the administrator starts the controlled synchronous tour mode, the position coordinates of all users on the virtual ground remain unchanged, and the body orientations of the digital avatars of all ordinary users are uniformly rotated to the same orientation as that of the administrator; at the same time, all users end their current playback content; The common clients that receive the controlled synchronous tour mode playback instruction all create an independent synchronous virtual spherical space for playing the panoramic picture. The size of the synchronous virtual spherical space is consistent with that of the basic virtual spherical space. The position of the administrator's digital human avatar in the basic virtual spherical space is the same as that of the common user's digital human avatar in its corresponding synchronous virtual spherical space. The server manages the panoramic picture currently played by the client and synchronizes it to the common client, and each common user plays the panoramic picture in the synchronized virtual spherical space created by each user; Only the administrator can click on the interactive hotspots in the panoramic picture, and ordinary users cannot click on them. However, both the management client and the ordinary client can play the playback materials of the interactive hotspots synchronously. When the digital human avatar of the administrator or the ordinary user moves on the virtual ground, the position of the digital human avatar of the ordinary user on the virtual ground is adjusted in real time, so that the position of the digital human avatar of the administrator in the basic virtual spherical space and the position of the digital human avatar of the ordinary user in the corresponding synchronous virtual spherical space always remain the same; When the administrator's digital human avatar rotates, the orientation of all ordinary users' digital humans remains unchanged, and their corresponding synchronized virtual spherical spaces rotate in the opposite direction by the same angle. When the orientation of ordinary users' digital humans rotate, their corresponding synchronized virtual spherical spaces follow their rotation direction and angle, so that the playback screen area and angle seen by the ordinary users' digital human avatars and the administrator's digital human avatars are completely consistent.

2. The multi-person panoramic virtual space interaction method according to claim 1, characterized in that: When the ordinary client plays the VR panoramic resources in a controlled synchronous viewing mode, the movement posture, walking posture and position data of the digital human avatars of the administrator and ordinary users in the virtual ground space are synchronized through the server, so that both the management client and the ordinary client can display the movement, posture and position of the digital human avatars of all users.

3. The multi-person panoramic virtual space interaction method according to claim 1 or 2, characterized in that: When the ordinary client adopts the controlled synchronous viewing mode to play the VR panoramic resources, the management client plays some interactive materials while playing the VR panoramic resources. The interactive materials include interactive questions and auxiliary playback materials. The auxiliary playback materials include audio, video, pictures or text.

4. The multi-person panoramic virtual space interaction method according to claim 1 or 2, characterized in that: When the common client adopts the controlled synchronous viewing mode, the administrator can select a new VR panoramic resource from the VR panoramic resource library of the server at any time to play it on the management client during the current VR panoramic resource playback process; At this time, all ordinary clients play the new VR panoramic resources synchronously.

5. The multi-person panoramic virtual space interaction method according to claim 1 or 2, characterized in that: The management client can require all common clients to enter a controlled synchronous viewing mode at any time when playing VR panoramic resources.

6. The multi-person panoramic virtual space interaction method according to claim 1 or 2, characterized in that: Assuming that the radius of the basic virtual spherical space is R, and the radius of the virtual ground is r, then 3r≤R≤10r.

7. The multi-person panoramic virtual space interaction method according to claim 1 or 2, characterized in that: The virtual ground is a transparent ground.

Images