A method, server and AR device for constructing a virtual three-dimensional conference scene

Abstract

The present invention discloses a method, server and AR device for constructing a virtual three-dimensional conference scene, wherein: if it is detected that the current user logs into the conference system, the three-dimensional model data of the current user is sent to the AR devices of other users except the current user, the material display position information and the current seating information set are sent to the AR devices of the participating users, so that each AR device determines the seating information of the corresponding user according to the current seating information set; the received action data is sent to the AR devices of other users except the corresponding user, the audio data of the participating users and the display material data of the conference speaker are sent to each AR device, so that each AR device drives the corresponding three-dimensional model movement based on the action data, displays the display material data at the corresponding position, and plays the audio data of the corresponding user. It is used to improve the immersion, reality and presence during remote conferences, as well as the ability to interact with the real world.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a method, a server and an AR device for constructing a virtual three-dimensional conference scene. Background Art

[0002] With the advancement and development of science and technology, the emergence of remote conferencing in automated office systems has brought great convenience to people's work. Traditional remote conferencing is generally conducted in the form of video or audio, which lacks a sense of presence and immersion, and cannot provide effective and realistic interactions.

[0003] In order to overcome this problem, virtual meetings have emerged in related technologies, such as holding meetings in a virtual three-dimensional scene using a preset virtual avatar image, with the display device being a PC (Personal Computer), a tablet, a mobile phone, or a VR (Virtual Reality) helmet. Although this form of virtual meeting has a strong sense of immersion and can also interact in a virtual scene, the character image is a preset virtual image, so the sense of reality and presence is poor.

[0004] In addition, in the related technology, three-dimensional reconstruction technology can also be used to collect and reconstruct the three-dimensional image of a real person for a meeting in a virtual scene. However, since the entire scene is virtualized in the form of a three-dimensional model, it is impossible to interact with the real world and the sense of presence is poor. Summary of the invention

[0005] In an exemplary embodiment of the present invention, a method, server and AR device for constructing a virtual three-dimensional conference scene are provided to improve the sense of immersion, reality and presence during remote conferences, as well as the ability to interact with the real world.

[0006] According to a first aspect of an exemplary embodiment, a method for constructing a virtual three-dimensional conference scene is provided, the method comprising:

[0007] If it is detected that the current user logs into the conference system, the 3D model data of the current user is sent to the AR devices of other users among the conference participants except the current user, and the material display location information and the current seating information set are sent to each AR device of the conference participants, so that each AR device determines the seating information of the corresponding user according to the current seating information set;

[0008] For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, receive the action data of the corresponding user acquired by the action acquisition camera; and receive the audio data of the participating user and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera;

[0009] For the action data of any one user, the action data is sent to the AR devices of other participating users except the one user, and the audio data of the participating user and the presentation material data of the conference speaker are sent to the respective AR devices of the participating users, so that each AR device drives the corresponding three-dimensional model to move based on the received action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0010] In an embodiment of the present application, after detecting that a user has logged into the conference system, since the current user's AR device usually already has its own three-dimensional model data, the three-dimensional model data of the user is sent to the AR devices of other participating users except the current user, and the material display location information and the seating information set at the current moment are sent to each AR device of the participating users. In this way, each AR device can determine the seating information of its own user based on the seating information set at the current moment, so as to ensure that the seating information of the participating users is not repeated, thereby avoiding the phenomenon of overlapping characters in the virtual conference scene. For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, the action data of the corresponding user collected by the action acquisition camera is received; as well as the audio data of the participating users and the presentation material data of the conference speaker. Since it is generally not necessary to see one's own actions in one's own VR device, the action data is sent to the AR devices of other users except the source user of the action data, and the audio data of the participating users and the presentation material data of the conference speaker are sent to each AR device of the participating users; each AR device drives the corresponding three-dimensional model movement based on the action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user. The whole process has a sense of reality and immersion, and the conference scene is real, and the action information of the participating users obtained by the application is used to drive the movement of the corresponding user's three-dimensional model, improving the sense of presence and the ability to interact with the real world.

[0011] According to a second aspect of the exemplary implementation, a method for constructing a virtual three-dimensional conference scene is provided, the method comprising:

[0012] If the user associated with the current AR device logs in to the conference system, the three-dimensional model data of other users except the user, as well as the material display location information and the current seating information set are received from the server;

[0013] Determine the seating information of the associated user according to the seating information set at the current moment;

[0014] Receive the action data of other users except the associated user among the conference participants from the server, the audio data of the conference participants and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera; the action data of each user is acquired by the action acquisition camera in the conference scene where the user is located and sent to the server;

[0015] Based on the received action data, the corresponding three-dimensional model is driven to move, the display material data is displayed at the position represented by the material display position information, and the audio data of the corresponding user is played.

[0016] In an embodiment of the present application, if the associated user of the current AR device logs into the conference system, the AR device receives the three-dimensional model data of other users except the associated user from the server, as well as the material display position information and the current seating information set. In this way, the device can determine the seating information of the user according to the current seating information set to ensure that the seating information of each participating user is not repeated, thereby avoiding the phenomenon of overlapping characters in the virtual conference scene. In this way, the current AR device receives the action data of other users except its own associated users from the server, as well as the audio data of the participating users and the display material data of the conference speaker. The current AR device drives the corresponding three-dimensional model movement based on the action data, displays the display material data at the position represented by the material display position information, and plays the audio data of the corresponding user. The current AR device is any AR device, so the whole process has a sense of reality and immersion, and the conference scene is real, and the action information of the participating users obtained by the application is used to drive the movement of the corresponding user's three-dimensional model, improve the sense of presence and the ability to interact with the real world.

[0017] According to a third aspect of an exemplary implementation, there is provided a server for constructing a virtual three-dimensional conference scene, the server comprising a processor, a memory, and at least one external communication interface, wherein the processor, the memory, and the external communication interface are all connected via a bus;

[0018] The external communication interface is configured to receive a login instruction from a user;

[0019] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0020] If it is detected that the current user logs into the conference system, the 3D model data of the current user is sent to the AR devices of other users among the conference participants except the current user, and the material display location information and the current seating information set are sent to each AR device of the conference participants, so that each AR device determines the seating information of the corresponding user according to the current seating information set;

[0021] For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, receive the action data of the corresponding user acquired by the action acquisition camera; and receive the audio data of the participating user and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera;

[0022] For the action data of any one user, the action data is sent to the AR devices of other participating users except the one user, and the audio data of the participating user and the presentation material data of the conference speaker are sent to the respective AR devices of the participating users, so that each AR device drives the corresponding three-dimensional model to move based on the received action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0023] According to a fourth aspect of an exemplary implementation, there is provided an AR device for constructing a virtual three-dimensional conference scene, the AR device comprising a processor, a memory, and at least one external communication interface, wherein the processor, the memory, and the external communication interface are all connected via a bus;

[0024] The external communication interface is configured to receive a notification message of successful login from the user;

[0025] The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0026] If the associated user of the current AR device logs in to the conference system, the 3D model data of other users except the associated user, as well as the material display location information and the current seating information set are received from the server;

[0027] Determine the seating information of the associated user according to the seating information set at the current moment;

[0028] Receive the action data of other users except the associated user among the conference participants from the server, the audio data of the conference participants and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera; the action data of each user is acquired by the action acquisition camera in the conference scene where the user is located and sent to the server;

[0029] Based on the received action data, the corresponding three-dimensional model is driven to move, the display material data is displayed at the position represented by the material display position information, and the audio data of the corresponding user is played.

[0030] According to a fifth aspect of an exemplary implementation, a device for constructing a virtual three-dimensional conference scene is provided, the device comprising:

[0031] A data sending module, for sending the 3D model data of the current user to AR devices of other users except the current user among the conference users, and sending the material display location information and the current seating information set to each AR device of the conference user if it is detected that the current user logs into the conference system, so that each AR device determines the seating information of the corresponding user according to the current seating information set;

[0032] A first data receiving module is used for receiving the motion data of the corresponding user collected by any motion acquisition camera after determining the seating information of the user corresponding to the motion acquisition camera; and receiving the audio data of the participating user and the presentation material data of the conference speaker; wherein a conference scene includes at least one motion acquisition camera;

[0033] The first scene construction module is used to send the action data of any one user to the AR devices of other participating users except the one user, and send the audio data of the participating user and the presentation material data of the conference speaker to the respective AR devices of the participating users, so that each AR device drives the corresponding three-dimensional model to move based on the received action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0034] According to a sixth aspect of an exemplary implementation, there is provided a device for constructing a virtual three-dimensional conference scene, the device comprising:

[0035] A second data receiving module is used to receive the three-dimensional model data of other users except the associated user from the server, as well as the material display position information and the current seating information set when the associated user of the current AR device logs in to the conference system;

[0036] A seating information determination module, used to determine the seating information of the associated user according to the seating information set at the current moment;

[0037] The third data receiving module is used to receive the action data of other users except the associated user among the conference participants from the server, the audio data of the conference participants and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera; the action data of each user is acquired by the action acquisition camera in the conference scene where the user is located and sent to the server;

[0038] The second scene construction module is used to drive the corresponding three-dimensional model to move based on the received action data, display the display material data at the position represented by the material display position information, and play the audio data of the corresponding user.

[0039] According to a seventh aspect of the exemplary implementation, a computer storage medium is provided, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the method for constructing a virtual three-dimensional conference scene as described in the first aspect and the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0041] Figure 1 A schematic diagram of a real conference scene when constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0042] Figure 2 A schematic diagram of a system for constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0043] Figure 3 A flowchart of a method for constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0044] Figure 4 A schematic diagram showing, by way of example, a conference area and a collection area in a conference scenario provided by an embodiment of the present invention;

[0045] Figure 5 A flowchart of a method for constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0046] Figure 6 A signaling flow chart of a method for constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0047] Figure 7 A schematic diagram of the structure of a device for constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0048] Figure 8 A schematic diagram of the structure of a device for constructing a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0049] Fig. 9 The schematic diagram of the structure of a server for building a virtual three-dimensional conference scene provided by an embodiment of the present invention is exemplarily shown;

[0050] Fig.10 The schematic diagram of the structure of an AR device provided by an embodiment of the present invention is exemplarily shown. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0052] For ease of understanding, the terms involved in the embodiments of the present application are explained below:

[0053] (1) 3D reconstruction: refers to the process of reconstructing 3D information based on single-view or multi-view images. For some static scenes and objects, 3D models of the scenes and objects are obtained through 3D reconstruction, and then they can be displayed in the 3D scene through rendering. However, for some dynamic objects, such as people, they often change dynamically, so it is necessary to dynamically establish a series of 3D models. These 3D models are rendered and displayed in sequence to describe the dynamic character movements. For dynamic 3D reconstruction technology, one application method is to transmit the 3D model reconstructed by the acquisition device to the terminal device for rendering and display through a 3D communication system. For example, virtual social networking based on 3D reconstruction is an important application scenario. Different users transmit their own dynamic 3D models to other users through 3D reconstruction. After receiving these data, other users render and display them on their own terminal devices to achieve an immersive social experience.

[0054] (2) AR (Augmented Reality) device: also known as AR head display device. AR technology is a technology that cleverly integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia, three-dimensional modeling, real-time tracking and registration, intelligent interaction, and sensing to simulate computer-generated text, images, three-dimensional models, music, videos and other virtual information, and apply them to the real world. The two types of information complement each other, thereby achieving "enhancement" of the real world. In the embodiment of the present application, the AR device can be AR glasses.

[0055] (3) RGBD camera: RGB refers to color and D refers to depth. The pictures taken by ordinary color cameras can see and record all objects within the camera's field of view, but the recorded data does not include the distance of these objects from the camera. The distance of the object can only be judged through semantic analysis of the image, but there is no exact data. The depth camera solves this problem. The data obtained by the depth camera can accurately know the distance of each point in the image from the camera. In this way, by adding the (x, y) coordinates of the point in the two-dimensional image, the three-dimensional spatial coordinates of each point in the image can be obtained. The three-dimensional coordinates can be used to restore the real scene and realize scene modeling.

[0056] The idea of ​​the embodiments of the present application is summarized below.

[0057] Traditional remote meetings can generally only be conducted through video or audio, which lacks a sense of presence and immersion, and cannot provide effective and realistic interactions. In recent years, remote meetings have emerged in the form of virtual meetings, where users can select some preset virtual Avatar images to hold meetings in a virtual three-dimensional scene. The display device can be a PC, tablet, mobile phone, or VR helmet. This form of virtual meeting has a strong sense of immersion and can also interact freely in the virtual scene, but because the character image is a preset virtual image, the sense of reality and presence is poor.

[0058] In addition, virtual meetings have a new form, that is, using 3D reconstruction technology to collect and reconstruct real-life 3D images for meetings in virtual scenes. This form of virtual remote meetings has a strong sense of immersion and reality, and can also perform some 3D interactions, so it has a good meeting effect. However, since the entire meeting scene is virtualized in the form of a 3D model, it cannot interact with the real world and has a poor sense of presence, so it has certain limitations in practical applications.

[0059] Therefore, it is very important to build a virtual meeting scene that has a strong sense of presence, immersion and reality, and can interact with the real world.

[0060] To this end, the embodiment of the present application provides a method for constructing a virtual three-dimensional conference scene, in which the image of the participant is a real-person three-dimensional image reconstructed using the collected real-person image or video data, so it has a sense of immersion and reality. In addition, the conference scene is real, and at least one action acquisition camera is set in each conference scene. The server sends the action data of the corresponding user collected by each action acquisition camera, the audio data and display material data of the conference speaker collected by the action acquisition camera or AR device to each AR device. In this way, each AR device drives the corresponding three-dimensional model movement based on the action data, displays the display material data at the position represented by the material display position information, and plays the audio data of the conference speaker. In this way, the actions of the three-dimensional image of the real person all follow the real actions of the participants. Therefore, the technical solution of the embodiment of the present application also achieves the effect of interacting with the real world, and the sense of presence is strong.

[0061] After introducing the design ideas of the embodiments of the present application, the following briefly introduces the application scenarios to which the technical solutions of the embodiments of the present application can be applied. It should be noted that the application scenarios introduced below are only used to illustrate the embodiments of the present application and are not limited. In specific implementation, the technical solutions provided by the embodiments of the present application can be flexibly applied according to actual needs.

[0062] refer to Figure 1 , shows a schematic diagram of a real conference scene when constructing a virtual three-dimensional conference scene, wherein, taking the four cities of Beijing, Shanghai, Guangzhou and Shenzhen as an example, the participants are located in their respective conference scenes (conference rooms). Each participant has a real conference scene, and in order to make the sense of presence, immersion and reality stronger, the spatial deployment information of each conference scene is the same, wherein the spatial deployment information includes the spatial size of the conference scene, the size and position of the conference table, and the size and position of the conference chairs. The spatial size of the conference scene includes, for example, length, width and height; the size of the conference table includes, for example, the length, width and height of the conference table; the position of the conference table includes, for example, the position of the conference table in the conference room; the size of the conference chair includes, for example, the length, width and height of the conference chair; the position of the conference chair includes, for example, the position of each conference chair in the conference room or relative to the conference table.

[0063] Figure 1In the figure, a conference table and four conference chairs are shown. The server sends the motion data of the corresponding users (except the users corresponding to its own AR device) received from the motion acquisition camera, as well as the audio data of the participating users (all users participating in the meeting) and the presentation material data of the conference speaker to each AR device, so that each AR device drives the corresponding three-dimensional model movement based on the motion data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user, thereby completing the construction of the virtual three-dimensional conference scene.

[0064] It should be noted that Figure 1 At least one motion acquisition camera is arranged on the conference table in the conference room to collect motion data of users in the corresponding conference room. Figure 1 Not shown.

[0065] in addition, Figure 2 A schematic diagram of a system for constructing a virtual three-dimensional conference scene is shown, wherein four conference scenes are shown, in each of which a motion acquisition camera is set on the conference table, and a participant in each conference scene wears an AR glasses. Each AR glasses interacts with the server to realize the construction of the virtual three-dimensional conference scene.

[0066] To further illustrate the technical solution provided by the embodiment of the present application, this is described in detail below in conjunction with the accompanying drawings and specific implementation methods. Although the embodiment of the present application provides the method operation steps shown in the following embodiments or drawings, more or fewer operation steps may be included in the method based on routine or no creative labor. In the steps where there is no necessary causal relationship logically, the execution order of these steps is not limited to the execution order provided in the embodiment of the present application.

[0067] Combine the following Figure 1 and Figure 2 , the application scenario shown, refer to Figure 3 The flowchart of a method for constructing a virtual three-dimensional conference scene is shown to illustrate the technical solution provided by the embodiment of the present application. The method is executed by a server and includes at least the following steps:

[0068] S301. If it is detected that the current user logs into the conference system, the three-dimensional model data of the current user is sent to the AR devices of other users among the conference participants except the current user, and the material display location information and the seating information set at the current moment are sent to each AR device of the conference users, so that each AR device determines the seating information of the corresponding user according to the seating information set at the current moment.

[0069] S302. For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, receive the action data of the corresponding user acquired by the action acquisition camera; and receive the audio data of the participating users and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera.

[0070] S303. For the action data of any user, the action data is sent to the AR devices of other users except one user among the conference participants, and the audio data of the conference participants and the presentation material data of the conference speaker are sent to the AR devices of the conference participants, so that each AR device drives the corresponding three-dimensional model to move based on the received action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0071] In an embodiment of the present application, after detecting that a user has logged into the conference system, since the current user's AR device usually already has its own three-dimensional model data, the three-dimensional model data of the user is sent to the AR devices of other participating users except the current user, and the material display location information and the seating information set at the current moment are sent to each AR device of the participating users. In this way, each AR device can determine the seating information of its own user based on the seating information set at the current moment, so as to ensure that the seating information of the participating users is not repeated, thereby avoiding the phenomenon of overlapping characters in the virtual conference scene. For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, the action data of the corresponding user collected by the action acquisition camera is received; as well as the audio data of the participating users and the presentation material data of the conference speaker. Since it is generally not necessary to see one's own actions in one's own VR device, the action data is sent to the AR devices of other users except the source user of the action data, and the audio data of the participating users and the presentation material data of the conference speaker are sent to each AR device of the participating users; each AR device drives the corresponding three-dimensional model movement based on the action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user. The whole process has a sense of reality and immersion, and the conference scene is real, and the action information of the participating users obtained by the application is used to drive the movement of the corresponding user's three-dimensional model, improving the sense of presence and the ability to interact with the real world.

[0072] First, various conference scenarios in the embodiments of this application are described by example:

[0073] Since the user's seat information can be determined by the desktop mark on the conference table, it is necessary that the position correspondence between each conference chair and the conference table is consistent with the pre-stored position relationship. Figure 4, the pre-stored position relationship is, for example, conference chair No. 1 is at a certain distance from desktop No. 1, conference chair No. 2 is at a certain distance from desktop No. 2, conference chair No. 3 is at a certain distance from desktop No. 3, and conference chair No. 4 is at a certain distance from desktop No. 4. Desktop No. 5 is used to determine the material display location information. Therefore, before detecting the user logging into the conference system, or before the latter user takes a seat, it is determined that the position correspondence between each conference chair and the conference table is consistent with the pre-stored position relationship. If it is inconsistent, an adjustment message is sent to the terminal device associated with the motion acquisition camera in the conference scene to prompt the adjustment of the position of each conference chair. In addition, the inconsistent situation also includes the situation where the identification on the conference chair or conference table is missing. The seat corresponding to the inconsistent conference chair will be marked as invalid, and the user cannot sit down. In the actual application process, it can be the adjustment of the participants or other personnel in the conference room.

[0074] In actual application, a collection area can be set within a certain range of the conference scene, such as a set area that must be passed before entering the conference room, and a collection area where the participating users enter the conference scene. The collection area can be a rectangular area, and four RGBD cameras are set at the four corners to collect images or video information of the participating users. In actual application, the collection area can also be configured with a PC or tablet computer connected to each camera, and the participating user can enter personal identity information in the PC or tablet computer to query whether the historical three-dimensional model data of the participating user is stored in the server.

[0075] In this way, it is ensured that the position correspondence between each conference chair and the conference table is consistent with the pre-stored position relationship, and the user's seating information can be determined by scanning the desktop mark on the conference table through the AR glasses.

[0076] It should be noted that the location relationship between the collection area and the meeting area can be referred to Figure 4 The collection area can also be set in other areas where participants pass through. This is just an example and does not constitute a specific limitation.

[0077] In S301, the conference speaker starts the conference system through a terminal (such as a smart phone), creates a conference room, and sends the conference room number and conference room password to the terminals of other users through the server for other users to log in to the conference system.

[0078] Exemplarily, each user wears an AR device, such as AR glasses. Taking one of the participating regions as an example, for example, the server detects that user A in Beijing logs into the conference system, and then sends the 3D model data of user A to the AR devices worn by other users except user A. The 3D model data can be obtained from pre-stored historical 3D model data, or generated based on the acquired image or video information.

[0079] Specifically, for any logged-in user, the 3D model data can be determined in the following ways:

[0080] In the first case, if historical three-dimensional model data of a logged-in user is pre-stored, the historical three-dimensional model data is determined to be the three-dimensional model data of the logged-in user.

[0081] In this case, the server pre-stores the historical 3D model data of the logged-in user, and does not need to send a prompt message to the user, but directly applies the historical 3D model data as the 3D model data of the logged-in user.

[0082] In the second case, if historical 3D model data of a logged-in user is pre-stored, the logged-in user is prompted whether to apply the historical 3D model data, and the historical 3D model data is determined to be the 3D model data of the logged-in user based on the received message indicating the application of the historical 3D model data.

[0083] In this case, the server pre-stores historical three-dimensional model data of the logged-in user. The server sends a message to the logged-in user whether to apply the historical three-dimensional model data. After receiving user feedback indicating that the historical three-dimensional model data should continue to be applied, the server determines that the historical three-dimensional model data is the three-dimensional model data of the logged-in user.

[0084] In the third case, if historical 3D model data of a logged-in user is pre-stored, the logged-in user is prompted whether to apply the historical 3D model data. After receiving a message indicating a refusal to apply the historical 3D model data, the 3D model data of the logged-in user is generated based on the acquired image or video information.

[0085] In this case, the server pre-stores the historical 3D model data of the logged-in user, and sends a message to the logged-in user whether to apply the historical 3D model data. After receiving the message from the user indicating that the historical 3D model data is rejected, the server generates the 3D model data of the logged-in user according to the acquired image or video information. The specific process of generating the 3D model data can be found in the description of the related art, which will not be described here.

[0086] In a fourth case, if there is no historical three-dimensional model data of the logged-in user, the three-dimensional model data of the logged-in user is generated based on the acquired image or video information.

[0087] In this case, if there is no historical three-dimensional model data of the logged-in user in the server, the three-dimensional model data of the logged-in user is directly generated based on the acquired image or video information.

[0088] In combination with the scenarios in the embodiments of the present application, the process of acquiring image or video information can be achieved by using the RGBD camera set in the acquisition area in the aforementioned embodiments.

[0089] In the above process, the server sends the 3D model data of the logged-in user to the AR devices of other users.

[0090] In addition, the server sends the material display location information to each AR device of the participating users. Specifically, the material display location information can be determined according to the deployment information of the conference scene. The material display location information can be determined by an identification code pre-pasted on the conference table (for example, Figure 4 5 desktop identification in the display), the identification code may be, for example, a QR code or a bar code, and the display height information in the display position information may be preset.

[0091] In addition, the server sends the current seating information set to each AR device of the participating users. Specifically, after each AR device obtains the seating information of the corresponding user, it sends it to the server in real time, and the server can update the seating information set in real time. The seating information set stores information about which user in which area sits in which seat. Therefore, after detecting that the user has logged into the conference system, the server sends the current seating information set to the user's AR device, so that the AR device can determine the user's seating information based on the current seating information set.

[0092] In actual application, for any AR device, the user wearing the AR device is called the associated user of the AR device. The seating information of the associated user is determined by the AR device associated with the associated user scanning the desktop mark on the conference table within a preset range after the associated user sits down; the seating information set at the current moment is the seating status of each conference chair at the current moment.

[0093] Specifically, since the AR device is worn on the user's head and has a built-in camera, after the user sits down, the corresponding AR device scans the desktop logo on the conference table within a preset range (for example, 0.5 meters in front) after the user sits down.

[0094] Among them, the AR device determines the seating information of the associated user based on the seating information set at the current moment, which can be specifically implemented in the following way: match the initial seating information of the associated user with the seating information set at the current moment, and determine whether the seat represented by the initial seating information is in an idle state. If so, determine the initial seating information as the seating information of the associated user; otherwise, determine the seat identifier that is in an idle state, and prompt the associated user according to the seat identifier.

[0095] In the above embodiment, although there are 4 seats in each conference scene, there are a total of 4 seats for the entire virtual conference scene. If a user is seated, the position will be unavailable, otherwise the characters will overlap. If the current seat is available, the augmented reality prompt "This seat is available" can be displayed above the seat logo. The user can sit down and operate the seat button in the AR glasses to complete the seat, notify the server to update the seat information set, and send the seat information to the motion acquisition camera in the current conference scene. If the current seat is not available, the enhanced display prompt "This seat is already occupied, please choose another seat" will be displayed above the logo. If you force yourself to sit in a seat that is already occupied, the AR glasses will not be able to operate the seat button, and the three-dimensional model of this user will not be sent to other users through the system, and other users will not be able to see the three-dimensional model image of the user through the AR glasses.

[0096] Specifically, for example, there are 4 conference chairs in each conference scene in each region, numbered 1-4, and the position relationship corresponds to the desktop identifiers 1-4 on the conference table, that is, conference chair No. 1 is located near desktop identifier No. 1, conference chair No. 2 is located near desktop identifier No. 2, conference chair No. 3 is located near desktop identifier No. 3, and conference chair No. 4 is located near desktop identifier No. 4. The current seating information set is, for example, that conference chairs No. 1 and No. 3 are idle, and conference chairs No. 2 and No. 4 are occupied.

[0097] For example, if the user scans the No. 2 desktop marker on the desktop, it means that the user is sitting on conference chair No. 2. The seating information is the user's initial seating information. Through the current seating information set, it is determined that conference chair No. 2 is not idle, and then the idle conference chairs are determined to be No. 1 and No. 3, and the user is prompted that No. 2 cannot be sat on. At this time, the user sits on an idle conference chair, such as No. 3, and No. 3 is determined to be the user's seating information.

[0098] For another example, if the user scans position 1 on the desktop, it means that the user is sitting in conference chair 1, and conference chair 1 is vacant, then 1 is determined to be the user's seating information.

[0099] In actual application, there is no necessary connection between a user logging into the conference system and the user taking a seat. However, for the sake of clarity, it can be assumed that the user logs into the user system while taking a seat.

[0100] As mentioned above, for any action capture camera, after the user's seating information in the conference room is determined, the server sends the user's seating information to the action capture camera in the conference scene where the user is located, so that the action capture camera adjusts the shooting direction according to the seating information. For example, if the participant in the conference scene in Shanghai sits in the conference chair No. 3, the corresponding action capture camera adjusts its shooting direction to face the No. 3 position. In this way, the action capture camera can start the shooting process.

[0101] In actual application, the motion capture camera on the conference table collects the following types of user limb motion data:

[0102] The first type is that the motion capture camera is set on the conference table, and can only collect the motion data of the user's upper body (the lower body is blocked by the conference table). Therefore, only the upper body movements are transmitted to the server, and other users only drive the upper body of the model after receiving the user's motion data. Since users sit in seats in most meetings, and the body movements of the upper body are more important than those of the lower body, this method can basically meet the needs.

[0103] The second type is that the motion acquisition cameras include at least two. For example, one is set on the desktop of the conference table to collect the user's upper body data, and the other is set under the desktop of the conference table to collect the user's lower body data. After the two motion acquisition cameras collect the data, they are fused and processed and sent to the server. After receiving the user's motion data, other users can drive the whole body of the three-dimensional model.

[0104] The third type is to set up a motion capture camera on the conference table to collect the motion data of the user's upper body. The user wears a motion capture sensor on the lower body, and the lower body data is captured by the sensor. The motion capture camera and sensor data are fused and processed and sent to the server. After receiving the user's motion data, other users can drive the whole body of the 3D model.

[0105] In S302, for any action capture camera, once the seating information of the user in the conference room is determined, the action capture camera can start shooting, and no matter whether it is the main speaker of the meeting, each participating user may have actions during the entire meeting. As for audio data, all participating users may generate audio data, even at the same time, there may be audio data of multiple users, and the display material data usually comes from the main speaker of the meeting. Therefore, the server receives the action data of the corresponding user collected by each action capture camera, as well as the audio data of the participating users and the display material data of the main speaker of the meeting. Exemplarily, the display material data is in various forms such as slides, documents, videos or images.

[0106] Among them, audio data can be collected by an AR device or by a motion capture camera in a corresponding conference scene. One of them can be selected as the audio capture device according to actual conditions, and there is no limitation here.

[0107] In S303, for the action data of any user, the action data is sent to the AR devices of other users except the user, and the audio data of the participating users and the presentation material data of the conference speaker are sent to each AR device, so that each AR device drives the corresponding three-dimensional model to move based on the action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0108] Among them, when each AR device drives the corresponding three-dimensional model to move based on the motion data, for example, at the same time, the arm of user M moves and the head of user N moves, the motion data of the two are used to determine the corresponding part of the corresponding three-dimensional model to move. At the same time, the display material data is displayed at the position represented by the material display position information, such as a certain distance above the material display mark on the corresponding conference table. In addition, an audio synchronization module can also be used to synchronously play the audio data of the participating users. In this way, the virtual display of each conference scene can synchronously display the meeting situation.

[0109] Therefore, the above-mentioned embodiment provides a complete virtual three-dimensional conference scene with real user three-dimensional image display, sound propagation and visual sharing of conference materials, and integration of virtual and real.

[0110] refer to Figure 5 A flowchart of a method for constructing a virtual three-dimensional conference scene is shown to illustrate the technical solution provided by the embodiment of the present application. The method is executed by an AR device. For any AR device, the method includes at least the following steps:

[0111] S501. If the user associated with the current AR device logs in to the conference system, the three-dimensional model data of other users except the associated user, as well as the material display location information and the current seating information set are received from the server.

[0112] S502: Determine the seating information of the associated user according to the seating information set at the current moment.

[0113] S503, receiving the action data of other users except the associated user among the participating users, the audio data of the participating users and the presentation material data of the conference speaker from the server; wherein a conference scene includes at least one action acquisition camera; the action data of each user is acquired by the action acquisition camera in the conference scene where the user is located and then sent to the server.

[0114] S504: Drive the corresponding three-dimensional model to move based on the received action data, display the display material data at the position represented by the material display position information, and play the audio data of the corresponding user.

[0115] The implementation process of the above steps can be found in the description of the aforementioned embodiment and will not be repeated here.

[0116] In an embodiment of the present application, if the associated user of the current AR device logs into the conference system, the AR device receives the three-dimensional model data of other users except the associated user from the server, as well as the material display position information and the current seating information set. In this way, the device can determine the seating information of the user according to the current seating information set to ensure that the seating information of each participating user is not repeated, thereby avoiding the phenomenon of overlapping characters in the virtual conference scene. In this way, the current AR device receives the action data of other users except its own associated users from the server, as well as the audio data of the participating users and the display material data of the conference speaker. The current AR device drives the corresponding three-dimensional model movement based on the action data, displays the display material data at the position represented by the material display position information, and plays the audio data of the corresponding user. The current AR device is any AR device, so the whole process has a sense of reality and immersion, and the conference scene is real, and the action information of the participating users obtained by the application is used to drive the movement of the corresponding user's three-dimensional model, improve the sense of presence and the ability to interact with the real world.

[0117] In order to make the technical solution of this application clearer, Figure 6 A signaling flow chart of a method for constructing a virtual three-dimensional conference scene is shown, wherein:

[0118] S601. The server detects whether there is a user logging into the user conference system.

[0119] S602. If it is detected that the current user logs into the conference system, for each logged-in user, the 3D model data of the logged-in user is sent to the AR devices of other participating users except the logged-in user, and the material display location information and the current seating information set are sent to each AR device.

[0120] S603: For each AR device, the AR device determines the seating information of its associated user according to the seating information set at the current moment.

[0121] S604: For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, the server receives the action data of the corresponding user acquired by the action acquisition camera, as well as the audio data of the participating users and the presentation material data of the conference speaker.

[0122] S605. For the action data of any user, the server sends the action data to the AR devices of other conference participants except the user, and sends the audio data of the conference participants and the presentation material data of the conference speaker to each AR device.

[0123] S606. Each AR device drives the corresponding three-dimensional model to move based on the action data, displays the display material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0124] The embodiments of the present application can solve the problem that traditional conference systems can only conduct conferences through video or audio and lack a sense of presence and immersion. It can also solve the problem that conventional virtual conference systems can only be conducted in completely virtual scenes and cannot interact with the real world.

[0125] like Figure 7 As shown, based on the same inventive concept, an embodiment of the present invention provides a device for constructing a virtual three-dimensional conference scene. The device is integrated in a server and includes a data sending module 71, a first data receiving module 72 and a first scene construction module 73.

[0126] The data sending module 71 is used to send the 3D model data of the current user to the AR devices of other users except the current user among the conference users, and send the material display location information and the current seating information set to each AR device of the conference users if it is detected that the current user logs into the conference system, so that each AR device determines the seating information of the corresponding user according to the current seating information set;

[0127] The first data receiving module 72 is used for receiving the motion data of the corresponding user collected by the motion collection camera after determining the seating information of the user corresponding to the motion collection camera for any motion collection camera; and receiving the audio data of the participating users and the presentation material data of the conference speaker; wherein a conference scene includes at least one motion collection camera;

[0128] The first scene construction module 73 is used to send the action data of any user to the AR devices of other users except one user among the participating users, and send the audio data of the participating users and the presentation material data of the conference speaker to the various AR devices of the participating users, so that each AR device drives the corresponding three-dimensional model to move based on the received action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0129] In some exemplary embodiments, a 3D model data determination module is further included, which is used to, before sending the action data to the AR devices of other users except one of the conference participants and sending the audio data of the conference participants and the presentation material data of the conference speaker to the AR devices of the conference participants:

[0130] If historical 3D model data of a logged-in user is pre-stored, determining the historical 3D model data as the 3D model data of the logged-in user, or prompting the logged-in user whether to apply the historical 3D model data, and determining the historical 3D model data as the 3D model data of the logged-in user based on a received message indicating the application of the historical 3D model data; or

[0131] If there is no historical three-dimensional model data of the logged-in user, the three-dimensional model data of the logged-in user is generated according to the acquired image or video information.

[0132] In some exemplary embodiments, a seating information sending module is further included, which is used to detect that after the current user logs into the conference system, before receiving the motion data of the corresponding user collected by the motion collection camera:

[0133] The user's seating information is sent to a motion capture camera in a conference scene where the user is located, so that the motion capture camera adjusts the shooting direction according to the seating information.

[0134] In some exemplary embodiments, the spatial deployment information of each conference scene is the same, wherein the spatial deployment information includes the spatial size of the conference scene, the size and position of the conference table, and the size and position of the conference chairs;

[0135] It also includes a position relationship determination module, which is used before detecting that the current user logs into the conference system:

[0136] Determine that the position correspondence between each conference chair and conference table is consistent with the pre-stored position relationship;

[0137] If they are inconsistent, a position adjustment message is sent; wherein the position adjustment message is used to adjust the position of each conference chair.

[0138] like Figure 8 As shown, based on the same inventive concept, an embodiment of the present invention provides a device for constructing a virtual three-dimensional conference scene, which is integrated in an AR device. The device includes a second data receiving module 81, a seating information determination module 82, a third data receiving module 83 and a second scene construction module 84.

[0139] The second data receiving module 81 is used to receive the three-dimensional model data of other users except the associated user from the server, as well as the material display location information and the current seating information set when the associated user of the current AR device logs in to the conference system;

[0140] A seating information determination module 82, configured to determine the seating information of the associated user according to the seating information set at the current moment;

[0141] The third data receiving module 83 is used to receive the action data of other users except the associated user among the conference participants from the server, the audio data of the conference participants and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera; the action data of each user is acquired by the action acquisition camera in the conference scene where the user is located and sent to the server;

[0142] The second scene construction module 84 is used to drive the corresponding three-dimensional model to move based on the received action data, display the display material data at the position represented by the material display position information, and play the audio data of the corresponding user.

[0143] In some exemplary embodiments, the seating information of the associated user is determined by scanning, by the AR device associated with the associated user, a desktop marker on a conference table within a preset range after the associated user sits down; the seating information set at the current moment is the seating status of each conference chair at the current moment;

[0144] The seating information determination module 82 is specifically used for:

[0145] Matching the initial seating information of the associated user with the seating information set at the current moment, determining whether the seat represented by the initial seating information is in an idle state, and if so, determining that the initial seating information is the seating information of the associated user;

[0146] Otherwise, determine the seat identification that is in an idle state, and prompt the associated user according to the seat identification.

[0147] Since the device is the device in the method in the embodiment of the present invention, and the principle of solving the problem by the device is similar to that of the method, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0148] like Fig. 9 As shown, based on the same inventive concept, a server for constructing a virtual three-dimensional conference scene is provided, the server comprising a processor 91, a memory 92 and at least one external communication interface 93, the processor 91, the memory 92 and the external communication interface 93 are all connected via a bus 94;

[0149] The external communication interface 93 is configured to receive a user's login instruction;

[0150] The memory 92 stores a computer program, and the processor 91 is configured to perform the following operations based on the computer program:

[0151] If it is detected that the current user logs into the conference system, the 3D model data of the current user is sent to the AR devices of other users among the conference participants except the current user, and the material display location information and the current seating information set are sent to each AR device of the conference participants, so that each AR device determines the seating information of the corresponding user according to the current seating information set;

[0152] For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, receive the action data of the corresponding user acquired by the action acquisition camera; and receive the audio data of the participating users and the presentation material data of the conference speaker; wherein a conference scene includes at least one action acquisition camera;

[0153] For the action data of any user, the action data is sent to the AR devices of other users except one user among the participants, and the audio data of the participating users and the presentation material data of the conference speaker are sent to the AR devices of each participating user, so that each AR device drives the corresponding three-dimensional model to move based on the received action data, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

[0154] In some exemplary embodiments, before sending the action data to the AR devices of other users except one of the conference users and sending the audio data of the conference users and the presentation material data of the conference speaker to the AR devices of the conference users, the processor 91 is further configured to execute:

[0155] If historical 3D model data of a logged-in user is pre-stored, determining the historical 3D model data as the 3D model data of the logged-in user, or prompting the logged-in user whether to apply the historical 3D model data, and determining the historical 3D model data as the 3D model data of the logged-in user based on a received message indicating the application of the historical 3D model data; or

[0156] If there is no historical three-dimensional model data of the logged-in user, the three-dimensional model data of the logged-in user is generated according to the acquired image or video information.

[0157] In some exemplary embodiments, after detecting that the current user logs into the conference system and before receiving the motion data of the corresponding user collected by the motion collection camera, the processor 91 is further configured to execute:

[0158] The seating information of the corresponding user is sent to the motion acquisition camera in the conference scene where the user is located, so that the motion acquisition camera adjusts the shooting direction according to the seating information.

[0159] In some exemplary embodiments, the spatial deployment information of each conference scene is the same, wherein the spatial deployment information includes the spatial size of the conference scene, the size and position of the conference table, and the size and position of the conference chairs;

[0160] Before detecting that a user logs into the conference system, the processor 91 is further configured to execute:

[0161] Determine that the position correspondence between each conference chair and conference table is consistent with the pre-stored position relationship;

[0162] If they are inconsistent, a position adjustment message is sent; wherein the position adjustment message is used to adjust the position of each conference chair.

[0163] like Fig.10 As shown, based on the same inventive concept, an AR device is provided, the AR device includes a processor 101, a memory 102 and at least one external communication interface 103, the processor 101, the memory 102 and the external communication interface 103 are all connected through a bus 104;

[0164] The external communication interface 103 is configured to receive a notification message of a user's successful login;

[0165] The memory 102 stores a computer program, and the processor is configured to perform the following operations based on the computer program:

[0166] If the associated user of the current AR device logs in to the conference system, the 3D model data of other users except the associated user, as well as the material display location information and the current seating information set are received from the server;

[0167] Determine the seating information of the associated user according to the seating information set at the current moment;

[0168] Receive the action data of other users except the associated user among the conference participants, the audio data of the conference participants and the presentation material data of the conference speaker from the server; wherein a conference scene includes at least one action acquisition camera; the action data of each user is acquired by the action acquisition camera in the conference scene where the user is located and sent to the server;

[0169] Based on the received motion data, the corresponding three-dimensional model is driven to move, the display material data is displayed at the position represented by the material display position information, and the audio data of the corresponding user is played.

[0170] In some exemplary embodiments, the seating information of the associated user is determined by scanning, by the AR device associated with the associated user, a desktop marker on a conference table within a preset range after the associated user sits down; the seating information set at the current moment is the seating status of each conference chair at the current moment;

[0171] The processor 101 is configured to execute:

[0172] Determine the seat information of the associated user according to the seat information set at the current moment, including:

[0173] Matching the initial seating information of the associated user with the seating information set at the current moment, determining whether the seat represented by the initial seating information is in an idle state, and if so, determining that the initial seating information is the seating information of the associated user;

[0174] Otherwise, determine the seat identification that is in an idle state, and prompt the associated user according to the seat identification.

[0175] The embodiment of the present invention further provides a computer storage medium, in which computer program instructions are stored. When the instructions are executed on a computer, the computer executes the steps of the method for constructing a virtual three-dimensional conference scene.

[0176] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0177] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0178] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0179] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0180] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for constructing a virtual three-dimensional conference scene, characterized in that: include: If it is detected that the current user logs into the conference system, the 3D model data of the current user is sent to the AR devices of other users among the conference participants except the current user, and the material display location information and the current seating information set are sent to each AR device of the conference participant, so that each AR device determines the seating information of the corresponding user according to the current seating information set; For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, the action data of the corresponding user's limbs acquired by the action acquisition camera is received; and the audio data of the participating users and the presentation material data of the conference speaker are received; wherein a conference scene includes at least one action acquisition camera; the action data of the user's limbs includes upper body data and / or lower body data; For the motion data of any user's limbs, the motion data is sent to the AR devices of other participating users except the one user, and the audio data of the participating user and the presentation material data of the conference speaker are sent to the respective AR devices of the participating users, so that each AR device determines the motion part indicated by the motion data according to the received motion data, drives the corresponding part of the corresponding three-dimensional model to move, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

2. The method according to claim 1, characterized in that Before sending the action data to AR devices of other users among the conference participants except the one user, and sending the audio data of the conference participants and the presentation material data of the conference speaker to the AR devices of the conference participants, the method further includes: If historical 3D model data of a logged-in user is pre-stored, determining that the historical 3D model data is the 3D model data of the logged-in user, or prompting the logged-in user whether to apply the historical 3D model data, and determining that the historical 3D model data is the 3D model data of the logged-in user based on a received message indicating the application of the historical 3D model data, or prompting the logged-in user whether to apply the historical 3D model data, and after receiving a message indicating a refusal to apply the historical 3D model data, generating the 3D model data of the logged-in user based on the acquired image or video information; or If there is no historical three-dimensional model data of the logged-in user, the three-dimensional model data of the logged-in user is generated according to the acquired image or video information.

3. The method according to claim 1, characterized in that After detecting that the current user logs into the conference system and before receiving the motion data of the corresponding user's limbs collected by the motion collection camera, the method further includes: The seating information of the corresponding user is sent to a motion acquisition camera in a conference scene where the user is located, so that the motion acquisition camera adjusts the shooting direction according to the seating information.

4. The method according to any one of claims 1 to 3, characterized in that: The spatial deployment information of each conference scene is the same, wherein the spatial deployment information includes the spatial size of the conference scene, the size and position of the conference table, and the size and position of the conference chairs; Before detecting that the current user logs into the conference system, the method further includes: Determine that the position correspondence between each conference chair and conference table is consistent with the pre-stored position relationship; If they are inconsistent, a position adjustment message is sent; wherein the position adjustment message is used to adjust the position of each conference chair.

5. A method for constructing a virtual three-dimensional conference scene, characterized in that: include: If the associated user of the current AR device logs in to the conference system, the 3D model data of other users except the associated user, as well as the material display location information and the current seating information set are received from the server; Determine the seating information of the associated user according to the seating information set at the current moment; Receive from the server the motion data of the limbs of other users participating in the conference except the associated user, the audio data of the participating users, and the presentation material data of the conference speaker; wherein a conference scene includes at least one motion acquisition camera; the motion data of each user's limb is acquired by the motion acquisition camera in the conference scene where the user is located and then sent to the server; the motion data of the user's limbs includes upper body data and / or lower body data; Determining, according to the received action data, an action part indicated by the action data; Driving corresponding parts of the corresponding three-dimensional model to move; The display material data is displayed at the position represented by the material display position information, and the audio data corresponding to the user is played.

6. The method according to claim 5, characterized in that The seating information of the associated user is determined by scanning a desktop marker on a conference table within a preset range after the associated user is seated by the current AR device; the seating information set at the current moment is the seating status of each conference chair at the current moment; The determining the seating information of the associated user according to the seating information set at the current moment includes: Matching the initial seating information of the associated user with the seating information set at the current moment, determining whether the seat represented by the initial seating information is in an idle state, and if so, determining that the initial seating information is the seating information of the associated user; Otherwise, determine the seat identification that is in an idle state, and prompt the associated user according to the seat identification.

7. A server for constructing a virtual three-dimensional conference scene, characterized in that: It includes a processor, a memory and at least one external communication interface, wherein the processor, the memory and the external communication interface are all connected via a bus; The external communication interface is configured to receive a login instruction from a user; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: If it is detected that the current user logs into the conference system, the 3D model data of the current user is sent to the AR devices of other users among the conference participants except the current user, and the material display location information and the current seating information set are sent to each AR device of the conference participants, so that each AR device determines the seating information of the corresponding user according to the current seating information set; For any action acquisition camera, after determining the seating information of the user corresponding to the action acquisition camera, the action data of the corresponding user's limbs acquired by the action acquisition camera is received; and the audio data of the participating users and the presentation material data of the conference speaker are received; wherein a conference scene includes at least one action acquisition camera; the action data of the user's limbs includes upper body data and / or lower body data; For the motion data of any user's limbs, the motion data is sent to the AR devices of other participating users except the one user, and the audio data of the participating user and the presentation material data of the conference speaker are sent to the respective AR devices of the participating users, so that each AR device determines the motion part indicated by the motion data according to the received motion data, drives the corresponding part of the corresponding three-dimensional model to move, displays the presentation material data at the position represented by the material display position information, and plays the audio data of the corresponding user.

8. The server according to claim 7, characterized in that: Before sending the action data to the AR devices of other users among the conference participants except the one user, and sending the audio data of the conference participants and the presentation material data of the conference speaker to the AR devices of the conference participants, the processor is further configured to execute: If historical 3D model data of a logged-in user is pre-stored, determining that the historical 3D model data is the 3D model data of the logged-in user, or prompting the logged-in user whether to apply the historical 3D model data, and determining that the historical 3D model data is the 3D model data of the logged-in user based on a received message indicating the application of the historical 3D model data; or If there is no historical three-dimensional model data of the logged-in user, the three-dimensional model data of the logged-in user is generated according to the acquired image or video information.

9. The server according to claim 7, characterized in that: After detecting that the current user logs into the conference system and before receiving the motion data of the corresponding user's limbs collected by the motion collection camera, the processor is further configured to execute: The seating information of the corresponding user is sent to a motion acquisition camera in a conference scene where the user is located, so that the motion acquisition camera adjusts the shooting direction according to the seating information.

10. An AR device for constructing a virtual three-dimensional conference scene, characterized in that: It includes a processor, a memory and at least one external communication interface, wherein the processor, the memory and the external communication interface are all connected via a bus; The external communication interface is configured to receive a notification message of successful login of a user associated with the current AR device; The memory stores a computer program, and the processor is configured to perform the following operations based on the computer program: If the associated user of the current AR device logs in to the conference system, the 3D model data of other users except the associated user, as well as the material display location information and the current seating information set are received from the server; Determine the seating information of the associated user according to the seating information set at the current moment; Receive from the server the motion data of the limbs of other users participating in the conference except the associated user, the audio data of the participating users, and the presentation material data of the conference speaker; wherein a conference scene includes at least one motion acquisition camera; the motion data of each user's limb is acquired by the motion acquisition camera in the conference scene where the user is located and then sent to the server; the motion data of the user's limbs includes upper body data and / or lower body data; Determining, according to the received action data, an action part indicated by the action data; Driving corresponding parts of the corresponding three-dimensional model to move; The display material data is displayed at the position represented by the material display position information, and the audio data corresponding to the user is played.

Images