A Semantic Preservation Oriented Virtual Conference Method and 3D Virtual Conference System

Abstract

The present invention relates to a virtual conference method and a three-dimensional virtual conference system oriented to semantic fidelity. The method includes: according to the semantic concept tree semantic structure; for each semantic concept of different levels in the semantic tree structure, setting a corresponding default Signal; extract the semantic description of the scene where the participants are in from the multimodal signal and update the private level semantic library; use the private level semantic library to update the public level semantic library; the server sends all the information in the public level semantic library to all clients The hierarchical semantic description composed of content, updates the corresponding private hierarchical semantic library on each client according to the hierarchical semantic description composed of all content; generates the virtual conference scene from the perspective of the participants; generates the virtual conference scene from the perspective of the participants during the meeting Virtual meeting scene. The method of the invention solves the problems of rough expressions and movements of virtual images, unreal meeting scenes, and difficult interaction between people and scenes, increases the scale of virtual meetings, and enhances the realism of meeting participants.

Description

technical field [0001] The invention belongs to the technical field of virtual conferences, and in particular relates to a semantic-fidelity-oriented virtual conference method and a three-dimensional virtual conference system. Background technique [0002] Affected by economic globalization and the COVID-19 pandemic, remote conferencing technology has been widely used in government, schools, companies and other scenarios, and has gradually replaced face-to-face conversations as the mainstream media of meetings. At present, the commonly used teleconferencing systems mainly include teleconferencing systems, video conferencing systems, and multimedia platform sharing systems. However, compared with face-to-face conversations, the existing teleconferencing systems have the problems of lack of realism, immersion, and communication means for participants. With the advancement of augmented display and virtual reality technology, virtual meeting technology emerges as the times requ...

AI Technical Summary

Problems solved by technology

There are two problems with this solution: 1) In order to make the human visual system feel real, the virtual reality video needs to reach 21K resolution 60 frames per second

At present, the theoretical bandwidth of 5G is only 10Gbps, which cannot meet the needs of normal meetings.

2) The existing algorithms for generating 3D models based on high-definition images are time-consuming and labor-intensive, and existing servers cannot meet the real-time computing power requirements of large-scale conferences

However, the preset virtual image cannot reflect important features such as the appearance and demeanor of the participants, resulting in a lack of realism in the meeting, and even causing resentment among the participants due to the uncanny valley effect

[0005] Therefore, there are two problems in the first solution: 1) In order to make the human visual system feel real, the virtual reality video needs to reach 21K resolution 60 frames per second

At present, the theoretical bandwidth of 5G is only 10Gbps, which cannot meet the needs of normal meetings.

2) The existing algorithms for generating 3D models based on high-definition images are time-consuming and labor-intensive, and existing servers cannot meet the real-time computing power requirements of large-scale conferences

In the second solution, the preset virtual image cannot reflect important features such as the appearance and demeanor of the participants, resulting in a lack of realism in the meeting, and even the problem of disgust among the participants due to the uncanny valley effect

[0007] Therefore, the shortcomings of existing solutions all stem from the failure to achieve semantic fidelity, that is, the failure to achieve semantic characterization, signal semantic extraction, transmission and reproduction, and user intention detection feedback.

Images