Multi-terminal linkage meta-universe digital human interaction system and method

Abstract

The invention relates to the field of man-machine interaction, and provides a multi-terminal linkage meta-universe digital human interaction system and method, and the method comprises the steps: receiving user voice, and converting the user voice into a text instruction; receiving a text instruction from the voice interaction terminal, obtaining a control instruction according to the text instruction, and distributing the control instruction to the display terminal; receiving a control instruction from the collaborative server, and displaying a three-dimensional image of the digital human according to the control instruction; and receiving a control instruction from the collaborative server, displaying the meta universe content according to the control instruction, independently receiving voice control, and performing response according to the voice control. According to the invention, the defect of single interactive experience of a single terminal in the prior art is overcome, and a multi-terminal collaborative interactive display form is realized.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a multi-terminal linked metaverse digital human interaction system and method. Background Art

[0002] With the rise of the concept of the metaverse, digital human technology has become a bridge between the real and virtual worlds. Digital human technology integrates a number of advanced technologies such as high-precision 3D modeling, natural language processing, emotion recognition and real-time rendering, enabling digital humans to simulate and replace human behavior and interaction in a virtual environment. This technology is not only widely used in the fields of entertainment and games, but also shows great potential in many industries such as exhibitions, education, and customer service.

[0003] At present, digital human technology can generate realistic human appearance and bone structure through high-precision 3D modeling, ensuring that the appearance and movements of digital humans in virtual environments are similar to those of real people. At the same time, the use of natural language processing (NLP) technology enables digital humans to understand and generate natural language, supporting voice and text interaction with users. In addition, by analyzing voice, facial expressions and body language, emotion recognition technology enables digital humans to express and respond to emotions, further enhancing the authenticity of interaction and user experience. The use of real-time rendering technology ensures the smoothness and visual effects of digital humans' movements, expressions and environmental interactions.

[0004] However, the current digital human interaction system is mainly limited to the interaction and experience on a single terminal, and fails to achieve linked interaction and display between multiple terminals, limiting the richness and breadth of the user experience. Summary of the invention

[0005] The present invention provides a multi-terminal linked metaverse digital human interaction system and method, which solves the defect of single terminal interaction experience in the prior art and realizes a multi-terminal collaborative interactive display form.

[0006] The present invention provides a multi-terminal linked metaverse digital human interaction system, comprising the following modules: The voice interaction terminal is used to receive user voice, convert the voice into text instructions and transmit them to the collaborative server; The collaborative server is used to generate a control instruction according to the text instruction and distribute the control instruction to the display end; the display end includes a digital human display end and a large screen display end; The digital human display terminal is used to display the three-dimensional image of the digital human according to the control instruction; The large-screen display terminal is used to display the metaverse content according to the control instructions, and independently receive voice control and respond according to the voice control.

[0007] According to a multi-terminal linked metaverse digital human interaction system provided by the present invention, the system also includes: a rendering server, which is used to receive control instructions from a collaborative server, process data from a digital human display terminal according to the control instructions, generate images and animations according to the processing results, and perform three-dimensional dynamic rendering and display of the digital human image in real time.

[0008] The present invention also provides a multi-terminal linked metaverse digital human interaction method, including the following method: Receiving user voice and converting the user voice into text instructions; Receiving a text instruction from a voice interaction terminal, acquiring a control instruction according to the text instruction and distributing the control instruction to a display terminal; receiving a control instruction from a collaborative server, and displaying a three-dimensional image of a digital human according to the control instruction; Receive control instructions from the collaborative server, display the metaverse content according to the control instructions, and independently receive voice control and respond according to the voice control.

[0009] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, the method also includes: receiving control instructions from a collaborative server, processing data from the digital human display terminal according to the control instructions, generating images and animations according to the processing results, and performing three-dimensional dynamic rendering and display of the digital human image in real time.

[0010] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, before receiving user voice, the method also includes: initializing various system components; turning on the collaborative server to enter a standby state to receive command requests at any time; turning on the rendering server to enter a standby state to receive rendering requests at any time; turning on the digital human display terminal and connecting to the rendering server to enter a ready state; turning on the large-screen display terminal and connecting to the collaborative server; turning on the voice interaction terminal and connecting to the collaborative server, completing all coordination links, and reaching a multi-terminal collaborative state.

[0011] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, the receiving of user voice and converting the user voice into text instructions specifically includes: receiving user voice, converting the voice into text instructions through a local voice-to-text service or an online conversion service, and then transmitting the text instructions to a collaborative server through network communication.

[0012] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, the independent reception of voice control and the response according to the voice control specifically include: receiving the user's voice control instructions, converting the voice control instructions into executable commands, and adjusting the display content, display effects or executing other operations requested by the user according to the commands.

[0013] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the multi-terminal linked metaverse digital human interaction method as described in any one of the above-mentioned methods is implemented.

[0014] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the multi-terminal linked metaverse digital human interaction method as described in any of the above.

[0015] The present invention also provides a computer program product, including a computer program, which, when executed by a processor, implements any of the multi-terminal linked metaverse digital human interaction methods described above.

[0016] The present invention provides a multi-terminal linkage metaverse digital human interaction system and method, including the following beneficial effects: through multi-terminal linkage, the present invention can take into account both individual and audience-oriented scenarios, and provide a rich and diverse interactive display form. Users can perform voice control through voice interaction terminals, making the interaction more natural and convenient, and improving the user experience. According to the control instructions of the collaborative server, the present invention can display the three-dimensional image of the digital human and the metaverse content, making the display content more diverse. The large-screen display terminal can independently receive voice control instructions and respond according to the instructions, which increases the flexibility and convenience of interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is one of the component structure diagrams of the multi-terminal linked metaverse digital human interaction system provided by the present invention.

[0019] Figure 2 It is a flow chart of the multi-terminal linked Metaverse digital human interaction method provided by the present invention.

[0020] Figure 3 This is the second component structure diagram of the multi-terminal linked metaverse digital human interaction system provided by the present invention.

[0021] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Digital human technology: Digital human technology integrates a number of advanced technologies, including high-precision 3D modeling, natural language processing, emotion recognition and real-time rendering. High-precision 3D modeling generates realistic human appearance and bone structure, ensuring that the appearance and movements of digital humans in virtual environments are similar to those of real people.

[0024] Through natural language processing (NLP), digital humans can understand and generate natural language, supporting voice and text interaction with users. By analyzing voice, facial expressions and body language, human emotional states can be identified and simulated. Emotion recognition technology enables digital humans to express and respond to emotions, enhancing the authenticity of interaction and user experience. Using graphics processing technology, high-quality digital human images and animations are generated in real time. Real-time rendering ensures the smoothness and visual effects of digital humans' movements, expressions and environmental interactions.

[0025] The combination of these technologies enables digital humans to simulate and replace human behaviors and interactions in various application scenarios, which not only improves the realism of virtual environments, but also brings innovation and change to various industries.

[0026] The current shortcomings are: the current voice digital human interaction is mainly the interaction and experience on a single terminal, and it is not to link multiple terminals for interaction and display; the display effect is single; voice interaction requires a position close to the interactive terminal for language dialogue, which is not convenient for the general public.

[0027] Combine the following Figure 1-Figure 4 Embodiments of the present invention are described in detail.

[0028] Figure 1 Schematic diagram of the composition of the multi-terminal linked Metaverse digital human interaction system provided by the present invention, such as Figure 1 As shown, the method includes the following modules: The voice interaction terminal 110 is used to receive user voice, convert the voice into text instructions and transmit them to the collaborative server; The collaborative server 120 is used to generate control instructions according to the text instructions and distribute the control instructions to the display terminals; the display terminals include the digital human display terminal and the large screen display terminal; The digital human display terminal 130 is used to display the three-dimensional image of the digital human according to the control instruction; The large-screen display terminal 140 is used to display the metaverse content according to the control instruction, and independently receive voice control and respond according to the voice control.

[0029] According to a multi-terminal linked metaverse digital human interaction system provided by the present invention, the system also includes: a rendering server 150, which is used to receive control instructions from a collaborative server, process data from a digital human display terminal according to the control instructions, generate images and animations according to the processing results, and perform three-dimensional dynamic rendering and display of the digital human image in real time.

[0030] Specifically, the voice interaction terminal: mainly serves as a medium for voice interaction between users and the system, and performs voice recognition and interaction through running applications; Digital human holographic display terminal: Displays the three-dimensional image of the digital human in a vivid form through special equipment; Large-screen display terminal: displays and presents rich metaverse content in a public manner and receives voice control and response; Collaborative server: supports data command synchronization of multiple terminals to ensure synchronization of multiple terminals; Rendering server: As a rendering capability provider for the digital human display end, it supports the three-dimensional dynamic real-time rendering display of the digital human image. It is not required when the digital human display has independent rendering capabilities.

[0031] The operation steps for the above-mentioned multi-terminal linkage Metaverse Digital Human Interaction System are as follows: 1) Turn on the collaborative server to enter standby mode and receive command requests from the voice input terminal at any time; 2) Turn on the rendering server and enter the standby state, ready to receive the request status from the digital human holographic display end at any time; 3) Open the large-screen display terminal and connect to the collaborative server; 4) Open the digital human holographic display terminal, connect to the rendering server, and enter the preparation state; 5) Open the voice interaction terminal, connect to the collaborative server, and connect to the digital human server; complete all coordination links; and reach a multi-terminal collaborative state; 6) Use the voice interaction terminal for voice interaction; 7) Use the large screen to display the operation terminal for operation; 8) Complete multi-terminal collaborative interaction and display.

[0032] The multi-terminal linked Metaverse digital human interaction method provided by the present invention is described below. The multi-terminal linked Metaverse digital human interaction method described below and the multi-terminal linked Metaverse digital human interaction system described above can be referenced to each other.

[0033] Figure 2: is a flow chart of the multi-terminal linked metaverse digital human interaction method provided by the present invention, such as Figure 2 As shown, the method comprises the following steps: S210: Receive user voice, and convert the user voice into text instructions.

[0034] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, before receiving user voice, the method also includes: initializing various system components; turning on the collaborative server to enter a standby state to receive command requests at any time; turning on the rendering server to enter a standby state to receive rendering requests at any time; turning on the digital human display terminal and connecting to the rendering server to enter a ready state; turning on the large-screen display terminal and connecting to the collaborative server; turning on the voice interaction terminal and connecting to the collaborative server, completing all coordination links, and reaching a multi-terminal collaborative state.

[0035] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, the receiving of user voice and converting the user voice into text instructions specifically includes: receiving user voice, converting the voice into text instructions through a local voice-to-text service or an online conversion service, and then transmitting the text instructions to a collaborative server through network communication.

[0036] Specifically, first, the system is initialized to start and initialize each component of the system to ensure that each component can work normally, including but not limited to the collaborative server, rendering server, digital human display terminal, large-screen display terminal and voice interaction terminal.

[0037] Turn on and connect each server and terminal. Turn on the collaborative server and enter the standby state so that it can receive and process command requests from each terminal at any time. At the same time, turn on the rendering server and enter the standby state, ready to receive rendering requests from the digital human display terminal at any time. Next, turn on the digital human display terminal and connect to the rendering server, enter the ready state, wait to receive rendering instructions and display the corresponding digital human image. Then, turn on the large-screen display terminal and connect to the collaborative server to display the interactive effects related to the digital human in the metaverse environment. Finally, turn on the voice interaction terminal and connect to the collaborative server to complete the coordinated link of all components so that the system can reach a multi-terminal collaborative state.

[0038] Receive and process user voice commands. When the user issues a voice command through the voice interaction terminal, the system starts to receive the user's voice. Then, the received voice is converted into text commands through the local voice-to-text service or the online conversion service. After the conversion is completed, the text command is transmitted to the collaborative server through network communication.

[0039] S220: Receive a text instruction from the voice interaction terminal, obtain a control instruction according to the text instruction, and distribute the control instruction to the display terminal.

[0040] Specifically, a stable communication connection is established to ensure that the data transmission between the voice interaction terminal and the collaborative server is correct. A monitoring module can be set in the collaborative server to monitor the text instructions from the voice interaction terminal. When the voice interaction terminal converts the user's voice into a text instruction, it is sent to the monitoring module of the collaborative server through the established communication connection.

[0041] After receiving the text instruction, the collaborative server performs necessary verification, such as instruction format checking and instruction integrity confirmation, to ensure that the received instruction is valid and executable, and then performs corresponding processing according to the instruction content, such as controlling the digital human's movements and expressions, or triggering certain events in the large-screen display terminal.

[0042] A command parsing module can be set up in the collaborative server to parse the text commands that have passed the verification. The command parsing module parses the text commands into control commands that can be recognized by the system according to the preset rules and algorithms. This process may involve technologies such as keyword matching and semantic analysis to ensure accurate understanding of user intent. After the parsing is completed, one or more specific control commands are generated, which will be directly used to control the behavior of the display end.

[0043] A command distribution module can be set up in the collaborative server to distribute the control commands generated by the analysis to the display end. The command distribution module first determines the target display end, which may be based on specific information in the text command, such as device ID, location information, etc., or according to the preset logic of the system.

[0044] Establish a communication connection between the collaborative server and the display terminal to ensure that the control command can be accurately transmitted to the display terminal. Send the control command to the target display terminal through the communication connection. After receiving the command, the display terminal performs corresponding operations according to the command content, such as displaying specific content, adjusting the device status, etc. In this way, the user's voice command can affect and control the performance and interaction of the digital human in the metaverse in real time.

[0045] Through the above steps, the present invention realizes a multi-terminal linked metaverse digital human interaction method, which enables users to control and influence the digital human image and interaction effect in the metaverse in real time through voice commands, thereby improving user experience and interactivity.

[0046] S230, receiving a control instruction from the collaborative server, and displaying a three-dimensional image of the digital human according to the control instruction.

[0047] Specifically, the digital human display terminal first establishes a stable communication connection with the collaborative server to ensure that it can receive control instructions from the collaborative server in real time. The communication connection can be a network connection based on the TCP / IP protocol or other communication protocols suitable for real-time data transmission.

[0048] The digital human display terminal continuously monitors the data message from the collaborative server through the established communication connection. When receiving the data message containing the control instruction, the digital human display terminal will immediately parse it to extract the specific content of the control instruction.

[0049] The digital human display terminal converts the received control instructions into a format that the digital human display terminal can recognize. During the parsing process, the instruction parsing module decodes and converts the control instructions according to the preset instruction set and protocol specifications to extract key information such as the digital human's movements, expressions, and postures. According to the parsed control instructions, the digital human display terminal will update the current state of the digital human, including movements, expressions, and postures. This process involves the playback of the digital human's skeletal animation, the rendering of facial expressions, and the adjustment of dynamic light and shadow effects. The digital human display terminal uses the built-in rendering engine to render the three-dimensional image of the digital human in real time according to the updated digital human state.

[0050] During the rendering process, the rendering engine will comprehensively consider factors such as the digital human's geometric model, texture mapping, lighting conditions, and camera viewing angle to generate realistic three-dimensional visual effects. After the rendering is completed, the digital human display terminal will display the generated three-dimensional image to the user or audience.

[0051] S240, receiving a control instruction from a collaborative server, displaying the metaverse content according to the control instruction, and independently receiving voice control, and responding according to the voice control.

[0052] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, the independent reception of voice control and the response according to the voice control specifically include: receiving the user's voice control instructions, converting the voice control instructions into executable commands, and adjusting the display content, display effects or executing other operations requested by the user according to the commands.

[0053] Specifically, the Metaverse linkage large screen first establishes a stable communication connection with the collaborative server to ensure that it can receive control instructions from the collaborative server in real time. When receiving the control instruction, the large screen will parse the instruction content and adjust the display content, display format or perform other related operations according to the instruction requirements. The Metaverse linkage large screen has a built-in microphone and voice recognition module, which can capture the voice instructions issued by the user in real time.

[0054] The voice recognition module converts the captured voice signal into text form for subsequent processing and response. The command processing module in the large screen is responsible for converting the text instructions output by the voice recognition module into executable commands. These commands include adjusting the order of displayed content, changing the display effect, zooming in / out displayed content, switching display scenes, etc. According to the converted commands, the large screen will adjust its display content, display effect or perform other operations requested by the user accordingly. After receiving and processing the control instructions and voice control instructions, the Metaverse linkage large screen will display the Metaverse content according to the user's requirements. The display content includes the three-dimensional image of the digital human, virtual scenes, interactive elements, etc. The large screen provides users with an immersive Metaverse experience through high-quality display technology and rendering effects.

[0055] According to a multi-terminal linked metaverse digital human interaction method provided by the present invention, control instructions from a collaborative server are received, data from a digital human display terminal is processed according to the control instructions, images and animations are generated according to the processing results, and three-dimensional dynamic rendering and display of the digital human image are performed in real time.

[0056] Specifically, in the process of receiving and processing voice control commands, the Metaverse Linked Large Screen will receive user feedback. Based on this feedback, the large screen can adjust the sensitivity and accuracy of its voice recognition module or optimize the logic of the command processing module to improve the user experience.

[0057] Although the Metaverse Interactive Large Screen has the ability to independently receive and process voice control commands, it still needs to coordinate with other terminals in the system (such as digital human display terminals, voice interaction terminals, etc.).

[0058] Through the collaborative server, the large screen can share data and synchronize status with other terminals to achieve a multi-terminal linked metaverse interactive experience.

[0059] Suppose a user stands in front of the Metaverse interactive screen and wants to change the displayed content through voice commands.

[0060] User says: "Show the action demonstration of the digital human."

[0061] The large screen captures the user's voice command and converts it into text through the voice recognition module: "Show the action demonstration of the digital human."

[0062] The instruction processing module converts the text instructions into executable commands and sends them to the display control module of the large screen. The display control module adjusts the display content according to the command and starts playing the action demonstration of the digital human. Users can experience the interactive experience of the metaverse by observing the display content on the large screen.

[0063] Through the above steps, the Metaverse linkage large screen can display the Metaverse content according to the control instructions of the collaborative server and the user's voice control instructions, and provide users with an immersive interactive experience. At the same time, the large screen also has the ability to work in coordination with other terminals to realize a multi-terminal linkage Metaverse digital human interactive system.

[0064] like Figure 3 The figure shows a digital human interaction system provided by an embodiment of the present invention, comprising: Voice input terminal: It faces the main interactive controller, receives user voice, and converts the voice into text instructions through local voice-to-text service or online conversion service, and transmits it to the voice and control service through text forwarding; the voice input terminal can be an interactive terminal such as a mobile phone, pad, smart glasses, etc. with functions such as microphone and network connection.

[0065] Voice and control services: After entering the receiving state, after receiving the text command from the voice input terminal, it is distributed to the digital human holographic display screen and the metaverse linkage large screen, so that the two can divide the work and cooperate in content; Digital Human Holographic Screen: It is used to display the appearance, voice and movements of the digital human to the main interactive personnel and audiences. It can be in various forms such as flat or three-dimensional holographic, and is mainly a data display terminal. Digital human rendering server: provides rendering capabilities for the digital human holographic screen. If the digital human holographic screen itself has rendering capabilities, this module can be omitted.

[0066] Metaverse linkage large screen: Facing the audience, after receiving the control command of the control service, the Metaverse content can be controlled and displayed, and its content can also be controlled independently; it can also be other display forms; it can be a visual terminal in the form of PC, digital screen, TV, etc.

[0067] The present invention has the following advantages: 1. It adopts a multi-terminal linkage model, taking into account scenarios for individuals and audiences, and provides a rich and diverse form of interactive display.

[0068] 2. It can not only facilitate voice interaction between interlocutors, but also provide a specific display effect for the audience, and at the same time reflect the specific image of the digital human.

[0069] 3. Solve the problem that voice interaction cannot be used directly at a distance on a large screen and there is no display prompt; provide a multi-terminal linkage display and interaction method with rich diversity. Provide a form in which a portable holographic terminal can display rich content.

[0070] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the multi-terminal linkage metaverse digital human interaction method, the method comprising: receiving user voice, converting the user voice into text instructions; receiving text instructions from a voice interaction terminal, obtaining control instructions according to the text instructions and distributing the control instructions to the display terminal; receiving control instructions from a collaborative server, and displaying the three-dimensional image of the digital human according to the control instructions; receiving control instructions from a collaborative server, and displaying the metaverse content according to the control instructions, and independently receiving voice control, and responding according to the voice control.

[0071] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0072] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-terminal linked metaverse digital human interaction method provided by the above methods, and the method includes: receiving user voice and converting the user voice into text instructions; receiving text instructions from a voice interaction terminal, obtaining control instructions according to the text instructions and distributing the control instructions to the display end; receiving control instructions from a collaborative server, and displaying the three-dimensional image of the digital human according to the control instructions; receiving control instructions from a collaborative server, and displaying the metaverse content according to the control instructions, and independently receiving voice control, and responding according to the voice control.

[0073] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, it is implemented to execute the multi-terminal linked metaverse digital human interaction method provided by the above-mentioned methods. The method includes: receiving user voice and converting the user voice into text instructions; receiving text instructions from a voice interaction terminal, obtaining control instructions according to the text instructions and distributing the control instructions to the display terminal; receiving control instructions from a collaborative server, displaying the three-dimensional image of the digital human according to the control instructions; receiving control instructions from a collaborative server, displaying the metaverse content according to the control instructions, and independently receiving voice control and responding according to the voice control.

[0074] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0075] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-terminal linked metaverse digital human interaction system, characterized in that: include: The voice interaction terminal is used to receive user voice, convert the voice into text instructions and transmit them to the collaborative server; The collaborative server is used to generate a control instruction according to the text instruction and distribute the control instruction to the display end; the display end includes a digital human display end and a large screen display end; The digital human display terminal is used to display the three-dimensional image of the digital human according to the control instruction; The large-screen display terminal is used to display the metaverse content according to the control instructions, and independently receive voice control and respond according to the voice control.

2. The multi-terminal linked Metaverse digital human interactive system according to claim 1, characterized in that: The system further comprises: The rendering server is used to receive control instructions from the collaborative server, process the data of the digital human display terminal according to the control instructions, generate images and animations according to the processing results, and perform three-dimensional dynamic rendering and display of the digital human image in real time.

3. A multi-terminal linked metaverse digital human interaction method, characterized in that: include: Receiving user voice and converting the user voice into text instructions; Receiving a text instruction from a voice interaction terminal, acquiring a control instruction according to the text instruction and distributing the control instruction to a display terminal; receiving a control instruction from a collaborative server, and displaying a three-dimensional image of a digital human according to the control instruction; Receive control instructions from the collaborative server, display the metaverse content according to the control instructions, and independently receive voice control and respond according to the voice control.

4. The multi-terminal linked metaverse digital human interaction method according to claim 3 is characterized in that: The method further comprises: Receive control instructions from the collaborative server, process the data of the digital human display terminal according to the control instructions, generate images and animations according to the processing results, and perform three-dimensional dynamic rendering and display of the digital human image in real time.

5. The multi-terminal linked metaverse digital human interaction method according to claim 3, characterized in that: Before receiving the user voice, the method further includes: Initialize system components; Turn on the collaborative server and put it into standby mode to receive command requests at any time; Turn on the rendering server and put it into standby mode to receive rendering requests at any time; Open the digital human display terminal and connect to the rendering server to enter the preparation state; Open the large-screen display terminal and connect to the collaborative server; Turn on the voice interaction terminal and connect to the collaboration server to complete all coordination links and achieve multi-terminal collaboration.

6. The multi-terminal linked metaverse digital human interaction method according to claim 3, characterized in that: The receiving user voice and converting the user voice into a text instruction specifically includes: Receive user voice, convert the voice into text instructions through local voice-to-text service or online conversion service, and then transmit the text instructions to the collaborative server through network communication.

7. The multi-terminal linked metaverse digital human interaction method according to claim 3, characterized in that: The independently receiving voice control and responding according to the voice control specifically includes: Receive the user's voice control instructions, convert the voice control instructions into executable commands, and adjust the display content, display effects or perform other operations requested by the user according to the commands.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the multi-terminal linked metaverse digital human interaction method as described in any one of claims 3 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, the multi-terminal linked metaverse digital human interaction method as described in any one of claims 3 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by the processor, the multi-terminal linked metaverse digital human interaction method as described in any one of claims 3 to 7 is implemented.

Images