Digital human hosting method and system based on large model and storage medium

Abstract

The application provides a digital human hosting method and system based on a large model and a storage medium. The method comprises the following steps: controlling a digital human to perform a hosting operation according to a hosting task; obtaining perception interaction data for the hosting operation based on a multi-modal large model, performing cognitive reasoning on the perception interaction data based on a vertical field large model to obtain interaction reasoning data; performing intelligent decision-making on the interaction reasoning data based on an intelligent agent to obtain an execution task chain, and controlling the digital human to perform a hosting business operation according to the execution task chain. According to the embodiment of the application, the perception interaction data for the hosting operation can be effectively obtained, the cognitive reasoning on the perception interaction data can be effectively performed, the interaction reasoning data can be effectively obtained, the intelligent agent can perform intelligent decision-making on the interaction reasoning data, the execution task chain can be automatically generated, the digital human can be effectively controlled to perform the hosting business operation based on the execution task chain, the interactivity between the digital human and the user is improved, and the use experience of the user is improved.

Description

Technical Field

[0001] This invention relates to the field of digital human technology, and in particular to a digital human hosting method, system and storage medium based on a large model. Background Technology

[0002] With the continuous development of virtual simulation, computer graphics rendering, and intelligent voice technology, digital human technology is maturing and is gradually being applied to various scenarios such as event broadcasting, online programs, and ceremony hosting. Relying on human posture simulation, facial expression-driven technology, and real-time speech synthesis, digital humans can achieve human-like visual presentation and speech expression. Digital human hosting, with its flexibility, efficiency, and strong adaptability, is gradually becoming an important form of audiovisual communication and scene display.

[0003] In existing digital human hosting scenarios, the digital human is typically controlled by a pre-prepared script, resulting in poor interaction with the user and a reduced user experience. Summary of the Invention

[0004] The purpose of this invention is to provide a digital human hosting method, system, and storage medium based on a large model, so as to solve the problem of poor interaction between digital humans and users in the prior art.

[0005] The present invention is implemented as follows: a digital human hosting method based on a large model, the method comprising:

[0006] Obtain the hosting task, and control the digital human to perform hosting operations according to the hosting task;

[0007] Based on a multimodal large model, perceptual interaction data for the hosting operation is obtained, and based on a vertical domain large model, cognitive reasoning is performed on the perceptual interaction data to obtain interaction reasoning data.

[0008] Based on the intelligent agent's intelligent decision-making on the interactive reasoning data, an execution task chain is obtained, and the digital human is controlled to perform hosting business operations according to the execution task chain.

[0009] Preferably, the perceptual interaction data for the hosting operation is obtained based on a multimodal large model, including:

[0010] Acquire voice interaction data, visual interaction data, gesture interaction data, and document interaction data for the hosting operation, and perform speech recognition on the voice interaction data according to the multimodal large model to obtain voice command text;

[0011] Visual interaction data is subjected to visual recognition to obtain visual instruction text, and gesture interaction data is subjected to gesture recognition to obtain gesture instruction text;

[0012] The perceptual interaction data is obtained by combining the voice command text, the visual command text, the gesture command text, and the document interaction data.

[0013] Preferably, cognitive reasoning is performed on the perceived interaction data based on a large model of the vertical domain to obtain interaction reasoning data, including:

[0014] The perceived interaction data is retrieved and enhanced to obtain enhanced interaction data, and the enhanced interaction data is input into the vertical domain big model for intent recognition to obtain the interaction intent;

[0015] The interaction relationships in the perceived interaction data are determined based on a preset knowledge graph, and the reasoning logic is determined based on the interaction intent.

[0016] Based on the interaction intent, the reasoning logic, and the interaction relationship, correlation reasoning is performed to obtain correlation reasoning data. Then, the correlation reasoning data is extended according to the intent extension rules to obtain the interaction reasoning data.

[0017] Preferably, the intelligent agent makes intelligent decisions based on the interactive reasoning data to obtain an execution task chain, including:

[0018] The intelligent agent is controlled to determine the decision target based on the interactive reasoning data, and the decision target is marked with basis, process and confidence level;

[0019] Candidate task chains are determined based on the interaction requirement types in the interaction reasoning data and the decision objective, and the task importance of the candidate task chains in the decision objective is obtained.

[0020] Based on the importance of the tasks, key task nodes are determined, and the candidate task chains are marked according to the key task nodes to obtain the execution task chain.

[0021] Preferably, controlling the digital human to perform hosting business operations according to the execution task chain includes:

[0022] The execution task chain is decomposed to obtain sub-task chains, and the display control data and business execution data corresponding to the sub-task chains are obtained.

[0023] The control data to be executed is determined based on the current display status of the digital human and the display control data, and the business execution instruction is determined based on the business execution data;

[0024] The digital human is controlled to change its form according to the control data to be executed, and business operations are performed according to the business execution instructions.

[0025] Preferably, the perceived interaction data is retrieved and enhanced to obtain enhanced interaction data, including:

[0026] Keyword recognition is performed on the voice command text, the visual command text, the gesture command text, and the document interaction data to obtain voice keywords, visual keywords, gesture keywords, and document keywords, and a first search index is generated based on the voice keywords, the visual keywords, the gesture keywords, and the document keywords;

[0027] The voice keywords, visual keywords, gesture keywords, and document keywords are combined to obtain combined keywords, and a second search index is generated based on the combined keywords.

[0028] Entity recognition is performed on the voice command text, the visual command text, the gesture command text, and the document interaction data to obtain text entities, and a third retrieval index is generated based on the text entities;

[0029] Data retrieval is performed based on the first retrieval index, the second retrieval index, and the third retrieval index to obtain interactive retrieval data. The interactive retrieval data and the perceived interactive data are then combined to obtain the interactive enhancement data.

[0030] Preferably, obtaining the task importance of the candidate task chain in the decision objective includes:

[0031] Obtain the task identifier of the candidate task chain, and determine the execution importance based on the task identifier;

[0032] Different task identifiers are combined to obtain a combined identifier, and the correlation importance of the corresponding candidate task chain is determined based on the combined identifier;

[0033] The importance of the task is determined based on the execution importance and the correlation importance.

[0034] Another objective of this invention is to provide a digital human hosting system based on a large model, the system comprising:

[0035] The hosting control module is used to acquire hosting tasks and control the digital human to perform hosting operations according to the hosting tasks.

[0036] The cognitive reasoning module is used to acquire perceptual interaction data for the hosting operation based on a multimodal large model, and to perform cognitive reasoning on the perceptual interaction data based on a vertical domain large model to obtain interaction reasoning data.

[0037] The business execution module is used to make intelligent decisions based on the interactive reasoning data of the intelligent agent, obtain the execution task chain, and control the digital human to perform hosting business operations according to the execution task chain.

[0038] Preferably, the cognitive reasoning module is further used for:

[0039] Acquire voice interaction data, visual interaction data, gesture interaction data, and document interaction data for the hosting operation, and perform speech recognition on the voice interaction data according to the multimodal large model to obtain voice command text;

[0040] Visual interaction data is subjected to visual recognition to obtain visual instruction text, and gesture interaction data is subjected to gesture recognition to obtain gesture instruction text;

[0041] The perceptual interaction data is obtained by combining the voice command text, the visual command text, the gesture command text, and the document interaction data.

[0042] In this embodiment of the invention, a multimodal large model can effectively acquire perceptual interaction data for hosting operations. A vertical domain large model can perform cognitive reasoning on the perceptual interaction data to effectively obtain interaction reasoning data. An intelligent agent can make intelligent decisions on the interaction reasoning data to automatically generate an execution task chain. Based on the execution task chain, the digital human can be effectively controlled to perform hosting business operations, thereby improving the interactivity between the digital human and the user during the hosting process and enhancing the user experience. Attached Figure Description

[0043] Figure 1 This is a flowchart of the digital human hosting method based on a large model provided in the first embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the structure of the digital human hosting system based on a large model provided in the second embodiment of the present invention;

[0045] Figure 3 This is a schematic diagram of the implementation framework of the digital human hosting system based on a large model provided in the second embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of the structure of the terminal device provided in the third embodiment of the present invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] To illustrate the technical solution described in this invention, specific embodiments are described below.

[0049] Example 1

[0050] Please see Figure 1 This is a flowchart of a large-model-based digital human hosting method provided in the first embodiment of the present invention. This large-model-based digital human hosting method can be applied to any device or system. The large-model-based digital human hosting method includes the following steps:

[0051] Step S10: Obtain the hosting task, and control the digital human to perform hosting operations according to the hosting task;

[0052] Specifically, host control data and host display data are determined based on the task chain in the host task, and the digital human is controlled to perform host operations based on the host control data and host display data.

[0053] Step S20: Obtain perceptual interaction data for the hosting operation based on the multimodal large model, and perform cognitive reasoning on the perceptual interaction data based on the vertical domain large model to obtain interaction reasoning data.

[0054] Among them, by acquiring perceptual interaction data for hosting operations through multimodal large models, it is possible to effectively extract user feedback information during the digital human hosting process. By performing cognitive reasoning on perceptual interaction data through vertical domain large models, the effect of accurate semantic analysis of perceptual interaction data can be achieved, so as to identify the user's interaction needs.

[0055] Optionally, perceptual interaction data for the hosting operation can be obtained based on a multimodal large model, including:

[0056] The system acquires voice interaction data, visual interaction data, gesture interaction data, and document interaction data related to the hosting operation, and performs speech recognition on the voice interaction data based on the multimodal large model to obtain voice command text. Specifically, by collecting user voice information, visual information, gesture information, and text input information, the system obtains voice interaction data, visual interaction data, gesture interaction data, and document interaction data. By performing speech recognition on the voice interaction data, the system can effectively convert the voice interaction data into text-formatted voice command text.

[0057] Visual interaction data is subjected to visual recognition to obtain visual instruction text; gesture interaction data is subjected to gesture recognition to obtain gesture instruction text; and the voice instruction text, visual instruction text, gesture instruction text, and document interaction data are combined to obtain the perceptual interaction data.

[0058] Furthermore, cognitive reasoning is performed on the perceived interaction data based on a large model of the vertical domain to obtain interaction reasoning data, including:

[0059] The perceived interaction data is retrieved and enhanced to obtain enhanced interaction data, which is then input into the vertical domain big model for intent recognition to obtain the interaction intent. In this process, by retrieving and enhancing the perceived interaction data, data associated with the perceived interaction data can be effectively obtained, and by inputting the enhanced interaction data into the vertical domain big model for intent recognition, the user's interaction intent can be obtained.

[0060] The interaction relationships in the perceived interaction data are determined according to a preset knowledge graph, and the reasoning logic is determined according to the interaction intent. The preset knowledge graph can be set according to needs. The interaction relationships are obtained by matching the keywords in the perceived interaction data with the preset knowledge graph. An interaction relationship matrix is ​​constructed based on the interaction relationship. The interaction relationship matrix is ​​vectorized to obtain the interaction relationship vector. The similarity between the interaction relationship vector and the preset logic vector in the logic database is calculated. The preset logic vector with the maximum similarity is determined as the target logic vector. The logic information corresponding to the target logic vector is determined as the reasoning logic corresponding to the interaction intent.

[0061] Based on the interaction intent, the reasoning logic, and the interaction relationship, association reasoning is performed to obtain association reasoning data. Then, the association reasoning data is extended according to the intent extension rule to obtain the interaction reasoning data. In this process, the interaction intent, reasoning logic, and interaction relationship are matched with the reasoning database to obtain association reasoning data. The intent extension rule can be set according to requirements. For example, the intent extension rule can be set to replace keywords, perform semantic similarity retrieval, etc.

[0062] Furthermore, the perceived interaction data is retrieved and enhanced to obtain enhanced interaction data, including:

[0063] Keyword recognition is performed on the voice command text, the visual command text, the gesture command text, and the document interaction data to obtain voice keywords, visual keywords, gesture keywords, and document keywords, and a first search index is generated based on the voice keywords, the visual keywords, the gesture keywords, and the document keywords;

[0064] The voice keywords, visual keywords, gesture keywords, and document keywords are combined to obtain combined keywords, and a second search index is generated based on the combined keywords; entity recognition is performed on the voice command text, visual command text, gesture command text, and document interaction data to obtain text entities, and a third search index is generated based on the text entities;

[0065] Data retrieval is performed based on the first retrieval index, the second retrieval index, and the third retrieval index to obtain interactive retrieval data. The interactive retrieval data and the perceived interactive data are then combined to obtain the interactive enhancement data.

[0066] Step S30: Based on the intelligent agent's intelligent decision-making on the interactive reasoning data, an execution task chain is obtained, and the digital human is controlled to perform hosting business operations according to the execution task chain;

[0067] Among them, by making intelligent decisions based on interactive reasoning data, intelligent agents can automatically generate execution task chains. Based on the execution task chains, the digital human can be effectively controlled to perform hosting business operations, thereby improving the interactivity between the digital human and the user during the hosting process.

[0068] Before making intelligent decisions based on the interactive reasoning data, this step further includes: inputting sample reasoning data into the intelligent agent to make intelligent decisions, obtaining sample task chains, and training the intelligent agent based on the standard task chain and the sample task chain until the intelligent agent converges.

[0069] Optionally, based on the intelligent agent's intelligent decision-making on the interactive reasoning data, an execution task chain is obtained, including:

[0070] The intelligent agent is controlled to determine the decision target based on the interactive reasoning data, and the decision target is marked with basis, process and confidence level; wherein, by controlling the intelligent agent to perform target analysis on the interactive reasoning data to obtain the decision target, and by marking the basis, process and confidence level of the decision target, the traceability of the decision target generation process is effectively facilitated;

[0071] Candidate task chains are determined based on the interaction requirement types in the interaction reasoning data and the decision objective. The task importance of each candidate task chain in the decision objective is obtained. Key task nodes are determined based on the task importance, and the candidate task chains are marked based on the key task nodes to obtain the execution task chain. In this process, the interaction requirement types and decision objectives are matched with the task chain database to obtain candidate task chains. By obtaining the task importance of each candidate task chain in the decision objective, the determination of subsequent key task nodes is effectively facilitated.

[0072] Preferably, obtaining the task importance of the candidate task chain in the decision objective includes:

[0073] Obtain the task identifier of the candidate task chain and determine the execution importance based on the task identifier; wherein, the task identifier is matched with an importance lookup table to obtain the execution importance, and the importance lookup table stores the correspondence between different task identifiers and their corresponding execution importance;

[0074] Different task identifiers are combined to obtain a combined identifier. The association importance of the corresponding candidate task chain is determined based on the combined identifier. The task importance is determined based on the execution importance and the association importance. The combined identifier is matched with an association lookup table to obtain the association importance.

[0075] Furthermore, controlling the digital human to perform hosting business operations according to the execution task chain includes:

[0076] The execution task chain is decomposed into sub-task chains, and the display control data and business execution data corresponding to the sub-task chains are obtained. Specifically, the execution task chain is decomposed into sub-task chains based on the task nodes of the execution task chain. The chain identifier of the sub-task chain is matched with the local database to obtain display control data and business execution data. The display control data is used to control the display changes of the digital human, and the business execution data is used to control the operation of the business operator.

[0077] Based on the current display state of the digital human and the display control data, control data to be executed is determined. Based on the business execution data, business execution instructions are determined. The digital human is then controlled to undergo shape changes based on the control data to be executed, and business operations are performed based on the business execution instructions. Specifically, combined data is obtained by combining the first and second digits of the current display state and the display control data. This combined data is then matched with a change database to obtain target change data. Finally, the target change data is combined with the display control data to obtain control data to be executed. Business operations are then performed through the business execution instructions, enabling the digital human to effectively perform business operations based on interaction with the user when hosting.

[0078] In this embodiment, a multimodal large model can effectively acquire perceptual interaction data for hosting operations. A vertical domain large model can perform cognitive reasoning on the perceptual interaction data to effectively obtain interaction reasoning data. An intelligent agent can make intelligent decisions on the interaction reasoning data to automatically generate execution task chains. Based on the execution task chains, the digital human can be effectively controlled to perform hosting business operations, which improves the interactivity between the digital human and the user during the hosting process and enhances the user experience.

[0079] Example 2

[0080] Please see Figure 2 This is a schematic diagram of the structure of a large-model-based digital human hosting system 100 provided in the second embodiment of the present invention, including:

[0081] The hosting control module 10 is used to acquire hosting tasks and control the digital human to perform hosting operations according to the hosting tasks.

[0082] The cognitive reasoning module 11 is used to acquire perceptual interaction data for the host operation based on a multimodal large model, and to perform cognitive reasoning on the perceptual interaction data based on a vertical domain large model to obtain interaction reasoning data.

[0083] Optionally, the cognitive reasoning module 11 is further configured to: acquire voice interaction data, visual interaction data, gesture interaction data and document interaction data for the hosting operation, and perform voice recognition on the voice interaction data according to the multimodal large model to obtain voice command text;

[0084] Visual interaction data is subjected to visual recognition to obtain visual instruction text, and gesture interaction data is subjected to gesture recognition to obtain gesture instruction text;

[0085] The perceptual interaction data is obtained by combining the voice command text, the visual command text, the gesture command text, and the document interaction data.

[0086] Furthermore, the cognitive reasoning module 11 is also used to: retrieve and enhance the perceptual interaction data to obtain enhanced interaction data, and input the enhanced interaction data into the vertical domain big model for intent recognition to obtain the interaction intent;

[0087] The interaction relationships in the perceived interaction data are determined based on a preset knowledge graph, and the reasoning logic is determined based on the interaction intent.

[0088] Based on the interaction intent, the reasoning logic, and the interaction relationship, correlation reasoning is performed to obtain correlation reasoning data. Then, the correlation reasoning data is extended according to the intent extension rules to obtain the interaction reasoning data.

[0089] Furthermore, the cognitive reasoning module 11 is also used to: perform keyword recognition on the voice command text, the visual command text, the gesture command text and the document interaction data to obtain voice keywords, visual keywords, gesture keywords and document keywords, and generate a first search index based on the voice keywords, the visual keywords, the gesture keywords and the document keywords;

[0090] The voice keywords, visual keywords, gesture keywords, and document keywords are combined to obtain combined keywords, and a second search index is generated based on the combined keywords.

[0091] Entity recognition is performed on the voice command text, the visual command text, the gesture command text, and the document interaction data to obtain text entities, and a third retrieval index is generated based on the text entities;

[0092] Data retrieval is performed based on the first retrieval index, the second retrieval index, and the third retrieval index to obtain interactive retrieval data. The interactive retrieval data and the perceived interactive data are then combined to obtain the interactive enhancement data.

[0093] The business execution module 12 is used to make intelligent decisions based on the interactive reasoning data of the intelligent agent, obtain the execution task chain, and control the digital human to perform hosting business operations according to the execution task chain.

[0094] Optionally, the business execution module 12 is further configured to: control the intelligent agent to determine the decision target based on the interactive reasoning data, and mark the decision target with basis, process and confidence level;

[0095] Candidate task chains are determined based on the interaction requirement types in the interaction reasoning data and the decision objective, and the task importance of the candidate task chains in the decision objective is obtained.

[0096] Based on the importance of the tasks, key task nodes are determined, and the candidate task chains are marked according to the key task nodes to obtain the execution task chain.

[0097] Furthermore, the business execution module 12 is also used to: decompose the execution task chain to obtain sub-task chains, and obtain the display control data and business execution data corresponding to the sub-task chains;

[0098] The control data to be executed is determined based on the current display status of the digital human and the display control data, and the business execution instruction is determined based on the business execution data;

[0099] The digital human is controlled to change its form according to the control data to be executed, and business operations are performed according to the business execution instructions.

[0100] Furthermore, the business execution module 12 is also used to: obtain the task identifier of the candidate task chain, and determine the execution importance based on the task identifier;

[0101] Different task identifiers are combined to obtain a combined identifier, and the correlation importance of the corresponding candidate task chain is determined based on the combined identifier;

[0102] The importance of the task is determined based on the execution importance and the correlation importance.

[0103] Please see Figure 3 This embodiment effectively constructs a five-layer architecture: "Perception and Interaction Layer - Cognitive Reasoning Layer - Intelligent Agent Decision-Making Layer - Digital Human Execution Layer - Memory Evolution Layer".

[0104] By using a multimodal large-scale model to perceive and collect real-time commands (voice input commands, visual / gesture commands, document input commands), and combining a vertical domain large-scale model with a knowledge graph, the Agent Planning Engine performs precise semantic analysis, autonomously breaking down complex tasks into executable sub-task chains. Dynamic planning and anomaly self-correction are performed based on an intelligent agent planning engine. A generative digital human engine drives a highly realistic avatar, which then controls the business system through tools such as MCP / Skill. Based on a digital life memory network, intergenerational experience transfer and multi-agent collaborative evolution are achieved, creating a truly capable digital employee / host with complete abilities to "understand, think, do, and remember." In this embodiment, by upgrading the large-scale model to an intelligent agent hub, and using Tool Use to operate the business system, the digital human transforms from "eloquent" to "capable of writing and doing." The multimodal large-scale model introduces multi-agent collaboration and hierarchical memory mechanisms, enabling intergenerational experience transfer and team collaboration, and constructing a new organizational paradigm of human-machine symbiosis.

[0105] In this embodiment, a multimodal large model can effectively acquire perceptual interaction data for hosting operations. A vertical domain large model can perform cognitive reasoning on the perceptual interaction data to effectively obtain interaction reasoning data. An intelligent agent can make intelligent decisions on the interaction reasoning data to automatically generate execution task chains. Based on the execution task chains, the digital human can be effectively controlled to perform hosting business operations, which improves the interactivity between the digital human and the user during the hosting process and enhances the user experience.

[0106] Example 3

[0107] Figure 4 This is a structural block diagram of a terminal device 2 provided in the third embodiment of this application. For example... Figure 4 As shown, the terminal device 2 in this embodiment includes a processor 20, a memory 21, and a computer program 22 stored in the memory 21 and executable on the processor 20, such as a program for a large-model-based digital human hosting method. When the processor 20 executes the computer program 22, it implements the steps in the various embodiments of the large-model-based digital human hosting method described above.

[0108] For example, the computer program 22 may be divided into one or more modules, which are stored in the memory 21 and executed by the processor 20 to complete this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 22 in the terminal device 2. The terminal device may include, but is not limited to, the processor 20 and the memory 21.

[0109] The processor 20 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0110] The memory 21 can be an internal storage unit of the terminal device 2, such as a hard drive or memory of the terminal device 2. The memory 21 can also be an external storage device of the terminal device 2, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the terminal device 2. Furthermore, the memory 21 can include both internal and external storage units of the terminal device 2. The memory 21 is used to store the computer program and other programs and data required by the terminal device. The memory 21 can also be used to temporarily store data that has been output or will be output.

[0111] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0112] If an integrated module is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. This computer-readable storage medium can be non-volatile or volatile. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of a computer-readable storage medium may be appropriately added to or subtracted from the contents as required by the legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, a computer-readable storage medium may not include electrical carrier signals and telecommunication signals.

[0113] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A digital human hosting method based on a large model, characterized in that, The method includes: Obtain the hosting task, and control the digital human to perform hosting operations according to the hosting task; Based on a multimodal large model, perceptual interaction data for the hosting operation is obtained, and based on a vertical domain large model, cognitive reasoning is performed on the perceptual interaction data to obtain interaction reasoning data. Based on the intelligent agent's intelligent decision-making on the interactive reasoning data, an execution task chain is obtained, and the digital human is controlled to perform hosting business operations according to the execution task chain.

2. The digital human hosting method based on a large model as described in claim 1, characterized in that, Based on a multimodal large model, perceptual interaction data for the hosting operation is obtained, including: Acquire voice interaction data, visual interaction data, gesture interaction data, and document interaction data for the hosting operation, and perform speech recognition on the voice interaction data according to the multimodal large model to obtain voice command text; Visual interaction data is subjected to visual recognition to obtain visual instruction text, and gesture interaction data is subjected to gesture recognition to obtain gesture instruction text; The perceptual interaction data is obtained by combining the voice command text, the visual command text, the gesture command text, and the document interaction data.

3. The digital human hosting method based on a large model as described in claim 2, characterized in that, Cognitive reasoning is performed on the perceived interaction data based on a large model in a vertical domain to obtain interaction reasoning data, including: The perceived interaction data is retrieved and enhanced to obtain enhanced interaction data, and the enhanced interaction data is input into the vertical domain big model for intent recognition to obtain the interaction intent; The interaction relationships in the perceived interaction data are determined based on a preset knowledge graph, and the reasoning logic is determined based on the interaction intent. Based on the interaction intent, the reasoning logic, and the interaction relationship, correlation reasoning is performed to obtain correlation reasoning data. Then, the correlation reasoning data is extended according to the intent extension rules to obtain the interaction reasoning data.

4. The digital human hosting method based on a large model as described in claim 1, characterized in that, Based on the intelligent agent's intelligent decision-making on the interactive reasoning data, an execution task chain is obtained, including: The intelligent agent is controlled to determine the decision target based on the interactive reasoning data, and the decision target is marked with basis, process and confidence level; Candidate task chains are determined based on the interaction requirement types in the interaction reasoning data and the decision objective, and the task importance of the candidate task chains in the decision objective is obtained. Based on the importance of the tasks, key task nodes are determined, and the candidate task chains are marked according to the key task nodes to obtain the execution task chain.

5. The digital human hosting method based on a large model as described in claim 1, characterized in that, Controlling the digital human to perform hosting business operations according to the execution task chain includes: The execution task chain is decomposed to obtain sub-task chains, and the display control data and business execution data corresponding to the sub-task chains are obtained. The control data to be executed is determined based on the current display status of the digital human and the display control data, and the business execution instruction is determined based on the business execution data; The digital human is controlled to change its form according to the control data to be executed, and business operations are performed according to the business execution instructions.

6. The digital human hosting method based on a large model as described in claim 3, characterized in that, The perceived interaction data is retrieved and enhanced to obtain enhanced interaction data, including: Keyword recognition is performed on the voice command text, the visual command text, the gesture command text, and the document interaction data to obtain voice keywords, visual keywords, gesture keywords, and document keywords, and a first search index is generated based on the voice keywords, the visual keywords, the gesture keywords, and the document keywords; The voice keywords, visual keywords, gesture keywords, and document keywords are combined to obtain combined keywords, and a second search index is generated based on the combined keywords. Entity recognition is performed on the voice command text, the visual command text, the gesture command text, and the document interaction data to obtain text entities, and a third retrieval index is generated based on the text entities; Data retrieval is performed based on the first retrieval index, the second retrieval index, and the third retrieval index to obtain interactive retrieval data. The interactive retrieval data and the perceived interactive data are then combined to obtain the interactive enhancement data.

7. The digital human hosting method based on a large model as described in claim 4, characterized in that, Obtaining the task importance of the candidate task chain in the decision objective includes: Obtain the task identifier of the candidate task chain, and determine the execution importance based on the task identifier; Different task identifiers are combined to obtain a combined identifier, and the correlation importance of the corresponding candidate task chain is determined based on the combined identifier; The importance of the task is determined based on the execution importance and the correlation importance.

8. A digital human hosting system based on a large model, characterized in that, The system includes: The hosting control module is used to acquire hosting tasks and control the digital human to perform hosting operations according to the hosting tasks. The cognitive reasoning module is used to acquire perceptual interaction data for the hosting operation based on a multimodal large model, and to perform cognitive reasoning on the perceptual interaction data based on a vertical domain large model to obtain interaction reasoning data. The business execution module is used to make intelligent decisions based on the interactive reasoning data of the intelligent agent, obtain the execution task chain, and control the digital human to perform hosting business operations according to the execution task chain.

9. The digital human hosting system based on a large model as described in claim 8, characterized in that, The cognitive reasoning module is also used for: Acquire voice interaction data, visual interaction data, gesture interaction data, and document interaction data for the hosting operation, and perform speech recognition on the voice interaction data according to the multimodal large model to obtain voice command text; Visual interaction data is subjected to visual recognition to obtain visual instruction text, and gesture interaction data is subjected to gesture recognition to obtain gesture instruction text; The perceptual interaction data is obtained by combining the voice command text, the visual command text, the gesture command text, and the document interaction data.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images