Growth virtual digital human method, system and product

Abstract

The invention discloses a growing virtual digital human method, system and product, and relates to the field of virtual digital. The method for growing the virtual digital human is realized based on a Dify framework, and comprises the following steps: acquiring user question data; generating a text data packet based on the user question data; a workflow engine is adopted, a workflow instance is generated and stored on the basis of the text data packet, intention recognition is conducted on the basis of the workflow instance in combination with service logic and service content stored in the workflow engine, and a recognition result is obtained; and generating virtual digital human appearance data and behavior data based on the identification result, and outputting the identification result based on the virtual digital human appearance data and behavior data. According to the invention, complex business processes can be supported, and end-to-end automation is realized.

Description

Technical Field

[0001] The present application relates to the field of virtual digital beings, and in particular to a method, system and product for growing virtual digital beings. Background Art

[0002] With the rapid development of information technology, digital transformation has become an important trend in all walks of life. In particular, workflow engine technology has been widely used in enterprise management and business process automation. Workflow engines help companies improve efficiency, reduce errors and optimize resource utilization by defining and executing business processes. At the same time, virtual digital human technology has also made significant progress in recent years. By combining technologies such as artificial intelligence, natural language processing and computer vision, virtual digital humans can provide intelligent services in a variety of scenarios, such as customer service, education and training, and entertainment interaction.

[0003] The complexity and diversity of business processes within an enterprise make it difficult for traditional manual operations to meet the requirements of efficiency and accuracy. Workflow engines allow users to define complex business processes and automatically execute them through graphical interfaces and programming interfaces. However, existing workflow engines still lack flexibility and adaptability, especially when faced with rapidly changing business needs, which often require a lot of manual adjustments and reconfigurations.

[0004] On the other hand, the development of virtual digital human technology provides new possibilities for business process automation. Virtual digital humans can not only simulate human appearance and behavior, but also interact with users through natural language processing technology to provide personalized services. However, existing virtual digital human systems are mostly limited to specific scenarios, lack support for complex business processes, and cannot achieve end-to-end automation. Summary of the invention

[0005] The purpose of this application is to provide a growable virtual digital human method, system and product that can support complex business processes and achieve end-to-end automation.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] In a first aspect, the present application provides a growable virtual digital human method, which is implemented based on the Dify framework and includes:

[0008] Get user question data;

[0009] generating a text data packet based on the user question data;

[0010] Using a workflow engine, generating and storing a workflow instance based on the text data packet, and performing intent recognition based on the workflow instance in combination with the business logic and business content stored in the workflow engine to obtain a recognition result;

[0011] Generate virtual digital human appearance data and behavior data based on the recognition result, and output the recognition result based on the virtual digital human appearance data and behavior data.

[0012] Optionally, generating a text data packet based on the user question data includes:

[0013] When the user question data is voice data, an automatic speech recognition model is used to convert the voice data into text content to generate the text data packet;

[0014] When the user question data is text data, the shielding word library stored in the workflow engine is used to shield the set words in the text data to generate the text data packet.

[0015] Optionally, converting the speech data into text data using an automatic speech recognition model includes:

[0016] The speech data is preprocessed; the preprocessing includes noise reduction processing and standardization processing.

[0017] Extracting feature vectors of the preprocessed speech data, and inputting the feature vectors into the automatic speech recognition model to obtain a text sequence;

[0018] The text data is obtained by performing spelling check and grammar correction on the text sequence.

[0019] Optionally, when the user question data is voice data, the duration of the voice data is not longer than a set duration; when the user question data is text data, the number of characters in the text data is not more than a set number of characters.

[0020] Optionally, performing intent recognition based on the workflow instance in combination with the business logic and business content stored inside the workflow engine to obtain a recognition result includes:

[0021] Using a large language model to identify the user intent based on the workflow instance, and obtaining an initial intent recognition result;

[0022] Classify the user questions according to the initial intent recognition result to obtain a user question classification result;

[0023] Determine a processing path based on the user problem classification result;

[0024] Based on the processing path, the business logic and business content corresponding to the processing path stored in the workflow engine are retrieved to obtain the recognition result.

[0025] In the second aspect, the present application provides a growable virtual digital human system, including: a user interaction layer, a virtual digital human layer, a workflow engine layer, a data management layer and a technical support layer;

[0026] The user interaction layer is used to obtain user question data and interact with the virtual digital human layer based on the user question data;

[0027] The workflow engine layer is connected to the user interaction layer; the workflow engine layer is used to generate a text data packet based on the user question data, and generate and store a workflow instance based on the text data packet, and perform intent recognition based on the workflow instance combined with the business logic and business content stored inside the workflow engine to obtain a recognition result;

[0028] The virtual digital human layer is used to generate virtual digital human appearance data and behavior data based on the recognition result, and output the recognition result based on the virtual digital human appearance data and behavior data;

[0029] The data management layer is connected to the user interaction layer and the workflow engine layer; the data management layer is used to store and manage user data, business data and system configuration data; the user data includes user problem data and user information data;

[0030] The technical support layer is connected to the workflow engine layer; the technical support layer is used to provide model support; the model includes one or more of a large language model, an automatic speech recognition model, a MuseTalk model and a Wav2Lip model.

[0031] Optionally, the user interaction layer includes a user interaction interface.

[0032] Optionally, the workflow engine layer includes:

[0033] An intention recognition module, used to use a large language model to recognize user intentions based on the workflow instance, obtain an initial intention recognition result, and classify user questions according to the initial intention recognition result to obtain a user question classification result;

[0034] A classification and routing module, connected to the intention recognition module, for determining a processing path based on the classification result of the user question;

[0035] The business logic or service module is connected to the classification and routing module and is used to retrieve the business logic and business content corresponding to the processing path stored inside the workflow engine based on the processing path to obtain the recognition result.

[0036] In a third aspect, the present application provides a computer program product, including an application program embedded with a workflow engine, which, when executed by a smart device, implements the above-mentioned growable virtual digital human method.

[0037] Optionally, the smart device is one or more of a desktop computer, a laptop computer, a smart phone, a tablet computer, an Internet of Things device, and a portable wearable device.

[0038] According to the specific embodiments provided in this application, this application has the following technical effects:

[0039] The present application provides a method, system and product for a growable virtual digital human. By adopting a workflow engine and combining the business logic and business content stored in the workflow engine to identify the intention of user questions, it can support virtual digital human Q&A for complex business processes. In addition, the workflow engine can automatically execute tasks according to the business process definition, and has dynamic task scheduling and exception handling capabilities to ensure the correctness and efficiency of the business logic, thereby achieving end-to-end automation. In addition, through the workflow instance obtained through real-time storage, the data stored in the workflow engine can be updated and learned in real time, thereby achieving the purpose of growth. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 A flowchart of a method for creating a growable virtual digital human provided in one embodiment of the present application;

[0042] Figure 2 A schematic diagram of a processing flow of a growable virtual digital human system provided in one embodiment of the present application;

[0043] Figure 3 A schematic diagram of a virtual digital human rendering result provided by another embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0046] In some embodiments, the growable virtual digital human method can be implemented independently by a server or a terminal. For example, the terminal can directly create a growable virtual digital human for the video to be processed, or the server can obtain the user questions to be processed from the data storage system and perform intent recognition on the user questions to be processed.

[0047] The terminals may be, but are not limited to, various desktop computers, laptops, smart phones, tablet computers, IoT devices and portable wearable devices. IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers, or a cloud server.

[0048] The Dify framework achieves a minimalist design in the construction of intelligent entities. Through the combination of rapid knowledge base construction and existing digital human technologies (such as MetaHuman components), the realization of virtual digital humans is more flexible and efficient. The Dify framework integrates a variety of advanced AI models to achieve real-time interactive functions of digital humans. The following are the applications and advantages of the Dify framework in various key technical fields:

[0049] (1) TTS model (Text-to-Speech):

[0050] Function: Convert text into natural and fluent speech, enabling virtual digital humans to "speak".

[0051] Advantages: The Dify framework supports a variety of high-quality TTS models, such as Google Text-to-Speech, Microsoft Azure TTS, etc., ensuring that the generated speech is highly natural and clear.

[0052] (2) ASR model (Automatic Speech Recognition):

[0053] Function: Convert the user's voice input into text so that the digital human can "understand" the user's instructions.

[0054] Advantages: The Dify framework integrates advanced ASR models, such as Google Speech-to-Text and Microsoft Azure ASR, which can accurately recognize various accents and dialects and improve the accuracy of speech recognition.

[0055] (3)MuseTalk Model:

[0056] Function: Generate natural and smooth digital human videos so that the digital human’s speech content is consistent with the lip shape.

[0057] Advantages:

[0058] (a) High-precision lip sync.

[0059] (b) Accurate audio-lip alignment: Through deep learning technology, the MuseTalk model can accurately align audio and lip animation to ensure that the digital human's lip movements are completely consistent with the speech content. Multi-language support: Supports lip synchronization in multiple languages, suitable for application needs in different language environments.

[0060] (c)Highly customizable.

[0061] (d) Multimodal fusion: supports fusion with other modalities (such as facial expressions, gestures, etc.) to make the digital human's performance richer and more natural.

[0062] (e) High efficiency performance:

[0063] (e1) Real-time generation: The MuseTalk model can generate high-quality dialogue content and lip animation in a short time, supports real-time interaction, and is suitable for online services and real-time application scenarios.

[0064] (e2) Low latency: Optimized algorithms and high-performance computing capabilities ensure low latency during the generation process and improve user experience.

[0065] (e3) Ease of use:

[0066] Cross-platform support: supports multiple operating systems and hardware platforms to ensure stable operation on different devices.

[0067] (4)Wav2Lip model.

[0068] Function: Generate corresponding lip animation according to audio to make the digital human's facial expression more natural.

[0069] Advantages: The Wav2Lip model uses deep learning technology to accurately synchronize audio and lip animation, so that the digital human's lip movements perfectly match the voice content, enhancing the user's immersion and interactive experience.

[0070] Based on the above description, in an exemplary embodiment, the present application provides a method for growing virtual digital human, which is executed by a computer device, specifically, a terminal or a server, or a terminal and a server. In the present embodiment, the method is applied to a terminal as an example. The method for growing virtual digital human is implemented based on the Dify framework, such as Figure 1 As shown, the method includes:

[0071] Step 100: Obtain user question data.

[0072] Step 101: Generate a text data package based on user question data.

[0073] Step 102: Using a workflow engine, a workflow instance is generated and stored based on the text data packet, and intent recognition is performed based on the workflow instance combined with the business logic and business content stored inside the workflow engine to obtain a recognition result.

[0074] Step 103: Generate virtual digital human appearance data and behavior data based on the recognition result, and output the recognition result based on the virtual digital human appearance data and behavior data.

[0075] In another exemplary embodiment of the present application, for different types of user question data, the implementation process of the above step 101 includes:

[0076] 11) When the user question data is voice data, an automatic speech recognition model is used to convert the voice data into text content to generate a text data packet.

[0077] 12) When the user question data is text data, the shielding word library stored in the workflow engine is used to shield the set words in the text data to generate a text data packet. The automatic speech recognition model is used to convert the speech data into text data, including:

[0078] (1) Preprocess the speech data. Preprocessing includes noise reduction and standardization. Third-party speech recognition and synthesis services such as Google Cloud Speech-to-Text and Text-to-Speech can be used. These services provide high-precision speech processing capabilities.

[0079] (2) Extract the feature vector of the preprocessed speech data and input the feature vector into the automatic speech recognition model to obtain a text sequence.

[0080] (3) Perform spelling check and grammar correction on the text sequence to obtain text data.

[0081] Further, when the user question data is voice data, the duration of the voice data is not longer than a set duration (eg, 30 seconds). When the user question data is text data, the number of characters in the text data is not more than a set number of characters (eg, 256 characters).

[0082] In another exemplary embodiment of the present application, after the text content converted by ASR or directly input by the user is submitted to the workflow engine as a data packet, in the above step 102 of the present application, intent recognition is performed based on the workflow instance combined with the business logic and business content stored in the workflow engine to obtain a recognition result, including:

[0083] 21) Use a large language model to identify user intent based on workflow instances and obtain initial intent recognition results.

[0084] 22) Classify user questions according to the initial intent recognition results to obtain user question classification results.

[0085] 23) Determine the processing path based on the user problem classification results.

[0086] 24) Based on the processing path, the business logic and business content corresponding to the processing path stored in the workflow engine are retrieved to obtain the recognition result.

[0087] The growable virtual digital human method provided in the present application collects user feedback and learns new dialogue patterns during operation, thereby optimizing its own performance and service quality, achieving self-evolution and adapting to application requirements in different scenarios, so as to continuously improve and ensure the provision of the best user experience. For example:

[0088] A. Knowledge question and answer route: Based on the questions asked, vectorized search is performed in the knowledge base. After the search content is obtained, the large language model is submitted. The large language model will complete the thinking and output the reply content based on the prompt words and the contextual content of the knowledge base (that is, the one set in the workflow engine to store business logic and business content). After the reply content is generated, the virtual digital human will receive the reply content and broadcast it.

[0089] B. Weather forecast route: After entering the weather forecast route, the date, address and other content in the question are extracted through parameter extraction. For example, "What will the weather be like in Beijing tomorrow?" The "address" and "time" are extracted, and then a weather query is performed. The obtained weather information is handed over to the large language model to complete the thinking and output the reply content, and then the reply content is broadcast through the virtual digital human.

[0090] Based on the above description, this application introduces a workflow engine that can iteratively and dynamically add processing route nodes, such as weather nodes, translation nodes, Feishu / DingTalk nodes, crawler nodes, interface request nodes, etc., which can all be integrated. The ability of a workflow will bring together the excellent capabilities of various large language models with a task and complete it excellently.

[0091] Based on the same inventive concept, the embodiment of the present application also provides a growable virtual digital human system for implementing the growable virtual digital human method involved above. The implementation scheme for solving the problem provided by the system is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more growable virtual digital human system embodiments provided below can refer to the limitations of the growable virtual digital human method above, and will not be repeated here.

[0092] In an exemplary embodiment, a growable virtual digital human system is provided, including: a user interaction layer, a virtual digital human layer, a workflow engine layer, a data management layer and a technical support layer.

[0093] The user interaction layer is used to obtain user question data and interact with the virtual digital human layer based on the user question data. Based on this setting, users can interact with the virtual digital human through client applications (such as web browsers, mobile applications, etc.).

[0094] The workflow engine layer is connected to the user interaction layer. The workflow engine layer is used to generate text data packets based on user question data, generate and store workflow instances based on text data packets, and perform intent recognition based on workflow instances combined with business logic and business content stored inside the workflow engine to obtain recognition results. The workflow engine layer is mainly responsible for managing and executing business processes to ensure the correctness and efficiency of business logic.

[0095] The virtual digital human layer is used to generate the virtual digital human appearance data and behavior data based on the recognition results, and output the recognition results based on the virtual digital human appearance data and behavior data. Among them, the virtual digital human layer is mainly responsible for generating and displaying the appearance and behavior of the virtual digital human, including facial expressions, body movements, etc. Figure 3 Shown is a rendering example of a virtual digital human.

[0096] The data management layer is connected to the user interaction layer and the workflow engine layer. The data management layer is used to store and manage user data, business data, and system configuration data. User data includes user problem data and user information data.

[0097] The technical support layer is connected to the workflow engine layer. The technical support layer is used to provide necessary technical support and services, such as large language models, automatic speech recognition models, MuseTalk models, and Wav2Lip models.

[0098] As an optional implementation, the user interaction layer includes a user interaction interface. Based on this, the user interaction layer provides a graphical user interface and supports microphone voice transmission, keyboard text and other input methods to display the response of the virtual digital human. Users can interact with the virtual digital human through voice, text and other methods. After completing the input, the digital human will perform video interaction, voice broadcast, gesture interaction, etc. according to the input content. The specific logic is:

[0099] The user inputs voice or text through a microphone or keyboard, and the ASR model converts the user's voice into text. Then, the text is submitted to the workflow engine to obtain the answer to the question. The TTS model then converts the answer text into voice. The voice is synthesized with the video to complete the broadcast of the virtual digital human. The whole process is streaming output. The working principle of this logic is as follows Figure 2 shown. Figure 2 In , WEBRTC is a set of technologies that support real-time voice or video conversations in web browsers. It is a project developed and open sourced by Google, which aims to enable developers to build high-quality communication applications without relying on plug-ins or additional downloads. Edge-tts is a text-to-speech service provided by Microsoft, which allows developers to convert natural language text into high-quality synthesized speech. This technology is widely used in various scenarios, such as smart assistants, audiobooks, navigation systems, accessibility features, etc. to enhance user experience and accessibility. GPT-SOVITS is a model that combines GPT (Generative Pre-trained Transformer) and SOVITS (Slice-based VITS, Variational Inference for Text-to-Speech) technologies. This combination aims to take advantage of the advantages of both to achieve more natural, fluent and expressive text-to-speech (TTS) synthesis. MuseTalk is an open source high-quality digital human mouth reasoning model, mainly used in virtual digital human technology. Ernerf is a trainable lightweight digital human reasoning model with a fast reasoning speed. Wav2Lip is a trainable digital human reasoning model that can adjust faces according to different facial proportions and customize the output of specific digital human reasoning videos.

[0100] Based on the above description, in this embodiment, the technical implementation process is as follows:

[0101] (1) Process definition: Use a graphical interface or programming interface to define business processes.

[0102] (2) Process execution: Automatically execute tasks according to the defined process.

[0103] (3) Task scheduling: Schedule tasks based on task priority and resource availability.

[0104] (4) Exception handling: handling exceptions that occur during process execution.

[0105] In addition to voice and text, gesture recognition function can also be added, using the camera to capture the user's gesture input and converting it into system instructions through gesture recognition algorithm.

[0106] As an optional implementation, the workflow engine layer includes: an intent recognition module, a classification and routing module, and a business logic or service module. The classification and routing module is connected to the intent recognition module. The business logic or service module is connected to the classification and routing module.

[0107] The intent recognition module is used to identify user intent based on workflow instances using a large language model to obtain initial intent recognition results, and classify user questions based on the initial intent recognition results to obtain user question classification results.

[0108] The classification and routing module is used to determine the processing path based on the user question classification results.

[0109] The business logic or service module is used to retrieve the business logic and business content corresponding to the processing path stored in the workflow engine based on the processing path to obtain the recognition result.

[0110] As an optional implementation, the implementation process of the growable virtual digital human method provided by the present application is described based on the system structure provided above, including:

[0111] Step 1: User input. The user interacts through a graphical interface and chooses to use a microphone for voice input or a keyboard for text input. Specifically:

[0112] Voice input: Supports Mandarin voice recognition. Users must use standard pronunciation to ensure recognition accuracy. The output duration of a single sentence is limited to 30 seconds. Audio that exceeds this duration will be truncated. The system performs quality checks on the audio (such as volume, clarity, etc.) to ensure that the audio meets the input standards of the ASR model.

[0113] Text input: Text input must be in simplified Chinese and the length is limited to 256 characters. The system has a blocked word library to automatically filter sensitive words and check whether the text conforms to grammatical rules and logical coherence.

[0114] Input verification: The system verifies whether the user input meets the above conditions. If not, the user is immediately prompted to re-enter.

[0115] Step 2: Audio to text conversion.

[0116] 1) When the input is detected as audio information, the system calls the pre-trained automatic speech recognition (ASR) model to convert it into text content. The specific conversion process is:

[0117] a. Audio preprocessing: including noise reduction, standardization and other operations to optimize audio quality.

[0118] b. Feature extraction: Extract feature vectors from audio as input to the ASR model.

[0119] c. Model reasoning: The ASR model predicts the most likely text sequence based on the extracted feature vectors. This step involves the joint work of the acoustic model, language model, and decoder.

[0120] d. Post-processing: The generated text is spell-checked and grammar-corrected to improve accuracy and readability.

[0121] 2) Convert audio into text and output: The converted text content is submitted to the workflow engine for the next step.

[0122] Step 3: Text submission and process start.

[0123] 1) Submission action: Submit the text content converted by the ASR model or directly input by the user as a data packet to the start node of the workflow engine.

[0124] 2) Process initialization: After receiving the text, the workflow engine initializes a new workflow instance and prepares to enter the intent recognition phase.

[0125] Step 4: Intent recognition and routing decision.

[0126] 1) Intent recognition: The workflow engine uses a large language model to recognize the user's intent. This step involves technical means such as keyword matching, semantic analysis, and context understanding.

[0127] 2) Classification and routing: Based on the identified user intent, the questions are classified and the next processing path is determined to direct different intents to specific business logic or service modules.

[0128] 3) Output: The workflow engine passes the text to the corresponding node for further processing. Each node represents a specific task or functional module.

[0129] Step 5: Growth and optimization of the virtual digital human system.

[0130] Collect user feedback, learn new conversation patterns, and optimize its own performance and service quality accordingly, achieve self-evolution and adapt to application needs in different scenarios, so as to be able to continuously improve and ensure the best user experience.

[0131] In summary, the system provided in this application can realize the automation and intelligence of business processes by integrating multiple advanced technologies and adopting a layered architecture design including user interaction layer, virtual digital human layer, workflow engine layer, data management layer and technical support layer.

[0132] The workflow engine can automatically execute tasks according to business process definitions, and has dynamic task scheduling and exception handling capabilities to ensure the correctness and efficiency of business logic.

[0133] Compared with the best existing technologies, this application also has the following advantages:

[0134] First of all, this application realizes the automation and intelligence of business processes by integrating workflow engines and virtual digital human technology, greatly improving the flexibility and adaptability of the system. Although the existing workflow engine technology can manage complex business processes, it is insufficient in combination with virtual digital human technology and cannot provide a highly realistic virtual interactive experience. However, this application uses the MetaHuman component to generate a highly realistic virtual digital human model, and combines the Wav2Lip model, deep learning model and motion capture data to generate natural expressions and movements, making the performance of virtual digital humans more vivid and natural.

[0135] Secondly, this application also has obvious advantages in user interaction and data management. Users can interact with virtual digital humans in a variety of ways (such as voice, text, gestures, etc.), and the system supports multimodal input and output, which improves the user experience. At the same time, the system adopts an efficient data management mechanism, supports data addition, deletion, modification and query operations, and has regular backup and multi-node synchronization functions to ensure data security and high availability. Existing technical solutions often have limitations in these aspects and cannot provide such comprehensive and efficient support. In summary, this application is superior to the best existing technologies in terms of flexibility, realism, user interaction and data management, providing users with a more intelligent and efficient virtual digital human system.

[0136] In an exemplary embodiment, a computer program product is provided, including an application (Application, APP) implanted with a workflow engine, which, when executed by a smart device, implements the above-mentioned growable virtual digital human method.

[0137] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0138] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0139] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0140] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for growing virtual digital human, characterized in that: The growable virtual digital human method is implemented based on the Dify framework; the growable virtual digital human method includes: Get user question data; generating a text data packet based on the user question data; Using a workflow engine, generating and storing a workflow instance based on the text data packet, and performing intent recognition based on the workflow instance in combination with the business logic and business content stored in the workflow engine to obtain a recognition result; Generate virtual digital human appearance data and behavior data based on the recognition result, and output the recognition result based on the virtual digital human appearance data and behavior data.

2. The method for creating a growable virtual digital human according to claim 1, characterized in that: Generating a text data packet based on the user question data includes: When the user question data is voice data, an automatic speech recognition model is used to convert the voice data into text content to generate the text data packet; When the user question data is text data, the shielding word library stored in the workflow engine is used to shield the set words in the text data to generate the text data packet.

3. The method for creating a growable virtual digital human according to claim 2, characterized in that: The voice data is converted into text data using an automatic speech recognition model, including: The speech data is preprocessed; the preprocessing includes noise reduction processing and standardization processing. Extracting feature vectors of the preprocessed speech data, and inputting the feature vectors into the automatic speech recognition model to obtain a text sequence; The text data is obtained by performing spelling check and grammar correction on the text sequence.

4. The method for growing virtual digital human according to claim 3, characterized in that: When the user question data is voice data, the duration of the voice data is not longer than the set duration; when the user question data is text data, the number of characters in the text data is not more than the set number of characters.

5. The method for growing virtual digital human according to claim 2, characterized in that: Intent recognition is performed based on the workflow instance in combination with the business logic and business content stored in the workflow engine to obtain recognition results, including: Using a large language model to identify the user intent based on the workflow instance, and obtaining an initial intent recognition result; Classify the user questions according to the initial intent recognition result to obtain a user question classification result; Determine a processing path based on the user problem classification result; Based on the processing path, the business logic and business content corresponding to the processing path stored in the workflow engine are retrieved to obtain the recognition result.

6. A growable virtual digital human system, characterized in that: The growable virtual digital human system includes: a user interaction layer, a virtual digital human layer, a workflow engine layer, a data management layer and a technical support layer; The user interaction layer is used to obtain user question data and interact with the virtual digital human layer based on the user question data; The workflow engine layer is connected to the user interaction layer; the workflow engine layer is used to generate a text data packet based on the user question data, and generate and store a workflow instance based on the text data packet, and perform intent recognition based on the workflow instance combined with the business logic and business content stored inside the workflow engine to obtain a recognition result; The virtual digital human layer is used to generate virtual digital human appearance data and behavior data based on the recognition result, and output the recognition result based on the virtual digital human appearance data and behavior data; The data management layer is connected to the user interaction layer and the workflow engine layer; the data management layer is used to store and manage user data, business data and system configuration data; the user data includes user problem data and user information data; The technical support layer is connected to the workflow engine layer; the technical support layer is used to provide model support; the model includes one or more of a large language model, an automatic speech recognition model, a MuseTalk model and a Wav2Lip model.

7. The growable virtual digital human system according to claim 6, characterized in that: The user interaction layer includes a user interaction interface.

8. The growable virtual digital human system according to claim 6, characterized in that: The workflow engine layer includes: An intention recognition module, used to use a large language model to recognize user intentions based on the workflow instance, obtain an initial intention recognition result, and classify user questions according to the initial intention recognition result to obtain a user question classification result; A classification and routing module, connected to the intention recognition module, for determining a processing path based on the classification result of the user question; The business logic or service module is connected to the classification and routing module and is used to retrieve the business logic and business content corresponding to the processing path stored inside the workflow engine based on the processing path to obtain the recognition result.

9. A computer program product, comprising an application program having a workflow engine embedded therein, characterized in that: When the application is executed by a smart device, the growable virtual digital human method described in any one of claims 1-5 is implemented.

10. The computer program product according to claim 9, characterized in that The smart device is one or more of a desktop computer, a laptop computer, a smart phone, a tablet computer, an Internet of Things device, and a portable wearable device.

Images