Digital human interaction method, device and electronic equipment

Abstract

The present invention provides a digital human interaction method, device, and electronic device, relating to the field of digital human technology. The method comprises the following steps: obtaining text information and historical conversations input by a user; extracting user intent labels and user vector semantics corresponding to the text information based on the text information; obtaining a response content list based on the user intent labels and a preset question-and-answer library; determining the response similarity between the user vector semantics and each response content in the response content list; determining a response weight corresponding to each response content based on historical keywords in the historical conversations; obtaining a corresponding response score based on the product of the response similarity corresponding to the response content and the response weight, and determining the highest-scoring user response content as the user response content; outputting the user response content to a digital human display unit when the user response content is textual; and generating interactive content for the digital human display unit based on the user response content when the user response content is multimodal. The present invention can improve the user experience.

Description

Technical Field

[0001] The present invention relates to the field of digital human technology, and in particular to a digital human interaction method, device and electronic equipment. Background Art

[0002] With the rapid development of artificial intelligence, computer graphics, and multimedia technologies, digital humans, with their highly realistic appearance and intelligent interactive capabilities, are being widely applied in virtual customer service, film and television production, education and training, healthcare, and other fields. In virtual customer service scenarios, digital humans can quickly respond to customer inquiries; in film and television production, digital humans can achieve special effects that are difficult to achieve with traditional filming; in education and training, digital humans can become intelligent instructors to provide personalized teaching; and in healthcare, digital humans can assist in surgical simulations and patient communication.

[0003] However, existing digital human interaction methods mostly rely on fixed scripts or simple keyword matching technology, which may lead to a lack of flexibility and naturalness in the interaction process. When faced with complex semantics, ambiguous questions or emergencies, digital humans often have deviations in their understanding of users, affecting the user experience. Summary of the Invention

[0004] The problem solved by the present invention is how to improve the user experience.

[0005] To solve the above problems, the present invention provides a digital human interaction method, device and electronic equipment.

[0006] In a first aspect, the present invention provides a digital human interaction method, which is applied to a digital human interaction system, wherein the digital human interaction system includes a digital human display unit, and the digital human interaction method includes:

[0007] Get the text information and historical conversations entered by the user;

[0008] According to the text information, the user intention label and user vector semantics corresponding to the text information are extracted through the trained semantic understanding model;

[0009] Matching the user intent tag with a preset question-and-answer library to obtain a response content list;

[0010] Determining the response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship;

[0011] Determining a response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation;

[0012] Obtaining a corresponding response score according to the product of the response similarity corresponding to the response content and the response weight, and determining the response content with the highest response score as the user response content corresponding to the text information;

[0013] When the user response content is in text type, outputting the user response content to the digital human display unit;

[0014] When the user response content is of a multimodal type, the interactive content of the digital human display unit is generated according to the header identifier of the user response content and a preset mapping mechanism.

[0015] Optionally, before obtaining the user's text information and historical conversations, the method further includes:

[0016] When the user input is voice information, extracting an acoustic feature sequence of the voice information;

[0017] Matching the acoustic feature sequence with a preset reference phoneme template to determine a phoneme sequence through a dynamic programming method;

[0018] The phoneme sequence is input into a trained language model to output the text information.

[0019] Optionally, obtaining a response content list by matching the user intention tag with a preset question and answer library includes:

[0020] A response content list corresponding to the user intention tag is determined based on a preset matching relationship between the user intention tag and the response content in the question and answer library, wherein the matching relationship includes a one-to-one correspondence between the preset intention tag and the response content, and the response content list includes at least one of the response contents.

[0021] Optionally, determining the response similarity between the user vector semantics and each response content in the response content list by using a preset cosine similarity relationship includes:

[0022] Obtaining the preset question corresponding to the response content in the question and answer library;

[0023] Extracting corresponding response vector semantics according to the preset question;

[0024] The response similarity between the user vector semantics and the response vector semantics is determined according to the cosine similarity relationship.

[0025] Optionally, determining the response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation includes:

[0026] Extracting the historical keywords according to the historical conversations, as well as the historical conversation sequence and historical frequency corresponding to each of the historical keywords;

[0027] Determining a historical weight according to the product of the historical conversation sequence and the historical frequency corresponding to the historical keyword;

[0028] Determining the response frequency corresponding to the historical keyword according to the response vector semantics corresponding to the response content;

[0029] The response weight corresponding to the response content is obtained by multiplying the response frequency corresponding to each of the historical keywords in the response content by the historical weight and then summing the results.

[0030] Optionally, generating interactive content of the digital human display unit according to the header identifier of the user response content and a preset mapping mechanism includes:

[0031] Determining an execution logic according to the type identifier of the header identifier and the mapping mechanism, wherein the mapping mechanism includes a preset one-to-one correspondence between the type identifier and the execution logic;

[0032] The digital human display unit is driven according to the execution logic to generate the interactive content.

[0033] Optionally, the training process of the semantic understanding model includes:

[0034] Obtaining a training data set, wherein the training data set includes training text information with intent labels;

[0035] The preset initial model is trained using the training data set, and the model parameters are optimized using a back-propagation algorithm to obtain the trained semantic understanding model.

[0036] Optionally, the initial model includes an intermediate layer and a classification layer, the intermediate layer is used to extract training vector semantics based on the vector representation of training text information, and the classification layer is used to output a predicted intent label based on the training vector semantics output by the intermediate layer.

[0037] In a second aspect, the present invention provides a digital human interaction device, comprising:

[0038] The acquisition module is used to obtain the text information and historical conversations entered by the user;

[0039] An extraction module is used to extract the user intent label and user vector semantics corresponding to the text information through a trained semantic understanding model based on the text information;

[0040] A matching module is used to match the user intention tag with a preset question and answer library to obtain a response content list;

[0041] a comparison module, configured to determine a response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship;

[0042] A weight module, configured to determine a response weight corresponding to each response content in the response content list according to historical keywords of the historical conversation;

[0043] a processing module, configured to obtain a corresponding response score according to a product of the response similarity corresponding to the response content and the response weight, and determine the response content with the highest response score as the user response content corresponding to the text information;

[0044] A first execution module, configured to output the user response content to a digital human display unit when the user response content is in text type;

[0045] The second execution module is configured to generate interactive content of the digital human display unit according to a header identifier of the user response content and a preset mapping mechanism when the user response content is of a multimodal type.

[0046] In a third aspect, the present invention provides an electronic device comprising a memory and a processor;

[0047] The memory is used to store computer programs;

[0048] The processor is configured to implement the digital human interaction method as described in the first aspect when executing the computer program.

[0049] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the digital human interaction method as described in the first aspect is implemented.

[0050] The digital human interaction method, device, and electronic device of the present invention have the following beneficial effects: by acquiring user input text information and historical conversations, they can fully understand the user's current needs and contextual content, thereby providing the user with more coherent and personalized interactive content. A trained semantic understanding model is used to extract user intent labels and vector semantics, enabling accurate capture of the user's true intent. This semantic information is then processed in a quantified manner, providing a solid foundation for subsequent matching and selection of response content. A response content list is generated by matching against a preset question-and-answer database, and a response score is calculated by combining cosine similarity and historical keyword weights. This comprehensively considers the influence of semantic matching and historical conversations, ultimately selecting the response content that best meets the user's needs, thereby improving the accuracy of the interaction and its relevance to contextual information. By extracting user intent labels and vector semantics using a trained semantic understanding model, cosine similarity is used to determine the similarity relationship between the user input information and the response content in the question-and-answer database, and historical conversation keywords are used to assign response weights. This achieves dynamic "semantic + context" matching, dynamically adjusting response priority, making replies more relevant to the current conversation context and avoiding mechanical responses, thereby enhancing the natural flow of interaction and user experience. Furthermore, a semantic understanding model deeply analyzes user input, converting fuzzy semantics into structured vectors to avoid literal comprehension bias. Furthermore, a multi-dimensional matching strategy combining cosine similarity with historical conversation keywords filters responses based on context and accurately locates core demands, thereby improving comprehension accuracy in complex scenarios. Furthermore, by outputting both text and multimodal response content, the output format can be flexibly adjusted based on the type of user response content, making digital human interactions more vivid, natural, and diverse. This interactive approach, which comprehensively considers semantic understanding, historical context, and multimodal output, not only improves the user experience but also enhances the intelligence and adaptability of the digital human system, enabling it to better meet the interaction needs of different users in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 A flowchart of a digital human interaction method according to an embodiment of the present invention;

[0052] Figure 2 Schematic diagram of the structure of a digital human interaction device according to an embodiment of the present invention;

[0053] Figure 3 The figure is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] To make the above-mentioned objects, features, and advantages of the present invention more clearly understood, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. Although certain embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0055] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0056] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to"; the term "based on" means "based at least in part on"; the term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optionally" means "optional embodiments". The relevant definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc. mentioned in the present invention are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0057] It should be noted that the modifications of "one" and "multiple" mentioned in the present invention are illustrative rather than restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0058] The names of the messages or information exchanged between multiple devices in the embodiments of the present invention are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0059] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0060] Embodiments of the present invention provide a digital human interaction method, device, and electronic equipment.

[0061] like Figure 1 As shown, an embodiment of the present invention provides a digital human interaction method, which is applied to a digital human interaction system. The digital human interaction system includes a digital human display unit.

[0062] Specifically, the digital human display unit, as the core of the human-computer interaction system, is responsible for presenting responses generated based on user input in the form of a digital human. This allows for intuitive user interaction through the digital human's voice, expressions, and movements. This entire process enables users to communicate naturally and smoothly with the digital human, as if they were interacting with a real person, resulting in an efficient and convenient human-computer interaction experience.

[0063] The digital human interaction method comprises:

[0064] S100, obtaining text information and historical conversations input by the user.

[0065] Specifically, the textual information entered by the user is the direct basis for the current interaction. Natural language processing technology can be used to parse this text, understand the user's intent, and generate a corresponding response. Furthermore, historical conversation records can be retrieved, containing previous exchanges between the user and the digital human. These records help the system better understand the context of the conversation and the user's needs and preferences. By combining the current text with historical conversations, the digital human can provide a more coherent, personalized, and contextually appropriate interactive experience, thereby improving user satisfaction and interaction efficiency.

[0066] S200: Extracting user intent labels and user vector semantics corresponding to the text information through a trained semantic understanding model based on the text information.

[0067] Specifically, in order to accurately understand the text information input by the user, the trained semantic understanding model can be used to process and extract the user intent labels and vector semantics in the text information. Among them, the intent labels can include keywords such as user emotions, user actions, and user instructions that represent user intent. The user vector semantics converts the text information input by the user into a high-density numerical vector, so that the computer can understand the semantic relationship through mathematical operations. The semantic understanding model is trained based on a large amount of text information annotated with intent labels and can perform in-depth analysis of the text. When the user inputs text information, the model can extract and output user vector semantics and user intent labels based on the text information, that is, to clarify the specific goals or needs that the user wants to achieve.

[0068] S300: Match the user intention tag with a preset question and answer library to obtain a response content list.

[0069] Specifically, the user's intent tag is matched with the preset question and answer library. The question and answer library is a database containing a large number of common questions and their corresponding answers. At the same time, the questions will also generate corresponding intent tags. By comparing the user's intent tag with the intent tags corresponding to the questions in the question and answer library, the intent tag and the corresponding question that best match the user's intent tag are found, and the corresponding answer, that is, the response content, is matched according to the question. A response content list is generated based on all matching response contents. This list may contain multiple response contents, which are sorted by the degree of match so that the most suitable answer for the user can be further screened out in the future, thereby achieving efficient and accurate interactive responses and meeting the diverse needs of users.

[0070] S400: Determine the response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship.

[0071] Specifically, in order to further optimize the selection of response content, the preset cosine similarity relationship is used to evaluate the response similarity between the user vector semantics and each response content in the response content list. Specifically, the user vector semantics is a high-dimensional semantic vector extracted by the semantic understanding model, and each response content in the response content list will also be converted into corresponding vector semantics. By calculating the cosine similarity between the user vector semantics and the vector semantics corresponding to each response content, a response similarity is obtained. The value of cosine similarity is between 0 and 1. The closer the value is to 1, the more similar the semantics of the user vector semantics is to the semantics of the response content. In this way, the correlation between each response content and the user vector semantics can be quantitatively evaluated, thereby providing a basis for the subsequent selection of the response content that best meets the user's intention, further improving the accuracy of the interaction and user experience.

[0072] S500: Determine a response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation.

[0073] Specifically, in order to better optimize the selection of response content based on historical conversation information, the response weight corresponding to each response content in the response content list is determined based on historical keywords in historical conversations. Historical keywords are representative and important words extracted from previous conversations between the user and the digital human. They can reflect the user's focus and the context of the conversation. The system assigns a weight value to each response content by analyzing the relevance of these historical keywords and the response content. The higher the weight value, the more consistent the response content is with the historical conversation and the more it conforms to the user's intention and context in the current conversation. In this way, the current input information and historical conversation content can be comprehensively considered to more accurately select the response content that best suits the current user, making the interaction more coherent and personalized.

[0074] Furthermore, assuming that the previous conversation between the user and the digital human revolved around "Beijing Travel Guide", historical keywords can be extracted as "Beijing Attractions", "Food Recommendations", and "Accommodation Budget". The current user's question is "What are the special snacks near the Forbidden City?" Count the number of times the historical keywords appear in each response content. For example, if a response content contains "Beijing Attractions" and "Food Recommendations", and two historical keywords appear, while another response content only contains "Food Recommendations", then the former has a higher correlation and a higher weight assignment. At the same time, use word vectors to calculate the semantic similarity between historical keywords and response content. If the semantic similarity between "accommodation budget" and a response content reaches 0.8, but is only 0.3 with another response, then the former has a higher weight than the latter. Ultimately, the weight of each response content can be determined by combining the two methods, and the content with high weight can be selected to reply, so that the interaction is more in line with user needs.

[0075] S600: Obtain a corresponding response score according to the product of the response similarity corresponding to the response content and the response weight, and determine the response content with the highest response score as the user response content corresponding to the text information.

[0076] Specifically, the response score for each response is calculated based on the product of response similarity and response weight. Response similarity reflects the degree of match between the user vector semantics and the response content semantics, while response weight is based on the correlation between historical keywords and response content, reflecting the impact of historical conversations on the current response. The response similarity of each response content is multiplied by the response weight to obtain the corresponding response score. Ultimately, the response content with the highest response score is selected as the final response content corresponding to the user's current input text information, ensuring that the interaction result not only meets the user's semantic needs but also fits the context of historical conversations, achieving an efficient and accurate interaction experience.

[0077] S700: When the user response content is in text type, output the user response content to the digital human display unit.

[0078] Specifically, when the user response is determined to be text, the text is transmitted to the digital human display unit, where it can be output via the digital human's voice. The digital human display unit can also be equipped with a text display function, presenting the response text to the user in a clear and easy-to-read format. For example, it can display the text in a dialog box on the digital human interface or through other visual methods, allowing the user to intuitively see the response. This output method not only ensures accurate information transmission but also enhances the intuitiveness of the interaction and user experience, allowing users to quickly obtain the required information, thereby achieving efficient human-computer interaction.

[0079] S800: When the user response content is of a multimodal type, generating interactive content of the digital human display unit according to the header identifier of the user response content and a preset mapping mechanism.

[0080] Specifically, when the user response content is multimodal, the interactive content for the digital human display unit is generated based on the user response header identifier (such as voice, expression, action, and other type tags) and a preset mapping mechanism. The mapping mechanism converts the multimodal response content into a set of rules for the digital human to display. For example, if the response content includes voice and expression, the mapping mechanism converts the speech text into the digital human's voice output and generates corresponding expression animation based on the sentiment analysis results. In this way, the digital human display unit can simultaneously present multiple interactive methods such as voice, expression, and action, making the interaction more vivid and natural, enhancing the user experience, and achieving richer and more expressive human-computer interaction effects.

[0081] In this embodiment, by acquiring user input text and historical conversations, a comprehensive understanding of the user's current needs and context is achieved, providing users with more coherent and personalized interactive content. A trained semantic understanding model is used to extract user intent labels and vector semantics, enabling accurate capture of the user's true intent. This semantic information is then processed in a quantitative manner, providing a solid foundation for subsequent matching and selection of response content. A response list is generated by matching responses with a pre-set Q&A database. Response scores are calculated by combining cosine similarity and historical keyword weights. This comprehensively considers the influence of semantic matching and historical conversations, ultimately selecting the response that best meets the user's needs. This improves interaction accuracy and relevance to context. By extracting user intent labels and vector semantics from a trained semantic understanding model, cosine similarity is used to determine the similarity between the user input and the Q&A database responses, and historical conversation keywords are used to assign weights to responses. This achieves dynamic "semantic + context" matching, dynamically adjusting response priorities to ensure responses are more relevant to the current conversation context and avoid mechanical responses, thereby enhancing the natural flow of interaction and user experience. Furthermore, a semantic understanding model deeply analyzes user input, converting fuzzy semantics into structured vectors to avoid literal comprehension bias. Furthermore, a multi-dimensional matching strategy combining cosine similarity with historical conversation keywords filters responses based on context and accurately locates core demands, thereby improving comprehension accuracy in complex scenarios. Furthermore, by outputting both text and multimodal response content, the output format can be flexibly adjusted based on the type of user response content, making digital human interactions more vivid, natural, and diverse. This interactive approach, which comprehensively considers semantic understanding, historical context, and multimodal output, not only improves the user experience but also enhances the intelligence and adaptability of the digital human system, enabling it to better meet the interaction needs of different users in various scenarios.

[0082] Optionally, before obtaining the user's text information and historical conversations, the method further includes:

[0083] When the user input is voice information, extracting an acoustic feature sequence of the voice information;

[0084] Matching the acoustic feature sequence with a preset reference phoneme template to determine a phoneme sequence through a dynamic programming method;

[0085] The phoneme sequence is input into a trained language model to output the text information.

[0086] This optional embodiment accepts multiple user input methods, such as text or voice. When the user input is voice, the system first extracts the acoustic feature sequence of the voice information. These features include key information such as pitch, timbre, and duration, which effectively characterize the physical characteristics of speech. This acoustic feature sequence can be extracted using conventional existing techniques. Subsequently, the acoustic feature sequence is matched with a preset reference phoneme template using a dynamic programming method. Dynamic programming is an efficient algorithm that finds the optimal correspondence between the acoustic feature sequence and the phoneme template by minimizing error or maximizing similarity, thereby determining the accurate phoneme sequence. Phonemes are the basic building blocks of speech, and the generation of phoneme sequences lays the foundation for subsequent text recognition. Finally, the phoneme sequence is input into a trained language model, which generates the corresponding text information based on the context of the phoneme sequence and linguistic rules. This process achieves efficient conversion from speech to text, enabling the digital human system to accurately understand the user's voice commands and provide clear text input for subsequent semantic analysis and interactive responses, thereby providing users with a more natural and smooth human-computer interaction experience. It should be noted that acoustic features are a set of parameters extracted from speech information that reflect the physical characteristics of speech. These features include, but are not limited to, Mel-Frequency Cepstral Coefficients (MFCCs), Linear Predictive Cepstral Coefficients (LPCCs), energy, and fundamental frequency. MFCCs are one of the most commonly used features, simulating the human ear's perception of different frequencies and effectively capturing the spectral information of speech. During the extraction process, the speech signal is first segmented into short time frames, each typically containing tens of milliseconds of speech data. Each frame is then subjected to a series of operations, including pre-emphasis, windowing, and Fast Fourier Transform (FFT), to extract a sequence of acoustic features. These feature sequences provide critical information for subsequent tasks such as speech recognition and speech synthesis, helping the system better understand and process the speech signal.

[0087] Optionally, obtaining a response content list by matching the user intention tag with a preset question and answer library includes:

[0088] A response content list corresponding to the user intention tag is determined based on a preset matching relationship between the user intention tag and the response content in the question and answer library, wherein the matching relationship includes a one-to-one correspondence between the preset intention tag and the response content, and the response content list includes at least one of the response contents.

[0089] In this optional embodiment, the question-answer library pre-stores a one-to-one matching relationship between intent tags and response content. The matching relationship is generated based on a large amount of user interaction data and scenario design. By searching the matching relationship in the question-answer library through the user intent tag, the response content corresponding to the user intent tag can be found, so that multiple possible response contents can be quickly and accurately found from the question-answer library according to the user's specific intention, providing users with a richer and more flexible interactive experience. Among them, the response content may include: (1) text type: plain text string (such as "Hello, the following is recommended content for you"); (2) multimodal type: link string with parameters, the format is [type identifier] + [parameter] + [resource path], for example: dance:style=jazz&speed=medium| / resource / dance_jazz.mp4 (dance action); picture:category=landscape&size=large| / image / scenery.jpg (picture display); video:topic=tech&duration=30s| / video / tech_update.mp4 (video playback). Each user intent tag may match multiple intent tags and corresponding multiple response contents in the question and answer library, and a response content list is generated based on all matched response contents.

[0090] Optionally, determining the response similarity between the user vector semantics and each response content in the response content list by using a preset cosine similarity relationship includes:

[0091] Obtaining the preset question corresponding to the response content in the question and answer library;

[0092] Extracting corresponding response vector semantics according to the preset question;

[0093] The response similarity between the user vector semantics and the response vector semantics is determined according to the cosine similarity relationship.

[0094] In this optional embodiment, the preset questions corresponding to each response content are first obtained from the question and answer library. These preset questions are standard question texts associated with the response content in the question and answer library, which are used to define the semantic background of the response content. Then, the response vector semantics of these preset questions can be extracted through the semantic understanding model. Subsequently, the response similarity between the user vector semantics and the response vector semantics is calculated through the preset cosine similarity relationship. Cosine similarity evaluates the similarity between two vectors by measuring the cosine value of the angle between them. The closer the value is to 1, the more similar the semantics of the user input is to the preset question. In this way, the degree of matching between each response content and the user input information can be quantitatively evaluated, thereby providing a basis for the subsequent selection of the most appropriate response, further improving the accuracy of the interaction and the user experience.

[0095] For example, the response similarity satisfies:

[0096] ;

[0097] Wherein, C is the response similarity, A is the vector representation for vector semantics, B is the vector representation of the response vector semantics, || ||2 is the Euclidean norm, and the vector representation can be obtained based on methods such as the co-occurrence matrix method or the word-to-vector (Word2Vec) model.

[0098] Optionally, determining the response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation includes:

[0099] Extracting the historical keywords according to the historical conversations, as well as the historical conversation sequence and historical frequency corresponding to each of the historical keywords;

[0100] Determining a historical weight according to the product of the historical conversation sequence and the historical frequency corresponding to the historical keyword;

[0101] Determining the response frequency corresponding to the historical keyword according to the response vector semantics corresponding to the response content;

[0102] The response weight corresponding to the response content is obtained by multiplying the response frequency corresponding to each of the historical keywords in the response content by the historical weight and then summing the results.

[0103] In this optional embodiment, historical keywords are extracted from historical conversations, and the order and frequency of each keyword's appearance in the conversation, known as the historical conversation order and historical frequency, are recorded. Historical keywords are important terms that reflect user focus and conversation topics. Their order of appearance reflects the flow of the conversation, while their frequency indicates the keyword's importance. The importance of each keyword in the historical conversation is quantified by calculating its "historical weight," the product of the historical conversation order and the historical frequency. Subsequently, based on the semantics of the response vector corresponding to the response content, the frequency of each historical keyword's appearance in the response content, known as the response frequency, is determined. Non-occurring keywords are marked as zero. Finally, the response frequency of each historical keyword is multiplied by its corresponding historical weight, and the products corresponding to all keywords are summed to obtain the response weight for that response content. This response weight comprehensively reflects the degree of fit between the response content and the historical conversation. A higher weight indicates that the response content more closely matches the context of the historical conversation and the user's focus, thus providing a basis for selecting the most appropriate response content and making the interaction more coherent and personalized.

[0104] Optionally, generating interactive content of the digital human display unit according to the header identifier of the user response content and a preset mapping mechanism includes:

[0105] Determining an execution logic according to the type identifier of the header identifier and the mapping mechanism, wherein the mapping mechanism includes a preset one-to-one correspondence between the type identifier and the execution logic;

[0106] The digital human display unit is driven according to the execution logic to generate the interactive content.

[0107] In this optional embodiment, the specific execution logic is determined based on the header identifier of the response content (e.g., type identifiers such as expressions and actions). The header identifier is a specific identifier located at the beginning of the response content that identifies the modal type. Its core function is to quickly distinguish the modal attributes of the response content (e.g., text, action, image, video, etc.). The header identifier can be presented in a structured format of "type identifier + parameters + resource path," with the type identifier being the core identification element of the header. The type identifier in the header identifier is used to specify the modal type of the response content, such as expression animation or body movement. The execution logic is determined through a preset mapping mechanism that defines a one-to-one correspondence between type identifiers and execution logic. For example, if the type identifier is "expression animation," the execution logic may be the expression generation module that drives the digital human. Based on the determined execution logic, the system drives the digital human's display unit to generate corresponding interactive content, such as video playback, expression display, or action execution. This process ensures that the digital human accurately generates corresponding multimodal interactive content based on the type of the response content, making the interaction more vivid and natural, and enhancing the user experience.

[0108] For example, when the head identifier type is an action type, such as a dance movement, the corresponding dance execution logic is generated according to the mapping mechanism. This execution logic drives the digital human display unit to generate corresponding interactive content, such as invoking the digital human's skeletal animation engine, matching the corresponding action parameters (such as dance style and speed), and driving the digital human to perform the specified action. When the head identifier type is an image or video type, the corresponding image and video execution logic is executed, such as controlling the digital human display unit to parse the corresponding resource path and invoking the rendering engine to load and display the image or play the video.

[0109] Optionally, the training process of the semantic understanding model includes:

[0110] Obtaining a training data set, wherein the training data set includes training text information with intent labels;

[0111] The preset initial model is trained using the training data set, and the model parameters are optimized using a back-propagation algorithm to obtain the trained semantic understanding model.

[0112] Optionally, the initial model includes an intermediate layer and a classification layer, the intermediate layer is used to extract training vector semantics based on the vector representation of training text information, and the classification layer is used to output a predicted intent label based on the training vector semantics output by the intermediate layer.

[0113] In this optional embodiment, the training dataset includes training text information with intent labels. This training text information and its corresponding intent labels form the basis for model learning, helping the model understand the intent expressed in different texts. A preset initial model is trained using the training dataset. This initial model is typically a deep learning architecture consisting of intermediate layers and classification layers. The intermediate layers convert the training text information into corresponding vector representations and extract the corresponding vector semantics, namely, training vector semantics. This process typically involves embedding layers, convolutional layers, and recurrent neural network layers to capture the contextual and semantic features of the text. The classification layer outputs a predicted intent label based on the training vector semantics output by the intermediate layers. This is typically implemented using a fully connected layer and an activation function (softmax), mapping the extracted semantic features to a probability distribution of intent labels. During training, the model parameters can be optimized using a backpropagation algorithm. Specifically, based on the difference (loss function) between the predicted intent label and the true intent label, gradients are calculated and model parameters are updated, gradually improving the model's ability to fit the training data, ultimately resulting in a trained semantic understanding model. This semantic understanding model can accurately output corresponding intent labels and vector semantics based on the input text information, providing core support for intent recognition and response generation in digital human interaction systems.

[0114] like Figure 2 As shown, an embodiment of the present invention provides a digital human interaction device 700, including:

[0115] Acquisition module 210, used to acquire text information and historical conversations input by the user;

[0116] Extraction module 220, configured to extract user intent labels and user vector semantics corresponding to the text information using a trained semantic understanding model based on the text information;

[0117] A matching module 230 is configured to match the user intention tag with a preset question-and-answer library to obtain a response content list;

[0118] a comparison module 240, configured to determine a response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship;

[0119] A weighting module 250 is configured to determine a response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation;

[0120] The processing module 260 is configured to obtain a corresponding response score based on the product of the response similarity corresponding to the response content and the response weight, and determine the response content with the highest response score as the user response content corresponding to the text information;

[0121] A first execution module 270 is configured to output the user response content to a digital human display unit when the user response content is in text type;

[0122] The second execution module 280 is configured to generate interactive content of the digital human display unit according to the header identifier of the user response content and a preset mapping mechanism when the user response content is of a multimodal type.

[0123] The digital human interaction of this embodiment is used to implement the digital human interaction method described above. Its advantages over the existing technology are the same as the advantages of the above-mentioned digital human interaction method over the existing technology, and will not be repeated here.

[0124] like Figure 3 As shown, an electronic device 300 provided by an embodiment of the present invention includes a memory 310 and a processor 320; the memory 310 is used to store computer programs; the processor 320 is used to implement the digital human interaction method described above when executing the computer program.

[0125] In other words, an electronic device 300 includes a memory 310 and a processor 320 coupled to the memory 310; the memory 310 is configured to store a computer program; and the processor 320 is configured to perform the following operations when executing the computer program:

[0126] Get the text information and historical conversations entered by the user;

[0127] According to the text information, the user intention label and user vector semantics corresponding to the text information are extracted through the trained semantic understanding model;

[0128] Matching the user intent tag with a preset question-and-answer library to obtain a response content list;

[0129] Determining the response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship;

[0130] Determining a response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation;

[0131] Obtaining a corresponding response score according to the product of the response similarity corresponding to the response content and the response weight, and determining the response content with the highest response score as the user response content corresponding to the text information;

[0132] When the user response content is in text type, outputting the user response content to the digital human display unit;

[0133] When the user response content is of a multimodal type, the interactive content of the digital human display unit is generated according to the header identifier of the user response content and a preset mapping mechanism.

[0134] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the digital human interaction method described above is implemented.

[0135] In other words, a non-volatile computer-readable storage medium stores a computer program, which, when executed by a processor, causes the processor to perform the following operations:

[0136] Get the text information and historical conversations entered by the user;

[0137] According to the text information, the user intention label and user vector semantics corresponding to the text information are extracted through the trained semantic understanding model;

[0138] Matching the user intent tag with a preset question-and-answer library to obtain a response content list;

[0139] Determining the response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship;

[0140] Determining a response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation;

[0141] Obtaining a corresponding response score according to the product of the response similarity corresponding to the response content and the response weight, and determining the response content with the highest response score as the user response content corresponding to the text information;

[0142] When the user response content is in text type, outputting the user response content to the digital human display unit;

[0143] When the user response content is of a multimodal type, the interactive content of the digital human display unit is generated according to the header identifier of the user response content and a preset mapping mechanism.

[0144] An electronic device 300 that can serve as a server or client of the present invention will now be described, which is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device 300 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 300 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0145] Electronic device 300 includes a computing unit that can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) or loaded from a storage unit into a random access memory (RAM). The RAM can also store various programs and data required for device operation. The computing unit, ROM, and RAM are interconnected via a bus. An input / output (I / O) interface is also connected to the bus.

[0146] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM). In this application, the units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network elements. Some or all of these units can be selected based on actual needs to achieve the objectives of the embodiments of the present invention. Furthermore, the functional units in the various embodiments of the present invention can be integrated into a single processing unit, each unit can exist physically separately, or two or more units can be integrated into a single unit. These integrated units can be implemented in either hardware or software functional units.

[0147] Although the present invention is disclosed as above, the protection scope of the present invention is not limited thereto. Those skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention, and these changes and modifications will fall within the protection scope of the present invention.

Claims

1. A digital human interaction method, characterized in that: Applied to a digital human interaction system, the digital human interaction system includes a digital human display unit, and the digital human interaction method includes: Get the text information and historical conversations entered by the user; Before obtaining the user's text information and historical conversations, the method further includes: when the user input is voice information, extracting an acoustic feature sequence of the voice information; matching the acoustic feature sequence with a preset reference phoneme template to determine a phoneme sequence through a dynamic programming method; inputting the phoneme sequence into a trained language model to output the text information; According to the text information, the user intention label and user vector semantics corresponding to the text information are extracted through the trained semantic understanding model; Matching the user intent tag with a preset question-and-answer library to obtain a response content list; Determining the response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship; Determining a response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation; The determining of the response weight corresponding to each response content in the response content list based on the historical keywords of the historical conversation includes: extracting the historical keywords, and the historical conversation sequence and historical frequency corresponding to each historical keyword based on the historical conversation; determining the historical weight based on the product of the historical conversation sequence and the historical frequency corresponding to the historical keyword; determining the response frequency corresponding to the historical keyword based on the response vector semantics corresponding to the response content; and multiplying the response frequency corresponding to each historical keyword in the response content by the historical weight and summing the results to obtain the response weight corresponding to the response content; Obtaining a corresponding response score according to the product of the response similarity corresponding to the response content and the response weight, and determining the response content with the highest response score as the user response content corresponding to the text information; When the user response content is in text type, outputting the user response content to the digital human display unit; When the user response content is of a multimodal type, the interactive content of the digital human display unit is generated according to the header identifier of the user response content and a preset mapping mechanism.

2. The digital human interaction method according to claim 1, characterized in that: The step of matching the user intent tag with a preset question-and-answer library to obtain a response content list includes: A response content list corresponding to the user intention tag is determined based on a preset matching relationship between the user intention tag and the response content in the question and answer library, wherein the matching relationship includes a one-to-one correspondence between the preset intention tag and the response content, and the response content list includes at least one of the response contents.

3. The digital human interaction method according to claim 1, characterized in that: The determining the response similarity between the user vector semantics and each response content in the response content list by using a preset cosine similarity relationship includes: Obtaining the preset question corresponding to the response content in the question and answer library; Extracting corresponding response vector semantics according to the preset question; The response similarity between the user vector semantics and the response vector semantics is determined according to the cosine similarity relationship.

4. The digital human interaction method according to claim 1, characterized in that: The generating of interactive content of the digital human display unit according to the header identifier of the user response content and a preset mapping mechanism includes: Determining an execution logic according to the type identifier of the header identifier and the mapping mechanism, wherein the mapping mechanism includes a preset one-to-one correspondence between the type identifier and the execution logic; The digital human display unit is driven according to the execution logic to generate the interactive content.

5. The digital human interaction method according to claim 1, characterized in that: The training process of the semantic understanding model includes: Obtaining a training data set, wherein the training data set includes training text information with intent labels; The preset initial model is trained using the training data set, and the model parameters are optimized using a back-propagation algorithm to obtain the trained semantic understanding model.

6. The digital human interaction method according to claim 5, characterized in that: The initial model includes an intermediate layer and a classification layer. The intermediate layer is used to extract training vector semantics based on the vector representation of training text information, and the classification layer is used to predict intent labels based on the training vector semantics output by the intermediate layer.

7. A digital human interaction device, characterized in that: include: The acquisition module is used to obtain the text information and historical conversations entered by the user; Before obtaining the user's text information and historical conversations, the method further includes: when the user input is voice information, extracting an acoustic feature sequence of the voice information; matching the acoustic feature sequence with a preset reference phoneme template to determine a phoneme sequence through a dynamic programming method; inputting the phoneme sequence into a trained language model to output the text information; An extraction module is used to extract the user intent label and user vector semantics corresponding to the text information through a trained semantic understanding model based on the text information; A matching module is used to match the user intention tag with a preset question and answer library to obtain a response content list; a comparison module, configured to determine a response similarity between the user vector semantics and each response content in the response content list through a preset cosine similarity relationship; A weight module, configured to determine a response weight corresponding to each response content in the response content list according to historical keywords of the historical conversation; Wherein, determining the response weight corresponding to each response content in the response content list according to the historical keywords of the historical conversation includes: extracting the historical keywords, and the historical conversation sequence and historical frequency corresponding to each historical keyword according to the historical conversation; determining the historical weight according to the product of the historical conversation sequence and the historical frequency corresponding to the historical keyword; determining the response frequency corresponding to the historical keyword according to the response vector semantics corresponding to the response content; and multiplying the response frequency corresponding to each historical keyword in the response content by the historical weight and summing the results to obtain the response weight corresponding to the response content; a processing module, configured to obtain a corresponding response score according to a product of the response similarity corresponding to the response content and the response weight, and determine the response content with the highest response score as the user response content corresponding to the text information; A first execution module, configured to output the user response content to a digital human display unit when the user response content is in text type; The second execution module is configured to generate interactive content of the digital human display unit according to a header identifier of the user response content and a preset mapping mechanism when the user response content is of a multimodal type.

8. An electronic device, characterized in that: including memory and processor; The memory is used to store computer programs; The processor is configured to implement the digital human interaction method according to any one of claims 1 to 6 when executing the computer program.

Images