A method and system for interactive live marketing of intelligent digital people

Abstract

The present application relates to the field of digital human interaction technology, and specifically discloses an intelligent digital human live broadcast marketing interaction method and system, which extracts user questions from the comment area, uses natural language processing technology based on deep learning to perform semantic analysis on user questions, extracts semantic feature representations of user questions, and semantically matches them with various template questions in a question library. Through a multi-granularity query matching mechanism, it intelligently determines the template questions that match the user questions, and then outputs the answers corresponding to the template questions as digital human interaction content, which can significantly improve the semantic understanding ability and interaction accuracy of the digital human live broadcast system, thereby providing users with more considerate and accurate answer services.

Description

Technical Field

[0001] The present application relates to the field of digital human interaction technology, and more specifically, to an intelligent digital human live marketing interaction method and system. Background Art

[0002] With the development of Internet technology and changes in consumer behavior, live streaming marketing has become an important online sales model. In this process, the anchor introduces products or services to the audience through live video, interacts with the audience, and answers their questions to promote sales conversion. However, the traditional manual live streaming method has many limitations, such as the limitation of the anchor's personal ability, fatigue caused by long live streaming, and the inability to handle a large number of audience questions at the same time.

[0003] In recent years, with the continuous advancement of artificial intelligence technology, digital humans have begun to be introduced into live broadcast marketing to assist or even replace real anchors in interacting with the audience. Digital humans can broadcast live without time or location restrictions, effectively reducing labor costs and improving live broadcast efficiency. However, the existing digital human live broadcast system mainly queries and answers questions raised by the audience based on keyword matching technology. It lacks in-depth semantic understanding of user questions, which may result in low matching accuracy, resulting in answers that may not be accurate or in line with the actual needs of users.

[0004] Specifically, the existing digital human live broadcast system lacks the ability to deeply understand user intentions and flexibly respond to complex problems. Therefore, in order to further improve the interactive experience of digital human live broadcast, an optimized intelligent digital human live broadcast marketing interaction method and system are needed to solve the above technical problems. Summary of the invention

[0005] The present application provides an intelligent digital human live broadcast marketing interaction method and system, which can significantly improve the semantic understanding ability and interaction accuracy of the digital human live broadcast system, thereby providing users with more considerate and accurate answer services.

[0006] In a first aspect, a method for interactive live marketing of an intelligent digital human is provided, comprising:

[0007] Extract user questions from the comment area and extract a collection of template questions from the question library;

[0008] Performing semantic coding on each template question in the set of the user question and the template question to obtain a set of user question semantic coding feature vectors and template question semantic coding feature vectors;

[0009] Performing semantic query matching on the set of the user question semantic encoding feature vector and the template question semantic encoding feature vector to obtain a user question-template question query matching implicit representation vector;

[0010] Determine a matching template question based on the user question-template question query matching implicit representation vector;

[0011] The answer to the matching template question is returned as digital human interaction content.

[0012] Among them, semantic query matching is performed on the set of semantic encoding feature vectors of the user question and the set of semantic encoding feature vectors of the template question, including: performing local semantic association optimization on the set of semantic encoding feature vectors of the template question to obtain a set of local context semantic association feature vectors of template questions; performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the user question, the set of semantic encoding feature vectors of the template question, and the set of local context semantic association feature vectors of the template question to obtain the implicit representation vector of the user question-template question query matching.

[0013] In a possible implementation, semantic encoding is performed on each template question in the set of the user question and the template question to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors, including:

[0014] The user questions and each template question in the set of template questions are respectively input into a semantic encoder based on the BERT model for semantic encoding to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors.

[0015] In a possible implementation, performing local semantic association optimization on the set of template question semantic encoding feature vectors to obtain a set of template question local context semantic association feature vectors includes:

[0016] One-dimensional convolution coding is performed on the set of the template question semantic encoding feature vectors to obtain the set of the template question local context semantic association feature vectors, wherein the scale of the one-dimensional convolution coding is equal to an integer multiple of the length of the template question semantic encoding feature vector.

[0017] In a possible implementation, the user question semantic encoding feature vector is subjected to attention matching query and multi-granularity query feature fusion with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain the user question-template question query matching implicit representation vector, including:

[0018] Performing query matching based on a multi-head self-attention mechanism on the user question semantic encoding feature vector, the set of template question semantic encoding feature vectors, and the set of template question local context semantic association feature vectors to obtain a set of user question-template question semantic query implicit vectors and a set of user question-template question local neighborhood granularity semantic query implicit vectors;

[0019] The set of user question-template question semantic query implicit vectors and the set of user question-template question local neighborhood granularity semantic query implicit vectors are fused to obtain the user question-template question query matching implicit representation vector.

[0020] In a possible implementation, query matching is performed based on a multi-head self-attention mechanism on the user question semantic encoding feature vector, the set of template question semantic encoding feature vectors, and the set of template question local context semantic association feature vectors to obtain a set of user question-template question semantic query implicit vectors and a set of user question-template question local neighborhood granularity semantic query implicit vectors, including:

[0021] Using a value embedding coding matrix, the user question semantic coding feature vector is processed to obtain a value feature vector;

[0022] Taking the user question semantic encoding feature vector as the query vector, the value feature vector as the value vector and the set of the template question semantic encoding feature vectors as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain the set of user question-template question semantic query implicit vectors;

[0023] Taking the user question semantic encoding feature vector as the query vector, the value feature vector as the value vector and the set of the template question local context semantic association feature vectors as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a second multi-head attention query module based on a converter structure to obtain a set of user question-template question local neighborhood granularity semantic query implicit vectors.

[0024] In a possible implementation, the set of implicit vectors of the user question-template question semantic query and the set of implicit vectors of the user question-template question local neighborhood granularity semantic query are merged to obtain the implicit representation vector of the user question-template question query match, including:

[0025] Respectively calculating the position mean vector of the set of user question-template question semantic query implicit vectors and the set of user question-template question local neighborhood granularity semantic query implicit vectors to obtain the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector;

[0026] The user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector are subjected to multiple interactive fusions to obtain the user question-template question query matching implicit representation vector.

[0027] In a possible implementation, the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector are subjected to multiple interactive fusion to obtain the user question-template question query matching implicit representation vector, including:

[0028] Calculate the point-by-point addition, point-by-point subtraction, and point-by-point multiplication between the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector to obtain a first user question-template question query feature fusion representation vector, a second user question-template question query feature fusion representation vector, and a third user question-template question query feature fusion representation vector;

[0029] After the first user question-template question query feature fusion representation vector, the second user question-template question query feature fusion representation vector and the third user question-template question query feature fusion representation vector are cascaded into a user question-template question multi-scale semantic fusion feature vector, one-dimensional convolution processing and maximum pooling processing are performed on it to obtain the user question-template question query matching implicit representation vector.

[0030] In a possible implementation, determining a matching template question based on the implicit representation vector of the user question-template question query matching includes:

[0031] Inputting the user question-template question query matching implicit representation vector into a classifier-based question matcher to obtain a matching result, wherein the matching result is used to represent a serial number of a matched template question;

[0032] Based on the matching result, the matched template question is determined from the set of template questions.

[0033] In the second aspect, an intelligent digital human live broadcast marketing interactive system is provided, including:

[0034] User question-template question extraction module, used to extract user questions from the comment area and extract a set of template questions from the question library;

[0035] A user question-template question semantic encoding module, used for semantically encoding each template question in the set of the user question and the template question to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors;

[0036] A user question-template question semantic query matching module, used to perform semantic query matching on a set of the user question semantic encoding feature vector and the template question semantic encoding feature vector to obtain a user question-template question query matching implicit representation vector;

[0037] A template question matching module, used to determine a matching template question based on the implicit representation vector of the user question-template question query matching;

[0038] The digital human interaction content determination module is used to return the answer to the matching template question as the digital human interaction content.

[0039] Among them, the user question-template question semantic query matching module includes: a local semantic association optimization unit, which is used to perform local semantic association optimization on the set of template question semantic encoding feature vectors to obtain a set of template question local context semantic association feature vectors; a multi-granularity query feature fusion processing unit, which is used to perform attention matching query and multi-granularity query feature fusion on the user question semantic encoding feature vector with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain the user question-template question query matching implicit representation vector.

[0040] In a possible implementation, the user question-template question semantic encoding module is used to:

[0041] The user questions and each template question in the set of template questions are respectively input into a semantic encoder based on the BERT model for semantic encoding to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors.

[0042] The present application has at least the following technical effects: Compared with the prior art, the present application provides an intelligent digital human live marketing interaction method and system, which extracts user questions from the comment area, uses natural language processing technology based on deep learning to perform semantic analysis on user questions, extracts the semantic feature representation of user questions, and semantically matches them with various template questions in the question library, and through a multi-granularity query matching mechanism, intelligently determines the template questions that match the user questions, and then outputs the answers corresponding to the template questions as digital human interaction content, which can significantly improve the semantic understanding ability and interaction accuracy of the digital human live broadcast system, thereby providing users with more considerate and accurate answer services. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings of the embodiments of the present application are briefly introduced below. Obviously, the drawings described below only relate to some embodiments of the present application, and are not intended to limit the present application.

[0044] Figure 1 This is a schematic flowchart of the intelligent digital human live marketing interaction method according to an embodiment of the present application.

[0045] Figure 2 A data flow diagram of the intelligent digital human live marketing interaction method according to an embodiment of the present application.

[0046] Figure 3 This is a schematic flowchart of performing semantic query matching on a set of semantic encoding feature vectors of user questions and semantic encoding feature vectors of template questions in the intelligent digital human live marketing interaction method of an embodiment of the present application.

[0047] Figure 4 In the intelligent digital human live marketing interaction method of an embodiment of the present application, an attention matching query and multi-granularity query feature fusion are performed on the user question semantic encoding feature vector with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain a schematic flowchart of the user question-template question query matching implicit representation vector.

[0048] Figure 5 This is a schematic flowchart of determining a matching template question based on the implicit representation vector of the user question-template question query matching in the intelligent digital human live marketing interaction method of an embodiment of the present application.

[0049] Figure 6 This is a schematic block diagram of the intelligent digital human live marketing interaction system of an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by ordinary technicians in this field without creative work also fall within the scope of protection of the present application.

[0051] With the advancement of AI technology, virtual digital people are becoming a new tool in live broadcast marketing. They can broadcast live at any time and any place, thereby reducing dependence on real people, reducing operating costs, and improving the flexibility and efficiency of live broadcasts. However, these virtual anchors currently rely mainly on keyword matching to respond to audience questions. This method often fails to accurately capture the true meaning of the questions, resulting in responses that may not be completely accurate or meet the actual needs of the audience. This means that the current system is not sufficient to understand complex user intentions and performs poorly when dealing with more complex or multi-faceted issues. Therefore, in order to improve this situation and improve the quality of interaction of digital people in live broadcasts, it is necessary to develop a more advanced method or system that can understand user intentions and interact with them flexibly.

[0052] In response to the above technical problems, the technical concept of the present application is: extract user questions from the comment area, use natural language processing technology based on deep learning to perform semantic analysis on user questions, extract the semantic feature representation of user questions, and semantically match them with various template questions in the question library. Through a multi-granularity query matching mechanism, intelligently determine the template questions that match the user questions, and then output the answers corresponding to the template questions as digital human interactive content, which can significantly improve the semantic understanding ability and interaction accuracy of the digital human live broadcast system, thereby providing users with more considerate and accurate answer services.

[0053] Figure 1 This is a schematic flowchart of the intelligent digital human live marketing interaction method according to an embodiment of the present application. Figure 2 This is a data flow diagram of the interactive method for live marketing of intelligent digital human in the embodiment of the present application. Figure 1 and Figure 2As shown, the intelligent digital human live marketing interaction method includes: S1, extracting user questions from the comment area, and extracting a set of template questions from the question library; S2, semantically encoding each template question in the set of user questions and template questions to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors; S3, performing semantic query matching on the set of user question semantic encoding feature vectors and template question semantic encoding feature vectors to obtain a user question-template question query matching implicit representation vector; S4, determining a matching template question based on the user question-template question query matching implicit representation vector; S5, returning the answer to the matching template question as the digital human interaction content.

[0054] In the above-mentioned intelligent digital human live marketing interactive method, S1 extracts user questions from the comment area and extracts a set of template questions from the question library. Specifically, extracting user questions from the comment area in real time can ensure the timeliness and pertinence of responses, and improve the quality of interaction and user experience. The question library provides a standard answer library for user questions, which covers various common questions that may be encountered in live marketing and their corresponding answers. By extracting a set of template questions from the question library, the digital human can find the closest standard answer to respond based on the user's specific questions, thereby ensuring the accuracy and relevance of the answers.

[0055] In the above-mentioned intelligent digital human live marketing interaction method, the S2 semantically encodes each template question in the set of the user questions and the template questions to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors. Specifically, in order to achieve in-depth semantic understanding of user questions and template questions, the present application further uses a pre-trained language model to semantically encode each template question in the set of user questions and template questions, so as to convert user questions and template questions into vector representations in a high-dimensional semantic space, and fully capture the semantic information in the original text, thereby providing a solid foundation for subsequent semantic matching. In a specific example of the present application, a BERT model is used to construct a semantic encoder, and the user questions and each template question are processed separately, so as to use the bidirectional attention mechanism of the BERT model to deeply understand and semantically encode the text, and fully mine the semantic information of each word in the text and the semantic association between words, thereby generating a contextual semantic feature representation of the question text, that is, a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors.

[0056] Optionally, in one embodiment of the present application, semantic encoding is performed on each template question in the set of user questions and template questions to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors, including: inputting each template question in the set of user questions and template questions into a semantic encoder based on the BERT model for semantic encoding to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors.

[0057] In the above-mentioned intelligent digital human live marketing interaction method, the S3 performs semantic query matching on the set of the semantic encoding feature vector of the user question and the semantic encoding feature vector of the template question to obtain the implicit representation vector of the user question-template question query matching. The user question semantic encoding feature vector and the set of the semantic encoding feature vector of the template question are further query matched to determine the template question that best matches the user question. Specifically, considering that there is semantic similarity or correlation between some template questions in the question library, in order to more accurately capture the potential connection between user questions and template questions, the present application proposes a multi-granularity query matching method, which uses the user question semantic encoding feature vector as a query feature, and performs multi-level semantic query matching on the set of the template question semantic encoding feature vector to achieve a more accurate semantic matching effect.

[0058] Optionally, in one embodiment of the present application, Figure 3 This is a schematic flow chart of performing semantic query matching on the set of semantic encoding feature vectors of user questions and the set of semantic encoding feature vectors of template questions in the interactive method of live marketing of intelligent digital human in the embodiment of the present application. Figure 3 As shown, semantic query matching is performed on the set of semantic encoding feature vectors of the user question and the set of semantic encoding feature vectors of the template question, including: S31, performing local semantic association optimization on the set of semantic encoding feature vectors of the template question to obtain a set of local context semantic association feature vectors of the template question; S32, performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the user question, the set of semantic encoding feature vectors of the template question, and the set of local context semantic association feature vectors of the template question to obtain the implicit representation vector of the user question-template question query matching.

[0059] Optionally, in one embodiment of the present application, the set of template question semantic encoding feature vectors is subjected to local semantic association optimization to obtain a set of template question local context semantic association feature vectors, including: performing one-dimensional convolution encoding on the set of template question semantic encoding feature vectors to obtain a set of template question local context semantic association feature vectors, wherein the scale of the one-dimensional convolution encoding is equal to an integer multiple of the length of the template question semantic encoding feature vector.

[0060] Optionally, in one embodiment of the present application, Figure 4 In the interactive method for live marketing of intelligent digital human in the embodiment of the present application, the user question semantic encoding feature vector is subjected to attention matching query and multi-granularity query feature fusion with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain the implicit representation vector of the user question-template question query matching. Figure 4 As shown, the user question semantic encoding feature vector is subjected to attention matching query and multi-granularity query feature fusion with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain the user question-template question query matching implicit representation vector, including: S321, the user question semantic encoding feature vector is subjected to query matching based on a multi-head self-attention mechanism with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain a set of user question-template question semantic query implicit vectors and a set of user question-template question local neighborhood granularity semantic query implicit vectors; S322, the set of user question-template question semantic query implicit vectors and the set of user question-template question local neighborhood granularity semantic query implicit vectors are fused to obtain the user question-template question query matching implicit representation vector.

[0061] Optionally, in one embodiment of the present application, the user question semantic encoding feature vector is subjected to query matching based on a multi-head self-attention mechanism with a set of the template question semantic encoding feature vectors and a set of the template question local context semantic association feature vectors to obtain a set of user question-template question semantic query implicit vectors and a set of user question-template question local neighborhood granularity semantic query implicit vectors, including: using a value embedding coding matrix to process the user question semantic encoding feature vector to obtain a value feature vector; using the user question semantic encoding feature vector as a query vector, the value feature vector as a value vector and the template question semantic encoding feature vector as a query vector; The set of code feature vectors is used as a set of key vectors, and the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the user question-template question semantic query; the user question semantic encoding feature vector is used as a query vector, the value feature vector is used as a value vector and the set of local context semantic association feature vectors of the template question is used as a set of key vectors, and the query vector, the value vector and the set of key vectors are input into a second multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the user question-template question local neighborhood granularity semantic query.

[0062] Optionally, in one embodiment of the present application, the set of implicit vectors of the user question-template question semantic query and the set of implicit vectors of the user question-template question local neighborhood granularity semantic query are fused to obtain the implicit representation vector of the user question-template question query matching, including: respectively calculating the positional mean vectors of the set of implicit vectors of the user question-template question semantic query and the set of implicit vectors of the user question-template question local neighborhood granularity semantic query to obtain a user question-template question semantic query matching feature vector and a user question-template question local neighborhood granularity semantic query matching feature vector; performing multiple interactive fusions on the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector to obtain the user question-template question query matching implicit representation vector.

[0063] Optionally, in one embodiment of the present application, the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector are subjected to multiple interactive fusions to obtain the user question-template question query matching implicit representation vector, including: calculating the position point addition, position point subtraction and position point multiplication between the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector to obtain a first user question-template question query feature fusion representation vector, a second user question-template question query feature fusion representation vector and a third user question-template question query feature fusion representation vector; after cascading the first user question-template question query feature fusion representation vector, the second user question-template question query feature fusion representation vector and the third user question-template question query feature fusion representation vector into a user question-template question multi-scale semantic fusion feature vector, one-dimensional convolution processing and maximum pooling processing are performed on them to obtain the user question-template question query matching implicit representation vector.

[0064] In summary, in the embodiment of the present application, the set of the user question semantic encoding feature vector and the template question semantic encoding feature vector is processed by the following query matching formula to obtain the user question-template question query matching implicit representation vector, wherein the query matching formula is:

[0065]

[0066] in, represents the set of semantically encoded feature vectors of the template question, , , and are respectively the first, second, and third in the set of semantic encoding feature vectors of the template question. and template question semantic encoding feature vector, The value of is the number of semantic encoding feature vectors of the template question, represents one-dimensional convolution processing, represents the length of the one-dimensional convolution kernel, represents the set of local context semantic association feature vectors of the template problem, , , and are respectively the first, second, and third in the set of local context semantic association feature vectors of the template question. and template question local context semantic association feature vector, The value of is the number of local context semantic association feature vectors of the template problem, represents the semantic encoding feature vector of the user's question, is the value embedding encoding matrix, is the bias term, is a value vector, is the normalized exponential function, The value of is the scale of the key vector, represents the transpose of a vector, Indicates User question-template question semantic query latent vector, Indicates User question-template question local neighborhood granularity semantic query implicit vector, Represents the user question-template question semantic query matching feature vector, Represents the user question-template question local neighborhood granularity semantic query matching feature vector, represents dot product, Indicates point addition, Indicates point reduction, Indicates cascade operation, represents the maximum pooling operation, represents a one-dimensional convolution operation, The implicit representation vector representing the user question-template question query matching.

[0067] Specifically, semantic query matching is performed on the set of semantic encoding feature vectors of the user question and the set of semantic encoding feature vectors of the template question; specifically, the set of semantic encoding feature vectors of the template question is firstly optimized for local semantic association through one-dimensional convolution processing to capture the potential semantic association relationship between template questions and generate a set of local context semantic association feature vectors of template questions. Here, the size of the convolution kernel is set to an integer multiple of the length of the semantic encoding feature vector of the template question to ensure that the contextual association information of the template question in the local neighborhood can be captured, and template questions with similar semantics are merged into the same feature subspace for subsequent matching screening to improve the generalization ability of the model. Then, a multi-head attention mechanism based on a transformer structure is used to perform query matching between user questions and template questions. The query vector and value vector are constructed based on the semantic encoding feature vector of the user question, and the set of the semantic encoding feature vector of the template question and the set of the local context semantic association feature vector of the template question are used as the set of key vectors respectively. The association matching relationship between user questions and template questions is learned from different perspectives, and the semantic information of each template question itself and the potential semantic connection between template questions are focused on, so as to more comprehensively capture the deep connection between user questions and template questions and improve the accuracy of query matching. Then, the query matching information of different granularity levels is summarized by calculating the position mean, and the user question-template question semantic query matching feature vectors of different granularity levels and the user question-template question local neighborhood granularity semantic query matching feature vectors are generated, and the two are subjected to multi-dimensional semantic interaction fusion to integrate the information of different granularity levels and generate the user question-template question query matching implicit representation vector, so as to fully reflect the multiple interactive association matching relationship between user questions and template questions, thereby providing a more accurate feature basis for the final answer selection.

[0068] In the above-mentioned intelligent digital human live marketing interaction method, the S4 determines the matching template question based on the implicit representation vector of the user question-template question query matching. Optionally, in one embodiment of the present application, Figure 5 This is a schematic flow chart of determining a matching template question based on the implicit representation vector of the user question-template question query matching in the interactive method of the live broadcast marketing of the intelligent digital human in the embodiment of the present application. Figure 5As shown, based on the implicit representation vector of the user question-template question query matching, determining the matching template question includes: S41, inputting the implicit representation vector of the user question-template question query matching into a classifier-based question matcher to obtain a matching result, and the matching result is used to represent the serial number of the matching template question; S42, based on the matching result, determining the matching template question from the set of template questions. Specifically, the classifier-based question matcher is used to perform feature classification mapping on the user question-template question query matching implicit representation vector, and by learning the template question matching information contained in the user question-template question query matching implicit representation vector, it is mapped to a preset category space to generate a matching score between each template question and the user question, thereby revealing the correlation between each template question and the user question, and outputting the serial number of the template question that best matches the user question. Then, using this serial number as an index, the corresponding template question is retrieved from the question library, and the answer corresponding to the template question is extracted and formatted, as the digital human interactive content, and output to the user in the form of voice or text through the digital human live broadcast system.

[0069] In the technical solution of the present application, considering that the set of template question semantic encoding feature vectors and the user question semantic encoding feature vectors respectively represent the text semantic encoding features of each template question and the text semantic encoding features of the user question, when performing attention query fusion based on 1D convolution local neighborhood optimization, the information redundancy between the template questions will make the query feature representation of the user question-template question query matching implicit representation vector have a diversified feature distribution with different local neighborhood query attention weight distribution structures.

[0070] Preferably, inputting the user question-template question query matching implicit representation vector into a classifier-based question matcher to obtain a matching result comprises:

[0071] Determine a user question-template question query matching probability value corresponding to the user question-template question query matching implicit representation vector, wherein the user question-template question query matching probability value represents a probability of matching a specific template question;

[0072] Calculate the product of the first user question-template question query matching feature value of the user question-template question query matching implicit representation vector and the user question-template question query matching probability value to obtain a first user question-template question query matching feature probability product;

[0073] Calculate the product of the second user question-template question query matching feature value of the user question-template question query matching implicit representation vector and the difference between one and the user question-template question query matching probability value to obtain a second user question-template question query matching feature inverse probability product;

[0074] Calculate the quotient of the first user question-template question query matching feature value divided by one minus the user question-template question query matching probability value to obtain a first user question-template question query matching feature inverse probability quotient;

[0075] Calculating the quotient of the second user question-template question query matching feature value and the user question-template question query matching probability value to obtain a second user question-template question query matching feature probability quotient;

[0076] After adding the first user question-template question query matching feature probability product and the second user question-template question query matching feature inverse probability product, subtract the difference between the first user question-template question query matching feature inverse probability quotient and the second user question-template question query matching feature probability quotient to obtain a user question-template question query matching correction feature value;

[0077] For each first user question-template question query matching feature value and second user question-template question query matching feature value of the user question-template question query matching implicit representation vector, matrix multiplying a user question-template question query matching correction feature matrix composed of the user question-template question query matching correction feature values ​​with the user question-template question query matching implicit representation vector to obtain a corrected user question-template question query matching implicit representation vector; and

[0078] The corrected user question-template question query matching implicit representation vector is input into a classifier-based question matcher to obtain a matching result.

[0079] Wherein, the user question-template question query matching feature vector The correction process is expressed as:

[0080]

[0081]

[0082]

[0083] in, represents the user question-template question query matching feature vector, a first user question-template question query matching feature value representing the user question-template question query matching implicit representation vector, a second user question-template question query matching feature value representing the user question-template question query matching implicit representation vector, represents the probability of matching a specific template problem, represents the user question-template question query matching correction feature value, A user question-template question query matching correction feature matrix representing the user question-template question query matching correction feature values, represents matrix multiplication, Represents the corrected user question-template question query matching feature vector.

[0084] Therefore, the present application targets the region-boundary integral relationship of the high-dimensional feature manifold of the implicit representation vector of the user question-template question query matching, and approximates the simply connected region representation of the high-dimensional manifold of the feature set of the implicit representation vector of the user question-template question query matching, which is composed of eigenvalue pairs, by defining the probability distribution of the entire feature set of the implicit representation vector of the user question-template question query matching as a manifold constraint boundary, so as to avoid the ambiguity in the mapping of the diversified feature representation of the implicit representation vector of the user question-template question query matching to the local manifold representation in the high-dimensional feature class convergence space, improve the execution iteration consistency of each local feature distribution in the mapping task, so as to improve the feature classification convergence effect, and improve the accuracy of the matching result obtained by the classifier-based question matcher of the implicit representation vector of the user question-template question query matching.

[0085] In the above-mentioned intelligent digital human live marketing interaction method, the S5 returns the answer to the matched template question as the digital human interaction content. Specifically, in the live interaction, the audience expects a quick response. Through template matching, the digital human can quickly find the right answer and respond to the user's question in a timely manner. By matching the question template, the digital human can provide richer and more specific interactive content to enhance the user's sense of participation and satisfaction. The use of templates can reduce the reliance on manual intervention and reduce labor costs while maintaining the continuity and professionalism of the live broadcast. Optionally, in one embodiment of the present application, the digital human presents the generated answer to the user in real time in the form of voice and text to complete the entire interaction process. In a further embodiment of the present application, after the digital human recognizes the user's question, in addition to providing a verbal answer, it can also enhance the vividness and interactivity of the interaction through a series of actions. These actions can be the digital human's facial expressions, gestures, posture changes, or certain operations in the virtual environment.

[0086] In summary, the interactive method for live marketing of intelligent digital human according to the embodiment of the present application is explained, which extracts user questions from the comment area, uses natural language processing technology based on deep learning to perform semantic analysis on user questions, extracts the semantic feature representation of user questions, and semantically matches them with each template question in the question library, and intelligently determines the template question that matches the user question through a multi-granularity query matching mechanism, and then outputs the answer corresponding to the template question as the interactive content of the digital human. In this way, the semantic understanding ability and interactive accuracy of the digital human live broadcast system can be significantly improved, thereby providing users with more intimate and accurate answer services.

[0087] Figure 6 Schematic block diagram of the intelligent digital human live marketing interactive system of the embodiment of the present application. Figure 6 As shown, the intelligent digital human live marketing interaction system 100 includes: a user question-template question extraction module 110, which is used to extract user questions from the comment area and extract a set of template questions from the question library; a user question-template question semantic encoding module 120, which is used to semantically encode each template question in the set of user questions and template questions to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors; a user question-template question semantic query matching module 130, which is used to perform semantic query matching on the user question semantic encoding feature vector and the set of template question semantic encoding feature vectors to obtain a user question-template question query matching implicit representation vector; a template question matching module 140, which is used to determine a matching template question based on the user question-template question query matching implicit representation vector; a digital human interaction content determination module 150, which is used to return the answer to the matching template question as the digital human interaction content.

[0088] Optionally, in one embodiment of the present application, the user question-template question semantic query matching module includes: a local semantic association optimization unit, used to perform local semantic association optimization on the set of template question semantic encoding feature vectors to obtain a set of template question local context semantic association feature vectors; a multi-granularity query feature fusion processing unit, used to perform attention matching query and multi-granularity query feature fusion on the user question semantic encoding feature vector with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain the user question-template question query matching implicit representation vector.

[0089] Optionally, in one embodiment of the present application, the user question-template question semantic encoding module is used to: input each template question in the set of the user question and the template question into a semantic encoder based on the BERT model for semantic encoding to obtain a set of the user question semantic encoding feature vector and the template question semantic encoding feature vector.

[0090] The basic principle of the present invention is described above in conjunction with specific embodiments. However, it should be pointed out that the advantages, strengths, effects, etc. mentioned in the present invention are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. must be possessed by each embodiment of the present invention. In addition, the specific details of the above embodiments are only for the purpose of illustration and facilitation of understanding, rather than limitation, and the above details do not limit the present invention to being implemented by adopting the above specific details.

[0091] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, please refer to the relevant description of other embodiments. In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the module division is only a logical function division, and there may be other division methods in actual implementation. The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0092] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0093] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference to a figure in a claim should not be considered as limiting the claim to which it relates.

[0094] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units stated in the device claim can also be implemented by one unit through software or hardware.

[0095] Finally, it should be noted that the above description has been given for the purpose of illustration and description. In addition, the above embodiments are only used to illustrate the technical solution of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent digital human live marketing interactive method, characterized in that: include: Extract user questions from the comment area and extract a collection of template questions from the question library; Performing semantic coding on each template question in the set of the user question and the template question to obtain a set of user question semantic coding feature vectors and template question semantic coding feature vectors; Performing semantic query matching on the set of the user question semantic encoding feature vector and the template question semantic encoding feature vector to obtain a user question-template question query matching implicit representation vector; Determine a matching template question based on the user question-template question query matching implicit representation vector; Returning the answer to the matching template question as digital human interaction content; Among them, semantic query matching is performed on the set of semantic encoding feature vectors of the user question and the set of semantic encoding feature vectors of the template question, including: performing local semantic association optimization on the set of semantic encoding feature vectors of the template question to obtain a set of local context semantic association feature vectors of the template question; performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the user question, the set of semantic encoding feature vectors of the template question, and the set of local context semantic association feature vectors of the template question to obtain the implicit representation vector of the user question-template question query matching; Among them, performing attention matching query and multi-granularity query feature fusion on the user question semantic encoding feature vector, the set of template question semantic encoding feature vectors, and the set of template question local context semantic association feature vectors to obtain the user question-template question query matching implicit representation vector, including: Performing query matching based on a multi-head self-attention mechanism on the user question semantic encoding feature vector, the set of template question semantic encoding feature vectors, and the set of template question local context semantic association feature vectors to obtain a set of user question-template question semantic query implicit vectors and a set of user question-template question local neighborhood granularity semantic query implicit vectors; Fusion of the user question-template question semantic query implicit vector set and the user question-template question local neighborhood granularity semantic query implicit vector set to obtain the user question-template question query matching implicit representation vector; Among them, the user question semantic encoding feature vector and the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors are respectively subjected to query matching based on a multi-head self-attention mechanism to obtain a set of user question-template question semantic query implicit vectors and a set of user question-template question local neighborhood granularity semantic query implicit vectors, including: Using a value embedding coding matrix, the user question semantic coding feature vector is processed to obtain a value feature vector; Taking the user question semantic encoding feature vector as the query vector, the value feature vector as the value vector and the set of the template question semantic encoding feature vectors as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain the set of user question-template question semantic query implicit vectors; Taking the user question semantic encoding feature vector as the query vector, the value feature vector as the value vector and the set of the template question local context semantic association feature vectors as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a second multi-head attention query module based on a converter structure to obtain a set of user question-template question local neighborhood granularity semantic query implicit vectors; The method of fusing the set of implicit vectors of the user question-template question semantic query and the set of implicit vectors of the user question-template question local neighborhood granularity semantic query to obtain the user question-template question query matching implicit representation vector includes: Respectively calculating the position mean vector of the set of user question-template question semantic query implicit vectors and the set of user question-template question local neighborhood granularity semantic query implicit vectors to obtain the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector; Performing multiple interactive fusions on the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector to obtain the user question-template question query matching implicit representation vector; Among them, performing multiple interactive fusions on the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector to obtain the user question-template question query matching implicit representation vector includes: Calculate the point-by-point addition, point-by-point subtraction, and point-by-point multiplication between the user question-template question semantic query matching feature vector and the user question-template question local neighborhood granularity semantic query matching feature vector to obtain a first user question-template question query feature fusion representation vector, a second user question-template question query feature fusion representation vector, and a third user question-template question query feature fusion representation vector; After the first user question-template question query feature fusion representation vector, the second user question-template question query feature fusion representation vector and the third user question-template question query feature fusion representation vector are cascaded into a user question-template question multi-scale semantic fusion feature vector, one-dimensional convolution processing and maximum pooling processing are performed on it to obtain the user question-template question query matching implicit representation vector.

2. The intelligent digital human live broadcast marketing interactive method according to claim 1 is characterized in that: Semantically encoding each template question in the set of the user question and the template question to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors, including: The user questions and each template question in the set of template questions are respectively input into a semantic encoder based on the BERT model for semantic encoding to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors.

3. The intelligent digital human live broadcast marketing interactive method according to claim 2 is characterized in that: The set of template question semantic encoding feature vectors is optimized for local semantic association to obtain a set of template question local context semantic association feature vectors, including: One-dimensional convolution coding is performed on the set of the template question semantic encoding feature vectors to obtain the set of the template question local context semantic association feature vectors, wherein the scale of the one-dimensional convolution coding is equal to an integer multiple of the length of the template question semantic encoding feature vector.

4. The intelligent digital human live broadcast marketing interactive method according to claim 3 is characterized in that: Determining a matching template question based on the implicit representation vector of the user question-template question query matching includes: Inputting the user question-template question query matching implicit representation vector into a classifier-based question matcher to obtain a matching result, wherein the matching result is used to represent a serial number of a matched template question; Based on the matching result, the matched template question is determined from the set of template questions.

5. An intelligent digital human live broadcast marketing interaction system, the intelligent digital human live broadcast marketing interaction system adopts the intelligent digital human live broadcast marketing interaction method as claimed in claim 4, characterized in that: include: User question-template question extraction module, used to extract user questions from the comment area and extract a set of template questions from the question library; A user question-template question semantic encoding module, used for semantically encoding each template question in the set of the user question and the template question to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors; A user question-template question semantic query matching module, used to perform semantic query matching on a set of the user question semantic encoding feature vector and the template question semantic encoding feature vector to obtain a user question-template question query matching implicit representation vector; A template question matching module, used to determine a matching template question based on the implicit representation vector of the user question-template question query matching; A digital human interaction content determination module, used to return the answer to the matching template question as the digital human interaction content; Among them, the user question-template question semantic query matching module includes: a local semantic association optimization unit, which is used to perform local semantic association optimization on the set of template question semantic encoding feature vectors to obtain a set of template question local context semantic association feature vectors; a multi-granularity query feature fusion processing unit, which is used to perform attention matching query and multi-granularity query feature fusion on the user question semantic encoding feature vector with the set of template question semantic encoding feature vectors and the set of template question local context semantic association feature vectors to obtain the user question-template question query matching implicit representation vector.

6. The intelligent digital human live broadcast marketing interactive system according to claim 5 is characterized in that: The user question-template question semantic encoding module is used to: The user questions and each template question in the set of template questions are respectively input into a semantic encoder based on the BERT model for semantic encoding to obtain a set of user question semantic encoding feature vectors and template question semantic encoding feature vectors.

Images