Digital human interaction system and method based on smart community

Abstract

The invention discloses a digital human interaction system and method based on a smart community, and the method comprises the steps: collecting the questioning voice of a user through a recording device, carrying out the voice recognition of the questioning voice of the user, and obtaining a text recognition result of the questioning voice of the user; a data processing and text semantic understanding algorithm based on artificial intelligence and a natural language processing technology is introduced at a rear end to perform semantic analysis on a user question voice text recognition result, so that user question semantics in the user question voice text recognition result are carefully and fully captured, and an answer text meeting user requirements is generated; and displaying the answer text in an answer window of the digital human object. In this way, the user problem can be understood and optimized, so that the generated interaction content has higher matching degree and user satisfaction.

Description

Technical Field

[0001] The present application relates to the field of intelligent interaction, and more specifically, to a digital human interaction system and method based on a smart community. Background Art

[0002] Smart community refers to a development model that uses information technology to manage and serve the community intelligently, thereby improving the quality of life and convenience of community residents. Smart communities use technologies such as the Internet of Things, big data, and artificial intelligence to achieve intelligent management of community facilities and equipment, intelligent provision of life services for residents, and intelligent decision-making for community operations, thereby building an intelligent, convenient, and efficient community ecosystem.

[0003] The digital human interaction system is an important part of the smart community. It is an interactive system based on artificial intelligence technology. It interacts with users through natural language dialogues through simulated humanoid images or virtual characters, and provides functions such as information query, service guidance, and question answering. The digital human interaction system can provide community residents with 24-hour online personalized services, improve the intelligent level of community management, and improve the living experience of residents.

[0004] However, the traditional digital human interaction system based on smart communities has some defects: the traditional system has limited semantic understanding of user questions, and can only generate answers based on keyword matching, and cannot accurately grasp the user's intention. In addition, since the digital human interaction system is essentially a dialogue system, the key to the dialogue system is to correctly understand the user's questions. When users ask questions, they often ask questions in an irregular and colloquial way, which affects the semantic understanding of the user's questions, thereby reducing the quality and matching of the generated interactive content.

[0005] Therefore, an optimized digital human interaction system based on a smart community is desired, which can optimize the understanding of user questions so that the generated interactive content can have a higher degree of matching and user satisfaction. Summary of the invention

[0006] In order to solve the above technical problems, this application is proposed. The embodiment of the present application provides a digital human interaction system and method based on a smart community, which collects user question voices through a recording device, and performs voice recognition on the user question voice to obtain the user question voice text recognition result, and then introduces data processing and text semantic understanding algorithms based on artificial intelligence and natural language processing technology at the back end to perform semantic analysis of the user question voice text recognition result, so as to more carefully and fully capture the user question semantics, thereby generating an answer text that meets the user's requirements, and displaying the answer text in the answer window of the digital human object. In this way, the user's questions can be understood and optimized so that the generated interactive content has a higher matching degree and user satisfaction.

[0007] According to one aspect of the present application, a digital human interaction method based on a smart community is provided, which includes:

[0008] Obtain the user's question voice collected by the recording device;

[0009] Inputting the user's question voice into a voice recognition module to obtain a user's question voice text recognition result;

[0010] Performing contextual semantic coding based on word granularity on the user question speech text recognition result to obtain a sequence of user question word granularity semantic coding feature vectors;

[0011] Using a semantic association enhancement module based on scale measurement to process the sequence of user question word granularity semantic encoding feature vectors to obtain a sequence of enhanced user question word granularity semantic encoding feature vectors;

[0012] Passing the sequence of the granularity embedding encoding vectors of the enhanced user question words through an answer generator based on a large language model to obtain an answer text;

[0013] The answer text is displayed in the answer window of the digital human object.

[0014] According to another aspect of the present application, a digital human interaction system based on a smart community is provided, which includes:

[0015] The user question voice acquisition module is used to acquire the user question voice collected by the recording device;

[0016] A speech recognition module, used for inputting the user's question speech into the speech recognition module to obtain a user's question speech text recognition result;

[0017] A context semantic encoding module, used for performing word-granularity-based context semantic encoding on the user question speech text recognition result to obtain a sequence of user question word-granularity semantic encoding feature vectors;

[0018] A semantic association strengthening module, used for processing the sequence of the user question word granularity semantic encoding feature vectors using the semantic association strengthening module based on scale measurement to obtain a sequence of strengthened user question word granularity semantic encoding feature vectors;

[0019] An answer text generation module, used for embedding the sequence of the granularity of the enhanced user question words into the encoding vector through an answer generator based on a large language model to obtain an answer text;

[0020] The visualization module is used to display the answer text in the answer window of the digital human object.

[0021] Compared with the prior art, the digital human interaction system and method provided by the present application based on the smart community collects the user's question voice through a recording device, and after performing voice recognition on the user's question voice to obtain the user's question voice text recognition result, a data processing and text semantic understanding algorithm based on artificial intelligence and natural language processing technology is introduced at the back end to perform semantic analysis of the user's question voice text recognition result, so as to more carefully and fully capture the user's question semantics, thereby generating an answer text that meets the user's requirements, and displaying the answer text in the answer window of the digital human object. In this way, the user's question can be understood and optimized so that the generated interactive content has a higher matching degree and user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0023] Figure 1 A flowchart of a digital human interaction method based on a smart community according to an embodiment of the present application;

[0024] Figure 2 A system architecture diagram of a digital human interaction method based on a smart community according to an embodiment of the present application;

[0025] Figure 3 It is a flowchart of the training phase of the digital human interaction method based on the smart community according to an embodiment of the present application;

[0026] Figure 4 It is a flowchart of sub-step S3 of the digital human interaction method based on the smart community according to an embodiment of the present application;

[0027] Figure 5It is a flowchart of sub-step S4 of the digital human interaction method based on the smart community according to an embodiment of the present application;

[0028] Figure 6 It is a block diagram of a digital human interaction system based on a smart community according to an embodiment of the present application. DETAILED DESCRIPTION

[0029] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0030] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0031] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0032] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0033] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0034] The traditional digital human interaction system based on smart communities has some defects: the traditional system has limited semantic understanding of user questions, and can only generate answers based on keyword matching, and cannot accurately grasp the user's intention. In addition, since the digital human interaction system is essentially a dialogue system, the key to the dialogue system is to correctly understand the user's questions. When users ask questions, they often ask questions in an irregular and colloquial way, which affects the semantic understanding of the user's questions, thereby reducing the quality and matching of the generated interactive content. Therefore, an optimized digital human interaction system based on smart communities is expected, which can optimize the understanding of user questions so that the generated interactive content can have a higher matching degree and user satisfaction.

[0035] In the technical solution of the present application, a digital human interaction method based on a smart community is proposed. Figure 1 The figure is a flowchart of a digital human interaction method based on a smart community according to an embodiment of the present application. Figure 2 : is a system architecture diagram of a digital human interaction method based on a smart community according to an embodiment of the present application. Figure 1 and Figure 2 As shown, according to the embodiment of the present application, the digital human interaction method based on the smart community includes the following steps: S1, obtaining the user question voice collected by the recording device; S2, inputting the user question voice into the voice recognition module to obtain the user question voice text recognition result; S3, performing contextual semantic encoding based on word granularity on the user question voice text recognition result to obtain a sequence of user question word granularity semantic encoding feature vectors; S4, using a semantic association enhancement module based on scale measurement to process the sequence of user question word granularity semantic encoding feature vectors to obtain a sequence of enhanced user question word granularity semantic encoding feature vectors; S5, passing the sequence of the enhanced user question word granularity embedded encoding vectors through an answer generator based on a large language model to obtain an answer text; S6, displaying the answer text in the answer window of the digital human object.

[0036] In particular, the S1 and S2 obtain the user's question voice collected by the recording device, and input the user's question voice into the voice recognition module to obtain the user's question voice text recognition result. Considering that the user's question voice contains the user's question semantics, it expresses the user's question semantic information, and the accuracy of voice recognition and processing technology is low, there are certain recognition and understanding errors. Compared with audio data, text data is easier to process and analyze. Based on this, in order to ensure that the system can accurately understand the user's question content, so as to generate a more accurate answer text, in the technical solution of the present application, the user's question voice is input into the voice recognition module to obtain the user's question voice text recognition result. Through the processing of the voice recognition module, the user's question voice input can be converted into text form to facilitate subsequent text processing and analysis, which is conducive to ensuring that the system can accurately understand the user's question content.

[0037] In particular, the step S3 performs context semantic encoding based on word granularity on the user question speech text recognition result to obtain a sequence of user question word granularity semantic encoding feature vectors. In particular, in a specific example of the present application, Figure 4 As shown, the S3 includes: S31, performing word segmentation processing on the user question speech text recognition result and then passing it through a word embedding layer based on word2vec to obtain a sequence of user question word granularity embedding coding vectors; S32, passing the sequence of user question word granularity embedding coding vectors through a context semantic encoder based on a Transformer layer to obtain a sequence of user question word granularity semantic coding feature vectors.

[0038] Specifically, the S31, after the word segmentation processing of the user question speech text recognition result, the word embedding layer based on word2vec is used to obtain a sequence of user question word granularity embedding coding vectors. It should be understood that since the user question speech text recognition result contains a large amount of semantic information, which reflects the user's questions and semantic information, these semantic information are composed of multiple words, and there is a semantic association of relevance between each word and word. Based on this, in order to be able to map words to vector representations in high-dimensional space and capture semantic information about each word in the user question speech text recognition result, in the technical solution of the present application, the user question speech text recognition result is further processed by word segmentation and then passed through a word embedding layer based on word2vec to obtain a sequence of user question word granularity embedding coding vectors. Through the processing based on the word2vec word embedding layer, the semantic understanding features of each word in the speech text recognition result can be captured, and each word is mapped to a vector representation in a high-dimensional space, so that words with similar semantics are close to each other in the vector space. In this way, the semantic relationship between each word in the text can be better captured, thereby improving the semantic understanding ability of user questions.

[0039] Specifically, in S32, the sequence of the user question word granularity embedding coding vectors is passed through a context semantic encoder based on a Transformer layer to obtain a sequence of the user question word granularity semantic coding feature vectors. Considering that in the user question voice text recognition result, each word does not exist independently, there is a correlation relationship and mutual influence between these words, that is, in natural language, the meaning of a word is often affected by the surrounding words. Based on this, in order to more accurately understand the contextual association features between the semantics of each word in the user question voice text recognition result, in the technical solution of the present application, the sequence of the user question word granularity embedding coding vectors is further passed through a context semantic encoder based on a Transformer layer to obtain a sequence of user question word granularity semantic coding feature vectors. Through the processing of the context semantic encoder based on the Transformer layer, the long-distance dependency between the embedding coding features of each word in the user question voice text recognition result can be captured, so as to better understand the complex relationship between words in the user question, thereby improving the overall semantic understanding ability of the user question, and better utilizing context information to enrich the semantic representation of the user question. More specifically, the sequence of user question word granularity embedding coding vectors is passed through a contextual semantic encoder based on a Transformer layer to obtain a sequence of user question word granularity semantic coding feature vectors, including: one-dimensionally arranging the sequence of user question word granularity embedding coding vectors to obtain a global user question word granularity embedding coding feature vector; calculating the global user question word granularity embedding coding feature vector and each user question word granularity embedding coding vector in the sequence of user question word granularity embedding coding vectors, respectively normalizing each self-attention association matrix in the multiple self-attention association matrices to obtain a plurality of standardized self-attention association matrices; passing each standardized self-attention association matrix in the multiple standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; and weighting each user question word granularity embedding coding vector in the sequence of user question word granularity embedding coding vectors using each probability value in the multiple probability values ​​as a weight to obtain a sequence of user question word granularity semantic coding feature vectors.

[0040] It is worth mentioning that in other specific examples of the present application, the user question speech text recognition result can also be contextually semantically encoded based on word granularity in other ways to obtain a sequence of user question word granularity semantic encoding feature vectors, for example: input the user question speech text recognition result; divide the user question speech text recognition result according to word granularity, and take each word in the recognition result as a basic unit; perform word embedding operation on each word, and map each word to a low-dimensional vector space so that the model can understand and process the semantic relationship between words; use recurrent neural network (RNN), long short-term memory network (LSTM), or Transformer and other models to capture the contextual relationship between words; in the contextual semantic encoding model, the word embedding vector of each word is integrated and propagated with its context information through the memory and learning of the model; after being processed by the contextual semantic encoding model, each word will be mapped to a semantic encoding feature vector; these word granularity semantic encoding feature vectors are arranged in the order of their order in the sentence to obtain a feature vector sequence, wherein each feature vector represents the semantic encoding of a word, so as to obtain the sequence of user question word granularity semantic encoding feature vectors.

[0041] In particular, the S4 uses a semantic association reinforcement module based on scale measurement to process the sequence of the user question word granularity semantic coding feature vectors to obtain a sequence of enhanced user question word granularity semantic coding feature vectors. Considering that in the actual semantic understanding process of the user question speech text recognition result, the semantic association between each word does not only exist in the global text content, but also in some specific content, and this local word granularity semantic association degree will decrease as the distance between words increases. Based on this, in order to further enhance the expression of the semantic association features between each word in the user question speech text recognition result, thereby further improving the system's semantic understanding ability of the user question speech text recognition result, in the technical solution of the present application, a semantic association reinforcement module based on scale measurement is further used to process the sequence of the user question word granularity semantic coding feature vectors to obtain a sequence of enhanced user question word granularity semantic coding feature vectors. By using the processing of the semantic association enhancement module based on scale measurement, it is possible to perform scale measurement based on the distance and correlation between the semantics of each word in the text, so as to enhance the expression ability of the semantic association between each word in the user's question speech text recognition result, thereby making the semantic information therein more accurate and rich. This helps the system better understand the intention and needs of the user's question, thereby improving the accuracy and efficiency of the system in answering questions or providing services. In particular, in a specific example of the present application, such as Figure 5As shown, the S4 includes: S41, calculating the semantic reinforcement factor based on the semantic similarity and word distance measurement between every two user question word granularity semantic coding feature vectors in the sequence of user question word granularity semantic coding feature vectors to obtain a semantic reinforcement factor vector composed of multiple semantic reinforcement factors; S42, using the semantic reinforcement factor at each position in the semantic reinforcement factor vector as a weight, and weightedly reinforcing each user question word granularity semantic coding feature vector in the sequence of user question word granularity semantic coding feature vectors to obtain the sequence of reinforced user question word granularity semantic coding feature vectors.

[0042] Specifically, the S41 calculates the semantic reinforcement factor based on the semantic similarity and word distance measurement between each two user question word granularity semantic encoding feature vectors in the sequence of the user question word granularity semantic encoding feature vectors to obtain a semantic reinforcement factor vector composed of multiple semantic reinforcement factors. In a specific example, the semantic reinforcement factor is calculated based on the semantic similarity and word distance measurement between each two user question word granularity semantic encoding feature vectors in the sequence of the user question word granularity semantic encoding feature vectors to obtain a semantic reinforcement factor vector composed of multiple semantic reinforcement factors, including: calculating the similarity between each two user question word granularity semantic encoding feature vectors in the sequence of the user question word granularity semantic encoding feature vectors to obtain a word semantic similarity vector composed of multiple word semantic similarities; calculating the number of feature vectors separated between each two user question word granularity semantic encoding feature vectors to obtain a word distance count vector composed of multiple word distance counts; respectively calculating the product between the word semantic similarity of each position in the word semantic similarity vector and the word distance count of the corresponding position in the word distance count vector to obtain the semantic reinforcement factor vector composed of multiple semantic reinforcement factors.

[0043] Specifically, the S42 uses the semantic reinforcement factor of each position in the semantic reinforcement factor vector as a weight to weight and reinforce each user question word granularity semantic encoding feature vector in the sequence of user question word granularity semantic encoding feature vectors to obtain the sequence of reinforced user question word granularity semantic encoding feature vectors. It should be understood that by weighting and reinforcing each user question word granularity semantic encoding feature vector in the sequence of user question word granularity semantic encoding feature vectors, the importance of each feature vector can be adjusted according to the semantic reinforcement factor of the corresponding position, so as to better represent the semantic information of the user question.

[0044] In summary, in the above embodiment, the sequence of the user question word granularity semantic coding feature vectors is processed using the semantic association enhancement module based on scale measurement to obtain a sequence of enhanced user question word granularity semantic coding feature vectors, including: using the semantic association enhancement module based on scale measurement to process the sequence of the user question word granularity semantic coding feature vectors using the following semantic association enhancement formula to obtain the sequence of enhanced user question word granularity semantic coding feature vectors; wherein the semantic association enhancement formula is:

[0045]

[0046] Among them, x i and x j The i-th and j-th user question word granular semantic coding feature vectors in the sequence of the user question word granular semantic coding feature vectors are represented respectively, g(x j ) represents x j With x i The number of eigenvectors between them, f(x i ,x j ) represents x j With x i similarity between them, M represents the total number of feature vectors in the sequence of the granular semantic encoding feature vectors of the user's question words, n is equal to M, v i Represents the i-th enhanced user question word granularity semantic encoding feature vector in the sequence of enhanced user question word granularity semantic encoding feature vectors.

[0047] It is worth mentioning that in other specific examples of the present application, the sequence of the user question word granular semantic coding feature vectors can also be processed by other methods using a semantic association enhancement module based on scale measurement to obtain a sequence of enhanced user question word granular semantic coding feature vectors, for example: for each feature vector in the user question word granular semantic coding feature vector sequence, calculate the scale measurement between it and other feature vectors. The scale measurement can be a measurement method such as cosine similarity and Euclidean distance, which is used to measure the degree of semantic association between feature vectors; the semantic association enhancement module is used to enhance the semantic association between feature vectors according to the scale measurement calculation result to obtain the sequence of enhanced user question word granular semantic coding feature vectors.

[0048] In particular, the S5 and S6 pass the sequence of the enhanced user question word granularity embedding coding vectors through the answer generator based on the large language model to obtain the answer text; and the answer text is displayed in the answer window of the digital human object. That is, the answer result is generated by using the contextual semantic association embedding coding features based on word granularity in the user question voice text recognition result after semantic association enhancement, so as to obtain the answer text. In this way, the user question can be understood and optimized so that the generated interactive content has a higher matching degree and user satisfaction. In this way, the user question semantics in the user question voice text can be captured more carefully and fully, so as to generate an answer text that meets the user's requirements, and the answer text is displayed in the answer window of the digital human object.

[0049] In the technical solution of the present application, in a preferred example, the sequence of the enhanced user question word granularity embedding encoding vectors is passed through an answer generator based on a large language model to obtain an answer text, specifically including:

[0050] Firstly, the sequence of the enhanced user question word granularity embedding encoding vectors is concatenated into an enhanced user question word granularity embedding joint encoding vector;

[0051] Calculate the self-association matrix of the joint encoding vector of the enhanced user question word granularity embedding, that is:

[0052]

[0053] Wherein, V represents the granularity embedding joint encoding vector of the enhanced user question word, T represents the transpose of the vector, Represents matrix multiplication, M c represents the autocorrelation matrix;

[0054] The enhanced user question word granularity embedding joint encoding vector is query-multiplied with the self-association matrix to obtain an intermediate feature vector, that is:

[0055]

[0056] Where V represents the joint encoding vector of the enhanced user question word granularity embedding, M c represents the autocorrelation matrix, represents matrix multiplication, V ′ represents the intermediate eigenvector;

[0057] The inner product and L2 distance of the i-th row vector and the j-th row vector of the autocorrelation matrix are calculated respectively to obtain a first weight matrix M1 and a second weight matrix M2, which are expressed as:

[0058]

[0059] in, and represents the i-th row vector and the j-th row vector of the autocorrelation matrix, T represents the transpose of the vector, Represents matrix multiplication, M c represents the autocorrelation matrix, M1 represents the first weight matrix, m 1-i,j represents the eigenvalue of the (i, j)th position of the first weight matrix, M2 represents the second weight matrix, and m 2-i,j represents the eigenvalue of the (i, j)th position of the second weight matrix, Indicates the L2 distance of the calculated vector;

[0060] After matrix multiplication of the first weight matrix M1 and the second weight matrix M2, further matrix multiplication is performed with the intermediate feature vector to obtain an optimized enhanced user question word granularity embedding joint encoding vector, that is:

[0061]

[0062] Where M1 represents the first weight matrix, M2 represents the second weight matrix, V ′ represents the intermediate eigenvector, represents matrix multiplication, V″ represents the optimized enhanced user question word granularity embedding joint encoding vector;

[0063] The optimized enhanced user question word granularity embedding joint encoding vector is split into a sequence of optimized enhanced user question word granularity embedding encoding vectors. Finally, the sequence of optimized enhanced user question word granularity embedding encoding vectors is passed through an answer generator based on a large language model to obtain an answer text.

[0064] That is, in order to improve the generative regression interpretability of the sequence of the enhanced user question word granular embedding coding vectors based on the multi-dimensional text semantic dimensional representation of its encoded text semantics in the local-global text semantic space domain based on the complex feature representation dimension, in the above steps, the characteristic values ​​of the sequence of the enhanced user question word granular embedding coding vectors are scattered bit by bit in the associated response space, and the deep embedding structure in the high-order associated space is obtained through the joint embedding space of the vector inner product embedding space and the vector independent embedding space, and the decomposable associated response sub-dimension of the characteristic value bit by bit scattered associated response space of the sequence of the enhanced user question word granular embedding coding vectors is represented as a set of spatial sub-dimensions to further perform the dependency space mapping of the decomposable associated response sub-dimension of the deep embedding structure of the sequence of the enhanced user question word granular embedding coding vectors, so as to improve the interpretability of the solution space of the generative regression solution of the sequence of the enhanced user question word granular embedding coding vectors based on the associated distribution through the associated response complex dimensional representation, thereby improving the text quality of the answer text obtained by the answer generator based on the large language model for the sequence of the enhanced user question word granular embedding coding vectors. In this way, an answer text that meets the user's requirements can be generated more accurately based on the semantics of the user's question voice text, and the answer text can be displayed in the answer window of the digital human object.

[0065] It should be understood that before using the above-mentioned neural network model for inference, it is necessary to train the word2vec-based word embedding layer, the Transformer-based contextual semantic encoder, the scale-metric-based semantic association reinforcement module, and the large language model-based answer generator. That is to say, in the digital human interaction method based on the smart community of the present application, a training phase is also included for training the word2vec-based word embedding layer, the Transformer-based contextual semantic encoder, the scale-metric-based semantic association reinforcement module, and the large language model-based answer generator.

[0066] Figure 3 FIG. 1 is a flow chart of the training phase of the digital human interaction method based on the smart community according to an embodiment of the present application. Figure 3As shown, according to the embodiment of the present application, the digital human interaction method based on the smart community includes: a training stage, including: S110, obtaining training data, wherein the training data includes a training user question voice collected by a recording device; S120, inputting the training user question voice into a voice recognition module to obtain a training user question voice text recognition result; S130, performing word segmentation processing on the training user question voice text recognition result and then passing it through a word embedding layer based on word2vec to obtain a sequence of training user question word granularity embedding coding vectors; S140, passing the sequence of training user question word granularity embedding coding vectors through a contextual semantic encoder based on a Transformer layer to obtain a sequence of training user question word granularity semantic coding feature vectors; S150, calculating a semantic reinforcement factor based on the semantic similarity and word distance measurement between each two training user question word granularity semantic coding feature vectors in the sequence of training user question word granularity semantic coding feature vectors to obtain a plurality of training user question word granularity semantic coding feature vectors. S160, using the semantic reinforcement factors at each position in the training semantic reinforcement factor vector as weights, weightedly reinforce each training user question word granularity semantic encoding feature vector in the sequence of training user question word granularity semantic encoding feature vectors to obtain a sequence of training reinforced user question word granularity semantic encoding feature vectors; S170, passing the sequence of training reinforced user question word granularity embedding encoding vectors through the answer generator based on the large language model to obtain an answer text; S180, calculating the cross entropy loss function value between the answer text and the real text to obtain a semantic difference cross entropy loss function value; S190, training the word2vec-based word embedding layer, the Transformer-based contextual semantic encoder, the scale metric-based semantic association reinforcement module and the answer generator based on the large language model based on the semantic difference cross entropy loss function value.

[0067] In summary, the digital human interaction method based on the smart community according to the embodiment of the present application is explained, which collects the user's question voice through a recording device, and performs voice recognition on the user's question voice to obtain the user's question voice text recognition result, and then introduces data processing and text semantic understanding algorithms based on artificial intelligence and natural language processing technology at the back end to perform semantic analysis of the user's question voice text recognition result, so as to more carefully and fully capture the user's question semantics, thereby generating an answer text that meets the user's requirements, and displaying the answer text in the answer window of the digital human object. In this way, the user's question can be understood and optimized so that the generated interactive content has a higher matching degree and user satisfaction.

[0068] Furthermore, a digital human interaction system based on a smart community is also provided.

[0069] Figure 6 FIG. 1 is a block diagram of a digital human interaction system based on a smart community according to an embodiment of the present application. Figure 6 As shown, according to the embodiment of the present application, the digital human interaction system 300 based on the smart community includes: a user question voice acquisition module 310, which is used to acquire the user question voice collected by the recording device; a voice recognition module 320, which is used to input the user question voice into the voice recognition module to obtain the user question voice text recognition result; a context semantic encoding module 330, which is used to perform context semantic encoding based on word granularity on the user question voice text recognition result to obtain a sequence of user question word granularity semantic encoding feature vectors; a semantic association enhancement module 340, which is used to process the sequence of user question word granularity semantic encoding feature vectors using a semantic association enhancement module based on scale measurement to obtain a sequence of enhanced user question word granularity semantic encoding feature vectors; an answer text generation module 350, which is used to embed the sequence of the enhanced user question word granularity into the encoding vector through an answer generator based on a large language model to obtain an answer text; a visualization module 360, which is used to display the answer text in the answer window of the digital human object.

[0070] As described above, the digital human interaction system 300 based on the smart community according to the embodiment of the present application can be implemented in various wireless terminals, such as a server with a digital human interaction algorithm based on the smart community. In one possible implementation, the digital human interaction system 300 based on the smart community according to the embodiment of the present application can be integrated into the wireless terminal as a software module and / or a hardware module. For example, the digital human interaction system 300 based on the smart community can be a software module in the operating system of the wireless terminal, or it can be an application developed for the wireless terminal; of course, the digital human interaction system 300 based on the smart community can also be one of the many hardware modules of the wireless terminal.

[0071] Alternatively, in another example, the smart community-based digital human interaction system 300 and the wireless terminal may also be separate devices, and the smart community-based digital human interaction system 300 may be connected to the wireless terminal via a wired and / or wireless network, and transmit interaction information in accordance with an agreed data format.

[0072] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A digital human interaction method based on a smart community, characterized in that: include: Obtain the user's question voice collected by the recording device; Inputting the user's question voice into a voice recognition module to obtain a user's question voice text recognition result; Performing contextual semantic coding based on word granularity on the user question speech text recognition result to obtain a sequence of user question word granularity semantic coding feature vectors; Using a semantic association enhancement module based on scale measurement to process the sequence of user question word granularity semantic encoding feature vectors to obtain a sequence of enhanced user question word granularity semantic encoding feature vectors; Passing the sequence of the granularity embedding encoding vectors of the enhanced user question words through an answer generator based on a large language model to obtain an answer text; The answer text is displayed in the answer window of the digital human object.

2. The digital human interaction method based on the smart community according to claim 1 is characterized in that: The user question speech text recognition result is subjected to context semantic coding based on word granularity to obtain a sequence of user question word granularity semantic coding feature vectors, including: After word segmentation processing is performed on the user question speech text recognition result, the result is passed through a word embedding layer based on word2vec to obtain a sequence of user question word granularity embedding coding vectors; The sequence of the user question word granularity embedding encoding vectors is passed through a contextual semantic encoder based on a Transformer layer to obtain a sequence of the user question word granularity semantic encoding feature vectors.

3. The digital human interaction method based on the smart community according to claim 2 is characterized in that: The sequence of the user question word granularity embedding encoding vectors is passed through a contextual semantic encoder based on a Transformer layer to obtain a sequence of the user question word granularity semantic encoding feature vectors, including: Arrange the sequence of the user question word granularity embedding coding vectors in one dimension to obtain a global user question word granularity embedding coding feature vector; Calculate the global user question word granularity embedding coding feature vector and each user question word granularity embedding coding vector in the sequence of the user question word granularity embedding coding vectors, and respectively perform normalization processing on each self-attention association matrix in the multiple self-attention association matrices to obtain multiple standardized self-attention association matrices; Passing each of the plurality of standardized self-attention association matrices through a Softmax classification function to obtain a plurality of probability values; Each user question word granularity embedding coding vector in the sequence of user question word granularity embedding coding vectors is weighted using each probability value in the multiple probability values ​​as a weight to obtain a sequence of user question word granularity semantic coding feature vectors.

4. The digital human interaction method based on the smart community according to claim 3 is characterized in that: The sequence of the user question word granularity semantic encoding feature vectors is processed using a semantic association enhancement module based on scale measurement to obtain a sequence of enhanced user question word granularity semantic encoding feature vectors, including: Calculating a semantic reinforcement factor based on the semantic similarity and word distance metric between every two user question word granularity semantic encoding feature vectors in the sequence of user question word granularity semantic encoding feature vectors to obtain a semantic reinforcement factor vector composed of multiple semantic reinforcement factors; Taking the semantic reinforcement factor at each position in the semantic reinforcement factor vector as a weight, each user question word granular semantic encoding feature vector in the sequence of user question word granular semantic encoding feature vectors is weightedly reinforced to obtain the sequence of reinforced user question word granular semantic encoding feature vectors.

5. The digital human interaction method based on the smart community according to claim 4 is characterized in that: The semantic reinforcement factor is calculated based on the semantic similarity and word distance measurement between each two user question word granularity semantic encoding feature vectors in the sequence of user question word granularity semantic encoding feature vectors to obtain a semantic reinforcement factor vector composed of multiple semantic reinforcement factors, including: Calculating the similarity between every two user question word granularity semantic encoding feature vectors in the sequence of user question word granularity semantic encoding feature vectors to obtain a word semantic similarity vector composed of multiple word semantic similarities; Calculate the number of feature vectors between each two user question word granularity semantic encoding feature vectors to obtain a word distance count vector composed of a plurality of word distance count quantities; The product of the word semantic similarity at each position in the word semantic similarity vector and the word distance count at the corresponding position in the word distance count vector is calculated respectively to obtain the semantic enhancement factor vector composed of multiple semantic enhancement factors.

6. The digital human interaction method based on the smart community according to claim 5 is characterized in that: It also includes a training step: for training the word2vec-based word embedding layer, the Transformer-based contextual semantic encoder, the scale metric-based semantic association reinforcement module, and the large language model-based answer generator.

7. The digital human interaction method based on smart community according to claim 6 is characterized in that: The training step comprises: Acquire training data, wherein the training data includes voice of a training user's question collected by a recording device; Inputting the training user's question voice into a voice recognition module to obtain a training user's question voice text recognition result; After word segmentation processing is performed on the speech text recognition results of the training user's questions, the results are passed through a word embedding layer based on word2vec to obtain a sequence of word granularity embedding encoding vectors of the training user's questions; The sequence of embedding encoding vectors of the training user question words is passed through a contextual semantic encoder based on a Transformer layer to obtain a sequence of semantic encoding feature vectors of the training user question words; Calculating a semantic reinforcement factor based on the semantic similarity and word distance metric between each two training user question word granularity semantic encoding feature vectors in the sequence of the training user question word granularity semantic encoding feature vectors to obtain a training semantic reinforcement factor vector composed of multiple semantic reinforcement factors; Using the semantic reinforcement factors at each position in the training semantic reinforcement factor vector as weights, weighted reinforcement is performed on each training user question word granularity semantic encoding feature vector in the sequence of training user question word granularity semantic encoding feature vectors to obtain a sequence of training reinforcement user question word granularity semantic encoding feature vectors; Passing the sequence of the training enhanced user question word granularity embedding encoding vectors through the large language model-based answer generator to obtain an answer text; Calculating a cross entropy loss function value between the answer text and the true text to obtain a semantic difference cross entropy loss function value; The word2vec-based word embedding layer, the Transformer-based contextual semantic encoder, the scale metric-based semantic association reinforcement module, and the large language model-based answer generator are trained based on the semantic difference cross entropy loss function value.

8. A digital human interaction system based on a smart community, characterized in that: include: The user question voice acquisition module is used to acquire the user question voice collected by the recording device; A speech recognition module, used for inputting the user's question speech into the speech recognition module to obtain a user's question speech text recognition result; A context semantic encoding module, used for performing word-granularity-based context semantic encoding on the user question speech text recognition result to obtain a sequence of user question word-granularity semantic encoding feature vectors; A semantic association strengthening module, used for processing the sequence of the user question word granularity semantic encoding feature vectors using the semantic association strengthening module based on scale measurement to obtain a sequence of strengthened user question word granularity semantic encoding feature vectors; An answer text generation module, used for embedding the sequence of the granularity of the enhanced user question words into the encoding vector through an answer generator based on a large language model to obtain an answer text; The visualization module is used to display the answer text in the answer window of the digital human object.

Images