Real-time voice stream and text dialogue interaction system based on large language model

Abstract

The invention discloses a real-time voice stream and text dialogue interaction system based on a large language model, relates to the technical field of dialogue interaction, and solves the technical problem that a self-designed corpus and a large language model are difficult to combine, so that a generated answer is difficult to quickly respond to a professional question in a targeted manner. According to the method, the pre-defined answers can be quickly returned for the questions of which the answers can be found in the corpus, so that the response time is shortened. By preferentially using the existing knowledge base, the burden of the large language model can be reduced, and the overall processing efficiency of the system is improved. The information obtained from the corpus is generally more professional data, which can improve the accuracy and reliability of the answer. Meanwhile, information better meeting the requirements or the background of the user is provided in combination with the user tag, so that the reply better fits the expectation of the user. And when the answer is not found in the corpus, the large language model can be called to generate the answer, so that the adaptability of the system to complex or new questions is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of dialogue interaction systems, and in particular is a real-time voice stream and text dialogue interaction system based on a large language model. Background Art

[0002] With the rapid development of artificial intelligence technology, especially the progress of large language models such as the GPT series and BERT, natural language processing (NLP) has been widely used in many fields. Real-time voice streaming and text dialogue interaction systems are the product of this trend. They combine speech recognition, natural language understanding and generation technology to provide users with a more intuitive and efficient way of interaction. The real-time voice streaming and text dialogue interaction system based on large language models can receive user voice input, convert it into text, analyze user intent, and generate corresponding answers based on the context, while supporting real-time feedback.

[0003] Most of the existing real-time voice streaming and text dialogue interaction systems of large language models directly generate responses through the large language model. It is difficult to combine the corpus and large language model designed by themselves, making it difficult for the generated answers to provide quick and targeted responses to professional questions. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a real-time voice stream and text dialogue interaction system based on a large language model, which is used to solve the technical problem that it is difficult to combine a corpus and a large language model designed by itself, making it difficult to generate answers to quickly respond to professional questions in a targeted manner.

[0005] To solve the above problems, the first aspect of the present invention provides a real-time voice stream and text dialogue interaction system based on a large language model, comprising:

[0006] Voice input module: used to receive voice or text information input by the client and convert the user's input voice into corresponding text information;

[0007] Time series analysis module: Through time series analysis, historical tracking of user attributes, user static logs and user problems is performed;

[0008] Perception tagging module: It uses NLP technology to extract key fields of user attributes, user static logs, and user questions tracked by the time series analysis module, and sets user tags based on the key fields of user attributes, user static logs, and user questions;

[0009] Answer generation module: used to query the answer in the corpus according to the user's user tag and the user question data currently input by the user. For questions whose answers are found in the corpus, answer data is generated based on the data in the corpus; for questions whose answers cannot be found in the corpus, the large language model is called to generate answer data;

[0010] Confidence analysis module: used to save the context information in the current conversation through the deep learning model, generate semantic features of the context information, analyze the confidence of the answer generated by the answer generation module, and output the answer that meets the confidence requirements through the confidence threshold mechanism;

[0011] Output module: Converts the final generated answer into natural language and synthesizes it to return to the user.

[0012] As a further solution of the present invention: the time series analysis module performs historical tracking of user attributes, user static logs and user questions through time series analysis, including the following steps:

[0013] Obtain user attributes and user static log data. User attribute data includes age, gender, and profession. User static log data includes the content of each conversation with the system and user satisfaction feedback on the system's answers.

[0014] Clean the acquired user attributes and user static log data to remove invalid data, such as duplicate records and missing values;

[0015] Standardize data from different sources and process them in a unified format;

[0016] The time series of user conversations are extracted through time series analysis technology, and user conversations with a frequency of 8 times or more per hour are considered to belong to the same cycle;

[0017] Filter the conversations in the five most recent cycles and the corresponding user static log data as the user static log data for tracking detection.

[0018] As a further solution of the present invention, the perception tagging module extracts the key fields of the user attributes, user static logs and user questions tracked by the time series analysis module through the NLP technology, and sets the user tags according to the key fields of the user attributes, user static logs and user questions, including the following steps:

[0019] Extract key fields of user attributes, user static logs, and user questions through the spaCy library of NLP technology;

[0020] According to the extracted user attributes, user static logs and key fields of user questions, the user attributes and key fields of user static logs are used as user tags;

[0021] Based on the filtered conversations of the last five cycles, a list of key fields of corresponding user questions is constructed, and corresponding key-value pairs are set for the list of key fields of user questions in each cycle, and the key-value pairs corresponding to the list are added to the user tag.

[0022] As a further solution of the present invention: the answer generation module searches for answers in the corpus according to the user tag of the user and the user question data currently input by the user, and generates answer data according to the data in the corpus for questions for which answers are found in the corpus, including the following steps:

[0023] Obtain the latest text data of professional books, academic papers and academic forums, set professional knowledge question and answer pairs based on the collected text data, and extract the key fields of each question and answer pair;

[0024] Extract key fields from user questions;

[0025] Based on the key fields in the user's question, search for relevant questions in the professional knowledge question-answer pairs in the corpus;

[0026] The key fields in the user's question and the key fields of the question-answer pairs in the corpus are converted into vector representations through the Word2Vec model;

[0027] Calculate the semantic matching degree between the user question and the question-answer pair in the corpus based on the cosine similarity of the word vectors between the key fields in the user question and the key fields in the question-answer pair in the corpus;

[0028] Among the question-answer pairs whose semantic matching degree between the user question and the question-answer pairs in the corpus is greater than a preset threshold, select the question-answer pair with the greatest semantic matching degree;

[0029] According to the question-answer pair with the greatest semantic matching degree, the corresponding answer data generates response data.

[0030] As a further solution of the present invention: the semantic matching degree between the user question and the question-answer pair in the corpus is calculated by the following formula:

[0031]

[0032] Among them, Ex is the semantic matching degree between the user question and the question-answer pair in the corpus, COSX i is the average value of the cosine similarity between the word vectors of the i-th key field in the user's question and the key field of the question-answer pair in the corpus, i∈(1,2,…,n), where n is the total number of key fields in the user's question;

[0033] As a further solution of the present invention: a confidence analysis module comprises:

[0034] Keyword generation unit: used to analyze the keyword vectors in the generated answer data through the LSTM layer and Transformer layer built in the deep learning model, and the attention weight of the keyword vector of each question and answer in the keyword vector sequence;

[0035] Analysis unit: used to analyze the confidence of the answer data generated by the answer generation module according to the obtained attention weight, and output the answer that meets the confidence requirement through the confidence threshold mechanism;

[0036] Among them, the confidence threshold mechanism includes: according to the confidence threshold of the response data, when it is detected that the confidence of the response data is lower than the confidence threshold, actively confirming the dialogue intention to the user, otherwise, not confirming the intention.

[0037] As a further solution of the present invention: the keyword generation unit analyzes the keyword vectors in the generated answer data through the LSTM layer and the Transformer layer constructed in the deep learning model, and accounts for the attention weight of the keyword vector of each question and answer in the keyword vector sequence, including the following steps:

[0038] Extract keywords from the context information of the current conversation and convert the keywords into vector representations;

[0039] Construct an LSTM layer, and input the keyword vector of context information. The LSTM layer processes the time when the keyword vector of the current conversation appears in the context through its internal memory unit and gating mechanism, generates a time series of the keyword vector, and adds a time series label to each keyword based on the time series of the keyword vector.

[0040] The output of the LSTM layer is a sequence of keyword vectors with time series labels added;

[0041] Construct a Transformer layer. The input of the Transformer layer is the output of the LSTM layer. The Transformer layer uses the Self-Attention mechanism to analyze the keyword vector sequence, the keyword vector in the latest generated response data of the current conversation, and the attention weight of the keyword vector sequence.

[0042] The Transformer layer outputs the keyword vector in the latest generated response data of the current conversation, accounting for the attention weight of the keyword vector sequence;

[0043] The output of the Transformer layer is input into the fully connected layer of the deep learning model to obtain the keyword vector in the latest generated answer data, accounting for the attention weight of the keyword vector of each question and answer in the keyword vector sequence.

[0044] As a further solution of the present invention: the analysis unit analyzes the confidence of the response data generated by the answer generation module according to the obtained attention weight, including the following steps:

[0045] According to the keyword vector sequence with time series labels added, the cosine similarity between the keyword vector in the response data and the word vector of the keyword vector sequence is analyzed, and the semantic matching degree between the keyword in the response data and the word vector of the keyword vector sequence is calculated;

[0046] The confidence of the response data generated by the answer generation module is analyzed based on the degree of semantic matching between the keywords in the response data and the word vectors of the keyword vector sequence and the attention weights generated by the keyword generation unit.

[0047] As a further solution of the present invention: the degree of semantic matching between the keywords in the response data and the word vectors of the keyword vector sequence is calculated by the following formula:

[0048]

[0049] Among them, Ey j is the semantic matching degree between the jth keyword and the word vector of the keyword vector sequence in the response data, COSX j is the average value of the cosine similarity between the jth keyword vector and the word vectors of the keyword vector sequence in the response data;

[0050] As a further solution of the present invention: analyzing the confidence of the response data generated by the answer generation module is performed by the following formula:

[0051]

[0052] Among them, C is the confidence of the answer data generated by the answer generation module, K j is the mean of the attention weights of the j-th keyword vector in the answer data over the keyword vectors of each question in the keyword vector sequence, j∈(1,2,…,m), where m is the total number of keywords in the answer data.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] The present invention reduces the response time by quickly returning predefined answers to questions that can be answered in the corpus. By giving priority to the use of existing knowledge bases, the burden of large language models can be reduced and the overall processing efficiency of the system can be improved. The information obtained from the corpus is usually more professional data, which can improve the accuracy and reliability of the answers. At the same time, combined with user tags, information that better meets their needs or background is provided, making the reply more in line with user expectations. When the answer cannot be found in the corpus, the large language model can be called to generate a response, which enhances the system's adaptability to complex or new problems.

[0055] This invention combines the advantages of deep learning technology in context understanding, semantic feature extraction, and confidence assessment, and uses a confidence threshold mechanism to ensure that the output content is both accurate and meets user needs. Ultimately, an efficient, intelligent, and user-centric dialogue system is realized, which not only improves the quality of interaction, but also enhances the system's ability to handle diverse scenarios and complex problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 It is a schematic diagram of the system framework of the present invention;

[0058] Figure 2 It is a schematic diagram of the framework of the confidence analysis module of the present invention. DETAILED DESCRIPTION

[0059] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] See also Figure 1-Figure 2 The first aspect of the present invention provides a real-time voice stream and text dialogue interaction system based on a large language model, including:

[0061] Voice input module: used to receive voice or text information input by the client and convert the user's input voice into corresponding text information;

[0062] Time series analysis module: Through time series analysis, historical tracking of user attributes, user static logs and user problems is performed;

[0063] Perception tagging module: It uses NLP technology to extract key fields of user attributes, user static logs, and user questions tracked by the time series analysis module, and sets user tags based on the key fields of user attributes, user static logs, and user questions;

[0064] Answer generation module: used to query the answer in the corpus according to the user's user tag and the user question data currently input by the user. For questions whose answers are found in the corpus, answer data is generated based on the data in the corpus; for questions whose answers cannot be found in the corpus, the large language model is called to generate answer data;

[0065] Confidence analysis module: used to save the context information in the current conversation through the deep learning model, generate semantic features of the context information, analyze the confidence of the answer generated by the answer generation module, and output the answer that meets the confidence requirements through the confidence threshold mechanism;

[0066] Output module: Converts the final generated answer into natural language and synthesizes it to return to the user.

[0067] Specifically, in this embodiment, the set perception tag module extracts the key fields of user attributes, user static logs and user questions tracked by the time series analysis module through NLP technology, and sets user tags according to the key fields of user attributes, user static logs and user questions;

[0068] By analyzing user attributes and user static logs, it is easier to provide personalized recommendations or answers that better meet their needs, thereby improving user satisfaction. By extracting key fields, it is possible to quickly understand the user's current problems and needs, thereby speeding up the provision of solutions. Combined with time series data, it is possible to better understand the focus of user conversations in a specific time period.

[0069] In this embodiment, the answer generation module searches for answers in the corpus based on the user's user tag and the user question data currently input by the user. For questions whose answers are found in the corpus, response data is generated based on the data in the corpus; for questions whose answers cannot be found in the corpus, the large language model is called to generate response data.

[0070] For questions that can be answered in the corpus, pre-defined answers are quickly returned, thus reducing response time. By giving priority to the use of existing knowledge bases, the burden of large language models can be reduced and the overall processing efficiency of the system can be improved.

[0071] The information obtained from the corpus is usually more professional data, which can improve the accuracy and reliability of the answers.

[0072] At the same time, user tags are combined to provide information that is more in line with their needs or background, making the response more in line with user expectations.

[0073] When the answer cannot be found in the corpus, the large language model can be called to generate a response, which enhances the system's adaptability to complex or new problems. The called large language model can be selected from the existing authorized large language models.

[0074] This hybrid approach not only improves the system's response speed and accuracy, but also allows for flexible handling of various types of questions. Combining the expertise from the corpus with the dynamic generation capabilities from the large language model, it can provide users with more comprehensive, personalized, and efficient services. This strategy can both ensure the consistency of information and adapt to changing needs.

[0075] In this embodiment, the confidence analysis module saves the context information in the current conversation through a deep learning model, generates semantic features of the context information, analyzes the confidence of the answer generated by the answer generation module, and outputs a response that meets the confidence requirement through a confidence threshold mechanism;

[0076] Saving conversation context information can help the model better understand the user's intentions and needs, thereby providing more relevant answers. By maintaining the context, misunderstandings or incorrect responses caused by lack of background information can be effectively avoided. Deep learning models can capture complex semantic relationships, analyze the confidence of generated answers, and facilitate the screening of answers with qualified confidence, thereby improving the overall response quality of the system.

[0077] The advantages of deep learning technology in context understanding, semantic feature extraction, and confidence assessment are combined, and the confidence threshold mechanism is used to ensure that the output content is both accurate and meets user needs. Ultimately, an efficient, intelligent, and user-centric dialogue system is implemented, which not only improves the quality of interaction, but also enhances the system's ability to handle diverse scenarios and complex problems.

[0078] In one embodiment of the present invention, the time series analysis module performs historical tracking of user attributes, user static logs, and user questions through time series analysis, including the following steps:

[0079] Obtain user attributes and user static log data. User attribute data includes age, gender, and profession. User static log data includes the content of each conversation with the system and user satisfaction feedback on the system's answers.

[0080] Clean the acquired user attributes and user static log data to remove invalid data, such as duplicate records and missing values;

[0081] Standardize data from different sources and process them in a unified format;

[0082] The time series of user conversations are extracted through time series analysis technology, and user conversations with a frequency of 8 times or more per hour are considered to belong to the same cycle;

[0083] Filter the conversations in the five most recent cycles and the corresponding user static log data as the user static log data for tracking detection.

[0084] In one embodiment of the present invention, the perception tagging module extracts key fields of user attributes, user static logs, and user questions tracked by the time series analysis module through NLP technology, and sets user tags according to the key fields of user attributes, user static logs, and user questions, including the following steps:

[0085] Extract key fields of user attributes, user static logs, and user questions through the spaCy library of NLP technology;

[0086] According to the extracted user attributes, user static logs and key fields of user questions, the user attributes and key fields of user static logs are used as user tags;

[0087] Based on the filtered conversations of the last five cycles, a list of key fields of corresponding user questions is constructed, and corresponding key-value pairs are set for the list of key fields of user questions in each cycle, and the key-value pairs corresponding to the list are added to the user tag.

[0088] In one embodiment of the present invention, the answer generation module searches for answers in a corpus according to the user tag of the user and the user question data currently input by the user, and generates answer data according to the data in the corpus for questions for which answers are found in the corpus, including the following steps:

[0089] Obtain the latest text data of professional books, academic papers and academic forums, set professional knowledge question and answer pairs based on the collected text data, and extract the key fields of each question and answer pair;

[0090] Extract key fields from user questions;

[0091] Based on the key fields in the user's question, search for relevant questions in the professional knowledge question-answer pairs in the corpus;

[0092] The key fields in the user's question and the key fields of the question-answer pairs in the corpus are converted into vector representations through the Word2Vec model;

[0093] Calculate the semantic matching degree between the user question and the question-answer pair in the corpus based on the cosine similarity of the word vectors between the key fields in the user question and the key fields in the question-answer pair in the corpus;

[0094] Among the question-answer pairs whose semantic matching degree between the user question and the question-answer pairs in the corpus is greater than a preset threshold, select the question-answer pair with the greatest semantic matching degree;

[0095] According to the question-answer pair with the greatest semantic matching degree, the corresponding answer data generates response data.

[0096] In one embodiment of the present invention, the semantic matching degree between the user question and the question-answer pairs in the corpus is calculated by the following formula:

[0097]

[0098] Among them, Ex is the semantic matching degree between the user question and the question-answer pair in the corpus, COSX i is the average value of the cosine similarity between the word vectors of the i-th key field in the user's question and the key field of the question-answer pair in the corpus, i∈(1,2,…,n), where n is the total number of key fields in the user's question;

[0099] In one embodiment of the present invention, the confidence analysis module includes:

[0100] Keyword generation unit: used to analyze the keyword vectors in the generated answer data through the LSTM layer and Transformer layer built in the deep learning model, and the attention weight of the keyword vector of each question and answer in the keyword vector sequence;

[0101] Analysis unit: used to analyze the confidence of the answer data generated by the answer generation module according to the obtained attention weight, and output the answer that meets the confidence requirement through the confidence threshold mechanism;

[0102] Among them, the confidence threshold mechanism includes: according to the confidence threshold of the response data, when it is detected that the confidence of the response data is lower than the confidence threshold, actively confirming the dialogue intention to the user, otherwise, not confirming the intention.

[0103] Specifically, in this embodiment, the confidence threshold of the initial response data is set to 0.5 based on the middle value of the average and lowest values ​​of the confidence of the response data in which the user did not respond to comprehension defects in the user's historical conversations. If there is no historical conversation, or there are less than 5 historical conversations, the confidence threshold of the initial response data can be preliminarily screened before the response is returned to the user.

[0104] In one embodiment of the present invention, the keyword generation unit analyzes the keyword vectors in the generated answer data through the LSTM layer and the Transformer layer constructed in the deep learning model, and accounts for the attention weight of the keyword vector of each question and answer in the keyword vector sequence, including the following steps:

[0105] Extract keywords from the context information of the current conversation and convert the keywords into vector representations;

[0106] Construct an LSTM layer, and input the keyword vector of context information. The LSTM layer processes the time when the keyword vector of the current conversation appears in the context through its internal memory unit and gating mechanism, generates a time series of the keyword vector, and adds a time series label to each keyword based on the time series of the keyword vector.

[0107] The output of the LSTM layer is a sequence of keyword vectors with time series labels added;

[0108] Construct a Transformer layer. The input of the Transformer layer is the output of the LSTM layer. The Transformer layer uses the Self-Attention mechanism to analyze the keyword vector sequence, the keyword vector in the latest generated response data of the current conversation, and the attention weight of the keyword vector sequence.

[0109] The Transformer layer outputs the keyword vector in the latest generated response data of the current conversation, accounting for the attention weight of the keyword vector sequence;

[0110] The output of the Transformer layer is input into the fully connected layer of the deep learning model to obtain the keyword vector in the latest generated answer data, accounting for the attention weight of the keyword vector of each question and answer in the keyword vector sequence.

[0111] In one embodiment of the present invention, the analyzing unit analyzes the confidence of the answer data generated by the answer generating module according to the obtained attention weight, comprising the following steps:

[0112] According to the keyword vector sequence with time series labels added, the cosine similarity between the keyword vector in the response data and the word vector of the keyword vector sequence is analyzed, and the semantic matching degree between the keyword in the response data and the word vector of the keyword vector sequence is calculated;

[0113] The confidence of the response data generated by the answer generation module is analyzed based on the degree of semantic matching between the keywords in the response data and the word vectors of the keyword vector sequence and the attention weights generated by the keyword generation unit.

[0114] In one embodiment of the present invention, the semantic matching degree between the keywords in the response data and the word vectors of the keyword vector sequence is calculated by the following formula:

[0115]

[0116] Among them, Ey j is the semantic matching degree between the jth keyword and the word vector of the keyword vector sequence in the response data, COSX j is the average value of the cosine similarity between the jth keyword vector and the word vectors of the keyword vector sequence in the response data;

[0117] In one embodiment of the present invention, the confidence of the response data generated by the answer generation module is analyzed by the following formula:

[0118]

[0119] Among them, C is the confidence of the answer data generated by the answer generation module, K j is the mean of the attention weights of the j-th keyword vector in the answer data over the keyword vectors of each question in the keyword vector sequence, j∈(1,2,…,m), where m is the total number of keywords in the answer data.

[0120] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A real-time voice stream and text dialogue interaction system based on a large language model, characterized in that: include: Voice input module: used to receive voice or text information input by the client and convert the user's input voice into corresponding text information; Time series analysis module: Through time series analysis, historical tracking of user attributes, user static logs and user problems is performed; Perception tagging module: It uses NLP technology to extract key fields of user attributes, user static logs, and user questions tracked by the time series analysis module, and sets user tags based on the key fields of user attributes, user static logs, and user questions; Answer generation module: used to query the answer in the corpus according to the user's user tag and the user question data currently input by the user. For questions whose answers are found in the corpus, answer data is generated according to the data in the corpus. For questions that cannot be answered in the corpus, a large language model is used to generate answer data; Confidence analysis module: used to save the context information in the current conversation through the deep learning model, generate semantic features of the context information, analyze the confidence of the answer generated by the answer generation module, and output the answer that meets the confidence requirements through the confidence threshold mechanism; Output module: Converts the final generated answer into natural language and synthesizes it to return to the user.

2. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 1, characterized in that: The time series analysis module performs historical tracking of user attributes, user static logs, and user questions through time series analysis, including the following steps: Obtain user attributes and user static log data. User attribute data includes age, gender, and profession. User static log data includes the content of each conversation with the system and user satisfaction feedback on the system's answers. Clean the acquired user attributes and user static log data to remove invalid data, such as duplicate records and missing values; Standardize data from different sources and process them in a unified format; The time series of user conversations are extracted through time series analysis technology, and user conversations with a frequency of 8 times or more per hour are considered to belong to the same cycle; Filter the conversations in the five most recent cycles and the corresponding user static log data as the user static log data for tracking detection.

3. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 2, characterized in that: The perception tagging module extracts the key fields of user attributes, user static logs and user questions tracked by the time series analysis module through NLP technology, and sets user tags according to the key fields of user attributes, user static logs and user questions, including the following steps: Extract key fields of user attributes, user static logs, and user questions through the spaCy library of NLP technology; According to the extracted user attributes, user static logs and key fields of user questions, the user attributes and key fields of user static logs are used as user tags; Based on the filtered conversations of the last five cycles, a list of key fields of corresponding user questions is constructed, and corresponding key-value pairs are set for the list of key fields of user questions in each cycle, and the key-value pairs corresponding to the list are added to the user tag.

4. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 1, characterized in that: The answer generation module searches for answers in the corpus according to the user tag and the user question data currently input by the user, and generates answer data according to the data in the corpus for questions for which answers are found in the corpus, including the following steps: Obtain the latest text data of professional books, academic papers and academic forums, set professional knowledge question and answer pairs based on the collected text data, and extract the key fields of each question and answer pair; Extract key fields from user questions; Based on the key fields in the user's question, search for relevant questions in the professional knowledge question-answer pairs in the corpus; The key fields in the user's question and the key fields of the question-answer pairs in the corpus are converted into vector representations through the Word2Vec model; Calculate the semantic matching degree between the user question and the question-answer pair in the corpus based on the cosine similarity of the word vectors between the key fields in the user question and the key fields in the question-answer pair in the corpus; Among the question-answer pairs whose semantic matching degree between the user question and the question-answer pairs in the corpus is greater than a preset threshold, select the question-answer pair with the greatest semantic matching degree; According to the question-answer pair with the greatest semantic matching degree, the corresponding answer data generates response data.

5. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 4, characterized in that: The semantic matching degree between the user question and the question-answer pair in the corpus is calculated by the following formula: Among them, Ex is the semantic matching degree between the user question and the question-answer pair in the corpus, COSX i is the average value of the cosine similarity between the word vectors of the i-th key field in the user question and the key field of the question-answer pair in the corpus, i∈(1,2,…,n), where n is the total number of key fields in the user question.

6. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 1, characterized in that: Confidence analysis module, including: Keyword generation unit: used to analyze the keyword vectors in the generated answer data through the LSTM layer and Transformer layer built in the deep learning model, and the attention weight of the keyword vector of each question and answer in the keyword vector sequence; Analysis unit: used to analyze the confidence of the answer data generated by the answer generation module according to the obtained attention weight, and output the answer that meets the confidence requirement through the confidence threshold mechanism; Among them, the confidence threshold mechanism includes: according to the confidence threshold of the response data, when it is detected that the confidence of the response data is lower than the confidence threshold, actively confirming the dialogue intention to the user, otherwise, not confirming the intention.

7. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 6, characterized in that: The keyword generation unit analyzes the keyword vectors in the generated answer data through the LSTM layer and the Transformer layer constructed in the deep learning model, and accounts for the attention weight of the keyword vector of each question and answer in the keyword vector sequence, including the following steps: Extract keywords from the context information of the current conversation and convert the keywords into vector representations; Construct an LSTM layer, and input the keyword vector of context information. The LSTM layer processes the time when the keyword vector of the current conversation appears in the context through its internal memory unit and gating mechanism, generates a time series of the keyword vector, and adds a time series label to each keyword based on the time series of the keyword vector. The output of the LSTM layer is a sequence of keyword vectors with time series labels added; Construct a Transformer layer. The input of the Transformer layer is the output of the LSTM layer. The Transformer layer uses the Self-Attention mechanism to analyze the keyword vector sequence, the keyword vector in the latest generated response data of the current conversation, and the attention weight of the keyword vector sequence. The Transformer layer outputs the keyword vector in the latest generated response data of the current conversation, accounting for the attention weight of the keyword vector sequence; The output of the Transformer layer is input into the fully connected layer of the deep learning model to obtain the keyword vector in the latest generated answer data, accounting for the attention weight of the keyword vector of each question and answer in the keyword vector sequence.

8. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 7, characterized in that: The analyzing unit analyzes the confidence of the answer data generated by the answer generating module according to the obtained attention weight, including the following steps: According to the keyword vector sequence with time series labels added, the cosine similarity between the keyword vector in the response data and the word vector of the keyword vector sequence is analyzed, and the semantic matching degree between the keyword in the response data and the word vector of the keyword vector sequence is calculated; The confidence of the response data generated by the answer generation module is analyzed based on the degree of semantic matching between the keywords in the response data and the word vectors of the keyword vector sequence and the attention weights generated by the keyword generation unit.

9. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 8, characterized in that: The degree of semantic matching between the keywords in the response data and the word vectors of the keyword vector sequence is calculated using the following formula: Among them, Ey j is the semantic matching degree between the jth keyword and the word vector of the keyword vector sequence in the response data, COSX j is the average value of the cosine similarity between the jth keyword vector and the word vectors of the keyword vector sequence in the response data.

10. A real-time voice stream and text dialogue interaction system based on a large language model according to claim 9, characterized in that: The confidence of the answer data generated by the answer generation module is analyzed using the following formula: Among them, C is the confidence of the answer data generated by the answer generation module, K j is the mean of the attention weights of the j-th keyword vector in the answer data over the keyword vectors of each question in the keyword vector sequence, j∈(1,2,…,m), where m is the total number of keywords in the answer data.

Images