Session output method and device based on large language model, storage medium and equipment

Abstract

The invention discloses a session output method and device based on a large language model, a storage medium and equipment, relates to the technical field of artificial intelligence and language processing, and mainly aims at solving the problem that an existing session output effect is poor. Comprising the steps of obtaining first voice compression data transmitted by a client based on a first slice data stream, and performing decompression conversion on the first voice compression data to obtain a first session text; outputting the first session text to the client, and updating the first session text based on second voice compression data obtained by a second slice data stream to obtain a second session text; and if the second session text is a question session text, searching a reply word corresponding to the question session text, processing the reply word based on the large language model which completes model training to obtain a reply session text, and outputting the reply session text to the client.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence and language processing technology, and in particular to a conversation output method and device based on a large language model, a storage medium, and a device. Background Art

[0002] With the rapid development of artificial intelligence, interactive dialogues have transformed from human conversations to intelligent robot conversations. For example, intelligent robots can provide customers with online conversation consultation and complaints 24 hours a day.

[0003] At present, existing conversation output usually uses artificial intelligence models to generate conversation content and output it. However, since artificial intelligence models for different application fields need to be learned and trained in advance, the data sets used for conversation learning have static biases and knowledge limitations, which makes the conversation content have a large response bias in the generation process, and cannot accurately avoid unclear or irrelevant and useless conversation content in this application field, resulting in poor conversation output effect. Summary of the invention

[0004] In view of this, the present invention provides a conversation output method and apparatus, storage medium, and device based on a large language model, the main purpose of which is to solve the problem of poor existing conversation output effect.

[0005] According to one aspect of the present invention, a conversation output method based on a large language model is provided, comprising:

[0006] Acquire first voice compression data transmitted by the client based on the first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text;

[0007] Outputting the first conversation text to the client, and updating the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text;

[0008] If the second conversation text is a question conversation text, the reply words corresponding to the question conversation text are searched, and the reply words are processed based on the large language model that has completed model training to obtain a reply conversation text, and the reply conversation text is output to the client.

[0009] Further, the decompressing and converting the first voice compression data to obtain the first conversation text includes:

[0010] Acquire a preset hot word library, wherein the preset hot word library stores a plurality of hot words pre-configured for conversation scenarios, and the hot words are used to represent words with the same pronunciation as non-conversation scenarios;

[0011] Performing text conversion on the voice data after decompressing the first voice compression data to obtain a plurality of reference text words;

[0012] If the pronunciation of the reference text word matches the pronunciation of the target hot word in the preset hot word library, a first conversation text is generated based on the hot word.

[0013] Further, generating the first conversation text based on the hot word includes:

[0014] Obtaining the hot word weight of each hot word in the preset hot word library;

[0015] If there are multiple hot words, weighted processing is performed based on the hot word weights to generate the first conversation text.

[0016] Further, acquiring first voice compression data transmitted by the client based on the first slice data stream includes:

[0017] Counting the number of times the first voice compression data is cached in the cache space according to the duration of the slice data stream;

[0018] If the number of cache times is greater than a preset cache threshold, the duration of the slice data stream is adjusted to instruct the client to transmit the collected voice compression data based on the adjusted duration of the slice data stream.

[0019] Further, the outputting the first conversation text to the client, and updating the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain the second conversation text includes:

[0020] Sending the first conversation text to the client, and instructing the client to display the first conversation text based on the duration of the slice data stream;

[0021] After acquiring the second voice compression data transmitted by the client based on the second slice data stream, converting the second voice compression data into text to obtain a third conversation text;

[0022] If the similarity value between the third conversation text and the first conversation text is less than a preset similarity threshold, the first conversation text is updated based on the third conversation text to obtain a second conversation text;

[0023] If the similarity value between the third conversation text and the first conversation text is greater than or equal to a preset threshold, the first conversation text is overwritten based on the third conversation text to be determined as the second conversation text.

[0024] Furthermore, before searching for reply words corresponding to the question conversation text and processing the reply words based on the large language model that has completed model training to obtain the reply conversation text, the method further includes:

[0025] Obtaining a response training sample that matches the conversation scenario and creating an initial large language model, wherein the initial large language model is constructed based on a combination of a Gaussian mixture network and a large language network;

[0026] The initial large language model is trained based on the conversation training samples to obtain a trained large language model.

[0027] Further, outputting the reply conversation text to the client includes:

[0028] If a target voice object is selected, voice broadcast data of the reply conversation text is generated and output to the client.

[0029] According to another aspect of the present invention, a conversation output device based on a large language model is provided, comprising:

[0030] A conversion module, configured to obtain first voice compression data transmitted by a client based on a first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text;

[0031] An updating module, configured to output the first conversation text to the client, and update the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text;

[0032] The output module is used to search for reply words corresponding to the question conversation text if the second conversation text is a question conversation text, and process the reply words based on the large language model that has completed model training to obtain a reply conversation text, and output the reply conversation text to the client.

[0033] Furthermore, the conversion module is specifically used to obtain a preset hot word library, which stores multiple hot words pre-configured for conversation scenarios, and the hot words are used to represent words with the same pronunciation as non-conversation scenarios; perform text conversion on the voice data after decompression of the first voice compression data to obtain multiple reference text words; if the pronunciation of the reference text word matches the pronunciation of the target hot word in the preset hot word library, generate a first conversation text based on the hot word.

[0034] Furthermore, the conversion module is specifically used to obtain the hot word weight of each hot word in the preset hot word library; if there are multiple hot words, weighted processing is performed based on the hot word weight to generate the first conversation text.

[0035] Furthermore, the conversion module is specifically used to count the number of times the first voice compression data is cached in the cache space according to the length of the slice data stream; if the number of cache times is greater than a preset cache threshold, the length of the slice data stream is adjusted to instruct the client to transmit the collected voice compression data based on the adjusted length of the slice data stream.

[0036] Further, the update module is used to send the first conversation text to the client and instruct the client to display the first conversation text based on the duration of the sliced ​​data stream; after obtaining the second voice compression data transmitted by the client based on the second sliced ​​data stream, the second voice compression data is converted into text to obtain a third conversation text; if the similarity value between the third conversation text and the first conversation text is less than a preset similarity threshold, the first conversation text is updated based on the third conversation text to obtain a second conversation text; if the similarity value between the third conversation text and the first conversation text is greater than or equal to a preset threshold, the first conversation text is overwritten based on the third conversation text and determined as the second conversation text.

[0037] Furthermore, the device also includes:

[0038] An acquisition module is used to acquire response training samples matching the conversation scenario and create an initial large language model, wherein the initial large language model is constructed based on a combination of a Gaussian mixture network and a large language network;

[0039] The training module is used to perform model training on the initial large language model based on the conversation training sample to obtain a trained large language model.

[0040] Furthermore, the output module is specifically used to generate voice broadcast data of the reply conversation text if a target voice object is selected, and output the voice broadcast data to the client.

[0041] According to another aspect of the present invention, a storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to perform operations corresponding to the conversation output method based on a large language model as described above.

[0042] According to another aspect of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus;

[0043] The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the above-mentioned conversation output method based on a large language model.

[0044] By means of the above technical solution, the technical solution provided by the embodiment of the present invention has at least the following advantages:

[0045] The present invention provides a conversation output method and device based on a large language model, a storage medium, and a device. Compared with the prior art, the embodiment of the present invention obtains first voice compression data transmitted by a client based on a first slice data stream, and decompresses and converts the first voice compression data to obtain a first conversation text; outputs the first conversation text to the client, and updates the first conversation text based on second voice compression data obtained from a second slice data stream to obtain a second conversation text; if the second conversation text is a question conversation text, searches for reply words corresponding to the question conversation text, and processes the reply words based on a large language model that has completed model training to obtain a reply conversation text, and outputs the reply conversation text to the client, thereby achieving a more superior conversation learning ability, reducing the deviation in the conversation content generation and broadcasting task, so as to avoid avoiding unclear or irrelevant and useless conversation content in this application field, thereby improving the conversation generation accuracy and output effect.

[0046] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0048] Figure 1 A flow chart of a conversation output method based on a large language model provided by an embodiment of the present invention is shown;

[0049] Figure 2 A block diagram of a conversation output device based on a large language model provided by an embodiment of the present invention is shown;

[0050] Figure 3 A schematic structural diagram of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0051] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0052] The embodiments of the present invention can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0053] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0054] The embodiment of the present invention provides a conversation output method based on a large language model, such as Figure 1 As shown, the method includes:

[0055] 101. Acquire first voice compression data transmitted by a client based on a first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text.

[0056] In an embodiment of the present invention, the server end, which is the current execution subject, obtains the voice compression data collected from the client end by transmitting data stream with the client end, so as to execute the method in steps 101-103. Among them, after the client end collects the voice information entered by the user, it compresses the voice information, and transmits it as the first voice compression data to the current execution end in a slice streaming manner. At this time, the first slice data stream is the data stream obtained by transmitting the first voice compression data in a slice streaming manner. Furthermore, after obtaining the first voice compression data, the current execution end decompresses the first voice compression data and converts it into text to obtain the first conversation text. At this time, the first conversation text is the content converted from the voice content entered by the user into text, which can be implemented using ASR voice-to-text technology, and the embodiment of the present invention does not make specific limitations.

[0057] It should be noted that the current execution end as a server can be an independent server, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. In addition, in the embodiment of the present invention, the client and the server are used as a voice processing system, which is suitable for application scenarios in which voice is input and converted into text, including but not limited to Zen dialogue applications, knowledge question and answer applications, etc., which are not specifically limited in the embodiment of the present invention.

[0058] 102. Output the first conversation text to the client, and update the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text.

[0059] In the embodiment of the present invention, after the current execution end obtains the first conversation text, it outputs the conversation text to the client. At this time, the same transmission method is adopted, that is, the first conversation text is output to the client in a sliced ​​streaming manner. After the client receives the first conversation text, the client displays the first conversation text in the front-end interface. At this time, the user outputs the voice information on the client again, and after compression, the compressed second voice compression data is sent to the current execution end based on the second slice data stream. At this time, in order to make the first conversation text have a more fluent text expression form, the current execution end updates the first conversation text based on the second voice compression data to obtain the second conversation text.

[0060] It should be noted that, in an embodiment of the present invention, when the first conversation text is updated based on the second voice compression data, the second voice compression data can be first decompressed and converted into text content using ASR sound-to-text technology, so that the first conversation text is updated based on the decompressed text content. It can be an overwrite update or an adjustment update, and the embodiment of the present invention does not make specific limitations.

[0061] 103. If the second conversation text is a question conversation text, then a reply word corresponding to the question conversation text is searched, and the reply word is processed based on the large language model that has completed model training to obtain a reply conversation text, and the reply conversation text is output to the client.

[0062] In the embodiment of the present invention, after conversion to the second conversation text, whether the second conversation text is a text conversation text can be identified based on the pre-configured question word content or based on a machine learning algorithm, which is not specifically limited in the embodiment of the present invention. After identifying the question conversation text, the current execution end searches for reply words that are paired with the question conversation text, including but not limited to matching specific question words in the question conversation text to determine the reply words, and then, based on the large language model that has completed model training, the reply words are processed to generate a conversation text for reply, which is sent to the client for output.

[0063] It should be noted that when searching for reply words, the current execution end can use advanced RAG retrieval technology to perform block segmentation, vectorization, retrieval and other processing on the question conversation text to determine the reply words. In addition, in an embodiment of the present invention, a large language model (LLM for short), as a deep learning model trained on a large amount of text data, can generate natural language text or understand the meaning of language text based on demand.

[0064] The embodiment of the present invention provides a conversation output method based on a large language model, which obtains first voice compression data transmitted by a client based on a first slice data stream, decompresses and converts the first voice compression data, and obtains a first conversation text; outputs the first conversation text to the client, and updates the first conversation text based on second voice compression data obtained from a second slice data stream to obtain a second conversation text; if the second conversation text is a question conversation text, searches for reply words corresponding to the question conversation text, processes the reply words based on the large language model that has completed model training, obtains a reply conversation text, and outputs the reply conversation text to the client, thereby achieving a more superior conversation learning ability, reducing the deviation in the conversation content generation and broadcasting task, so as to avoid avoiding unclear or useless conversation content that is irrelevant to this application field, thereby improving the conversation generation accuracy and output effect.

[0065] In another embodiment of the present invention, for further definition and explanation, the step of decompressing and converting the first voice compression data to obtain the first conversation text includes:

[0066] Get the preset hot word library;

[0067] Performing text conversion on the voice data after decompressing the first voice compression data to obtain a plurality of reference text words;

[0068] If the pronunciation of the reference text word matches the pronunciation of the target hot word in the preset hot word library, a first conversation text is generated based on the hot word.

[0069] In order to make the conversation content more scene-specific when converting sound into words, thereby improving the adaptability to the application scenario, the current execution end first obtains the preset hot word library. Among them, the preset hot word library stores multiple hot words pre-configured for conversation scenarios. Hot words are used to represent words with the same pronunciation as non-conversational scenarios. For example, the conversational scenario is industrial problem consultation, and the hot word is "petroleum". The non-conversational scenario is other than industrial problem consultation. Words with the same pronunciation can be "roommate" or "cooking oil", etc., which are not specifically limited in the embodiment of the present invention. Furthermore, after receiving the first language compressed data, it is decompressed to obtain language data and perform text conversion. At this time, ASR sound-to-word technology can be used for conversion to obtain multiple reference text words, which are not specifically limited in the embodiment of the present invention. Specifically, after the sound-to-character conversion, the current execution end obtains multiple reference text words, and the obtained multiple text words are used as reference text words, and compared with the hot words in the preset hot word library. If the pronunciation of the reference text words matches the pronunciation of the target hot words in the preset hot word library, the first conversation text is generated based on the hot words. At this time, the reference text words can be pronounced and compared with the pronunciation of the target hot words in the hot word library. If they match, the target hot words are replaced with the reference text words to obtain the first conversation text. Among them, pronunciation recognition can establish a search space based on a trained HMM acoustic model, language model, pronunciation dictionary, etc., and match multiple pronunciations that can be used as references according to the search algorithm, so as to compare them according to the preset hot word pronunciations in the preset hot word library. The embodiment of the present invention does not make specific limitations.

[0070] In another embodiment of the present invention, for further definition and explanation, the step of generating a first conversation text based on the hot word includes:

[0071] Obtaining the hot word weight of each hot word in the preset hot word library;

[0072] If there are multiple hot words, weighted processing is performed based on the hot word weights to generate the first conversation text.

[0073] In order to make the conversation text obtained after the sound-to-character conversion more suitable for the application scenario and to improve the conversion effectiveness of the conversation text, the current execution end pre-configures the hot word weights for the hot words summarized in the preset hot word library based on different conversation scenarios, so as to weight and judge multiple hot words based on this weight to generate the first conversation text. Specifically, the preset hot word library is configured based on the conversation scenario, and at the same time, different hot words are pre-configured with a hot word weight between 0 and 1 based on the conversation requirements, so that when matching multiple hot words with the same pronunciation, in order to accurately determine a replaceable hot word, the hot words found are respectively calculated with the maximum weight value to select the most suitable hot word to generate the first conversation text.

[0074] For example, the weight values ​​of hot words a, b, and c are 0.3, 0.6, 0.2, etc., respectively, and the formula is used. Among them, x is the weight value, k is the quantitative representation of similar hot words, and y+z is used to represent the addition of the weight values ​​of hot words b and hot words c when calculating hot word a, that is, Then, the weighted result of hot word a is calculated. Similarly, the weighted results of hot words b and hot word c are calculated and compared, and the hot word corresponding to the largest weighted result is selected to generate the first conversation text.

[0075] In another embodiment of the present invention, for further definition and description, the step of obtaining first voice compression data transmitted by the client based on the first slice data stream includes:

[0076] Counting the number of times the first voice compression data is cached in the cache space according to the duration of the slice data stream;

[0077] If the number of cache times is greater than a preset cache threshold, the duration of the slice data stream is adjusted to instruct the client to transmit the collected voice compression data based on the adjusted duration of the slice data stream.

[0078] In order to speed up the transmission efficiency of voice data and thus improve the efficiency of voice-to-text conversion, when the current execution end transmits the first voice compression data, specifically, firstly, the number of cache times belonging to the first voice compression data in the cache space is counted according to the duration of the sliced ​​data stream. Among them, the client is pre-configured with a cache space for storing voice compression data. Each time the voice compression data transmitted by the sliced ​​data stream is received, it is cached in the cache space and the ID of the corresponding voice compression data is marked. The sliced ​​data stream is a data stream obtained by the client after collecting and compressing the voice and using the slice transmission method. The slice transmission party can set the duration, and when the duration is reached, the slice transmission is performed again to avoid the loss of voice compression data transmission. Therefore, the cache space will store multiple IDs for caching the first voice compression data, and then the number of cache times can be counted. The embodiment of the present invention does not make specific limitations.

[0079] It should be noted that after the current execution end counts the number of cache times, it is compared with the preset cache threshold. If the number of cache times is greater than the preset cache threshold, the duration of the slice data stream is adjusted to instruct the client to transmit the collected voice compression data based on the adjusted slice data stream duration. Specifically, when the client uses the slice data stream to transmit voice compression data, the number of cache times of the corresponding compressed package in the cache space is limited. For example, after exceeding 3 times, the slice data stream duration is readjusted to avoid data accumulation in the cache space, and the size of the compressed package is dynamically adjusted to achieve efficient and stable transmission. Among them, the adjustment of the slice data stream duration can be adjusted according to the size of the cache space in the cache space. For example, according to the accumulation of the cache pool, the duration of 200ms is adjusted to 1 second for a slice data stream transmission. The embodiment of the present invention does not make specific limitations.

[0080] In another embodiment of the present invention, for further definition and description, the step of outputting the first conversation text to the client, and updating the first conversation text based on the second voice compression data obtained from the second slice data stream, to obtain the second conversation text includes:

[0081] Sending the first conversation text to the client, and instructing the client to display the first conversation text based on the duration of the slice data stream;

[0082] After acquiring the second voice compression data transmitted by the client based on the second slice data stream, converting the second voice compression data into text to obtain a third conversation text;

[0083] If the similarity value between the third conversation text and the first conversation text is less than a preset similarity threshold, the first conversation text is updated based on the third conversation text to obtain a second conversation text;

[0084] If the similarity value between the third conversation text and the first conversation text is greater than or equal to a preset threshold, the first conversation text is overwritten based on the third conversation text to be determined as the second conversation text.

[0085] In order to realize the callback of the conversation content, to improve the recognition accuracy of the sound-to-character conversion, to avoid the conversion of homophones, etc., and to realize the recognition and correction of the intermediate result words based on the complete sentence, the current execution end first sends the first conversation text to the client, and instructs the client to display the first conversation text based on the slice data stream duration. After the current execution end determines the first conversation text, it sends it to the client. After the client receives the first conversation text, it displays the first conversation text in the front-end interface according to the slice data stream duration (or the adjusted slice data stream duration). Furthermore, the voice content entered by the user is re-collected in the client, and compressed to obtain the second language compressed data, which is uploaded to the current execution end in the form of the second slice data stream. At this time, the second slice data stream duration is the same as the first slice data stream duration, or the same as the adjusted first slice data stream duration, and the embodiment of the present invention does not make specific restrictions. After the current execution end obtains the second voice compression data transmitted by the client based on the second slice data stream, the second voice compression data is decompressed and converted into text to obtain the third conversation text. At this time, the same decompression and sound-to-character conversion technology as mentioned above is adopted, and the embodiment of the present invention does not make specific restrictions.

[0086] It should be noted that, since the user may repeat or adjust the recorded voice content when recording the voice, the current execution end compares the third conversation text of the voice-to-character conversion after receiving the user's recorded voice with the first conversation text. When the similarity value calculated between the third conversation text and the first conversation text is less than the preset similarity threshold, it means that the third conversation text is not related or similar to the first conversation text. Therefore, the current execution end updates the first conversation text based on the third conversation text to obtain the second conversation text. At this time, updating the first conversation text based on the third conversation text means directly changing the first conversation text according to the third conversation text, generating the second conversation text as a new conversation content, and outputting it. If the similarity value between the third conversation text and the first conversation text is greater than or equal to the preset threshold, it means that the third conversation text is related or similar to the first conversation text. Therefore, the current execution end covers the first conversation text based on the third conversation text and determines it as the second conversation text. At this time, the words in the third conversation text that are different from those in the first conversation text are replaced with the first conversation text to obtain the second conversation text, thereby achieving the purpose of callback for the error in the previous sentence conversation content.

[0087] In another embodiment of the present invention, for further definition and explanation, before the step of searching for reply words corresponding to the question conversation text and processing the reply words based on the large language model that has completed model training to obtain the reply conversation text, the method further includes:

[0088] Obtain response training samples that match the conversation scenario and create an initial large language model;

[0089] The initial large language model is trained based on the conversation training samples to obtain a trained large language model.

[0090] In order to achieve the purpose of automatic and effective conversation generation, after searching for the reply words, the current execution end uses the large language model to generate a reply conversation text that matches the reply words, and outputs it to the client for broadcast. Specifically, the current execution end first obtains the reply training samples that match the conversation scene, and creates an initial large language model at the same time. Among them, the initial large language model is constructed based on the combination of a Gaussian mixture network and a large language network. The reply training samples that match the conversation scene include but are not limited to special words in the field of Buddhism in the Zen master's dialogue scene, and the embodiment of the present invention does not make specific restrictions. Among them, the Gaussian mixture model is GMMs (Gaussian Mixture Models), which is used to calculate the text clustering probability, and the large language model is an LLM model. When performing model training, the reply training sample is a text vector (x), and its probability of belonging to the (k)th Gaussian distribution is expressed as: p(x|k)=N(x; μ k ,∑k), where N is a multivariate Gaussian distribution, μ k is the mean vector, ∑k is the covariance matrix, and the overall probability distribution is a weighted combination, expressed as π k Represents the mixed weight of the kth Gaussian distribution, so that the probability calculated based on the Gaussian distribution is input into the large language model as the text clustering probability in the large language model, and the corresponding reply conversation text is generated in combination with the reply words. The embodiment of the present invention does not make specific limitations.

[0091] It should be noted that, in the embodiment of the present invention, in order to improve the retrieval ability of reply words and increase the accuracy of generating reply conversation text, the current execution end adopts data preprocessing methods such as private domain data extraction, text segmentation, vectorization and data storage in the early data preparation stage. Among them, advanced RAG chunking (Chunking) and vectorization (Vectorisation) conversion technology can be used, and the reply words can be preprocessed in combination with retrieval, reranking and filtering. Among them, the chunking strategy of RAG (retrieval enhanced generation) refers to dividing a large text corpus into smaller, more manageable parts or paragraphs. Each block is an independent information unit and can be indexed and retrieved separately. The retrieval process is that the user inputs the question, and the server performs data retrieval, including similarity retrieval, that is, calculating the similarity score between the query vector and the stored vector, and returning the record with a high score. It is also possible to search by full-text search, for example, by building an inverted index by keywords, and searching for reply words by full-text search by keywords during retrieval. In addition, the most relevant knowledge retrieved can be added with prompt injection, including task description, background knowledge, task instructions, etc. Other instructions can be appropriately added according to the task scenario and the performance of the large model to optimize the final output of the large language model. At this time, the reply conversation text finally generated by the large language model LLM is called back to the client for output.

[0092] In the embodiment of the present invention, the knowledge limitation is broken by dividing RAG and the large language model LLM, and the static snapshot problem of the data set is solved. RAG can retrieve relevant information from various data sources. At the same time, the pre-trained large language LLM is further trained on a smaller and specific data set. For example, the conversation scene is a scene of a conversation with a Zen master, and the data set is concentrated in the field of Buddhism, which reduces the possibility of the general large model talking nonsense seriously, avoids the generation of conversation hallucinations, and avoids the answer bias of sensitive questions to a certain extent.

[0093] In another embodiment of the present invention, for further definition and explanation, the step of outputting the reply conversation text to the client comprises:

[0094] If a target voice object is selected, voice broadcast data of the reply conversation text is generated and output to the client.

[0095] In order to meet the flexibility adaptation requirements of different conversation scenarios, when the current execution end outputs the reply conversation text to the client, specifically, first determine whether there is a selected target voice object. At this time, the target voice object can be selected by the client user, or it can be pre-selected and configured by the server based on the conversation scenario, for example, the Zen master object in the Zen master dialogue scene, which is not specifically limited in the embodiment of the present invention. Further, when the target voice object is selected, the voice broadcast data of the reply conversation text is generated and sent to the client for broadcast. Among them, the client can use TTS voice broadcast technology when broadcasting. The delay of TTS voice broadcast is mainly concentrated in the case of semantic processing of long text. The large model can be called to use streaming SSE (Server-Sent Events) server to push data to the client in one direction. The client assembles the received reply conversation text into a single sentence with complete semantics. At the same time, a cache pool is set up to call TTS sentence by sentence for real-time voice broadcast to ensure that the processing delay of the large voice model is within 2 seconds, and the delay of TTS is negligible. The embodiment of the present invention does not make specific limitations. In addition, the broadcast timbre of the Zen master object can be the voice of the old monk trained in advance, and with an appropriate accent, so as to make the human-computer dialogue more immersive and achieve the purpose of realistic human-computer dialogue. In addition, the one-way push in the embodiment of the present invention refers to the one-way push of text from the server to the client after the client establishes a connection with the server, for example, a few words at a time. This one-way SSE method is lighter than traditional websocket or socket communication. Moreover, each push is a small part of the text (including punctuation), and the client establishes a buffer pool for text reception, which can be assembled into a sentence or a few sentences, and then sent to the TTS engine for audio synthesis and broadcasting in multiple times to reduce the delay of the large language model. In addition, TTS also has a word length limit for the text to be synthesized, which avoids the risk of word crossing the boundary, and can improve semantic understanding, so that the tone and tone of the synthesized audio are closer to real people.

[0096] It should be noted that in a specific implementation scenario, during the voice broadcast process, the user can interrupt the operation so that the user can re-enter the voice content on the client to form a data stream. Among them, for the scenario where the large language model outputs the reply conversation text for TTS voice broadcast, if the user does not want to continue listening to the TTS broadcast, he can directly speak to "interrupt" the broadcast. At this time, when the ASR recognizes the presence of voice, TTS stops broadcasting immediately. If the synthesized audio is not completed, all download steps will also stop downloading and enter a new round of dialogue. When there are multiple rounds of human-computer dialogues, the large voice model will understand according to the context to ensure the coherence of the content of the multi-round conversation and the accuracy of semantic understanding. When a single or multiple rounds of dialogue are completed, the dialogue ends and the dialogue record list result interface is reviewed and viewed, that is, the user can see all the texts of the human-computer dialogue in the list interface, and can also re-listen to the audio of both parties, which is helpful for data collection and feedback training of the training data set of the large language model using the review text.

[0097] An embodiment of the present invention provides another conversation output method based on a large language model, which obtains first voice compression data transmitted by a client based on a first slice data stream, and decompresses and converts the first voice compression data to obtain a first conversation text; outputs the first conversation text to the client, and updates the first conversation text based on second voice compression data obtained from a second slice data stream to obtain a second conversation text; if the second conversation text is a question conversation text, searches for reply words corresponding to the question conversation text, and processes the reply words based on the large language model that has completed model training to obtain a reply conversation text, and outputs the reply conversation text to the client, thereby achieving a more superior conversation learning ability, reducing the deviation in the conversation content generation and broadcasting task, so as to avoid avoiding unclear or useless conversation content that is irrelevant to this application field, thereby improving the conversation generation accuracy and output effect.

[0098] Furthermore, as a response to the above Figure 1 The embodiment of the present invention provides a conversation output device based on a large language model, such as Figure 2 As shown, the device comprises:

[0099] The conversion module 21 is used to obtain the first voice compression data transmitted by the client based on the first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text;

[0100] An updating module 22, configured to output the first conversation text to the client, and update the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text;

[0101] The output module 23 is used to search for reply words corresponding to the question conversation text if the second conversation text is a question conversation text, and process the reply words based on the large language model that has completed model training to obtain a reply conversation text, and output the reply conversation text to the client.

[0102] Furthermore, the conversion module is specifically used to obtain a preset hot word library, which stores multiple hot words pre-configured for conversation scenarios, and the hot words are used to represent words with the same pronunciation as non-conversation scenarios; perform text conversion on the voice data after decompression of the first voice compression data to obtain multiple reference text words; if the pronunciation of the reference text word matches the pronunciation of the target hot word in the preset hot word library, generate a first conversation text based on the hot word.

[0103] Furthermore, the conversion module is specifically used to obtain the hot word weight of each hot word in the preset hot word library; if there are multiple hot words, weighted processing is performed based on the hot word weight to generate the first conversation text.

[0104] Furthermore, the conversion module is specifically used to count the number of times the first voice compression data is cached in the cache space according to the length of the slice data stream; if the number of cache times is greater than a preset cache threshold, the length of the slice data stream is adjusted to instruct the client to transmit the collected voice compression data based on the adjusted length of the slice data stream.

[0105] Further, the update module is used to send the first conversation text to the client and instruct the client to display the first conversation text based on the duration of the sliced ​​data stream; after obtaining the second voice compression data transmitted by the client based on the second sliced ​​data stream, the second voice compression data is converted into text to obtain a third conversation text; if the similarity value between the third conversation text and the first conversation text is less than a preset similarity threshold, the first conversation text is updated based on the third conversation text to obtain a second conversation text; if the similarity value between the third conversation text and the first conversation text is greater than or equal to a preset threshold, the first conversation text is overwritten based on the third conversation text and determined as the second conversation text.

[0106] Furthermore, the device also includes:

[0107] An acquisition module is used to acquire response training samples matching the conversation scenario and create an initial large language model, wherein the initial large language model is constructed based on a combination of a Gaussian mixture network and a large language network;

[0108] The training module is used to perform model training on the initial large language model based on the conversation training sample to obtain a trained large language model.

[0109] Furthermore, the output module is specifically used to generate voice broadcast data of the reply conversation text if a target voice object is selected, and output the voice broadcast data to the client.

[0110] The embodiment of the present invention provides a conversation output device based on a large language model, which obtains first voice compression data transmitted by a client based on a first slice data stream, and decompresses and converts the first voice compression data to obtain a first conversation text; outputs the first conversation text to the client, and updates the first conversation text based on second voice compression data obtained from a second slice data stream to obtain a second conversation text; if the second conversation text is a question conversation text, searches for reply words corresponding to the question conversation text, and processes the reply words based on the large language model that has completed model training to obtain a reply conversation text, and outputs the reply conversation text to the client, so as to achieve a more superior conversation learning ability, reduce the deviation in the conversation content generation and broadcasting task, so as to avoid avoiding unclear or useless conversation content that is irrelevant to the application field, thereby improving the conversation generation accuracy and output effect.

[0111] According to an embodiment of the present invention, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the conversation output method based on a large language model in any of the above method embodiments.

[0112] Figure 3 A schematic diagram of the structure of a computer device provided according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0113] like Figure 3 As shown, the computer device may include: a processor 302 , a communication interface 304 , a memory 306 , and a communication bus 308 .

[0114] The processor 302 , the communication interface 304 , and the memory 306 communicate with each other via the communication bus 308 .

[0115] The communication interface 304 is used to communicate with other devices such as clients or other servers.

[0116] The processor 302 is used to execute the program 310, and specifically can execute the relevant steps in the above embodiment of the conversation output method based on the large language model.

[0117] Specifically, the program 310 may include program codes, which include computer operation instructions.

[0118] The processor 302 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0119] The memory 306 is used to store the program 310. The memory 306 may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory.

[0120] The program 310 may be specifically configured to enable the processor 302 to perform the following operations:

[0121] Acquire first voice compression data transmitted by the client based on the first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text;

[0122] Outputting the first conversation text to the client, and updating the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text;

[0123] If the second conversation text is a question conversation text, the reply words corresponding to the question conversation text are searched, and the reply words are processed based on the large language model that has completed model training to obtain a reply conversation text, and the reply conversation text is output to the client.

[0124] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0125] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A conversation output method based on a large language model, characterized in that: include: Acquire first voice compression data transmitted by the client based on the first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text; Outputting the first conversation text to the client, and updating the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text; If the second conversation text is a question conversation text, the reply words corresponding to the question conversation text are searched, and the reply words are processed based on the large language model that has completed model training to obtain a reply conversation text, and the reply conversation text is output to the client.

2. The method according to claim 1, characterized in that The decompressing and converting the first voice compression data to obtain the first conversation text comprises: Acquire a preset hot word library, wherein the preset hot word library stores a plurality of hot words pre-configured for conversation scenarios, and the hot words are used to represent words with the same pronunciation as non-conversation scenarios; Performing text conversion on the voice data after decompressing the first voice compression data to obtain a plurality of reference text words; If the pronunciation of the reference text word matches the pronunciation of the target hot word in the preset hot word library, a first conversation text is generated based on the hot word.

3. The method according to claim 2, characterized in that Generating the first conversation text based on the hot word includes: Obtaining the hot word weight of each hot word in the preset hot word library; If there are multiple hot words, weighted processing is performed based on the hot word weights to generate the first conversation text.

4. The method according to claim 1, characterized in that: The acquiring first voice compression data transmitted by the client based on the first slice data stream includes: Counting the number of times the first voice compression data is cached in the cache space according to the duration of the slice data stream; If the number of cache times is greater than a preset cache threshold, the duration of the slice data stream is adjusted to instruct the client to transmit the collected voice compression data based on the adjusted duration of the slice data stream.

5. The method according to claim 1, characterized in that The step of outputting the first conversation text to the client and updating the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain the second conversation text comprises: Sending the first conversation text to the client, and instructing the client to display the first conversation text based on the duration of the slice data stream; After acquiring the second voice compression data transmitted by the client based on the second slice data stream, converting the second voice compression data into text to obtain a third conversation text; If the similarity value between the third conversation text and the first conversation text is less than a preset similarity threshold, the first conversation text is updated based on the third conversation text to obtain a second conversation text; If the similarity value between the third conversation text and the first conversation text is greater than or equal to a preset threshold, the first conversation text is overwritten based on the third conversation text to be determined as the second conversation text.

6. The method according to claim 1, characterized in that Before searching for reply words corresponding to the question conversation text and processing the reply words based on the large language model that has completed model training to obtain the reply conversation text, the method further includes: Obtaining a response training sample that matches the conversation scenario and creating an initial large language model, wherein the initial large language model is constructed based on a combination of a Gaussian mixture network and a large language network; The initial large language model is trained based on the conversation training samples to obtain a trained large language model.

7. The method according to any one of claims 1 to 6, characterized in that: Outputting the reply conversation text to the client comprises: If a target voice object is selected, voice broadcast data of the reply conversation text is generated and output to the client.

8. A conversation output device based on a large language model, characterized in that: include: A conversion module, configured to obtain first voice compression data transmitted by a client based on a first slice data stream, and decompress and convert the first voice compression data to obtain a first conversation text; An updating module, configured to output the first conversation text to the client, and update the first conversation text based on the second voice compression data obtained from the second slice data stream to obtain a second conversation text; The output module is used to search for reply words corresponding to the question conversation text if the second conversation text is a question conversation text, and process the reply words based on the large language model that has completed model training to obtain a reply conversation text, and output the reply conversation text to the client.

9. A storage medium storing at least one executable instruction, wherein the executable instruction enables a processor to perform operations corresponding to the conversation output method based on a large language model as described in any one of claims 1 to 7.

10. A computer device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the conversation output method based on a large language model as described in any one of claims 1-7.

Images