Voice session processing method and device based on large model, equipment and medium

Abstract

The invention relates to the technical field of insurance and natural language processing, and discloses a voice session processing method and device based on a large model, equipment and a medium, and the method comprises the steps: carrying out the fine adjustment of a pre-trained insurance large model through an instruction fine adjustment data set generated according to the historical record data of a user, the trained prediction model has pertinence to the user; the trained prediction model is utilized to generate a response text verbal skill for the user voice verbal skill of the user in a streaming mode, in the process of generating the response text verbal skill in the streaming mode, the response text verbal skill is cut into response text short sentences to be sequentially stored, and in the process of sequentially storing the response text short sentences, the response text short sentences are asynchronously and sequentially processed into response voice short sentences to be output. The reply speed is increased, the reply accuracy is improved, the fluency of man-machine interaction is improved, and the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of insurance and natural language processing technology, and in particular to a voice conversation processing method, device, equipment and medium based on a large model. Background Art

[0002] With the development of the times, scientific progress and people's increased risk awareness, insurance as an economic security system has been greatly developed. With the rapid development of artificial intelligence technology, human-computer interactive conversation systems have entered people's daily lives more and more. Therefore, artificial intelligence insurance conversation systems came into being.

[0003] The use of artificial intelligence insurance conversation system can realize intelligent human-computer insurance conversation. However, the existing conversation system based on generative model uses general samples to train the model, which leads to insufficient targeting for users and low accuracy of reply texts. Due to the different lengths of reply texts, longer reply texts will consume longer generation time, resulting in customers waiting for the conversation system during the human-computer interaction process, making the human-computer interaction unsmooth.

[0004] Therefore, there is an urgent need for a voice conversation processing method that can effectively improve the accuracy and fluency of human-computer interaction. Summary of the invention

[0005] The present invention provides a voice conversation processing method, device, equipment and medium based on a large model. By fine-tuning the pre-trained insurance large model, a prediction model suitable for the user is obtained, and the user's voice words are accurately replied through streaming output, thereby improving the reply accuracy, improving the fluency of human-computer interaction, and thus improving the user experience.

[0006] In a first aspect, a method for processing a speech conversation based on a large model is provided, the method comprising: Obtain the user's historical record data, generate an instruction fine-tuning dataset based on the historical record data, and use the instruction fine-tuning dataset to fine-tune the pre-trained insurance model to obtain a trained prediction model; Obtain the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate the response text speech in a streaming manner; In the process of streaming the response text speech, the response text speech is sequentially cut into at least one response text short sentence, and the at least one response text short sentence is sequentially stored; As at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted.

[0007] In a second aspect, a large model-based voice conversation processing device is provided, the device comprising: The model fine-tuning unit is used to obtain the user's historical record data, generate an instruction fine-tuning data set according to the historical record data, and use the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model; A response text speech generation unit is used to obtain the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate the response text speech in a streaming manner; A response text short sentence cutting unit is used to cut the response text speech into at least one response text short sentence in sequence during the process of generating the response text speech in a streaming manner, and to store at least one response text short sentence in sequence; The answering voice sentence output unit is used to store at least one answering text sentence, sequentially retrieve at least one answering text sentence, convert it into an answering voice sentence and output it, and delete the outputted answering text sentence.

[0008] In a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned large model-based voice conversation processing method when executing the computer program.

[0009] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned large model-based voice conversation processing method are implemented.

[0010] In the scheme implemented by the above-mentioned big model-based voice conversation processing method, device, equipment and medium, the user's historical record data is obtained, an instruction fine-tuning data set is generated according to the historical record data, and the pre-trained insurance big model is fine-tuned using the instruction fine-tuning data set to obtain a trained prediction model; the user's voice speech is received, the user voice speech is converted into user text speech, and then input into the trained prediction model for streaming generation of response text speech; in the process of streaming generation of response text speech, the response text speech is cut into at least one response text sentence in sequence, and at least one response text sentence is stored in sequence; as at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted.

[0011] In the present invention, the pre-trained insurance model is fine-tuned using the instruction fine-tuning data set, so that the trained prediction model is targeted at users, thereby improving the accuracy of the reply; the trained prediction model is used to generate the reply text speech in a streaming manner, and the reply text is stored in sequence in the form of short sentences, and is asynchronously processed in sequence during the sequential storage process, which greatly improves the reply speed. The method of the present invention improves the fluency of human-computer interaction and improves the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0013] Figure 1 It is a schematic diagram of an application environment of a voice conversation processing method based on a large model in one embodiment of the present invention; Figure 2 It is a flowchart of a method for processing a voice conversation based on a large model in one embodiment of the present invention; Figure 3 It is a structural diagram of a voice conversation processing device based on a large model in one embodiment of the present invention; Figure 4 is a schematic diagram of the structure of a computer device in one embodiment of the present invention; Figure 5 It is a schematic diagram of the structure of a computer device in another embodiment of the present invention. DETAILED DESCRIPTION

[0014] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0015] The following describes the implementation scenario of the embodiment of the present invention.

[0016] In the insurance field, in order to sell and expand insurance business, insurance companies are usually required to actively communicate with users. Active communication may include but is not limited to: calling users using SIM cards, calling users using instant messaging software, etc.

[0017] In traditional forms, when insurance companies actively communicate with users, they use humans to conduct conversations with users. However, with the development of artificial intelligence, artificial intelligence insurance conversation systems can be used to replace humans to achieve interactive communication with users.

[0018] Therefore, the voice conversation processing method based on the large model provided by the embodiment of the present invention can be applied in Figure 1When the insurance company calls the user and the call is connected, a network communication connection between the server and the client is formed. The server represents the insurance company, which can be equipped with an artificial intelligence insurance conversation system; the client represents the user.

[0019] The client can be, but is not limited to, a personal computer, a laptop, a smart phone, a tablet computer, a portable wearable device, or other devices with voice reception and input devices. The server can be implemented by a server or a service platform consisting of multiple servers.

[0020] The server can be equipped with an artificial intelligence insurance conversation system, which obtains the user's historical record data, generates an instruction fine-tuning data set based on the historical record data, and uses the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model; receives the user's voice speech from the client, converts the user's voice speech into user text speech, and then inputs the trained prediction model to stream generate response text speech; in the process of streaming generation of response text speech, cuts the response text speech into at least one response text sentence in sequence, and stores at least one response text sentence in sequence; as at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence and output to the client, and the output response text sentence is deleted.

[0021] The present invention is described in detail below through specific embodiments.

[0022] See also Figure 2 As shown, Figure 2 A flow chart of a voice conversation processing method based on a large model provided in an embodiment of the present invention is provided. The flow chart is executed by a server and includes the following steps: Step S210, obtaining the user's historical record data, generating an instruction fine-tuning data set according to the historical record data, and using the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model.

[0023] Since the embodiment of the present invention is applied to the active communication scenario of the insurance company in the insurance field, the prediction model for the user can be trained on the server side before establishing a communication connection with the client. Take the outbound call scenario in which the server uses the SIM card to make a call to the client as an example. After the server determines the client that needs to establish a communication connection, it determines the user that needs to establish a communication connection based on the phone number information, SIM card information, etc.

[0024] The server obtains the user's historical record data, which may be text data. The historical record data may include, but is not limited to, the user's historical insurance data and the user's historical session data, etc. The user's historical insurance data may further include, but is not limited to, the user's personal information data, the user's historical insurance data, the user's browsing preference data, etc. The server may obtain the user's historical record data from the insurance company's own database, or obtain the user's historical record data from public channels within the scope of authority.

[0025] After obtaining the historical record data, the server generates an instruction fine-tuning dataset based on the historical record data. The instruction fine-tuning dataset includes multiple instruction fine-tuning training samples, each of which includes the following three elements: Command input, used to provide text input for a given task; Hint information is used to provide specific information or context clues, which may include but are not limited to: task summary, problem description, mapping relationship between input and output, supplementary information, etc. The expected output is used to provide the target output for a given task as the ground truth. The pre-trained large language model is evaluated and optimized for fine-tuning based on this ground truth.

[0026] By instructing fine-tuning the data set, the server fine-tunes the pre-trained insurance model to obtain a trained prediction model.

[0027] Input the instruction fine-tuning dataset into the pre-trained insurance model for supervised fine-tuning, thereby obtaining a user-specific prediction model and improving the accuracy of the response; Step S220, receiving the user's voice speech, converting the user's voice speech into user text speech, and then inputting the trained prediction model to generate the response text speech in a streaming manner.

[0028] After the trained prediction model is prepared, the server can send an active communication to the client to establish a communication connection between the server and the client. After the communication connection is established, the server receives the user's voice speech and converts the user's voice speech into user text speech.

[0029] Using speech-to-text technology, the server converts the received user speech into a text sequence of user text speech, and generates the response text speech through the user text speech using the trained prediction model. The response text speech is generated in a streaming manner. That is, the response text speech is generated in a step-by-step manner, rather than generating the entire text content of the response text speech at one time.

[0030] Existing generative models are usually autoregressive generative methods, that is, given a series of observed word sequences, the goal of the generative model is to predict the next word in the sequence. This process can be viewed as a sequence of conditional probabilities, in which the probability of occurrence of each word depends on all the words before it. Depending on the historical context, the length of the reply text generated by the model is different. The longer the reply text, the more time-consuming it is, which causes the user to wait in human-computer interaction. The present invention uses a trained prediction model to stream-generate response text speech, which can improve the generation efficiency of response text speech and further improve the fluency of human-computer conversation.

[0031] Step S230, in the process of streaming generation of response text speech, the response text speech is cut into at least one response text short sentence in sequence, and at least one response text short sentence is stored in sequence.

[0032] In the present invention, although the response text speech is generated in a streaming manner, when storing, neither all the streaming generated results are stored nor only the final response text speech is stored. Instead, at least one response text short sentence is stored based on the long sentence cutting short sentence rule.

[0033] That is, in the process of streaming the response text speech, the process of storing at least one response text short sentence is as follows: Start generating streaming results. Once the rule for cutting the answer text short sentences is met, store the generated streaming results as an answer text short sentence. Continue to generate subsequent streaming results, and once the rule for cutting the answer text short sentence is met again, store the subsequently generated streaming results as the next answer text short sentence; Until the tail-end streaming result is generated, the generated tail-end streaming result is stored as the last response text sentence.

[0034] That is, the process of storing at least one short answer text sentence and the process of generating the short answer text words in a streaming manner are performed simultaneously. When the short answer text words are generated in a streaming manner, the storage of at least one short answer text sentence is completed. The streaming result is stored in the form of a short answer text sentence, so that the short answer text sentence can be retrieved and processed later.

[0035] Step S240, as at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted.

[0036] As at least one response text sentence is stored, the server retrieves the response text sentences in sequence according to the storage order, converts them into response voice sentences for output, and deletes the response text sentences that have been output.

[0037] That is to say, the process of storing at least one short answer text sentence and the process of retrieving at least one short answer text sentence can be performed simultaneously, but the stored and retrieved short answer text sentences are different. Even if the streaming generation of the short answer text words, the storage of at least one short answer text sentence and the processing of at least one short answer text sentence can be performed asynchronously, the fluency of human-computer interaction is greatly improved.

[0038] from Figure 2 It can be seen from the method shown that the voice conversation processing method based on the big model provided by the present invention obtains the historical record data of the user, generates an instruction fine-tuning data set according to the historical record data, and uses the instruction fine-tuning data set to fine-tune the pre-trained insurance big model to obtain a trained prediction model; receives the user's voice speech, converts the user's voice speech into user text speech, and then inputs the trained prediction model to generate the response text speech in a streaming manner; in the process of generating the response text speech in a streaming manner, the response text speech is sequentially cut into at least one response text short sentence, and at least one response text short sentence is sequentially stored; as at least one response text short sentence is stored, at least one response text short sentence is sequentially retrieved and converted into a response voice short sentence output, and the output response text short sentence is deleted. The present invention uses the instruction fine-tuning data set to fine-tune the pre-trained insurance big model, so that the trained prediction model has pertinence to the user, thereby improving the accuracy of the reply; the present invention uses the trained prediction model to generate the response text speech in a streaming manner, and stores it in the form of response text short sentences in sequence, and asynchronously processes it in sequence during the sequential storage process, which greatly improves the speed of the reply; the method of the present invention improves the fluency of human-computer interaction and improves the user experience.

[0039] In some optional embodiments, in the above method, step S210, obtaining the user's historical record data, generating an instruction fine-tuning data set based on the historical record data, and using the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model, includes: obtaining the user's historical insurance data and historical session data; generating input instructions based on the historical insurance data, generating prompt information based on the historical session data, and generating expected outputs based on the historical insurance data and historical session data; forming an instruction fine-tuning data set based on the input instructions, prompt information and expected outputs, and inputting the instruction fine-tuning data set into the pre-trained insurance model for fine-tuning to obtain a trained prediction model.

[0040] The server can obtain the user's historical insurance data and historical session data. The user's historical insurance data may include, but is not limited to: the user's personal information data, such as the user's age, gender, occupation, etc.; the user's historical insurance data, such as the user's insurance purchase record, the user's insurance claim record, etc.; the user's browsing preference data, such as the web pages the user has browsed within a certain period of time, etc. The user's historical session data may include, but is not limited to: the text of the telephone recording between the user and the insurance customer service, the online question and answer record between the user and the insurance customer service, etc.

[0041] After acquiring the historical insurance data and historical session data, an input instruction is generated according to the historical insurance data, a prompt message is generated according to the historical session data, and an expected output is generated according to the historical insurance data and the historical session data. The server can automatically generate input instructions according to the historical insurance data, generate prompt messages according to the historical session data, and generate expected outputs according to the historical insurance data and the historical session data; or the server can provide the historical insurance data and the historical session data to the staff so that the staff can manually arrange the input instructions, prompt messages, and expected outputs, thereby generating input instructions, prompt messages, and expected outputs. In some cases, the server can combine the two methods, automatically generate an input instruction through the historical insurance data, automatically generate a prompt message through the historical session data, provide the historical insurance data and the historical session data to the staff so that the staff can manually arrange an expected output, thereby forming the input instruction, prompt message, and expected output into an instruction fine-tuning training sample, and in this way, form multiple instruction fine-tuning training samples as instruction fine-tuning data sets.

[0042] For example: By obtaining user A's personal information data, historical insurance data, browsing preference data, etc., the server can generate the following input command: "Suppose you are an experienced insurance salesperson who sells relevant insurance products over the phone with given user-related information and can reasonably answer customers' questions about the products. The customer's name is user A, gender is female, date of birth is February 2, 2000, 23 years old this year, address is Xinjiang, has 1 insurance policy, and completed a medical insurance B for me on June 30, 2023. Social insurance information is unknown, and I have recently browsed medical insurance B." User A’s historical conversation data can be used to transcribe a recording of a phone call between the user and insurance customer service: “Insurance customer service: Hello, is this user A? User: Hi, where are you? Insurance Customer Service: Hello, ma’am. I’m the insurance follow-up specialist. I’m calling you because I saw that you completed an insurance policy for yourself on June 30th. Do you still remember Medical Insurance B? User: Um, what? Insurance Customer Service: Yes, because we have both electronic and paper policies here. Have you received both? User: No, I don’t know that.

[0043] Insurance customer service: No, um, you tell me, okay.

[0044] User: Yes, where is the electronic insurance policy? Insurance Customer Service: Has your service specialist added your WeChat through the official corporate WeChat before? User: This is a lie, right? This section of historical session data is used as prompt information.

[0045] Through historical insurance data and historical session data, the following expected output is compiled: "Oh, you can rest assured about the authenticity. The company is a listed company and the content is true and reliable."

[0046] Thus, the input instruction, prompt information, and expected output are formed into an instruction fine-tuning training sample. Based on multiple instruction fine-tuning training samples, an instruction fine-tuning data set is formed, and the instruction fine-tuning data set is input into the pre-trained insurance large model for fine-tuning to obtain a trained prediction model.

[0047] In some optional embodiments, in the above method, the insurance big model is pre-trained by the following method: obtaining insurance field data, preprocessing the insurance field data to obtain sample data; constructing a basic model, and passing the sample data to the basic model; calculating the predicted output of the basic model through forward propagation, calculating the error value between the predicted output and the actual result through the loss function, calculating the gradient of the basic model parameters through back propagation, and updating the basic model according to the gradient calculation results, repeating the iteration until the error value is less than the preset value, and obtaining the pre-trained insurance big model.

[0048] The following two steps are required to obtain a trained prediction model: the first is to use sample data to train the basic model to obtain a pre-trained insurance model; the second is to use the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model.

[0049] The above has explained the process of fine-tuning the pre-trained insurance model using the instruction fine-tuning dataset. The following describes the process of pre-training the basic model using sample data.

[0050] The server obtains insurance data and preprocesses it to obtain sample data. The server first obtains insurance data. Insurance data may include, but is not limited to, text information such as product descriptions, claim terms, insurance contracts, claim cases, and insurance Q&A. After obtaining insurance data, the server preprocesses the insurance data to obtain sample data. Preprocessing may include, but is not limited to, data cleaning and word embedding.

[0051] After obtaining the sample data, the server builds a basic model and passes the sample data to the basic model. The basic model can be, but is not limited to, a Seq2Seq model or a Transformer model. The parameters of the basic model are initialized, and the initialization can include, but is not limited to, initialization settings of weights, biases, memory, etc. After the basic model is built, the server passes the sample data to the basic model for pre-training of the basic model.

[0052] During the forward propagation process, sample data is passed through each layer of the neural network until a predicted output is obtained. The predicted output is compared with the actual result to calculate the value of the loss function. During the backpropagation process, the gradient of the loss function with respect to each weight is calculated (i.e., starting from the output layer, backpropagating along the hierarchy of the network to the input layer, and calculating the gradient of each weight). These gradients represent the contribution of the weight to the loss function and can therefore guide the update of the weight. After calculating the gradients of all weights, the weights are updated using an optimization algorithm. During this process, the weights are adjusted according to the gradient value to reduce the value of the loss function. After repeated iterations, until the error value is less than the preset value (i.e., the loss function is minimized), the pre-trained insurance model is obtained.

[0053] In addition, before the pre-trained insurance big model is deployed and applied, the performance of the pre-trained insurance big model can also be evaluated. The evaluation may include, but is not limited to, the following aspects: evaluating the conversation generation quality of the pre-trained insurance big model, evaluating the long context processing capability of the pre-trained insurance big model, etc. If the performance evaluation result of the pre-trained insurance big model reaches the expected level, the pre-trained insurance big model can be deployed for fine-tuning using the instruction fine-tuning dataset.

[0054] The present invention trains the basic model through sample data generated by insurance field data to form a pre-trained insurance model; fine-tunes the pre-trained insurance model through the instruction fine-tuning data set generated by the user's historical record data to form a trained prediction model. The trained prediction model is targeted at users and can provide users with more accurate responses, thereby improving the user experience.

[0055] In some optional embodiments, in the above method, step S220, receiving the user's voice speech, converting the user's voice speech into user text speech, and then inputting the trained prediction model to stream generate the response text speech, includes: receiving the user's voice speech, converting the user's voice speech into raw text data; segmenting the raw text data to form a user text speech of a text sequence; inputting the user text speech into the trained prediction model to stream generate the response text speech.

[0056] After receiving the user's voice speech, the server converts the user's voice speech into original text data through the voice-to-text function. For example, through the voice-to-text function, the original text data "Yes, where is the electronic insurance policy?" is obtained.

[0057] The server performs sentence segmentation processing on the original text data and forms a text sequence. Since the original text data is all listed in Chinese characters, it needs to be segmented and formed into a text sequence. For example: "Yes, where is the electronic insurance policy?" is segmented into "Yes, where is the electronic insurance policy?", and "Yes, where is the electronic insurance policy?" is formed into a text sequence to obtain "Yes, / electronic insurance policy / where / is? / ". It should be noted that " / " is not an actual symbol, but a separator inserted to indicate the formation of a text sequence.

[0058] After the server inputs the user's text into the trained prediction model, the prediction model generates the response text in a streaming manner.

[0059] For example, after the user text phrase "Is this a scam?" is input into the trained prediction model, the following results are output in streaming format: oh\ oh,\ Oh, authenticity. Oh, the authenticity of you\ Oh, authenticity you can totally \ Oh, you can rest assured of the authenticity. Oh, you can rest assured of the authenticity, the company\ Oh, you can rest assured of the authenticity, the company is\ Oh, you can rest assured of the authenticity, the company is a listed company\ Oh, you can rest assured of the authenticity, the company is a listed company,\ Oh, you can rest assured of the authenticity, the company is a listed company, the content\ Oh, you can rest assured about the authenticity, the company is a listed company and the content is true\ Oh, you can rest assured about the authenticity, the company is a listed company, and the content is true and reliable\ Oh, you can rest assured about the authenticity, the company is a listed company and the content is true and reliable. \” It should be noted that "\" is not an actual symbol, but a separator inserted to indicate the flow-generated response text dialogue.

[0060] In addition, after the server establishes a communication connection with the client, before the server obtains the user's user voice speech, the server can first provide the client with an opening speech to greet the user, thereby improving the user's connection experience. The opening speech can be preset and fixed, and does not need to be directly output in response to the user sending a voice. For example, when the server establishes a communication connection with the client, it first outputs the opening speech of "Hello, is this user A?" to the client, and then responds to the user voice speech sent by the user, converts the user voice speech into user text speech, and then inputs the trained prediction model to stream generate the response text speech.

[0061] In some optional embodiments, in the above method, step S230, in the process of streaming generation of response text speech, the response text speech is cut into at least one response text short sentence in sequence, and at least one response text short sentence is stored in sequence, including: in the process of streaming generation of response text speech, when the end punctuation is not generated, if the transition punctuation is generated and the number of characters not stored before the transition punctuation reaches a preset number, the transition punctuation and the characters not stored before the transition punctuation are stored as a response text short sentence; when the end punctuation is generated, the end punctuation and the characters not stored before the end punctuation are stored as a response text short sentence.

[0062] The above process is described below by taking an example. In this example, the preset number is 5.

[0063] During the streaming output of results: When outputting "Oh, you can be assured of the authenticity," the rule of cutting the short sentences of the answer text is satisfied. Then "Oh, you can be assured of the authenticity," is stored as the first short sentence of the answer text; When the output is "Oh, you can be assured of the authenticity, the company is a listed company," the rule for cutting the answer text short sentences is met. Then "The company is a listed company," is stored as the second answer text short sentence; When the output is "Oh, you can be assured of the authenticity. The company is a listed company and the content is authentic and reliable.", the rule of cutting the short answer text sentences is met. Then "The content is authentic and reliable." is stored as the third short answer text sentence.

[0064] That is, along with the streaming output results, three short response text sentences, "Oh, you can be assured of the authenticity," "The company is a listed company," and "The content is authentic and reliable," are stored in sequence.

[0065] In some optional embodiments, in the above method, step S240, as at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted, including: once at least one response text sentence begins to be stored, at least one response text sentence is sequentially retrieved in the order of storage; the retrieved response text sentence is converted into a response voice sentence and then output; the output response text sentence is deleted until at least one stored response text sentence is empty.

[0066] The above process is explained below by taking an example.

[0067] Once the first response text sentence "Oh, you can be assured of the authenticity" is stored, the first response text sentence can be retrieved. The first response text sentence is converted into a first response voice sentence using text-to-speech technology and output to the client. After being output to the client, the stored first response text sentence is deleted.

[0068] After the first answer text sentence is stored, regardless of how it is processed (retrieved, text-to-speech, output and deleted), the result is continuously streamed and stored: the second answer text sentence "The company is a listed company," and the third answer text sentence "The content is authentic and reliable."

[0069] After the first answer text sentence is processed and the second answer text sentence is stored and ready, the second answer text sentence is processed. After the second answer text sentence is processed and the third answer text sentence is stored and ready, the third answer text sentence is processed.

[0070] After all converted response voice sentences are output in sequence, there are no stored response text sentences.

[0071] The above process of the present invention does not need to wait for the trained prediction model to generate a complete response text before storage, nor does it need to wait for the complete response text to be stored before retrieval, text-to-speech conversion and output. Instead, the response text short sentences are stored and processed in real time during the process of the trained prediction model streaming output results. The response speed is greatly improved, the fluency of human-computer interaction is improved, and the user experience is better.

[0072] In some optional real-time methods, the above method also includes: as at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted: if new user voice speech begins to be received, the new user voice speech is cached in real time; after all the response voice sentences are output, if new user voice speech is not continuously received, the new user voice speech is converted into new user text speech and then input into the trained prediction model to stream generate new response text speech; if new user voice speech is continuously received, the new user voice speech is received after the new user voice speech is received. Afterwards, convert the new user voice words into new user text words and then input the trained prediction model to stream generate new response text words; and / or, after the step of obtaining the user's voice words, in the process before the step of sequentially calling at least one response text short sentence to convert it into a response voice short sentence output, and deleting the output response text short sentence, if the new user voice words are started to be received, the current process is stopped immediately, the new user voice words are continuously received, and after the new user voice words are received, the new user voice words are converted into new user text words and then input the trained prediction model to stream generate new response text words.

[0073] When the server and the client are connected in communication, the human-computer interaction can be divided into the following states: 1. The server outputs at least one response voice sentence to the client in sequence, 2. The client outputs user voice scripts to the server, 3. Waiting for the server to output at least one response voice sentence in sequence, 4. Waiting for the client to output user voice scripts without timeout, 5. Waiting for the client to output user voice scripts after timeout.

[0074] The above-mentioned states are described below respectively.

[0075] State 1: The server outputs at least one short answering voice sentence to the client in sequence. In this state, the server is sequentially retrieving at least one short answering voice sentence and outputting it to the client.

[0076] State 2: The client outputs the user voice script to the server. In this state, the user of the client is outputting the user voice script to the server.

[0077] State 3, waiting for the server to output at least one answering voice sentence in sequence. In this state, the server is in the process of obtaining the user's voice speech, sequentially calling at least one answering text sentence to convert it into an answering voice sentence and outputting it, and deleting the outputted answering text sentence.

[0078] State 4: Waiting for the client to output the user voice speech has not timed out. In this state, after the server has no response text speech to be output, the time for the client not to output the user voice speech to the server does not exceed the preset time.

[0079] State 5: Timeout waiting for the client to output the user voice speech. In this state, after the server has no response text speech to be output, the client has not output the user voice speech to the server for a period of time exceeding the preset time. When this state occurs, the server can provide the client with a connection speech in the hope that the user will continue to output the user voice speech, thereby continuing the conversation.

[0080] Ideally, there is no overlap between the above five states.

[0081] After state 1 ends, the system enters state 4. In state 4, if the user speaks, the system enters state 2. If the user does not speak, the system enters state 5. After state 2 ends, the system enters state 3. After state 3 ends, the system enters state 1.

[0082] In special cases, there is overlap among the above five states.

[0083] If state 2 occurs in state 1: the user's voice words are cached in real time. After state 1 ends, determine whether state 2 continues. If state 2 ends, enter state 3. If state 2 continues, continue to cache the user's voice words until state 2 ends, and enter state 3.

[0084] If state 2 occurs in state 3: state 3 is stopped immediately and the stored content of this round of prediction is discarded, the user's voice speech is cached in real time until state 2 ends, and then state 3 is entered.

[0085] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0086] In one embodiment, a voice conversation processing device based on a large model is provided, and the device can be deployed in a server. The voice conversation processing device based on a large model corresponds to the voice conversation processing method based on a large model in the above embodiment. Figure 3 As shown, the voice conversation processing device based on the large model includes: The model fine-tuning unit 310 is used to obtain the historical record data of the user, generate an instruction fine-tuning data set according to the historical record data, and use the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model; The answering text speech generating unit 320 is used to obtain the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate the answering text speech in a streaming manner; The response text short sentence cutting unit 330 is used to cut the response text speech into at least one response text short sentence in sequence during the process of generating the response text speech in a streaming manner, and to store the at least one response text short sentence in sequence; The response voice sentence output unit 340 is used to sequentially retrieve at least one response text sentence and convert it into a response voice sentence for output along with at least one response text sentence stored, and delete the output response text sentence.

[0087] In some optional embodiments, in the above-mentioned device 300, the model fine-tuning unit 310 is specifically used to: obtain the user's historical insurance data and historical session data; generate input instructions based on the historical insurance data, generate prompt information based on the historical session data, and generate expected outputs based on the historical insurance data and historical session data; form an instruction fine-tuning data set based on the input instructions, prompt information and expected outputs, and input the instruction fine-tuning data set into the pre-trained insurance large model for fine-tuning to obtain a trained prediction model.

[0088] In some optional embodiments, in the above-mentioned device 300, the model fine-tuning unit 310 is also specifically used to: obtain insurance field data, pre-process the insurance field data to obtain sample data; construct a basic model, and pass the sample data to the basic model; calculate the predicted output of the basic model through forward propagation, calculate the error value between the predicted output and the actual result through the loss function, calculate the gradient of the parameters of the basic model through back propagation, and update the parameters of the basic model according to the gradient calculation results, repeat the iteration until the error value is less than the preset value, and obtain the pre-trained insurance large model.

[0089] In some optional embodiments, in the above-mentioned device 300, the response text speech generation unit 320 is specifically used to: obtain the user's voice speech, convert the user's voice speech into original text data; segment the original text data to form a user text speech in a text sequence; input the user's text speech into a trained prediction model to stream-generate the response text speech.

[0090] In some optional embodiments, in the above-mentioned device 300, the response text sentence cutting unit 330 is specifically used to: in the process of streaming generation of response text speech, when the end punctuation is not generated, if the transition punctuation is generated and the number of characters not stored before the transition punctuation reaches a preset number, the transition punctuation and the characters not stored before the transition punctuation are stored as a response text sentence; when the end punctuation is generated, the end punctuation and the characters not stored before the end punctuation are stored as a response text sentence.

[0091] In some optional embodiments, in the above-mentioned device 300, the response voice sentence output unit 340 is specifically used to: once at least one response text sentence begins to be stored, at least one response text sentence is sequentially retrieved in the order of storage; the retrieved response text sentence is converted into a response voice sentence and then output; the output response text sentence is deleted until at least one stored response text sentence is empty.

[0092] In some optional embodiments, the above-mentioned device 300 also includes: a real-time caching unit, which is used to: during the steps of storing at least one response text sentence, sequentially retrieving at least one response text sentence to convert it into a response voice sentence output, and deleting the output response text sentence: if a new user voice speech starts to be received, the new user voice speech is cached in real time; after all the response voice sentences are output, if the new user voice speech is not continuously received, the caching ends; if the new user voice speech is continuously received, the caching ends after the new user voice speech is received.

[0093] In some optional embodiments, the above-mentioned device 300 also includes: a switching unit, which is used to: after the step of obtaining the user's voice speech, in the process before the step of sequentially calling at least one response text sentence to convert it into a response voice sentence output and deleting the output response text sentence: if new user voice speech is started to be received, the current process is stopped immediately, and the new user voice speech is continuously received until the new user voice speech is received.

[0094] For the specific definition of the voice conversation processing device based on the large model, please refer to the definition of the voice conversation processing method based on the large model above, which will not be repeated here. The various modules in the above-mentioned voice conversation processing device based on the large model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0095] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the server side of a voice conversation processing method based on a large model.

[0096] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen, an input device and an audio transmission device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps on the client side of a voice conversation processing method based on a large model.

[0097] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: Obtain the user's historical record data, generate an instruction fine-tuning dataset based on the historical record data, and use the instruction fine-tuning dataset to fine-tune the pre-trained insurance model to obtain a trained prediction model; Obtain the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate the response text speech in a streaming manner; In the process of streaming the response text speech, the response text speech is sequentially cut into at least one response text short sentence, and the at least one response text short sentence is sequentially stored; As at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted.

[0098] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Obtain the user's historical record data, generate an instruction fine-tuning dataset based on the historical record data, and use the instruction fine-tuning dataset to fine-tune the pre-trained insurance model to obtain a trained prediction model; Obtain the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate the response text speech in a streaming manner; In the process of streaming the response text speech, the response text speech is sequentially cut into at least one response text short sentence, and the at least one response text short sentence is sequentially stored; As at least one response text sentence is stored, at least one response text sentence is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted.

[0099] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0100] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0101] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0102] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for processing speech conversation based on a large model, characterized in that: The method comprises: Acquire the user's historical record data, generate an instruction fine-tuning data set according to the historical record data, and use the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model; Acquire the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate a response text speech in a streaming manner; In the process of streaming the response text speech, the response text speech is sequentially cut into at least one response text short sentence, and at least one response text short sentence is sequentially stored; As at least one of the response text sentences is stored, at least one of the response text sentences is sequentially retrieved and converted into a response voice sentence for output, and the output response text sentence is deleted.

2. The method for processing speech conversation based on a large model as claimed in claim 1, characterized in that: The acquiring of the historical record data of the user, generating an instruction fine-tuning data set according to the historical record data, and fine-tuning the pre-trained insurance model using the instruction fine-tuning data set to obtain a trained prediction model, includes: Obtain the user's historical insurance data and historical session data; generating an input instruction according to the historical insurance data, generating prompt information according to the historical session data, and generating an expected output according to the historical insurance data and the historical session data; The instruction fine-tuning data set is formed according to the input instruction, the prompt information and the expected output, and the instruction fine-tuning data set is input into the pre-trained insurance large model for fine-tuning to obtain the trained prediction model.

3. The method for processing speech conversation based on a large model according to claim 1 or 2, characterized in that: The insurance big model is pre-trained by the following method: Acquire insurance field data, and pre-process the insurance field data to obtain sample data; Building a basic model, and passing the sample data to the basic model; The predicted output of the basic model is calculated through forward propagation, the error value between the predicted output and the actual result is calculated through the loss function, the gradient of the parameters of the basic model is calculated through back propagation, and the parameters of the basic model are updated according to the gradient calculation results. Repeat the iteration until the error value is less than the preset value to obtain the pre-trained insurance model.

4. The method for processing speech conversation based on a large model as claimed in claim 1, characterized in that: The step of obtaining the user's voice speech, converting the user's voice speech into user text speech, and then inputting the trained prediction model to generate the response text speech in a streaming manner includes: Acquire the user's voice speech, and convert the user's voice speech into original text data; The user text speech is formed into a text sequence after segmenting the original text data; The user text speech is input into the trained prediction model to stream-generate the response text speech.

5. The method for processing speech conversation based on a large model as claimed in claim 1, characterized in that: In the process of streaming the response text speech, the response text speech is sequentially cut into at least one response text short sentence, and at least one response text short sentence is sequentially stored, including: In the process of generating the response text speech in a streaming manner, When the end punctuation is not generated, if a transition punctuation is generated and the number of characters not stored before the transition punctuation reaches a preset number, the transition punctuation and the characters not stored before the transition punctuation are stored as a short sentence of the response text; When the end punctuation mark is generated, the end punctuation mark and the characters not stored before the end punctuation mark are stored as a short sentence of the response text.

6. The method for processing speech conversation based on a large model according to claim 1, characterized in that: The step of storing at least one of the response text sentences, sequentially retrieving at least one of the response text sentences, converting the response text sentences into response voice sentences and outputting the response text sentences, and deleting the output response text sentences includes: Once at least one of the response text sentences begins to be stored, at least one of the response text sentences is retrieved in sequence according to the storage order; Convert the retrieved response text short sentence into the response voice short sentence and then output it; The outputted response text short sentences are deleted until at least one stored response text short sentence is empty.

7. The method for processing speech conversation based on a large model according to claim 1, characterized in that: The method further comprises: In the process of storing at least one of the response text sentences, sequentially retrieving at least one of the response text sentences and converting them into response voice sentences for output, and deleting the output response text sentences: If new user voice speech is started to be received, the new user voice speech is cached in real time; after all the response voice sentences are output, if the new user voice speech is not continuously received, the new user voice speech is converted into new user text speech and then input into the trained prediction model to generate new response text speech in a streaming manner; if the new user voice speech is continuously received, after the new user voice speech is completely received, the new user voice speech is converted into new user text speech and then input into the trained prediction model to generate new response text speech in a streaming manner; and / or, After the step of obtaining the user's voice speech, in the process before the step of sequentially calling at least one of the response text sentences to convert into a response voice sentence and outputting, and deleting the output response text sentence: If the new user voice speech is started to be received, the current process is stopped immediately, and the new user voice speech is continuously received. After the new user voice speech is completely received, the new user voice speech is converted into the new user text speech and then input into the trained prediction model to stream-generate the new response text speech.

8. A speech conversation processing device based on a large model, characterized in that: The device comprises: A model fine-tuning unit, used to obtain historical record data of the user, generate an instruction fine-tuning data set according to the historical record data, and use the instruction fine-tuning data set to fine-tune the pre-trained insurance model to obtain a trained prediction model; A response text speech generating unit is used to obtain the user's voice speech, convert the user's voice speech into user text speech, and then input the trained prediction model to generate the response text speech in a streaming manner; A response text short sentence cutting unit is used to cut the response text speech into at least one response text short sentence in sequence during the process of streaming generation of the response text speech, and to store at least one response text short sentence in sequence; The answering voice sentence output unit is used to sequentially retrieve at least one of the answering text sentences and convert them into answering voice sentences for output along with at least one of the answering text sentences being stored, and to delete the answering text sentences that have been output.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the large model-based voice conversation processing method as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the large model-based voice conversation processing method as described in any one of claims 1 to 7 are implemented.

Images