Credit auditing verbal skill automatic generation system and credit auditing question and answer method

Abstract

The invention discloses an auditing verbal skill automatic generation system, and the system comprises a collection unit which is used for collecting an answer of a credit auditing user for a current question, converting the voice number of the answer into text data, and recording the text data in a credit auditing question and answer text; an input unit extracts a dialogue text of a credit review user in a previous specified round from a credit review question and answer text, sequentially traverses the dialogue text based on a sliding window with the size of k, inputs k character strings in the current sliding window into the trained Qwen2-7B model, outputs prediction characters and prediction probabilities thereof by the trained Qwen2-7B model, and outputs the predicted characters and prediction probabilities of the predicted characters after traversal of the dialogue text; and outputting the character string with the highest prediction probability as a question of the next round to a credit review user, and recording the question in a credit review question and answer text. According to the method, a Qwen2-72B large model is trained through historical dialogues between a credit auditor and a credit auditing user, a Qwen2-72B model is trained by means of the Qwen2-72B large model, a next round of questions can be automatically put forward on line based on answers of the credit auditing user through the Qwen2-7B model, and the flexibility of the automatic question-asking verbal skill is greatly improved.

Description

Technical Field

[0001] The present invention belongs to the field of credit review technology, and more specifically, the present invention relates to a credit review speech automatic generation system and a credit review question and answer method. Background Art

[0002] The current customer groups in the credit market are becoming increasingly diverse, including individuals from different industries, professional backgrounds, economic conditions and credit histories. The traditional conversational robot's speech system and its corresponding speech are based on experience, and can only ask targeted questions about some key issues and industries with high proportions. This limitation makes it possible that the speech may not effectively cover all customer groups in actual applications, especially when facing emerging markets and special customer groups.

[0003] In addition, in-depth questions in the credit review dialogue (Q&A) are crucial to understanding the client's qualifications. As the customer base gradually moves down, more low-income or incomplete credit record customers will be exposed. The qualifications of these customers are different from those of high-end customers, and more detailed and differentiated language is needed to explore their qualifications and provide a basis for approval.

[0004] However, traditional conversational robots often lack the ability to ask in-depth questions based on customer answers. They can only ask questions that follow a fixed process, and their overall dialogue is rigid and lacks specificity. Summary of the invention

[0005] The present invention provides a system for automatically generating speech for letter review, aiming to improve the above-mentioned problem.

[0006] The present invention is implemented as follows: a system for automatically generating a letter review script, the system comprising:

[0007] An acquisition unit, an input unit connected to the acquisition unit, and a Qwen2-7B model connected to the input unit;

[0008] A collection unit is used to collect the answers of the credit review user to the current question, convert the voice data of the answer into text data and record it in the credit review question and answer text;

[0009] The input unit extracts the dialogue text of the credit review user in the previous specified round from the credit review question and answer text, traverses the dialogue text in sequence based on a sliding window of size k, inputs the k character strings in the current sliding window into the trained Qwen2-7B model, and the trained Qwen2-7B model outputs the predicted characters and their predicted probabilities. After completing the traversal of the dialogue text, the character string with the highest prediction probability is output to the credit review user as the question for the next round and recorded in the credit review question and answer text.

[0010] Furthermore, the online Qwen2-7B model is trained with the help of the offline trained Qwen2-72B model.

[0011] Furthermore, the training process of the Qwen2-7B model is as follows:

[0012] Connect the offline trained Qwen2-72B model with the Qwen2-7B model to be trained;

[0013] The sample data are respectively input into the trained Qwen2-72B model and the Qwen2-7B model to be trained, and the predicted characters and predicted probabilities output by the Qwen2-72B model are input into the Qwen2-7B model to train the Qwen2-7B model.

[0014] The sample data uses the dialogue text of the credit review user in the previous designated round in the credit review process as features, and the next round of questions given by the credit reviewer to the credit review user as labels.

[0015] Furthermore, the sample data construction process is as follows:

[0016] The review call data during the review question and answer process is recorded, and the voice data in the recording is converted into text data. The conversation text of the review user in the previous specified round is extracted from the text data as the sample feature, and the next round of questions given by the reviewer is used as the sample label. The sample data is constructed based on the above method.

[0017] Furthermore, the model loss language_loss(θ 72B ) are as follows:

[0018] language_loss(θ 72B )=Σ i logP 72B (s i |c i ,...,c i+k-1 θ 72B );

[0019] Among them, P 72B (s i |c i ,...,c i+k-1 θ 72B ) means that the string c in the sample feature i ,...,c i+k-1 The input model parameters are θ 72B Qwen2-72B model, Qwen2-72B model output character s i The predicted probability of character s i is the i-th character in the label.

[0020] Furthermore, the training process is as follows:

[0021] (1) Construct a sliding window with a window size of k, and the sliding window starts traversing from the first character in the sample feature;

[0022] (2) The k characters c in the current sliding window i ,...,c i+k-1 and its corresponding label character s i Input Qwen2-72B model, based on model loss language_loss(θ 72B ) Adjust the model parameters of the Qwen2-72B model;

[0023] (3) The sliding window moves backward one character and executes step (2) until the last character in the entire sample feature is traversed.

[0024] Furthermore, the loss function of the Qwen2-7B model is as follows:

[0025] final_loss(θ 7B )=α*language_loss(θ 7B )+(1-α)*teacher_loss(θ 7B );

[0026] language_loss(θ 7B )=∑ i logP 7B (s i| c i ,...,c i+k-1 θ 7B );

[0027] teacher_loss(θ 7B )=KL(P 7B (s i| c i ,...,c i+k-1 θ 7B ),P 72B (s i |c i ,...,c i+k-1 θ 72B ));

[0028] Among them, α is the weight value, KL(P 7B (s i| c i ,...,c i+k-1 θ 7B ),P 72B (s i|c i ,...,c i+k-1 θ 72B )) represents the probability P 7B (s i |c i ,...,c i+k-1 θ 7B ) and probability KL divergence of 7 means to convert the string c in the sample feature i ,...,c i+k-1 The input model parameters are θ 7B The Qwen2-7B model outputs the character s i The probability, P 72B (s i |c i ,...,c i+k-1 θ 72B ) means to convert the string c i ,...,c i+k-1 The input model parameters are θ 72B Qwen2-72B model, Qwen2-72B model output character s i The probability of character s i is the i-th character in the label.

[0029] The present invention is implemented as follows: a letter review question and answer method of the above-mentioned letter review speech automatic generation system, the method is specifically as follows:

[0030] (1) The conversational robot reads fixed questions from the question list in sequence, converts the fixed questions into speech, and sends them to the credit review user;

[0031] (2) After receiving the answer from the credit review user, execute step (1) until the credit review user answers the last question in the question list;

[0032] (3) The automatic generation system of credit review speech outputs the next round of questions based on the historical questions and answers between the dialogue robot and the credit review user, converts them into voice data, and sends them to the credit review user;

[0033] (4) After receiving the answer from the credit review user, execute step (3) until the number of questions generated by the credit review script automatic generation system reaches the number threshold, and the speech robot sends a closing statement to the credit review user, terminating the credit review question and answer process.

[0034] Furthermore, the answers given by the credit review user to the questions output by the speech robot are converted into text data, and the answers are recorded together with the robot's questions in the credit review question and answer text in the form of questions and answers.

[0035] The present invention uses the historical conversations between credit reviewers and credit review users to train the Qwen2-72B large model, and then uses the Qwen2-72B large model to train the Qwen2-7B model. The Qwen2-7B model can automatically raise the next round of questions online based on the credit review user's answers, greatly improving the flexibility of automatic questioning techniques. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A schematic diagram of the structure of a system for automatically generating letter review speech provided by an embodiment of the present invention;

[0037] Figure 2 A flowchart of a letter review question and answer method based on an automatic letter review speech generation system provided in an embodiment of the present invention. Implementation

[0039] The specific implementation modes of the present invention are further explained in detail below by describing the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0040] The present invention uses Qwen2-72B and Qwen2-7B models as basic models and uses sample data for training. The Qwen2 model is based on the Transformer architecture, which processes text-type sequence data through a self-attention mechanism, can perform parallel calculations, effectively improves training efficiency, and can capture long-distance dependencies. It is pre-trained on massive Internet texts. This large-scale data volume helps the model learn rich language patterns and knowledge, and it demonstrates excellent capabilities in multiple fields such as dialogue understanding, text generation, code understanding and generation.

[0041] Figure 1 The structural diagram of the automatic generation system of the letter review speech provided by the embodiment of the present invention, for the convenience of explanation, only the part related to the embodiment of the present invention is shown, and the system includes:

[0042] An acquisition unit, an input unit connected to the acquisition unit, and a Qwen2-7B model connected to the input unit;

[0043] A collection unit is used to collect the answers of the credit review user to the current question, convert the voice data of the answer into text data and record it in the credit review question and answer text;

[0044] The input unit extracts the dialogue text of the credit review user in the previous specified round from the credit review question and answer text, traverses the dialogue text in sequence based on a sliding window of size k, inputs the k character strings in the current sliding window into the trained Qwen2-7B model, and the trained Qwen2-7B model outputs the predicted characters and their predicted probabilities. After completing the traversal of the dialogue text, the character string with the highest prediction probability is output to the credit review user as the question for the next round and recorded in the credit review question and answer text.

[0045] In an embodiment of the present invention, at the starting moment, the first question is first asked based on the starting identifier. After receiving the answer to the first question from the credit review user, the first round of dialogue is completed. The second question is generated based on the first round of dialogue. After receiving the answer to the second question from the credit review user, the second round of dialogue is completed. Then, the third question is generated based on the first two rounds of dialogue. After receiving the answer to the third question from the credit review user, the third round of dialogue is completed. Then, the fourth question is generated based on the second and third rounds of dialogue, and so on.

[0046] In an embodiment of the present invention, the online Qwen2-7B model is trained with the help of the offline trained Qwen2-72B model. The Qwen2-72B model (large model) has excellent performance, but its huge model structure causes it to take a long time to output a round of questions. In the scenario of online voice calls, users have to wait for a long time, which reduces the user experience. Therefore, the Qwen2-72B model can only be used offline. The Qwen2-7B model (small model) is used as the model for online use. Through knowledge distillation, it has performance close to that of the Qwen2-72B model, but greatly reduces the inference time.

[0047] In the embodiment of the present invention, the training process of the Qwen2-7B model is as follows:

[0048] (1) Construct sample data and train the Qwen2-72B model based on the sample data. The sample data uses the conversation text of the credit reviewer in the previous designated round during the credit review process as the sample features, and the next round of questions given by the credit reviewer as the sample label;

[0049] The review call data in the review question and answer process is recorded, and the voice data in the recording is converted into text data. The dialogue text of the review user in the previous specified round is extracted from the text data as the sample feature, and the next round of questions given by the reviewer is used as the sample label. The sample data is constructed based on the above method. The sample data is divided into training samples and test samples. The Qwen2-72B model is trained based on the training samples, and the Qwen2-72B model is tested based on the test samples until the prediction accuracy of the Qwen2-72B model reaches the set standard, and the training of the Qwen2-72B model is completed.

[0050] In the embodiment of the present invention, the Qwen2-72B model adjusts the model parameter θ based on the model loss during the training process. 72B , the model loss is calculated based on the loss function, the loss function of the Qwen2-72B model is language_loss(θ 72B ) are as follows:

[0051] language_loss(θ 72B )=∑ i logP 72B (s i| c i , … ,c i+k-1 θ 72B );

[0052] Among them, P 72B (s i |c i ,...,c i+k-1 θ 72B ) means that the string c in the sample feature i ,...,c i+k-1 The input model parameters are θ 72B Qwen2-72B model, Qwen2-72B model output character s i The predicted probability of character s i is the i-th character in the label. The training process is as follows:

[0053] (1) Construct a sliding window with a window size of k, which starts from the first character in the sample feature; (2) The k characters c in the current sliding window are traversed. i ,...,c i+k-1 and its corresponding label character s i Input Qwen2-72B model, based on model loss language_loss(θ 72B ) for the model parameter θ of the Qwen2-72B model 72BMake adjustments; (3) the sliding window moves back one character and executes step (2) until the last character in the entire sample feature is traversed, and the Qwen2-72B model completes the learning of the current sample.

[0054] It should be noted that after converting the voice data into text data, it is also necessary to perform operations such as text sentence correction, text terminology standardization and merging of similar sentences on the converted text data. Based on the above operations, a batch of high-quality credit review call data texts are obtained, and sample data is constructed based on the credit review call data texts for dialogue training of the large model Qwen2-72B model.

[0055] (2) After the Qwen2-72B model is trained, the offline trained Qwen2-72B model is connected to the Qwen2-7B model to be trained;

[0056] (3) The sample data are respectively input into the trained Qwen2-72B model and the Qwen2-7B model to be trained, and the predicted characters and their predicted probabilities output by the Qwen2-72B model are input into the Qwen2-7B model to train the Qwen2-7B model.

[0057] In the embodiment of the present invention, the input of the Qwen2-7B model consists of two parts. The first part is the historical questions and answers of the credit reviewer and the credit review user, and the second part is the predicted answers and their probabilities output by the Qwen2-72B model for the historical questions and answers of the credit reviewer and the credit review user. Therefore, its loss function final_los(sθ 7B ) also consists of two parts, as follows:

[0058] final_loss(θ 7B )=α*language_loss(θ 7B )+(1-α)*teacher_loss(θ 7B );

[0059] language_loss(θ 7B )=Σ i logP 7B (s i |c i ,…,c i+k-1 θ 7B );

[0060] teacher_loss(θ 7B )=KL(P 7B (s i |c i ,...,c i+k-1 θ 7B),P 72B (s i |c i ,...,c i+k-1 θ 72B ));

[0061] Among them, α is the weight value, KL(P 7B (s i |c i ,...,c i+k-1 θ 7B ),P 72B (s i |c i ,…,c i+k-1 θ 72B )) represents the probability P 7B (s i |c i ,...,c i+k-1 θ 7B ) and probability KL divergence of 7 means to convert the string c in the sample feature i , … ,c i+k-1 The input model parameters are θ 7B The Qwen2-7B model outputs the character s i The probability, P 72B (s i |c i ,...,c i+k-1 θ 72B ) means to convert the string c i ,...,c i+k-1 The input model parameters are θ 72B Qwen2-72B model, Qwen2-72B model output character s i The probability of character s i is the i-th character in the label. The training process of the Qwen2-7B model is as follows: (1) construct a sliding window with a window size of k, which starts from the first character in the sample feature; (2) the k characters c in the current sliding window are traversed. i ,...,c i+k-1 and its corresponding label character s i Input the trained Qwen2-72B model and the Qwen2-7B model to be trained, and adjust the model parameters θ of the Qwen2-7B model based on the model loss. 7B Make adjustments; (3) the sliding window moves back one character and executes step (2) until the last character in the entire sample feature is traversed, and the Qwen2-7B model completes the learning of the current sample.

[0062] Teacher_loss learns the probability distribution of the Qwen2-72B model output. This probability distribution is more suitable for model generalization than the true learning result in language_loss learning, and can also learn the similarity between different characters.

[0063] The present invention uses the above-mentioned automatic generation system of credit review speech as a supplement to the dialogue robot. The robot only needs to ask some fixed questions. When the fixed process questions are finished, it starts to call the above-mentioned automatic generation system of credit review speech to ask questions based on the credit review user's answers to complete the dialogue with the credit review user.

[0064] Figure 2 A flowchart of a letter review question and answer method based on a letter review speech automatic generation system provided in an embodiment of the present invention, the method is specifically as follows:

[0065] (1) The conversational robot reads fixed questions from the question list in sequence, converts the fixed questions into speech, and sends them to the credit review user;

[0066] (2) After receiving the answer from the credit review user, execute step (1) until the credit review user answers the last question in the question list;

[0067] (3) The automatic generation system of credit review speech outputs the next round of questions based on the historical questions and answers between the dialogue robot and the credit review user, converts them into voice data, and sends them to the credit review user;

[0068] (4) After receiving the answer from the credit review user, execute step (3) until the number of questions generated by the credit review script automatic generation system reaches the number threshold, and the dialogue robot sends a closing message to the credit review user, terminating the credit review question and answer process.

[0069] In an embodiment of the present invention, the answer given by the credit review user to the question output of the speech robot is converted into text data, and the question of the robot is recorded in the form of question and answer in the credit review question and answer text, as shown in Table 1. The model in Table 1 is the credit review speech automatic generation system. After starting the credit review speech automatic generation system, the credit review speech automatic generation system extracts the first k character strings.

[0070] Table 1 Examples of letter review question and answer texts

[0071]

[0072]

[0073] The automatic generation system of credit review scripts provided by the present invention uses the historical conversations between credit reviewers and credit review users to train the Qwen2-72B large model, and then uses the Qwen2-72B large model to train the Qwen2-7B model. The Qwen2-7B model can automatically raise the next round of questions online based on the credit review user's answers, greatly improving the flexibility of automatic questioning scripts.

[0074] The present invention has been described exemplarily. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A system for automatically generating credit review scripts, characterized in that: The system comprises: An acquisition unit, an input unit connected to the acquisition unit, and a Qwen2-7B model connected to the input unit; A collection unit is used to collect the answers of the credit review user to the current question, convert the voice data of the answer into text data and record it in the credit review question and answer text; The input unit extracts the dialogue text of the credit review user in the previous specified round from the credit review question and answer text, traverses the dialogue text in sequence based on a sliding window of size k, inputs the k character strings in the current sliding window into the trained Qwen2-7B model, and the trained Qwen2-7B model outputs the predicted characters and their predicted probabilities. After completing the traversal of the dialogue text, the character string with the highest prediction probability is output to the credit review user as the question for the next round and recorded in the credit review question and answer text.

2. The automatic generation system for credit review scripts as claimed in claim 1, characterized in that: The online Qwen2-7B model is trained with the help of the offline trained Qwen2-72B model.

3. The automatic generation system of credit review scripts as claimed in claim 1, characterized in that: The training process of the Qwen2-7B model is as follows: Connect the offline trained Qwen2-72B model with the Qwen2-7B model to be trained; The sample data are respectively input into the trained Qwen2-72B model and the Qwen2-7B model to be trained, and the predicted characters and predicted probabilities output by the Qwen2-72B model are input into the Qwen2-7B model to train the Qwen2-7B model. The sample data uses the dialogue text of the credit review user in the previous designated round in the credit review process as features, and the next round of questions given by the credit reviewer to the credit review user as labels.

4. The automatic generation system for credit review scripts as claimed in claim 3, characterized in that: The sample data construction process is as follows: The review call data during the review question and answer process is recorded, and the voice data in the recording is converted into text data. The conversation text of the review user in the previous specified round is extracted from the text data as the sample feature, and the next round of questions given by the reviewer is used as the sample label. The sample data is constructed based on the above method.

5. The automatic generation system of credit review scripts as claimed in claim 3, characterized in that: The model loss of Qwen2-72B model is language_loss(θ 72B ) are as follows: language_loss(θ 72B )=∑ i logP 72B (s i |c i ,...,c i+k-1 ;θ 72B ); Among them, P 72B (s i |c i , …, c i+k-1 θ 72B ) means that the string c in the sample feature i , ..., c i+k-1 The input model parameters are θ 72B Qwen2-72B model, Qwen2-72B model output character s i The predicted probability of character s i is the i-th character in the label.

6. The automatic generation system of credit review scripts as claimed in claim 3, characterized in that: The training process is as follows: (1) Construct a sliding window with a window size of k, and the sliding window starts traversing from the first character in the sample feature; (2) The k characters c in the current sliding window i , ..., c i+k-1 and its corresponding label character s i Input Qwen2-72B model, based on model loss language_loss(θ 72B ) Adjust the model parameters of the Qwen2-72B model; (3) The sliding window moves backward one character and executes step (2) until the last character in the entire sample feature is traversed.

7. The automatic generation system for credit review scripts as claimed in claim 3, characterized in that: The loss function of the Qwen2-7B model is as follows: final_loss(θ 7B )=α*language_loss(θ 7B )+(1-α)*teacher_loss(θ 7B ); language_loss(θ 7B )=∑ i logP 7B (s i |c i ,...,c i+k-1 ;θ 7B ); teacher_loss(θ 7B )=KL(P 7B (s i |c i ,...,c i+k-1 ;θ 7B ),P 72B (s i |c i ,...,c i+k-1 ;θ 72B )); Among them, α is the weight value, KL(P 7B (s i |c i , ..., c i+k-1 θ 7B ), P 72B (s i |c i , ..., c i+k-1 θ 72B )) represents the probability P 7B (s i |c i , …, c i+k-1 θ 7B ) and probability P 72B (s i |c i , …, c i+k-1 θ 72B )’s KL divergence; P 7B (s i |c i , …, c i+k-1 θ 7B ) means that the string c in the sample feature i , ..., c i+k-1 The input model parameters are θ 7B The Qwen2-7B model outputs the character s i The probability, P 72B (s i |c i , …, c i+k-1 θ 72B ) means to convert the string c i , ..., c i+k-1 The input model parameters are θ 72B Qwen2-72B model, Qwen2-72B model output character s i The probability of character s i is the i-th character in the label.

8. A credit review question-answering method based on the credit review speech automatic generation system according to any one of claims 1 to 7, characterized in that: The method is specifically as follows: (1) The conversational robot reads fixed questions from the question list in sequence, converts the fixed questions into speech, and sends them to the credit review user; (2) After receiving the answer from the credit review user, execute step (1) until the credit review user answers the last question in the question list; (3) The automatic generation system of credit review speech outputs the next round of questions based on the historical questions and answers between the dialogue robot and the credit review user, converts them into voice data, and sends them to the credit review user; (4) After receiving the answer from the credit review user, execute step (3) until the number of questions generated by the credit review script automatic generation system reaches the number threshold, and the dialogue robot sends a closing message to the credit review user, terminating the credit review question and answer process.

9. The information questioning and answering method according to claim 8, characterized in that: The answers given by the credit review user to the questions output by the dialogue robot are converted into text data, and recorded in the credit review question and answer text in the form of questions and answers together with the robot's questions.

Images