Training and usage methods of user intention decision model

By training and verifying the user intention decision model of bank equipment, the accuracy and efficiency problems of bank equipment when dealing with complex user intentions are solved, local instant response and resource conservation are achieved, and user experience and service efficiency are improved.

CN118503712BActive Publication Date: 2024-12-03JINKE INTELLIGENT TECHNOLOGY INNOVATION CENTER (TIANJIN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410953913.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-12-03
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

Existing banking equipment is inaccurate and efficient when dealing with complex or fuzzy user intentions, and relying on cloud computing leads to latency and privacy leakage risks, cloud computing resources are consumed and costs are increased.

Method used

By dividing the pre-acquisitioned sample dataset into training business dictionary and verification business dictionary, the big model is fully fine-tuned using the Q&A token_ids and low-rank matrix to generate a fine-tuned big model, and the model qualification is verified based on the cross entropy loss value and performance indicators. , and finally build a user intention decision model to achieve local instant processing and response.

Benefits of technology

It improves the accuracy and speed of user intention identification, avoids privacy leakage, reduces the consumption of cloud computing resources, improves user interaction experience and service efficiency of banking equipment, and expands the coverage of outlets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118503712B_ABST
    Figure CN118503712B_ABST
Patent Text Reader

Abstract

The present invention provides a training method and a method for using a user intention decision model, including dividing a training business dictionary D1 and a verification business dictionary D2, confirming the question and answer token_ids and low-rank matrix E' corresponding to D1; fine-tuning the preset model weights of the preset large model according to the question and answer token_ids and E' to obtain a fine-tuned large model; inputting the question and answer token_ids into the fine-tuned large model to determine the cross entropy loss value L; verifying whether the fine-tuned large model is qualified according to L; calculating the performance index of the first decision model according to D2, wherein the first decision model is the fine-tuned large model after conversion based on the preset format; verifying whether the first decision model is qualified according to the performance index. This technology can process and respond to user intentions locally and instantly and accurately, and provide corresponding services to improve the recognition accuracy and speed of user intentions, avoid user privacy leakage, reduce the consumption of cloud computing resources, improve user interaction experience, improve the service efficiency of bank equipment, and expand the coverage of outlets.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of financial vertical technology, and in particular to a training method and a use method of a user intention decision model. Background Art

[0002] With the development of financial technology, the automation and intelligence of banking business processing have become the key to improving service efficiency and user experience.

[0003] Most current banking devices rely on database query and string fuzzy matching technology to process user requests. This method may be able to meet the needs when processing simple and direct queries, but the database and fuzzy matching method has obvious limitations in understanding complex user intentions, and cannot achieve high generalization and accurate recognition. When faced with more complex or ambiguous user intentions, its accuracy and efficiency will be greatly reduced.

[0004] At present, sometimes users' query requests are uploaded to the cloud to use more powerful computing resources for intent analysis and decision-making. Although this method can provide more complex decision logic, it will bring higher latency and increase the risk of privacy leakage during data transmission; in addition, this method will consume a lot of cloud computing resources, resulting in increased costs. Summary of the invention

[0005] Based on this, the purpose of the present invention is to provide a training method and a method for using a user intention decision model, which can process and respond to user intentions locally and instantly and accurately and provide corresponding services, so as to improve the accuracy and speed of identifying user intentions, avoid user privacy leakage, reduce the consumption of cloud computing resources, improve user interaction experience, enhance the service efficiency of banking equipment, and expand the coverage of outlets.

[0006] In a first aspect, an embodiment of the present invention provides a training method for a user intention decision model, the training method comprising: S1: dividing a pre-collected sample data set into a training business dictionary D1 and a verification business dictionary D2, and confirming the question and answer token_ids and the low-rank matrix E' corresponding to the training business dictionary D1; S2: fine-tuning the preset model weights of the preset large model according to the question and answer token_ids and the low-rank matrix E' to obtain a fine-tuned large model; S3: inputting the question and answer token_ids into the fine-tuned large model, and determining the cross-entropy loss value L according to the predicted value; S4: verifying whether the fine-tuned large model is qualified according to the cross-entropy loss value L; S5: if the fine-tuned large model is qualified, calculating the performance indicators of the first decision model according to the verification business dictionary D2, wherein the performance indicators include accuracy, precision, recall and F1 score, wherein the first decision model is the fine-tuned large model converted based on the preset format; S6: verifying whether the first decision model is qualified according to the performance indicators; S7: if the first decision model is qualified, the first decision model is the final user intention decision model.

[0007] Furthermore, S1 includes: S11: integrating and cleaning the data streams of the pre-collected initial data to obtain a sample data set; S12: dividing the sample data set into a training business dictionary D1 and a verification business dictionary D2 based on a preset ratio; S13: binding the user requirements of the training business dictionary D1 with business labels to generate a question and answer training data set; S14: inputting the question and answer training data set into the model vocabulary file of the preset large model to convert the question and answer training data set into a question and answer tokens sequence; S15: based on the preset tiktoken tool, translating the question and answer tokens sequence into question and answer token_ids; S16: constructing the training business dictionary D1 into a K-dimensional low-rank matrix E' according to the preset text2vec model and the current or preset matrix dimension K.

[0008] Furthermore, S2 includes: S21: dimensionality adjustment of the architecture of the preset large model; S22: inputting the K-dimensional low-rank matrix E' into the large model after dimension adjustment, and outputting the embedding vector matrix E"; S23: inputting the question and answer token_ids into the embedding vector matrix E", to fully fine-tune the model weight parameters to obtain the fine-tuned large model, wherein the model weight parameters include the parameters of the weights in the embedding layer, multiple hidden layers, the fully connected layer, and the softmax layer.

[0009] Furthermore, S3 includes: S31: inputting the question and answer token_ids into the fine-tuned large model, and the question and answer token_ids are sequentially passed through multiple hidden layers, a softmax layer, and a fully connected layer of the fine-tuned large model to obtain the predicted business label and predicted probability p for each question and answer token_ids; S32: calculating the cross entropy loss value L based on all the predicted probabilities p.

[0010] Further, S5 includes: S51: rewriting the pre-acquired initial GGUF model converter based on the model weight parameters in the fine-tuned large model to obtain a GGUF model converter; S52: writing the model weight file, model vocabulary file and model configuration information file in the fine-tuned large model into the GGUF model converter to obtain a first decision model in a preset format; S53: inputting the user requirements in the verification business dictionary D2 into the first decision model, so that the first decision model predicts the business category of each user requirement in the verification business dictionary D2 and outputs a predicted business label;

[0011] S54: Determine the performance indicator of the first decision model according to the service label in the verification service dictionary D2 and the predicted service label of the first decision model.

[0012] Furthermore, if the fine-tuned large model is unqualified, S2-S4 are repeatedly executed until L<=a preset loss threshold.

[0013] Furthermore, if the first decision model is unqualified, S1-S6 are repeatedly executed until all performance indicators are within a qualified threshold range.

[0014] In a second aspect, an embodiment of the present invention provides a method for using a user intention decision model, the method comprising: S102: cleaning, denoising, and formatting the user's current needs to obtain a target data set; S104: inputting the target data set into the model vocabulary file to obtain a target tokens sequence; S106: based on a preset tiktoken tool, translating the target tokens sequence into target token_ids; S108: the target token_ids pass through multiple hidden layers, a softmax layer, and a fully connected layer in sequence to obtain multiple predicted business labels and corresponding predicted probabilities p; S110: taking the predicted business label corresponding to the maximum predicted probability p as the user's intention.

[0015] The beneficial effects of the present invention are as follows:

[0016] The present invention provides a training method for a user intention decision model, comprising dividing a pre-collected sample data set into a training business dictionary D1 and a verification business dictionary D2, and confirming the question and answer token_ids and low-rank matrix E' corresponding to the training business dictionary D1; fine-tuning the preset model weights of a preset large model according to the question and answer token_ids and the low-rank matrix E' to obtain a fine-tuned large model; inputting the question and answer token_ids into the fine-tuned large model, and determining a cross entropy loss value L according to a predicted value; verifying whether the fine-tuned large model is qualified according to L; calculating performance indicators of a first decision model according to the verification business dictionary D2, wherein the performance indicators include accuracy, precision, recall and F1 score, wherein the first decision model is a fine-tuned large model converted based on a preset format; verifying whether the first decision model is qualified according to the performance indicators; the first decision model is the final user intention decision model. This technology can process and respond to user intentions locally and instantly and accurately and provide corresponding services to improve the accuracy and speed of identifying user intentions, avoid user privacy leakage, reduce the consumption of cloud computing resources, improve user interaction experience, enhance the service efficiency of banking equipment, and expand the coverage of outlets.

[0017] Other features and advantages of the present invention will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present invention.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are specifically listed below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 A flowchart for training a user intention decision model;

[0021] Figure 2 Flowchart for implementing S1;

[0022] Figure 3 Flowchart for implementing S2;

[0023] Figure 4 Flowchart for implementing S5;

[0024] Figure 5A flowchart of a decision model using user intention. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0026] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] Example 1

[0028] This embodiment provides a method for training a user intention decision model. Figure 1 The specific contents are as follows:

[0029] S1: Divide the pre-collected sample data set into a training business dictionary D1 and a verification business dictionary D2, and confirm the question and answer token_ids and low-rank matrix E' corresponding to the training business dictionary D1.

[0030] Figure 2 This is a specific description of S1.

[0031] S11: Integrate and clean the pre-collected initial data to obtain a sample data set.

[0032] S1-1-1: Collect initial data, including (1) user needs; (2) customer service responses; (3) business tags; and (4) business functions.

[0033] S1-1-2: Integrate the initial data belonging to the same group into one banking business data flow.

[0034] Specifically, the banking business data flow is: user makes a request → customer service responds to the request → user selects a business tag → calls a business function.

[0035] Table 1 is a specific example of banking business data flow.

[0036] Table 1

[0037]

[0038] S1-1-3: Clean, denoise and format the banking business data stream to obtain a sample data set.

[0039] The beneficial effect of S1-1 is that this step is the process of data preprocessing. In layman's terms, it is to package each business data stream into banking business data, and clean the collected banking business data, including denoising, formatting, and removing useless information, to ensure data quality and prepare for subsequent model training. Data cleaning significantly improves data quality and model prediction accuracy, and improves user service efficiency and satisfaction. At the same time, automated data processing and model reasoning reduce banks' operating costs. Step 1-1 effectively converts user utterances into accurate business intents by collecting and cleaning banking business data and user intent data: This method can greatly improve the accuracy and availability of data and avoid formatting problems that may be encountered when processing large models. This step is the key to successfully transforming the general large model Qianwen 1.8B into a large model specifically for banking business, ensuring that the model can fully cover and accurately understand the complex user needs in the banking field.

[0040] S12: Based on a preset ratio, the sample data set is divided into a training business dictionary D1 and a verification business dictionary D2.

[0041] Specifically, for example, if 70% of the sample data set is formatted into the training business dictionary D1, then 30% of the sample data set is formatted into the verification business dictionary D2. This ratio can be preset or randomly assigned, which is not required in this embodiment.

[0042] S13: Bind the user requirements of the training business dictionary D1 with the business labels to generate a question-answering training data set.

[0043] S14: Input the question-answering training dataset into the model vocabulary file of the preset large model to convert the question-answering training dataset into a sequence of question-answering tokens.

[0044] Specifically, the above-mentioned preset large model can be QWen1.8B, in which an initial model weight file, a model vocabulary file and a model configuration information file are set, and the model weights include the weights of the parameters in the embedding layer, multiple hidden layers, the fully connected layer, and the softmax layer; wherein the weight file format is (.pth), the model vocabulary file format is (qwen.tiktoken), and the model configuration information file format is (config.json).

[0045] S15: Based on the preset tiktoken tool, the question and answer tokens sequence is translated into question and answer token_ids.

[0046] Specifically, S15 converts the newly generated tokens sequence into question-and-answer token_ids through the convert_tokens_to_ids method of the tiktoken tool. Token_ids is the final processing result of the question-and-answer training data set conversion.

[0047] S16: According to the preset text2vec model and the current or preset matrix dimension K, the training business dictionary D1 is constructed into a K-dimensional low-rank matrix E'.

[0048] S161: Use the preset text2vec model to convert each group of data streams in the training business dictionary D1 into a vector of fixed length and fixed dimension, and construct a vector matrix E.

[0049] Specifically, the fixed dimension here is generally 100 dimensions.

[0050] S162: Decompose the vector matrix E to obtain three matrices U, Σ, and V', where E≈U×Σ×V'.

[0051] S163: Based on a preset matrix dimension K, extract the first K singular values ​​and their corresponding singular vectors to construct a low-rank matrix E', where E'=U_K×Σ_K×V'_K.

[0052] Specifically, by generating a low-rank matrix E' through SVD technology, the data processing efficiency is significantly improved, the computational complexity is reduced, the adaptability to the low performance of banking equipment is enhanced, and the generalization ability of the model is enhanced.

[0053] Specifically, SVD is a matrix decomposition technique called singular value decomposition. For any matrix, SVD can decompose it into the product of three matrices.

[0054] S2: Fine-tune the preset model weights of the preset large model according to the question and answer token_ids and the low-rank matrix E' to obtain the fine-tuned large model.

[0055] Figure 3 It is a specific description of S2.

[0056] Specifically, the preset large model can be QWen1.8B, which is provided with an initial model weight file, a model vocabulary file and a model configuration information file. The model weights include the weights of the parameters in the embedding layer, multiple hidden layers, the fully connected layer and the softmax layer; among which, the weight file format is (.pth), the model vocabulary file format is (qwen.tiktoken), and the model configuration information file format is (config.json).

[0057] Specifically, full fine-tuning refers to adjusting the parameter weights in the model's embedding layer, multiple hidden layers, fully connected layer, softmax layer, etc.; the fine-tuned large model can be the banking business large model QBank1.8B.

[0058] S21: Dimensionally adjust the architecture of the preset large model.

[0059] Specifically, this step is used to change the input dimension of a preset large model, such as QWen1.8B, to K dimensions so that it can accept and process K-dimensional data.

[0060] S22: Input the K-dimensional low-rank matrix E' into the large model after dimension adjustment, and output the embedded vector matrix E".

[0061] S23: Input the question and answer token_ids into the embedding vector matrix E' to fully fine-tune the model weight parameters to obtain a fine-tuned large model, where the model weight parameters include the parameters of the weights in the embedding layer, multiple hidden layers, the fully connected layer, and the softmax layer.

[0062] In S2, the low-rank matrix E' is embedded into the QWen1.8B preset model to generate the QBank1.8B process, which increases the weight of the banking-related content of the QWen1.8B preset model, expands the knowledge base of the model, and better generalizes the knowledge in the banking field.

[0063] S3: Input the question-answer token_ids into the fine-tuned large model and determine the cross entropy loss value L based on the predicted value.

[0064] S3 includes:

[0065] S31: The question and answer token_ids are input into the fine-tuned large model. The question and answer token_ids are sequentially passed through multiple hidden layers, a softmax layer, and a fully connected layer of the fine-tuned large model to obtain the predicted business label and predicted probability p for each question and answer token_ids.

[0066] Specifically, the number of sample data sets in the training business dictionary D1 (assuming there is a C1 group) can be used to obtain the predicted business label of the C1 group and the predicted probability p of the C1 group.

[0067] S32: Calculate the cross entropy loss value L based on all predicted probabilities p.

[0068] The calculation process of L is as follows:

[0069]

[0070] Among them, C1 is the total number of business labels, that is, the number of sample data sets in the training business dictionary D1; the value range of i is 1-C1, i represents the calculation of the loss value of any question and answer token_ids; yi is a Boolean indicator, if the business label prediction for the i-th token_ids is correct, then yi=1, otherwise yi=0.

[0071] Specifically, the value of yi is 0 or 1. Here, it is necessary to compare the predicted business label with the highest prediction probability p with the content of the original business label in the sample data set to draw a conclusion on whether it is correct or not.

[0072] S4: Verify whether the fine-tuned large model is qualified according to the cross entropy loss value L.

[0073] If L<=the preset loss threshold, it means that the model embedding fine-tuning is successful, that is, the fine-tuned large model obtained in S2 is qualified, and then continue to execute S5; otherwise, if L>the preset loss threshold, then repeat S2. When repeating S2, the "preset large model" refers to the fine-tuned large model that is currently unqualified in the verification.

[0074] Specifically, after the embedding fine-tuning is successful, it is necessary to save the corrected large model QWen1.8B, save all weight files and model configuration files, generate the banking business large model QBank1.8B, that is, the qualified fine-tuning large model, and then execute S5.

[0075] S5: Calculate performance indicators of the first decision model according to the verification business dictionary D2, wherein the performance indicators include accuracy, precision, recall and F1 score, wherein the first decision model is a fine-tuned large model converted based on a preset format.

[0076] For details on S5, see Figure 4 .

[0077] S51: Rewrite the pre-acquired initial GGUF model converter based on the model weight parameters in the fine-tuned large model to obtain the GGUF model converter.

[0078] Specifically, S51 can be implemented with the tool Llama.cpp.

[0079] Specifically, such an operation can ensure the accuracy and completeness of the generated GGUF model file (ie, the file in the first decision model).

[0080] S52: Write the model weight file, model vocabulary file and model configuration information file in the fine-tuned large model into the GGUF model converter to obtain a first decision model in a preset format.

[0081] Specifically, the preset large model can be QWen1.8B, which is provided with an initial model weight file, a model vocabulary file and a model configuration information file. The model weight parameters include the weights of the parameters in the embedding layer, multiple hidden layers, a fully connected layer and a softmax layer.

[0082] Beneficial effects of S51-52: The conversion process enables the model to adapt to the banking machine environment without GPU graphics cards. The GGUF conversion tool is used to convert the quantized large model into a banking business GGUF decision model: This conversion process enables the model to adapt to the banking machine environment without GPU graphics cards, solving the problem of high hardware requirements for traditional large models. This technology has excellent adaptability and flexibility, and can run on various banking machines without sacrificing performance, effectively solving the problem of hardware limitations. The banking business GGUF decision model significantly improves the ability to understand and generalize user intentions. Compared with traditional methods that rely on simple string matching, the GGUF decision model can more accurately identify and process user business demand dialogues; this not only solves the problem of low recognition accuracy, but also significantly improves the response speed of the device by reducing dependence on cloud computing resources. In the end, this technical solution achieved the goal of effectively reducing operation and maintenance costs and optimizing resource allocation while improving user satisfaction.

[0083] S53: Inputting the user requirements in the verification service dictionary D2 into the first decision model, so that the first decision model predicts the service category of each user requirement in the verification service dictionary D2 and outputs a predicted service label.

[0084] S54: Determine the performance indicator of the first decision model according to the service label in the verification service dictionary D2 and the predicted service label of the first decision model.

[0085] S54 includes:

[0086] Specifically, the number of sample data sets of the verification business dictionary D2 is defined as C2.

[0087] Table 2 is a specific example of sample business labels and predicted business labels in D2.

[0088] Table 2

[0089]

[0090]

[0091] It can be seen that predictions 1, 3, and 4 are correct, while predictions 2 and 5 are wrong.

[0092] Specific calculation of performance indicators (accuracy, precision, recall and F1 score):

[0093] 1. Accuracy A

[0094] A=3 / 5=0.6

[0095] 2. Precision P

[0096] There are C2=5 groups of sample data, the number of types of sample business labels (the third column of Table 2) is x=5, and the number of predicted label types of the large model (the fourth column of Table 2) is y=3 ("loan" and "deposit" were not predicted, so the number of types is 3).

[0097] Since the number of sample label types x=5, 5 Ps can be obtained.

[0098] P transfer = 1 transfer should be predicted / 1 transfer actually predicted = 1 / 1 = 1.

[0099] PLoan = 1 loan should be predicted / 0 loan is actually predicted, so the result is NA.

[0100] P account management = 1 account management should be predicted / 2 account management actually predicted = 1 / 2 = 0.5.

[0101] P deposit = should be predicted to be 1 deposit / actual predicted deposit is 0, so the result is NA.

[0102] P withdrawal = 1 withdrawal should be predicted / 2 withdrawals actually predicted = 1 / 2 = 0.5.

[0103] Paverage = (1 + 0.5 + 0.5) / y = 2 / 3 = 0.67.

[0104] 3. Recall Rate R

[0105] R transfer = 1 transfer is actually predicted and the prediction is correct / 1 transfer should be predicted = 1 / 1 = 1.

[0106] Rloan = actual predicted 0 loans / should have predicted 1 loan = 0 / 1 = 0.

[0107] R account management = 2 account managements were actually predicted, of which 1 was predicted correctly / 1 loan should have been predicted = 1 / 1 = 1.

[0108] R deposit = actual predicted 0 deposit / should be predicted 1 deposit = 0 / 1 = 0.

[0109] R withdrawal = 2 withdrawals are actually predicted, 1 of which is correct / 1 withdrawal should be predicted = 1 / 1 = 1

[0110] Raverage = 3 / x = 3 / 5 = 0.6.

[0111] 4. F1 score

[0112] F1=2×[P×R / (P+R)]=0.63.

[0113] S6: Verify whether the first decision model is qualified according to the performance indicator.

[0114] Specifically, a corresponding qualified threshold range is preset for each performance indicator.

[0115] If any performance indicator of the first decision model is not within the qualified threshold range, it is necessary to loop S16, adjust the matrix dimension K, and repeat S2, S3, S4, and S5 until all performance indicators are within the qualified threshold range; if all four indicators of the first decision model reach the corresponding qualified threshold range, it means that the first decision model is qualified, and then execute S7.

[0116] S7: The first decision model is the final user intention decision model.

[0117] Specifically, if any item is not within the corresponding qualified threshold range, S1 is executed in a loop, that is, the value of the matrix dimension K is changed to repeat the training.

[0118] This embodiment improves recognition accuracy through model fine-tuning, performance indicator verification, format conversion and other means; significantly improves the response speed of the device by reducing dependence on cloud computing resources; while improving user satisfaction, it effectively reduces operation and maintenance costs and optimizes resource allocation.

[0119] Example 2

[0120] This embodiment provides a method for using a user intention decision model, such as Figure 5 As shown, specifically including:

[0121] S102: Clean, denoise, and format the user's current needs to obtain a target data set.

[0122] S104: Input the target data set into the model vocabulary file to obtain the target tokens sequence.

[0123] S106: Based on the preset tiktoken tool, the target tokens sequence is translated into target token_ids.

[0124] S108: The target token_ids passes through multiple hidden layers, a softmax layer, and a fully connected layer in sequence to obtain multiple predicted business labels and corresponding predicted probabilities p.

[0125] S110: Taking the predicted business label corresponding to the maximum predicted probability p as the user's intention.

[0126] The method for using the user intention decision model provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned embodiment. For the sake of brief description, for parts not mentioned in this embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for training a user intention decision model, characterized in that: The training method comprises: S1: Divide the pre-collected sample data set into a training business dictionary D1 and a verification business dictionary D2, and confirm the question and answer token_ids and low-rank matrix E' corresponding to the training business dictionary D1; S2: fine-tune the preset model weights of the preset large model according to the question and answer token_ids and the low-rank matrix E' to obtain the fine-tuned large model; S3: Input the question-answer token_ids into the fine-tuned large model and determine the cross entropy loss value L based on the predicted value; S4: Verify whether the fine-tuned large model is qualified according to the cross entropy loss value L; S5: If the fine-tuned large model is qualified, the performance indicators of the first decision model are calculated according to the verification business dictionary D2, wherein the performance indicators include accuracy, precision, recall and F1 score, wherein the first decision model is the fine-tuned large model converted based on the preset format; S6: Verify whether the first decision model is qualified according to the performance indicator; S7: If the first decision model is qualified, the first decision model is the final user intention decision model; S1 includes: S11: Integrate and clean the data stream of the pre-collected initial data to obtain a sample data set; S12: dividing the sample data set into a training business dictionary D1 and a verification business dictionary D2 based on a preset ratio; S13: Bind the user requirements of the training business dictionary D1 with the business labels to generate a question-answering training data set; S14: inputting the question-answering training data set into the model vocabulary file of the preset large model to convert the question-answering training data set into a sequence of question-answering tokens; S15: Based on the preset tiktoken tool, the question and answer tokens sequence is translated into question and answer token_ids; S16: constructing the training business dictionary D1 into a K-dimensional low-rank matrix E' according to the preset text2vec model and the current or preset matrix dimension K; S2 includes: S21: Dimensional adjustment of the architecture of the preset large model; S22: Input the K-dimensional low-rank matrix E' into the large model after dimension adjustment, and output the embedded vector matrix E''; S23: Input the question and answer token_ids into the embedding vector matrix E'' to fully fine-tune the model weight parameters to obtain a fine-tuned large model, where the model weight parameters include the parameters of the weights in the embedding layer, multiple hidden layers, the fully connected layer, and the softmax layer; S3 includes: S31: Input the question and answer token_ids into the fine-tuned large model. The question and answer token_ids pass through multiple hidden layers, a softmax layer, and a fully connected layer of the fine-tuned large model in turn to obtain the predicted business label and predicted probability p for each question and answer token_ids; S32: Calculate the cross entropy loss value L based on all predicted probabilities p.

2. The method for training a user intention decision model according to claim 1, characterized in that S5 include: S51: rewrite the pre-acquired initial GGUF model converter based on the model weight parameters in the fine-tuned large model to obtain a GGUF model converter; S52: writing the model weight file, model vocabulary file and model configuration information file in the fine-tuned large model into the GGUF model converter to obtain a first decision model in a preset format; S53: inputting the user requirements in the verification service dictionary D2 into the first decision model, so that the first decision model predicts the service category of each user requirement in the verification service dictionary D2 and outputs a predicted service label; S54: Determine the performance indicator of the first decision model according to the service label in the verification service dictionary D2 and the predicted service label of the first decision model.

3. The training method of the user intention decision model according to claim 1, characterized in that: If the fine-tuned large model is unqualified, S2-S4 are repeatedly executed until L<=preset loss threshold.

4. The method for training a user intention decision model according to claim 1, characterized in that: If the first decision model is unqualified, steps S1 to S6 are repeated until all performance indicators are within a qualified threshold range.

5. A method for using a user intention decision model, characterized in that: The user intention decision model is obtained based on the training method of the user intention decision model according to any one of claims 1 to 4; The method of use includes: S102: Clean, denoise, and format the user's current needs to obtain a target data set; S104: Input the target data set into the model vocabulary file to obtain the target tokens sequence; S106: Based on the preset tiktoken tool, the target tokens sequence is translated into target token_ids; S108: The target token_ids passes through multiple hidden layers, a softmax layer, and a fully connected layer in sequence to obtain multiple predicted business labels and corresponding predicted probabilities p; S110: Taking the predicted business label corresponding to the maximum predicted probability p as the user's intention.