Data processing method, device and equipment and readable storage medium

Abstract

The invention discloses a data processing method, device and equipment and a readable storage medium, and the method comprises the steps: inputting business sample data into an initial large language model, predicting the business sample data through the initial large language model, and obtaining M lexical element prediction vectors; based on the M lexical element prediction vectors, generating a sample text result and a sample confidence coefficient corresponding to the sample text result; generating a correctness reward value based on a sample label corresponding to the business sample data and the sample text result, and generating a confidence coefficient calibration reward value based on the sample confidence coefficient, the sample label and the sample text result; and based on the confidence calibration reward value and the correctness reward value, performing reinforcement learning training on the initial large language model to obtain a target large language model. By adopting the method, the confidence of model output and the accuracy of a prediction result can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and readable storage medium. Background Technology

[0002] Currently, the output of large language models is mainly based on generated text tokens, which can form the text answer output by the large language model. However, existing large language models do not have the ability to evaluate the confidence of the text answer.

[0003] Existing solutions often involve using large language models (MLMs) to output predicted probabilities of text tokens via prompts, thought chains, or decoding strategies. The confidence score of the text answer is then generated based on these probabilities. However, the training objective of MLMs is to minimize the loss between the text answer and the sample labels; confidence scores are not involved in the training process, and their generation is not optimized. Therefore, when applying MLMs, these confidence scores are not calibrated during training. The output predicted probabilities are essentially statistical frequencies and have no correlation with confidence scores. The predicted probabilities may be artificially inflated due to the "fluency" of the learned sample data or excessively low due to rare exceptions in the learned sample data.

[0004] Because the predicted probability cannot accurately represent the confidence level of the large language model in the text answer, the text token selected based on the predicted probability cannot represent the correct text answer. The lack of judgment regarding correctness and authenticity leads to poor interpretability, reliability, and accuracy of the model's output text answer. Therefore, without a sufficiently accurate predicted probability, the accuracy of the confidence level calculated based on the predicted probability cannot be guaranteed. Summary of the Invention

[0005] This application provides a data processing method, apparatus, device, and readable storage medium that can improve the confidence level of model output and the accuracy of prediction results.

[0006] One embodiment of this application provides a data processing method, including: Obtain business sample data, input the business sample data into the initial large language model, and use the initial large language model to predict the business sample data to obtain M word prediction vectors; M is a positive integer; The words indicated by the M word prediction vectors are determined as the sample text results corresponding to the business sample data. Based on the prediction probabilities corresponding to the M word prediction vectors, the sample confidence scores corresponding to the sample text results are generated. Based on the sample labels and sample text results corresponding to the business sample data, a correctness reward value is generated, and based on the sample confidence, sample labels and sample text results, a confidence calibration reward value is generated. Based on the confidence calibration reward value and the correctness reward value, the initial large language model is trained by reinforcement learning to obtain the target large language model; the target large language model is used to generate the predicted text results corresponding to the business data and the prediction confidence of the predicted text results.

[0007] Among them, based on the sample labels and sample text results corresponding to the business sample data, a correctness reward value is generated, including: The feature similarity between the sample labels and sample text results corresponding to the generated business sample data; If the feature similarity is greater than or equal to the similarity threshold, then the first reward value is determined as the correctness reward value; If the feature similarity is less than the similarity threshold, the second reward value is determined as the correctness reward value; the first reward value is greater than the second reward value.

[0008] Among them, based on the sample confidence level, sample label, and sample text results, a confidence level calibration reward value is generated, including: The feature similarity between the sample labels and sample text results corresponding to the generated business sample data; The calibration value is obtained based on feature similarity, and a confidence calibration reward value is generated based on the calibration value and the sample confidence.

[0009] The process involves obtaining calibration values ​​based on feature similarity, and generating confidence calibration reward values ​​based on target matching values ​​and sample confidence, including: If the feature similarity is greater than or equal to the similarity threshold, the maximum value within the confidence interval to which the sample confidence belongs is determined as the calibration value; if the feature similarity is less than the similarity threshold, the minimum value within the confidence interval is determined as the calibration value. The difference between the calibration value and the sample confidence level is determined as the matching error value; The matching error value is exponentially calculated to obtain the mean squared error value. The negative of the mean squared error value is then used as the confidence calibration reward value.

[0010] Specifically, based on the confidence-calibrated reward value and the correctness reward value, the initial large language model is trained using reinforcement learning to obtain the target large language model, including: The confidence calibration reward value and the correctness reward value are weighted to obtain the comprehensive reward value; Based on the comprehensive reward value, the model parameters of the initial large language model are adjusted. When the initial large language model meets the training convergence condition, the initial large language model that meets the training convergence condition is determined as the target large language model.

[0011] Specifically, the initial large language model is used to predict business sample data, resulting in M ​​word prediction vectors, including: Feature extraction is performed on the business sample data using an initial large language model to obtain the sample feature sequence; Based on the candidate lexical units in the business dictionary of the initial large language model, probability matching is performed on the sample feature sequence to obtain M lexical prediction vectors.

[0012] The initial large language model includes an encoding layer; features are extracted from the business sample data using the initial large language model to obtain a sample feature sequence, including: The business sample data is segmented into N unit words by the encoding layer in the initial large language model; N is a positive integer. Encode each of the N unit words to obtain N unit word encoding vectors. Based on the positional order of each unit word in the business sample data, generate N positional encoding vectors. Based on N word encoding vectors and N position encoding vectors, N unit feature vectors are generated. The N unit feature vectors are then concatenated to obtain the sample feature sequence.

[0013] Specifically, the business sample data is segmented into N unit words by the encoding layer in the initial large language model, resulting in the following: Obtain the business dictionary and initial length threshold from the initial large language model; Based on the character arrangement order in the business sample data, candidate strings with the same character length as the initial length threshold are obtained from the business sample data; If a candidate string matches the business dictionary, the candidate string is determined as a unit word. If the candidate string does not match the business dictionary, the initial length threshold is decremented to obtain a new initial length threshold. Based on the new initial length threshold, a new candidate string is determined in the business sample data. If the new candidate string matches the business dictionary, the new candidate string is determined as a unit word. Characters other than unit words in the business sample data are identified as text to be matched. New unit words are then identified in the text to be matched based on the initial length threshold until all business sample data has been traversed and N unit words are obtained.

[0014] Specifically, based on candidate lexical units in the business dictionary of the initial large language model, probability matching is performed on the sample feature sequence to obtain M lexical unit prediction vectors, including: Obtain a sequence of candidate lexical units composed of candidate lexical units from the business dictionary in the initial large language model; Cross-attention processing is performed on the candidate word sequence and the sample feature sequence to obtain the matching probability corresponding to each candidate word. The feature vector corresponding to the candidate word with the highest matching probability is determined as the first target word. The sample feature sequence is updated based on the first target word to obtain a new sample feature sequence. Probability matching is then performed in the new sample feature sequence to obtain the next target word of the first target word. This process continues until the next target word meets the generation termination condition. The feature vectors corresponding to the M target words are then determined as the M word prediction vectors.

[0015] This also includes: The matching probabilities corresponding to the M word prediction vectors are determined as the prediction probabilities corresponding to the M word prediction vectors. Based on the prediction probabilities corresponding to the M word prediction vectors, the sample confidence scores corresponding to the sample text results are generated, including: The average of the prediction probabilities corresponding to the M word prediction vectors is used to determine the sample confidence level of the sample text result.

[0016] This includes: Obtain business data and threshold prompt words, input the business data and threshold prompt words into the target large language model, extract features from the business data through the target large language model to obtain a business feature sequence, and extract features from the threshold prompt words to obtain a prompt word feature sequence; Obtain the matching probability of each candidate word element in the target large language model and the business feature sequence respectively, and obtain the target confidence threshold indicated by the prompt word based on the prompt word feature sequence; Candidate words with matching probabilities greater than or equal to the target confidence threshold are identified as R undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the words indicated by the T target word prediction vectors are determined as the predicted text results corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text results is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0017] This includes: Acquire business data, input the business data into the target large language model, and extract features from the business data through the target large language model to obtain the business feature sequence; Obtain the matching probability of each candidate word in the target large language model and the business feature sequence, perform domain identification on the business feature sequence to obtain the business domain to which the business data belongs, and obtain the target confidence threshold corresponding to the business domain in the confidence mapping table; the confidence thresholds of different business domains in the confidence mapping table are different; Candidate words with matching probabilities greater than or equal to the target confidence threshold are identified as S undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the words indicated by the T target word prediction vectors are determined as the predicted text results corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text results is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0018] One embodiment of this application provides a data processing apparatus, including: The prediction module is used to acquire business sample data, input the business sample data into the initial large language model, and predict the business sample data through the initial large language model to obtain M word prediction vectors; M is a positive integer. The result calculation module is used to determine the words indicated by the M word prediction vectors as the sample text results corresponding to the business sample data, and generate the sample confidence scores corresponding to the sample text results based on the prediction probabilities corresponding to the M word prediction vectors respectively. The reward calculation module is used to generate a correctness reward value based on the sample labels and sample text results corresponding to the business sample data, and to generate a confidence calibration reward value based on the sample confidence, sample labels and sample text results. The training module is used to perform reinforcement learning training on the initial large language model based on confidence and correctness reward values ​​to obtain the target large language model. The target large language model is used to generate the predicted text results corresponding to the business data and the prediction confidence of the predicted text results.

[0019] In one possible implementation, when the reward calculation module generates a correctness reward value based on the sample labels and sample text results corresponding to the business sample data, it specifically performs the following operations: The feature similarity between the sample labels and sample text results corresponding to the generated business sample data; If the feature similarity is greater than or equal to the similarity threshold, then the first reward value is determined as the correctness reward value; If the feature similarity is less than the similarity threshold, the second reward value is determined as the correctness reward value; the first reward value is greater than the second reward value.

[0020] In one possible implementation, the reward calculation module, when generating a confidence-calibrated reward value based on sample confidence, sample labels, and sample text results, specifically performs the following operations: In one possible implementation, when the reward calculation module is used for…, it is specifically used to perform the following operations: The feature similarity between the sample labels and sample text results corresponding to the generated business sample data; The calibration value is obtained based on feature similarity, and a confidence calibration reward value is generated based on the calibration value and the sample confidence.

[0021] In one possible implementation, the reward calculation module is used to obtain calibration values ​​based on feature similarity. When generating confidence-calibrated reward values ​​based on target matching values ​​and sample confidence, it specifically performs the following operations: If the feature similarity is greater than or equal to the similarity threshold, the maximum value within the confidence interval to which the sample confidence belongs is determined as the calibration value; if the feature similarity is less than the similarity threshold, the minimum value within the confidence interval is determined as the calibration value. The difference between the calibration value and the sample confidence level is determined as the matching error value; The matching error value is exponentially calculated to obtain the mean squared error value. The negative of the mean squared error value is then used as the confidence calibration reward value.

[0022] In one possible implementation, the training module is used to perform reinforcement learning training on the initial large language model based on confidence and correctness reward values. When the target large language model is obtained, it is specifically used to perform the following operations: The confidence calibration reward value and the correctness reward value are weighted to obtain the comprehensive reward value; Based on the comprehensive reward value, the model parameters of the initial large language model are adjusted. When the initial large language model meets the training convergence condition, the initial large language model that meets the training convergence condition is determined as the target large language model.

[0023] In one possible implementation, the prediction module is used to predict business sample data using an initial large language model. Specifically, when obtaining M word prediction vectors, it performs the following operations: Feature extraction is performed on the business sample data using an initial large language model to obtain the sample feature sequence; Based on the candidate lexical units in the business dictionary of the initial large language model, probability matching is performed on the sample feature sequence to obtain M lexical prediction vectors.

[0024] In one possible implementation, the initial large language model includes an encoding layer; the prediction module is used to extract features from the business sample data using the initial large language model. When obtaining the sample feature sequence, it is specifically used to perform the following operations: The business sample data is segmented into N unit words by the encoding layer in the initial large language model; N is a positive integer. Encode each of the N unit words to obtain N unit word encoding vectors. Based on the positional order of each unit word in the business sample data, generate N positional encoding vectors. Based on N word encoding vectors and N position encoding vectors, N unit feature vectors are generated. The N unit feature vectors are then concatenated to obtain the sample feature sequence.

[0025] In one possible implementation, when the prediction module performs word segmentation on the business sample data through the encoding layer in the initial large language model to obtain N unit words corresponding to the business sample data, it specifically performs the following operations: Obtain the business dictionary and initial length threshold from the initial large language model; Based on the character arrangement order in the business sample data, candidate strings with the same character length as the initial length threshold are obtained from the business sample data; If a candidate string matches the business dictionary, the candidate string is determined as a unit word. If the candidate string does not match the business dictionary, the initial length threshold is decremented to obtain a new initial length threshold. Based on the new initial length threshold, a new candidate string is determined in the business sample data. If the new candidate string matches the business dictionary, the new candidate string is determined as a unit word. Characters other than unit words in the business sample data are identified as text to be matched. New unit words are then identified in the text to be matched based on the initial length threshold until all business sample data has been traversed and N unit words are obtained.

[0026] In one possible implementation, the prediction module, when performing probability matching on the sample feature sequence based on candidate lexical units in the business dictionary of the initial large language model to obtain M lexical unit prediction vectors, specifically performs the following operations: Obtain a sequence of candidate lexical units composed of candidate lexical units from the business dictionary in the initial large language model; Cross-attention processing is performed on the candidate word sequence and the sample feature sequence to obtain the matching probability corresponding to each candidate word. The feature vector corresponding to the candidate word with the highest matching probability is determined as the first target word. The sample feature sequence is updated based on the first target word to obtain a new sample feature sequence. Probability matching is then performed in the new sample feature sequence to obtain the next target word of the first target word. This process continues until the next target word meets the generation termination condition. The feature vectors corresponding to the M target words are then determined as the M word prediction vectors.

[0027] In one possible implementation, the result calculation module is also used to perform the following operations: The matching probabilities corresponding to the M word prediction vectors are determined as the prediction probabilities corresponding to the M word prediction vectors. The result calculation module is used to generate the sample confidence score corresponding to the sample text result based on the prediction probabilities corresponding to the M word prediction vectors. Specifically, it performs the following operations: The average of the prediction probabilities corresponding to the M word prediction vectors is used to determine the sample confidence level of the sample text result.

[0028] In one possible implementation, the result calculation module is also used to perform the following operations: Obtain business data and threshold prompt words, input the business data and threshold prompt words into the target large language model, extract features from the business data through the target large language model to obtain a business feature sequence, and extract features from the threshold prompt words to obtain a prompt word feature sequence; Obtain the matching probability of each candidate word element in the target large language model and the business feature sequence respectively, and obtain the target confidence threshold indicated by the prompt word based on the prompt word feature sequence; Candidate words with matching probabilities greater than or equal to the target confidence threshold are identified as R undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the words indicated by the T target word prediction vectors are determined as the predicted text results corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text results is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0029] In one possible implementation, the result calculation module is also used to perform the following operations: Acquire business data, input the business data into the target large language model, and extract features from the business data through the target large language model to obtain the business feature sequence; Obtain the matching probability of each candidate word in the target large language model and the business feature sequence, perform domain identification on the business feature sequence to obtain the business domain to which the business data belongs, and obtain the target confidence threshold corresponding to the business domain in the confidence mapping table; the confidence thresholds of different business domains in the confidence mapping table are different; Candidate words with matching probabilities greater than or equal to the target confidence threshold are identified as S undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the words indicated by the T target word prediction vectors are determined as the predicted text results corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text results is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0030] One embodiment of this application provides a computer device, including: a processor, a memory, and a network interface; The processor is connected to a memory and a network interface. The network interface is used to provide data communication functions, and the memory is used to store computer programs. When the computer program is executed by the processor, the computer device performs the method provided in the embodiments of this application.

[0031] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method provided in this application.

[0032] One embodiment of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the method provided in this application embodiment.

[0033] This application embodiment predicts business sample data to obtain M word prediction vectors. The words indicated by the M word prediction vectors are determined as sample text results. The sample confidence score corresponding to the sample text result is generated based on the prediction probabilities corresponding to the M word prediction vectors. During the initial training process of the large language model, a correctness reward value generated based on sample labels and sample text results is introduced, as well as a confidence score calibration reward value generated based on sample confidence score, sample labels, and sample text results. The accuracy reward value characterizes the accuracy of the initial large language model's output answer (e.g., by analyzing whether the sample labels and sample text results are the same). The confidence calibration reward value characterizes the error between the sample confidence value output by the initial large language model and the actual confidence value (e.g., the actual confidence value is the degree of matching between the sample labels and sample text results). Through the accuracy reward value and the confidence calibration reward value, the output answer and output confidence can be synergistically optimized, thereby establishing a direct link between the output answer and output confidence. This allows the initial large language model to learn the intrinsic correlation between the output answer and output confidence, enabling the prediction probabilities corresponding to the M word prediction vectors to accurately reflect the model's confidence in the output. Furthermore, the sample confidence value output by the model can be calibrated using sample labels, further improving the accuracy and reliability of the model's output confidence value. Meanwhile, by training reinforcement learning through confidence-calibrated reward values, a confidence correction mechanism can be established in the initial large language model. This allows the initial large language model to self-evaluate the accuracy of its confidence through prediction probabilities during the prediction process. By optimizing the output prediction results through the corrected confidence, the prediction results, as well as their accuracy and reliability, can be improved. Attached Figure Description

[0034] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0035] Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application; Figure 2 This is a schematic diagram of a data processing scenario provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 ; Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 ; Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 ; Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0037] Please see Figure 1 , Figure 1 This is a schematic diagram of a network architecture provided in an embodiment of this application. For example... Figure 1 As shown, the network architecture may include a service server 100 and a terminal device cluster. The terminal device cluster may include terminal devices 10a, 10b, ..., 10n. Each terminal device in the terminal device cluster may have a communication connection with the service server 100. For example, terminal device 10a may have a communication connection with the service server 100, and terminal device 10b may have a communication connection with the service server 100. The communication connection is not limited to a specific method. It may be a direct or indirect connection via wired communication, a direct or indirect connection via wireless communication, or other methods. This application does not impose any restrictions on these methods.

[0038] Each terminal device in the terminal device cluster can include: smartphones, tablets, laptops, desktop computers, intelligent voice interaction devices, smart home appliances (e.g., smart TVs), wearable devices, in-vehicle terminals, aircraft, and other intelligent terminals with data processing capabilities. In-vehicle terminals can be terminal devices used in intelligent transportation scenarios and assisted driving scenarios. It should be understood that, for example... Figure 1 Each terminal device in the terminal device cluster shown can be equipped with an application client that has data processing capabilities. When the application client runs on each terminal device, it can interact with the aforementioned... Figure 1 Data interaction is performed between the business servers 100 shown.

[0039] Specifically, the application client may include: in-vehicle client, smart home client, entertainment client (e.g., game client), multimedia client (e.g., video client), social client, and information client (e.g., news client), etc. In this embodiment, the application client may be integrated into a client (e.g., a social client) or may be a standalone client (e.g., a news client). This embodiment does not limit the type of application client.

[0040] like Figure 1 As shown, for ease of understanding, terminal device 10a is used as an example. Terminal device 10a can obtain business data input by the user through the application client. The business data can be text data, such as question text. Terminal device 10a can then send the business data to business server 100.

[0041] The business server 100 may deploy a target large language model. The business server 100 can input business data into the target large language model, which then generates text from the business data, obtaining the predicted text result corresponding to the business data and the prediction confidence score for the predicted text result. The prediction confidence score represents the target large language model's confidence level in the output predicted text result. Users can consider whether to ask further questions or modify their question text based on the prediction confidence score. Optionally, the terminal device 10a can be integrated with the business server 100 into a single device; this embodiment does not impose such limitations. The process of generating the predicted text result and prediction confidence score described above can be found below. Figure 2 The specific details of the corresponding embodiments.

[0042] Please see also Figure 2 , Figure 2 This is a schematic diagram of a data processing scenario provided in an embodiment of this application. For example... Figure 2 As shown, users can enter the question text in the application client installed on their terminal device. The terminal device can be one of the aforementioned... Figure 1 Any terminal device in the terminal device cluster, for example, terminal device 10a.

[0043] The terminal device can send the problem text to the business server, which can be one of the aforementioned... Figure 1 The business server 100 is configured to deploy a target large language model. The business server can input the question text into the target large language model, and then use the target large language model to perform probability prediction on the business sample data, obtaining the matching probability corresponding to each candidate word in the business dictionary within the target large language model.

[0044] Taking the input question text and the prediction of the next word in the question text as an example, the business server can obtain candidate words. Matching probability between the question text and the question text Candidate word elements Matching probability between the question text and the question text Candidate word elements Matching probability between the question text and the question text Etc. Matching probability Matching probability and matching probability Both are used to represent the probability that the target large language model predicts the next token in the input question text. Among them, the matching probability... Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of.

[0045] The business server can obtain the target confidence threshold, which can be obtained through prompt words, thought chains, or automatic recognition. This application embodiment does not impose any restrictions on this.

[0046] Taking obtaining the target confidence threshold through prompt words as an example, the computer device can acquire the threshold prompt words, input them into the target large language model, and extract features from the threshold prompt words through the target large language model to obtain the prompt word feature sequence. The target confidence threshold indicated by the threshold prompt words can then be obtained through the prompt word feature sequence. For example, the threshold prompt word could be "The confidence interval is [0,1], please output a prediction result with a confidence greater than 0.8".

[0047] Optionally, taking the automatic identification of the target confidence threshold as an example, the computer device can perform domain identification on the business feature sequence to obtain the business domain to which the business data belongs, and obtain the target confidence threshold corresponding to the business domain in the confidence mapping table. The confidence thresholds for different business domains in the confidence mapping table are different. For example, the confidence threshold corresponding to the audit domain is greater than the confidence threshold for the classification domain.

[0048] The business server can determine the matching probability corresponding to each candidate word as a word prediction vector. Taking the first word prediction vector as an example, the first word prediction vector is the word prediction vector. In word prediction vectors In this process, the business server can determine the prediction vector for each candidate word based on its matching probability. The indicated target lexical and its corresponding predicted probability. For example, the lexical prediction vector. Including matching probability Matching probability and matching probability Etc. If the matching probability and matching probability If the confidence threshold is greater than or equal to the confidence threshold, then the computer device can identify the lexical units. and word elements If it is determined to be an undetermined word element, and the matching probability is... Less than the matching probability Then the computer device can detect word elements. , identified as target lexical, lexical The corresponding matching probability is the word prediction vector. The corresponding predicted probability.

[0049] The business server can continue probability matching based on the question text and the first predicted target word. When the predicted target word meets the generation termination condition, the business server can obtain the target word and prediction probability corresponding to M word prediction vectors respectively. The business server can determine the target word indicated by each word prediction vector as the answer result corresponding to the question text, and determine the average of the M prediction probabilities as the confidence level for the answer result. This confidence level can be a score B, and the score B can be greater than or equal to the target confidence level threshold.

[0050] This application embodiment can dynamically adjust the target confidence threshold through mechanisms such as prompt words, thought chains, or automatic recognition, allowing the target large language model to flexibly adjust the balance between accuracy and recall according to different task requirements. For example, in tasks with high accuracy requirements, the system can set a higher confidence threshold to ensure output results with a low error rate; while in tasks with high recall, the threshold adjustment mechanism can ensure that a wider range of candidate answers are considered, improving the system's recall capability. This flexibility makes this application embodiment more applicable to various business scenarios in practical applications, such as review scenarios, classification scenarios, and question-answering scenarios. Taking the review scenario as an example, the target large language model can determine the confidence threshold under different review scenarios. For example, the confidence threshold in original video review can be higher than the confidence threshold in video tag review, flexibly adapting to the confidence requirements of different scenarios.

[0051] Please see Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 This data processing method can be executed by a computer device, which can be, for example, Figure 1The terminal device 10a shown is an integrated device with the business server 100. The following description will use the example of this data processing method being executed by a computer device. This data processing method may include at least the following steps S101-S104: Step S101: Obtain business sample data, input the business sample data into the initial large language model, and predict the business sample data through the initial large language model to obtain M word prediction vectors; M is a positive integer; Specifically, the computer device can acquire business sample data, which can be text data (also referred to as text sample data). The computer device can input the business sample data into an initial large language model. The computer device can then extract features from the business sample data using the initial large language model to obtain a sample feature sequence, which can be represented as […]. ].in, All of these are unit feature vectors corresponding to word segments obtained by splitting business sample data into word segments. The number of word segments and unit feature vectors is t-1.

[0052] The initial large language model can integrate a business dictionary, which is the semantic vector space of the initial large language model, including the mapping relationship between candidate words and their corresponding word vectors. Computer devices can perform probability matching on sample feature sequences based on candidate words in the business dictionary of the initial large language model to obtain M word prediction vectors.

[0053] Step S102: Determine the words indicated by the M word prediction vectors as the sample text results corresponding to the business sample data, and generate the sample confidence level corresponding to the sample text results based on the prediction probabilities corresponding to the M word prediction vectors respectively. Specifically, the computer device can obtain the prediction probabilities corresponding to the M word prediction vectors. These prediction probabilities are the conditional probability distributions calculated by the initial large language model decoder when outputting word prediction vectors, and can be expressed as follows: .in, This represents the next word predicted based on the input sample feature sequence. The M word prediction vectors can include word prediction vectors... lexical prediction vector ..., word prediction vector .

[0054] For ease of understanding, let's take the word prediction vectors from the M word prediction vectors as an example. For example, word prediction vector This can include the matching probability corresponding to each candidate word in the business dictionary of the initial large language model, for example, including candidate words. Corresponding matching probability Candidate word elements Corresponding matching probability Candidate word elements Corresponding matching probability Etc. Matching probability Matching probability and matching probability Both are used to represent the probability that the initial large language model predicts the next word in the input question text. Among them, the matching probability... Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of.

[0055] Computer devices can predict word vectors The candidate word with the highest matching probability is determined as the target word, and the matching probability of the target word is determined as the word prediction vector. The corresponding predicted probability. For example, the word prediction vector. Matching probability If the maximum value is reached, then the word prediction vector will be... The corresponding target lexical is the lexical unit. lexical prediction vector The corresponding predicted probability is the probability. That is, the next word predicted based on the input sample feature sequence. as a word element .

[0056] The computer device can obtain the target words and prediction probabilities corresponding to M word prediction vectors, and determine the target word indicated by each word prediction vector as the sample text result corresponding to the business sample data. The computer device can generate the sample confidence score for the sample text result based on the M prediction probabilities, as shown in formula (1): Formula (1) Where q represents the sample confidence level of the sample text result. This represents the matching probability of the i-th candidate word in the j-th word prediction vector. This represents the maximum matching probability in the j-th word prediction vector, which is the prediction probability corresponding to the j-th word prediction vector.

[0057] The confidence level of a sample can also be obtained through instructions such as prompts, chains of thought (CoT), or decoding strategies, and this embodiment of the application does not impose any limitations on these methods. Obtaining the prediction probability through prompts can mean explicitly requiring the model to output the prediction probability in the prompt, for example: "Please answer the input question text and output the confidence level of the answer at the end." Obtaining the prediction probability through chains of thought can mean exposing the model's judgment process through step-by-step reasoning; for example, the text corresponding to the chain of thought could be: "Please analyze the question text step by step and give an answer, and finally evaluate the reliability of each step and the overall confidence level." Obtaining the prediction probability through decoding strategies can mean adjusting the decoder's decoding process, such as adjusting it to non-greedy decoding (non-greedy decoding can retain the normalized probability distribution, i.e., the matching probability of each candidate word, and the decoder can output the maximum matching probability at each position as the confidence level of the word at that position).

[0058] Step S103: Based on the sample labels and sample text results corresponding to the business sample data, generate a correctness reward value, and based on the sample confidence, sample labels and sample text results, generate a confidence calibration reward value. Specifically, the computer equipment can extract features from the sample labels corresponding to the business sample data to obtain sample label vectors. It then concatenates the predicted vectors of M words to obtain the sample text vector. Vector similarity is calculated between the sample label vector and the sample text vector to obtain the feature similarity between the sample label and sample text results. The vector similarity can be cosine similarity, Euclidean distance, Manhattan distance, etc., and this embodiment of the application is not limited thereto. Feature similarity reflects the degree of matching between the sample label vector and the sample text vector.

[0059] Computer devices can generate a correctness reward value based on feature similarity. If the feature similarity is greater than or equal to a similarity threshold, the first reward value is determined as the correctness reward value; if the feature similarity is less than the similarity threshold, the second reward value is determined as the correctness reward value. The first reward value is greater than the second reward value. The above calculation process can be shown in formula (2): Formula (2) in, This represents the reward value for correctness. This represents the first reward value, for example, 1. This represents the second reward value, for example, 0. Indicates feature similarity. This represents the similarity threshold. When the similarity threshold is 1, it means that a numerical value is applied only if the sample text result is exactly the same as the sample label. The correctness reward value.

[0060] Computer devices can use Brier scores as an evaluation metric for confidence calibration, where the Brier score is a measure of the accuracy of probabilistic predictions. The Brier score measures the reliability of confidence by calculating the difference between the sample confidence score q of the initial large language model output and the sample label. A larger reward is given for correct answers with high confidence; a penalty is given for incorrect answers or inaccurate confidence.

[0061] Taking the Brier score as an example, the computer device can obtain the calibration value based on feature similarity, and generate the confidence calibration reward value based on the calibration value and sample confidence. The process is shown in formula (3): Formula (3) in, This represents the confidence level calibration reward value, also known as the confidence level error penalty value, used to represent the error between the sample confidence level and the actual confidence level. The actual confidence level is... , It can be determined based on feature similarity, and can also be called a calibration value. This value is used when the sample text matches the sample label. The larger the value, the better. When the sample text result does not match the sample label, The smaller the value, the better. q represents the sample confidence score output by the initial large language model. Optionally, when the sample confidence score q is too small, It can also be set to a smaller value.

[0062] Step S104: Based on the confidence score and the correctness score, perform reinforcement learning training on the initial large language model to obtain the target large language model; the target large language model is used to generate the predicted text results corresponding to the business data and the prediction confidence score for the predicted text results.

[0063] Specifically, the computer equipment can weight the confidence calibration reward value and the correctness reward value to obtain a comprehensive reward value, as shown in formula (4): Formula (4) in, This represents the overall reward value. The weights for confidence level calibration reward values ​​are preset adjustment parameters used to adjust the influence between confidence level calibration reward values ​​and accuracy reward values.

[0064] The computer device can perform reinforcement learning training on the initial large language model based on the comprehensive reward value. The computer device can generate the gradient of the initial large language model for the business sample data, and adjust the parameters of the initial large language model through the gradient. When the initial large language model meets the training convergence condition, the initial large language model that meets the training convergence condition is determined as the target large language model. The process is shown in formula (5): Formula (5) in, This represents the model parameters after parameter adjustment. This represents the model parameters before parameter adjustment. This represents the preset learning rate. The initial large language model uses this formula to adjust its model parameters to optimize answer generation and confidence calibration.

[0065] It is understood that the training convergence condition may refer to the following during the training process: if the overall reward value does not decrease significantly after multiple training epochs or training batches, or the training batch reaches a preset minimum value, or a preset performance metric (such as recall, accuracy, etc.) is achieved. This application embodiment does not impose any restrictions on the training convergence condition.

[0066] This application embodiment predicts business sample data to obtain M word prediction vectors. The words indicated by the M word prediction vectors are determined as sample text results. The sample confidence score corresponding to the sample text result is generated based on the prediction probabilities corresponding to the M word prediction vectors. During the initial training process of the large language model, a correctness reward value generated based on sample labels and sample text results is introduced, as well as a confidence score calibration reward value generated based on sample confidence score, sample labels, and sample text results. The accuracy reward value characterizes the accuracy of the initial large language model's output answer (e.g., by analyzing whether the sample labels and sample text results are the same). The confidence calibration reward value characterizes the error between the sample confidence value output by the initial large language model and the actual confidence value (e.g., the actual confidence value is the degree of matching between the sample labels and sample text results). Through the accuracy reward value and the confidence calibration reward value, the output answer and output confidence can be synergistically optimized, thereby establishing a direct link between the output answer and output confidence. This allows the initial large language model to learn the intrinsic correlation between the output answer and output confidence, enabling the prediction probabilities corresponding to the M word prediction vectors to accurately reflect the model's confidence in the output. Furthermore, the sample confidence value output by the model can be calibrated using sample labels, further improving the accuracy and reliability of the model's output confidence value. Meanwhile, by training reinforcement learning through confidence-calibrated reward values, a confidence correction mechanism can be established in the initial large language model. This allows the initial large language model to self-evaluate the accuracy of its confidence through prediction probabilities during the prediction process. By optimizing the output prediction results through the corrected confidence, the prediction results, as well as their accuracy and reliability, can be improved.

[0067] Please see Figure 4 , Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 This data processing method can be executed by a computer device, which can be, for example, Figure 1 The terminal device 10a shown is an integrated device with the business server 100. The following description will use the example of this data processing method being executed by a computer device. This data processing method may include at least the following steps S201-S208: Step S201: Obtain business sample data; Specifically, the computer device can obtain a training batch dataset from the training dataset, which can also be called business sample data. In this embodiment of the application, the batch size is not limited.

[0068] Step S202: Generate sample text results and sample confidence scores; Specifically, the computer device can input business sample data into an initial large language model. This initial large language model may include an encoder. The computer device can use the encoder in the initial large language model to perform token segmentation on the business sample data, obtaining N unit tokens corresponding to the business sample data. Token segmentation can be performed by using a tokenizer to split the business sample data into tokens. A token is the smallest basic unit obtained after token segmentation of the business sample data. The token segmentation method can be word-based, character-based, or subword-based; this embodiment does not impose any limitations on this method.

[0069] Taking word segmentation as an example, word segmentation methods can include rule-based segmentation methods, statistical segmentation methods, deep learning-based segmentation methods, etc., and this application embodiment does not impose any limitations. For ease of understanding, the inverse maximum matching method in rule-based segmentation methods will be used as an example for explanation. The computer device can obtain the business dictionary (preset dictionary) and the initial length threshold in the initial large language model. The computer device can obtain candidate strings with the same character length as the initial length threshold in the business sample data according to the arrangement order of characters in the business sample data. For example, the business sample data can be represented as the text "Let's do tokenization", and the initial length threshold can be 8. Then the computer device can scan from right to left to obtain strings with a character length of 8 in the business text, and obtain the candidate string "nization".

[0070] The computer device can search and match the candidate string "nization" in the business dictionary. If the candidate string matches the business dictionary, the computer device can identify the candidate string as a business term. If the candidate string does not match the business dictionary, the computer device will decrease the initial length threshold to obtain a new initial length threshold. For example, the new initial length threshold can be 7.

[0071] The computer device can determine new candidate strings in the business sample data based on a new initial length threshold, resulting in the new candidate string "ization". If the new candidate string matches the business dictionary, the computer device can identify the new candidate string as a candidate word. If the new candidate string does not match the business dictionary, the computer device can continue to decrease the new initial length threshold until the new candidate string matches the business dictionary. For example, the candidate word obtained by this scan could be "zation".

[0072] The computer device can identify the text in the business text, excluding candidate words, as the text to be matched. For example, the text to be matched could be "Let's do token". The computer device can then continue scanning the text to be matched, based on an initial length threshold, from right to left to obtain strings of length 8, resulting in the candidate string "sdotoken". The computer device can then continue searching and matching the candidate string "sdotoken" in the business dictionary, adjusting the initial length threshold until a new candidate string matches the business dictionary; for example, the candidate word obtained in this scan could be "token".

[0073] When the business sample data has been traversed, the N unit tokens obtained by splitting can include [Let,'s], [do], [token], and [ization].

[0074] The computer device can encode the N unit words obtained from word segmentation, resulting in vector representations corresponding to each of the N unit words, i.e., the N word word encoding vectors corresponding to the business sample data. The computer device can also generate N positional encoding vectors based on the positional order of each unit word in the business sample data. These positional encoding vectors represent the positional features of the word words.

[0075] Computer equipment can concatenate each word encoding vector with its corresponding positional encoding vector to obtain N unit feature vectors. For example, the word encoding vector corresponding to the word [Let] will be concatenated with the positional encoding vector corresponding to the word [Let] to obtain the unit feature vector corresponding to the word [Let]. Computer equipment can concatenate N unit feature vectors to obtain a sample feature sequence. The sample feature sequence can be represented as [ ].in, Both represent the unit feature vectors corresponding to the word segments obtained by splitting business data into word segments, and the number of word segments and unit feature vectors is t-1.

[0076] Computer devices can add start-of-sequence markers to sample feature sequences. These start-of-sequence markers can be represented as... <bos>Logits are calculated through the attention processing layer in the decoder. Taking the processing of the attention processing layer as an example, the computer device can obtain the candidate word sequence composed of candidate word words in the business dictionary of the initial large language model, and perform cross-attention processing on the candidate word sequence and the sample feature sequence.

[0077] Computer devices can combine sample feature sequences with start tags with query parameter matrices. Perform a dot product operation to obtain the query vector Q, and then combine the candidate word sequence with the key parameter matrix. Perform a dot product operation to obtain the key vector K, and then combine the candidate word sequence with the value parameter matrix. Performing a dot product operation yields a value vector V, which the computer device can then query. The result of the transpose operation with the key vector K Perform a dot product operation to obtain the attention score vector. This is based on the dimension value of the key vector K. square root The attention score matrix is ​​reduced in dimensionality, and the reduced attention score vector is then normalized. The normalized vector is obtained by multiplying the normalized vector with the value vector V to obtain the attention score corresponding to each candidate word. The process can be shown in formula (6): Formula (6) in, This represents the attention score between the i-th candidate word and the sample feature sequence.

[0078] The computer device can generate the matching probability corresponding to each candidate word based on the attention score corresponding to each candidate word. The computer device can determine the feature vector corresponding to the candidate word with the highest matching probability as the first target word of the initial large language model. The output process can be shown in formula (7): Formula (7) in, This represents the first target word predicted based on the input sample feature sequence.

[0079] The M word prediction vectors can include word prediction vectors lexical prediction vector ..., word prediction vector For ease of understanding, let's take the word prediction vectors from the M word prediction vectors as an example. For example, word prediction vector This can include the matching probability corresponding to each candidate word in the business dictionary of the initial large language model, for example, including candidate words. Corresponding matching probability Candidate word elements Corresponding matching probability Candidate word elements Corresponding matching probability Etc. Matching probability Matching probability and matching probability Both are used to represent the probability that the initial large language model predicts the next word in the input question text. Among them, the matching probability... Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability. Computer devices can predict word vectors. The candidate word with the highest matching probability is determined as the target word, and the matching probability of the target word is determined as the word prediction vector. The corresponding predicted probability. For example, the word prediction vector. Matching probability If the maximum value is reached, then the word prediction vector will be... The corresponding target lexical is the lexical unit. lexical prediction vector The corresponding predicted probability is the probability. That is, the next word predicted based on the input sample feature sequence. as a word element .

[0080] The computer device can update the sample feature sequence based on the first target word to obtain a new sample feature sequence. For example, the unit feature vector corresponding to the first target word can be concatenated to the sample feature sequence to obtain a new sample feature sequence. The computer device can continue to perform probability matching in the new sample feature sequence to obtain the next target word of the first target word, until the obtained next target word meets the generation termination condition. Then, the feature vectors corresponding to the M target words are determined as M word prediction vectors. The generation termination condition can refer to the obtained next target word being a stop marker, or reaching the maximum output length of the initial large language model. This embodiment of the application does not impose any limitations on this.

[0081] The computer device can acquire the target word and prediction probability corresponding to M word prediction vectors, and determine the word indicated by each word prediction vector as the sample text result corresponding to the business sample data. The computer device can generate a sample confidence score for the sample text result based on the M prediction probabilities. The sample confidence score can be the average of the M prediction probabilities or a weighted average, which is not limited in this embodiment.

[0082] Step S203: Generate a correctness reward value; Specifically, computer equipment can generate a correctness reward value based on the sample labels and sample text results corresponding to the business sample data. The process can be found above. Figure 3 The specific description of step S103 in the corresponding embodiment will not be repeated here.

[0083] Step S204: Generate confidence calibration reward value; Specifically, the computer equipment can extract features from the sample labels corresponding to the business sample data to obtain sample label vectors, and concatenate the M word prediction vectors to obtain sample text vectors. Vector similarity is then calculated between the sample label vectors and the sample text vectors to obtain the feature similarity between the sample labels and the sample text results. The vector similarity can be cosine similarity, Euclidean distance, Manhattan distance, etc., and this embodiment of the application does not impose any limitations.

[0084] Taking cosine similarity as an example, the computer device can generate the first modulus parameter of the sample text vector and the second modulus parameter corresponding to the sample label vector. The sample text vector and the sample label vector are multiplied to obtain the dot product parameter. Based on the first modulus parameter, the second modulus parameter and the dot product parameter, the cosine similarity between the sample text vector and the sample label vector is generated. The process is shown in formula (8): Formula (8) in, The cosine similarity between sample text vectors and sample label vectors is represented by... Represents the sample text vector. The first modulus parameter represents the sample text vector. Represents the sample label vector. The second modulus parameter represents the sample label vector.

[0085] If the feature similarity is greater than or equal to the similarity threshold, the computer device determines the maximum value within the confidence interval to which the sample confidence belongs as the calibration value. Taking the confidence interval to which the sample confidence belongs as [0,1] as an example, if the feature similarity is greater than or equal to the similarity threshold, the computer device can set the calibration value to 1. If the feature similarity is less than the similarity threshold, the minimum value within the confidence interval is determined as the calibration value, for example, it can be 0. The computer device can determine the difference between the calibration value and the sample confidence as the matching error value, perform a power operation on the matching error value to obtain the mean square error value, and determine the negative of the mean square error value as the confidence calibration reward value. The process is shown in formula (9): Formula (9) in, This represents the confidence level calibration reward value, also known as the confidence level error penalty value, which is used to represent the error between the sample confidence level and the actual confidence level. The cosine similarity between sample text vectors and sample label vectors is represented by... This represents the similarity threshold corresponding to the cosine similarity. q represents the sample confidence score output by the initial large language model. Optionally, when the sample confidence score q is too small, It can be set to 0.

[0086] It is understandable that the above Figure 3 The calibration value in formula (3) in the corresponding embodiment The value can be 0 or 1, depending on whether the sample text result matches the sample label. The value is 1. This occurs when the sample text result does not match the sample label. The value is 0. When the calibration value... When the value of is 0 or 1, formula (3) is the same as formula (9).

[0087] Step S205: Generate the comprehensive reward value; Specifically, the computer equipment can weight the confidence calibration reward value and the correctness reward value to obtain a comprehensive reward value. The calculation process for the comprehensive reward value can be as described above. Figure 3 The specific description of step S104 in the corresponding embodiment will not be repeated here.

[0088] Step S206: Update model parameters; Step S207: Determine whether to terminate model training; Step S208: When the model training is terminated, the target large language model is obtained.

[0089] Specifically, the computer device can perform reinforcement learning training on the initial large language model based on the comprehensive reward value. The computer device can generate gradients for the initial large language model in relation to the business sample data, and adjust the parameters of the initial large language model using these gradients. When the initial large language model meets the training convergence condition, the model training is terminated, and the initial large language model that meets the training convergence condition is determined as the target large language model. The training convergence condition can refer to the following: during the training process, if the comprehensive reward value does not significantly decrease after multiple training cycles or training batches, or if the training batch reaches a preset minimum value, or if a preset performance metric (such as recall or accuracy) is reached. This embodiment of the application does not impose limitations on the training convergence condition.

[0090] This application embodiment predicts business sample data to obtain M word prediction vectors. The words indicated by the M word prediction vectors are determined as sample text results. The sample confidence score corresponding to the sample text result is generated based on the prediction probabilities corresponding to the M word prediction vectors. During the initial training process of the large language model, a correctness reward value generated based on sample labels and sample text results is introduced, as well as a confidence score calibration reward value generated based on sample confidence score, sample labels, and sample text results. The accuracy reward value characterizes the accuracy of the initial large language model's output answer (e.g., by analyzing whether the sample labels and sample text results are the same). The confidence calibration reward value characterizes the error between the sample confidence value output by the initial large language model and the actual confidence value (e.g., the actual confidence value is the degree of matching between the sample labels and sample text results). Through the accuracy reward value and the confidence calibration reward value, the output answer and output confidence can be synergistically optimized, thereby establishing a direct link between the output answer and output confidence. This allows the initial large language model to learn the intrinsic correlation between the output answer and output confidence, enabling the prediction probabilities corresponding to the M word prediction vectors to accurately reflect the model's confidence in the output. Furthermore, the sample confidence value output by the model can be calibrated using sample labels, further improving the accuracy and reliability of the model's output confidence value. Meanwhile, by training reinforcement learning through confidence-calibrated reward values, a confidence correction mechanism can be established in the initial large language model. This allows the initial large language model to self-evaluate the accuracy of its confidence through prediction probabilities during the prediction process. By optimizing the output prediction results through the corrected confidence, the prediction results, as well as their accuracy and reliability, can be improved.

[0091] On the other hand, this application's embodiments optimize the training process of a large language model by combining reinforcement learning and a confidence calibration mechanism. This allows the model to not only output answers but also provide corresponding confidence scores, thereby adjusting the accuracy and recall of the review results based on the confidence level. Reinforcement learning training is performed using correctness reward values ​​and confidence calibration reward values. During reinforcement learning, the model not only optimizes its output answers based on the task's reward signals but also reflects on and corrects its own confidence, gradually improving future predictions. A confidence correction mechanism and a self-reflection mechanism can be established in the initial large language model, enabling the model to not only update parameters based on task feedback but also self-evaluate the accuracy of its confidence during prediction and adjust the confidence generation strategy based on feedback. This significantly improves the performance of the target large language model in tasks with various confidence outputs, especially in scenarios requiring high precision and high reliability.

[0092] Please see Figure 5 , Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 3 This data processing method can be executed by a computer device, which can be, for example, Figure 1 The terminal device 10a shown is an integrated device with the business server 100. The following description will use the example of this data processing method being executed by a computer device. This data processing method will at least include the following steps S301-S305: Step S301: Obtain business data, input the business data into the target large language model, and extract features from the business data through the target large language model to obtain a business feature sequence; Specifically, computer equipment can acquire business data, input the business data into a target large language model, and then extract features from the business data using the target large language model to obtain a business feature sequence. The process of obtaining the business feature sequence can be found above. Figure 4 The process of obtaining the sample feature sequence in step S201 of the corresponding embodiment will not be repeated here.

[0093] Step S302: Obtain the matching probability of each candidate word in the target large language model and the business feature sequence respectively; Specifically, the computer device can obtain the matching probability of each candidate word in the target large language model and the business feature sequence. The process of obtaining the matching probability of each candidate word can be found above. Figure 4 The process of step S202 in the corresponding embodiment will not be described again in this application embodiment.

[0094] Step S303: Obtain the target confidence threshold; Specifically, the computer device can obtain the target confidence threshold indicated when applying the target large language model. The target confidence threshold can be obtained through prompt words, thought chains, or automatic recognition, and this application embodiment does not impose any limitations on this.

[0095] Taking obtaining the target confidence threshold through prompt words as an example, the computer device can acquire the threshold prompt words, input the threshold prompt words into the target large language model, and extract features from the threshold prompt words through the target large language model to obtain the prompt word feature sequence. The target confidence threshold indicated by the threshold prompt words can then be obtained through the prompt word feature sequence. For example, the threshold prompt words could be "The confidence interval is [0,1], please output a prediction result with a confidence greater than 0.8".

[0096] Taking the automatic identification of target confidence thresholds as an example, computer equipment can perform domain identification on business feature sequences to determine the business domain to which the business data belongs, and obtain the target confidence threshold corresponding to the business domain in the confidence mapping table. The confidence thresholds for different business domains are different in the confidence mapping table. For example, the confidence threshold corresponding to the audit domain is higher than the confidence threshold for the classification domain.

[0097] Step S304: The candidate words with matching probabilities greater than or equal to the confidence threshold are determined as R undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is determined as the target word prediction vector; R is a positive integer. Specifically, taking the prediction of the next term in business data as an example, computer devices can acquire candidate terms. Matching probability with business data Candidate word elements Matching probability with business data Candidate word elements Matching probability with business data Etc. Matching probability Matching probability and matching probability Both are used to represent the probability that the target large language model predicts the next word in the input business data. Among them, the matching probability... Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of matching Indicates that the next lexical is a candidate lexical. The probability of.

[0098] Computer devices can match probabilities Matching probability and matching probability The resulting vector is determined as the first target word prediction vector. Within this first target word prediction vector, the computer device can determine the target word indicated by the first target word prediction vector and its corresponding prediction probability based on the matching probability of each candidate word. If the matching probability... and matching probability If the confidence threshold is greater than or equal to the confidence threshold, then the computer device can identify the lexical units. and word elements If it is determined to be an undetermined word element, and the matching probability is... Less than the matching probability Then the computer device can detect word elements. The target word corresponding to the first target word prediction vector is determined. The corresponding matching probability is the prediction probability corresponding to the prediction vector of the first target word.

[0099] Step S305: When T target word prediction vectors are obtained, the word indicated by the T target word prediction vectors is determined as the predicted text result corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text result is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0100] Specifically, when T target word prediction vectors are obtained, the computer device can determine the target words indicated by the T target word prediction vectors as the predicted text result corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, a prediction confidence score is generated for the predicted text result. The calculation method for the prediction confidence score can be found above. Figure 4 The method for calculating sample confidence in step S202 of the corresponding embodiment will not be repeated here.

[0101] This application's embodiments dynamically adjust the target confidence threshold through mechanisms such as prompt words, thought chains, or automatic recognition. A dynamic confidence threshold adjustment mechanism is designed, enabling the review system to dynamically adjust the confidence threshold according to task requirements. The system sets different confidence thresholds based on different task types. For example, when handling legal or medical-related review tasks, a higher confidence threshold may be needed, accepting answers only when the model's confidence is sufficiently high. In tasks that tolerate a certain level of error, the system can lower the confidence threshold to include more candidate answers and ensure a higher recall rate. During model training, a multi-task learning method is used, allowing the model to not only train on a specific task but also dynamically adjust the confidence threshold based on task feedback. Task feedback includes user error feedback and model self-evaluation. The system can adjust the model's confidence range based on this feedback, thereby continuously optimizing the system's performance in different tasks.

[0102] On the other hand, this application's embodiments optimize the training process of large language models by combining reinforcement learning and confidence calibration mechanisms. This allows the model to not only output answers but also provide corresponding confidence scores. The model not only optimizes its output answers based on the task's reward signals but also reflects on and corrects its own confidence, thereby gradually improving future predictions. A confidence correction mechanism and a self-reflection mechanism can be established in the initial large language model. Through these mechanisms, the model can continuously adjust its confidence evaluation criteria based on feedback, thus gradually improving the accuracy and reliability of predictions over long-term training. Experimental results show that the model's calibration and accuracy continuously improve with deeper training. This means that in complex, multi-step reasoning tasks, the model can effectively avoid the propagation of initial errors and improve its prediction and evaluation capabilities through continuous self-reflection.

[0103] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 6 As shown, the data processing device 1 includes a prediction module 610, a result calculation module 620, a reward calculation module 630, and a training module 640.

[0104] Prediction module 610 is used to acquire business sample data, input the business sample data into the initial large language model, and predict the business sample data through the initial large language model to obtain M word prediction vectors; M is a positive integer; The result calculation module 620 is used to determine the words indicated by the M word prediction vectors as the sample text results corresponding to the business sample data, and generate the sample confidence level corresponding to the sample text results based on the prediction probabilities corresponding to the M word prediction vectors respectively. The reward calculation module 630 is used to generate a correctness reward value based on the sample labels and sample text results corresponding to the business sample data, and to generate a confidence calibration reward value based on the sample confidence, sample labels and sample text results. Training module 640 is used to perform reinforcement learning training on the initial large language model based on confidence and correctness reward values ​​to obtain the target large language model; the target large language model is used to generate the predicted text results corresponding to the business data and the prediction confidence of the predicted text results.

[0105] In one possible implementation, when the reward calculation module 630 generates a correctness reward value based on the sample labels and sample text results corresponding to the business sample data, it specifically performs the following operations: The feature similarity between the sample labels and sample text results corresponding to the generated business sample data; If the feature similarity is greater than or equal to the similarity threshold, then the first reward value is determined as the correctness reward value; If the feature similarity is less than the similarity threshold, the second reward value is determined as the correctness reward value; the first reward value is greater than the second reward value.

[0106] In one possible implementation, when the reward calculation module 630 generates a confidence-calibrated reward value based on the sample confidence, sample label, and sample text results, it specifically performs the following operations: In one possible implementation, when the reward calculation module 630 is used for…, it is specifically used to perform the following operations: The feature similarity between the sample labels and sample text results corresponding to the generated business sample data; The calibration value is obtained based on feature similarity, and a confidence calibration reward value is generated based on the calibration value and the sample confidence.

[0107] In one possible implementation, the reward calculation module 630 is used to obtain a calibration value based on feature similarity, and when generating a confidence calibration reward value based on the target matching value and sample confidence, it is specifically used to perform the following operations: If the feature similarity is greater than or equal to the similarity threshold, the maximum value within the confidence interval to which the sample confidence belongs is determined as the calibration value; if the feature similarity is less than the similarity threshold, the minimum value within the confidence interval is determined as the calibration value. The difference between the calibration value and the sample confidence level is determined as the matching error value; The matching error value is exponentially calculated to obtain the mean squared error value. The negative of the mean squared error value is then used as the confidence calibration reward value.

[0108] In one possible implementation, the training module 640 is used to perform reinforcement learning training on the initial large language model based on confidence-calibrated reward values ​​and correctness reward values. When the target large language model is obtained, it is specifically used to perform the following operations: The confidence calibration reward value and the correctness reward value are weighted to obtain the comprehensive reward value; Based on the comprehensive reward value, the model parameters of the initial large language model are adjusted. When the initial large language model meets the training convergence condition, the initial large language model that meets the training convergence condition is determined as the target large language model.

[0109] In one possible implementation, when the prediction module 610 predicts the business sample data using the initial large language model and obtains M word prediction vectors, it specifically performs the following operations: Feature extraction is performed on the business sample data using an initial large language model to obtain the sample feature sequence; Based on the candidate lexical units in the business dictionary of the initial large language model, probability matching is performed on the sample feature sequence to obtain M lexical prediction vectors.

[0110] In one possible implementation, the initial large language model includes an encoding layer; the prediction module 610 is used to extract features from the business sample data using the initial large language model, and when obtaining the sample feature sequence, it is specifically used to perform the following operations: The business sample data is segmented into N unit words by the encoding layer in the initial large language model; N is a positive integer. Encode each of the N unit words to obtain N unit word encoding vectors. Based on the positional order of each unit word in the business sample data, generate N positional encoding vectors. Based on N word encoding vectors and N position encoding vectors, N unit feature vectors are generated. The N unit feature vectors are then concatenated to obtain the sample feature sequence.

[0111] In one possible implementation, when the prediction module 610 performs word segmentation on the business sample data through the encoding layer in the initial large language model to obtain N unit words corresponding to the business sample data, it specifically performs the following operations: Obtain the business dictionary and initial length threshold from the initial large language model; Based on the character arrangement order in the business sample data, candidate strings with the same character length as the initial length threshold are obtained from the business sample data; If a candidate string matches the business dictionary, the candidate string is determined as a unit word. If the candidate string does not match the business dictionary, the initial length threshold is decremented to obtain a new initial length threshold. Based on the new initial length threshold, a new candidate string is determined in the business sample data. If the new candidate string matches the business dictionary, the new candidate string is determined as a unit word. Characters other than unit words in the business sample data are identified as text to be matched. New unit words are then identified in the text to be matched based on the initial length threshold until all business sample data has been traversed and N unit words are obtained.

[0112] In one possible implementation, when the prediction module 610 performs probability matching on the sample feature sequence based on candidate lexical units in the business dictionary of the initial large language model to obtain M lexical unit prediction vectors, it specifically performs the following operations: Obtain a sequence of candidate lexical units composed of candidate lexical units from the business dictionary in the initial large language model; Cross-attention processing is performed on the candidate word sequence and the sample feature sequence to obtain the matching probability corresponding to each candidate word. The feature vector corresponding to the candidate word with the highest matching probability is determined as the first target word. The sample feature sequence is updated based on the first target word to obtain a new sample feature sequence. Probability matching is then performed in the new sample feature sequence to obtain the next target word of the first target word. This process continues until the next target word meets the generation termination condition. The feature vectors corresponding to the M target words are then determined as the M word prediction vectors.

[0113] In one possible implementation, the result calculation module 620 is also used to perform the following operations: The matching probabilities corresponding to the M word prediction vectors are determined as the prediction probabilities corresponding to the M word prediction vectors. The result calculation module 620 is used to generate the sample confidence score corresponding to the sample text result based on the prediction probabilities corresponding to the M word prediction vectors. Specifically, it performs the following operations: The average of the prediction probabilities corresponding to the M word prediction vectors is used to determine the sample confidence level of the sample text result.

[0114] In one possible implementation, the result calculation module 620 is also used to perform the following operations: Obtain business data and threshold prompt words, input the business data and threshold prompt words into the target large language model, extract features from the business data through the target large language model to obtain a business feature sequence, and extract features from the threshold prompt words to obtain a prompt word feature sequence; Obtain the matching probability of each candidate word element in the target large language model and the business feature sequence respectively, and obtain the target confidence threshold indicated by the prompt word based on the prompt word feature sequence; Candidate words with matching probabilities greater than or equal to the target confidence threshold are identified as R undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the words indicated by the T target word prediction vectors are determined as the predicted text results corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text results is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0115] In one possible implementation, the result calculation module 620 is also used to perform the following operations: Acquire business data, input the business data into the target large language model, and extract features from the business data through the target large language model to obtain the business feature sequence; Obtain the matching probability of each candidate word in the target large language model and the business feature sequence, perform domain identification on the business feature sequence to obtain the business domain to which the business data belongs, and obtain the target confidence threshold corresponding to the business domain in the confidence mapping table; the confidence thresholds of different business domains in the confidence mapping table are different; Candidate words with matching probabilities greater than or equal to the target confidence threshold are identified as S undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the words indicated by the T target word prediction vectors are determined as the predicted text results corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence corresponding to the predicted text results is generated; T is a positive integer, and the prediction confidence is greater than or equal to the target confidence threshold.

[0116] This application embodiment predicts business sample data to obtain M word prediction vectors. The words indicated by the M word prediction vectors are determined as sample text results. The sample confidence score corresponding to the sample text result is generated based on the prediction probabilities corresponding to the M word prediction vectors. During the initial training process of the large language model, a correctness reward value generated based on sample labels and sample text results is introduced, as well as a confidence score calibration reward value generated based on sample confidence score, sample labels, and sample text results. The accuracy reward value characterizes the accuracy of the initial large language model's output answer (e.g., by analyzing whether the sample labels and sample text results are the same). The confidence calibration reward value characterizes the error between the sample confidence value output by the initial large language model and the actual confidence value (e.g., the actual confidence value is the degree of matching between the sample labels and sample text results). Through the accuracy reward value and the confidence calibration reward value, the output answer and output confidence can be synergistically optimized, thereby establishing a direct link between the output answer and output confidence. This allows the initial large language model to learn the intrinsic correlation between the output answer and output confidence, enabling the prediction probabilities corresponding to the M word prediction vectors to accurately reflect the model's confidence in the output. Furthermore, the sample confidence value output by the model can be calibrated using sample labels, further improving the accuracy and reliability of the model's output confidence value. Meanwhile, by training reinforcement learning through confidence-calibrated reward values, a confidence correction mechanism can be established in the initial large language model. This allows the initial large language model to self-evaluate the accuracy of its confidence through prediction probabilities during the prediction process. By optimizing the output prediction results through the corrected confidence, the prediction results, as well as their accuracy and reliability, can be improved.

[0117] On the other hand, this application's embodiments optimize the training process of a large language model by combining reinforcement learning and a confidence calibration mechanism. This allows the model to not only output answers but also provide corresponding confidence scores, thereby adjusting the accuracy and recall of the review results based on the confidence level. Reinforcement learning training is performed using correctness reward values ​​and confidence calibration reward values. During reinforcement learning, the model not only optimizes its output answers based on the task's reward signals but also reflects on and corrects its own confidence, gradually improving future predictions. A confidence correction mechanism and a self-reflection mechanism can be established in the initial large language model, enabling the model to not only update parameters based on task feedback but also self-evaluate the accuracy of its confidence during prediction and adjust the confidence generation strategy based on feedback. This significantly improves the performance of the target large language model in tasks with various confidence outputs, especially in scenarios requiring high precision and high reliability.

[0118] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0119] Please see Figure 7 , Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 7 As shown, the computer device 1000 may include a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include a user interface 1003 and at least one communication bus 1002. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 7 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application.

[0120] In such Figure 7 In the computer device 1000 shown, the network interface 1004 provides network communication elements; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve: Obtain business sample data, input the business sample data into the initial large language model, and use the initial large language model to predict the business sample data to obtain M word prediction vectors; M is a positive integer; The words indicated by the M word prediction vectors are determined as the sample text results corresponding to the business sample data. Based on the prediction probabilities corresponding to the M word prediction vectors, the sample confidence scores corresponding to the sample text results are generated. Based on the sample labels and sample text results corresponding to the business sample data, a correctness reward value is generated, and based on the sample confidence, sample labels and sample text results, a confidence calibration reward value is generated. Based on the confidence calibration reward value and the correctness reward value, the initial large language model is trained by reinforcement learning to obtain the target large language model; the target large language model is used to generate the predicted text results corresponding to the business data and the prediction confidence of the predicted text results.

[0121] It should be understood that the computer device 1000 described in the embodiments of this application can execute the foregoing text. Figure 3 , Figure 4 and Figure 5 The description of the data processing method in any corresponding embodiment will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated.

[0122] Furthermore, it should be noted that this application embodiment also provides a computer-readable storage medium, which stores a computer program. When the processor executes the computer program, it can execute the aforementioned... Figure 3 , Figure 4 and Figure 5 The description of the data processing method in any corresponding embodiment is already provided, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer-readable storage medium embodiments related to this application, please refer to the description of the method embodiments of this application.

[0123] The aforementioned computer-readable storage medium can be an internal storage unit of the data processing apparatus or computer device provided in any of the foregoing embodiments, such as a hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been displayed or will be displayed.

[0124] Furthermore, it should be noted that this application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the aforementioned... Figure 3 , Figure 4 and Figure 5 The method provided in any of the corresponding embodiments.

[0125] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0126] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the foregoing description as a network element. Whether these network elements are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the described network elements using different methods for each specific application, but such implementation should not be considered beyond the scope of this application.

[0127] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0128] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0129] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0130] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.< / bos>

Claims

1. A data processing method, characterized in that, include: Obtain business sample data, input the business sample data into an initial large language model, and use the initial large language model to predict the business sample data to obtain M word prediction vectors; M is a positive integer; The words indicated by the M word prediction vectors are determined as the sample text results corresponding to the business sample data. Based on the prediction probabilities corresponding to the M word prediction vectors, the sample confidence level corresponding to the sample text results is generated. Based on the sample labels corresponding to the business sample data and the sample text results, a correctness reward value is generated, and based on the sample confidence, the sample labels and the sample text results, a confidence calibration reward value is generated. Based on the confidence calibration reward value and the correctness reward value, the initial large language model is trained using reinforcement learning to obtain the target large language model; the target large language model is used to generate predicted text results corresponding to business data and prediction confidence values ​​for the predicted text results.

2. The method according to claim 1, characterized in that, The step of generating a correctness reward value based on the sample tags corresponding to the business sample data and the sample text results includes: Generate the feature similarity between the sample labels corresponding to the business sample data and the sample text results; If the feature similarity is greater than or equal to the similarity threshold, then the first reward value is determined as the correctness reward value; If the feature similarity is less than the similarity threshold, then the second reward value is determined as the correctness reward value; the first reward value is greater than the second reward value.

3. The method according to claim 1, characterized in that, The step of generating a confidence calibration reward value based on the sample confidence level, the sample label, and the sample text result includes: Generate the feature similarity between the sample labels corresponding to the business sample data and the sample text results; A calibration value is obtained based on the feature similarity, and a confidence calibration reward value is generated based on the calibration value and the sample confidence.

4. The method according to claim 1, characterized in that, The process of obtaining a calibration value based on the feature similarity and generating a confidence calibration reward value based on the target matching value and the sample confidence includes: If the feature similarity is greater than or equal to the similarity threshold, the maximum value within the confidence interval to which the sample confidence belongs is determined as the calibration value; if the feature similarity is less than the similarity threshold, the minimum value within the confidence interval is determined as the calibration value. The difference between the calibration value and the sample confidence level is determined as the matching error value; The matching error value is exponentially calculated to obtain the mean squared error value. The negative of the mean squared error value is then determined as the confidence calibration reward value.

5. The method according to claim 1, characterized in that, The step of training the initial large language model using reinforcement learning based on the confidence calibration reward value and the correctness reward value to obtain the target large language model includes: The confidence calibration reward value and the correctness reward value are weighted to obtain a comprehensive reward value; Based on the comprehensive reward value, the model parameters of the initial large language model are adjusted. When the initial large language model meets the training convergence condition, the initial large language model that meets the training convergence condition is determined as the target large language model.

6. The method according to claim 1, characterized in that, The process of predicting the business sample data using the initial large language model yields M word prediction vectors, including: The initial large language model is used to extract features from the business sample data to obtain a sample feature sequence; Based on the candidate lexical units in the business dictionary of the initial large language model, probability matching is performed on the sample feature sequence to obtain M lexical unit prediction vectors.

7. The method according to claim 6, characterized in that, The initial large language model includes an encoding layer; the step of extracting features from the business sample data using the initial large language model to obtain a sample feature sequence includes: The business sample data is segmented into N unit words by the encoding layer in the initial large language model; N is a positive integer. The N unit words are encoded to obtain N word word encoding vectors. Based on the positional order of each unit word in the business sample data, N positional encoding vectors are generated. Based on the N word encoding vectors and the N position encoding vectors, N unit feature vectors are generated, and the N unit feature vectors are concatenated to obtain the sample feature sequence.

8. The method according to claim 7, characterized in that, The step involves segmenting the business sample data using the encoding layer in the initial large language model to obtain N unit words corresponding to the business sample data, including: Obtain the business dictionary and initial length threshold from the initial large language model; According to the order of characters in the business sample data, candidate strings with the same character length as the initial length threshold are obtained from the business sample data; If the candidate string matches the business dictionary, the candidate string is determined as a unit word. If the candidate string does not match the business dictionary, the initial length threshold is decremented to obtain a new initial length threshold. Based on the new initial length threshold, a new candidate string is determined in the business sample data. If the new candidate string matches the business dictionary, the new candidate string is determined as the unit word. Characters other than the unit words in the business sample data are identified as text to be matched. New unit words are then identified in the text to be matched based on the initial length threshold until the business sample data has been traversed and N unit words are obtained.

9. The method according to claim 6, characterized in that, Based on the candidate lexical units in the business dictionary of the initial large language model, probability matching is performed on the sample feature sequence to obtain M lexical unit prediction vectors, including: Obtain the candidate lexical sequence composed of candidate lexical units from the business dictionary in the initial large language model; Cross-attention processing is performed on the candidate word sequence and the sample feature sequence to obtain the matching probability corresponding to each candidate word. The feature vector corresponding to the candidate word with the highest matching probability is determined as the first target word. Based on the first target word, the sample feature sequence is updated to obtain a new sample feature sequence. Probability matching is then performed on the new sample feature sequence to obtain the next target word of the first target word. This process continues until the next target word meets the generation termination condition. The feature vectors corresponding to the M target words are then determined as M word prediction vectors.

10. The method according to claim 9, characterized in that, Also includes: The matching probabilities corresponding to the M word prediction vectors are determined as the prediction probabilities corresponding to the M word prediction vectors. The step of generating the sample confidence score corresponding to the sample text result based on the prediction probabilities corresponding to the M word prediction vectors includes: The average value of the prediction probabilities corresponding to the M word prediction vectors is determined as the sample confidence level corresponding to the sample text result.

11. The method according to claim 1, characterized in that, include: Obtain business data and threshold prompt words, input the business data and threshold prompt words into a target large language model, extract features from the business data through the target large language model to obtain a business feature sequence, and extract features from the threshold prompt words to obtain a prompt word feature sequence; Obtain the matching probability of each candidate word in the target large language model and the business feature sequence respectively, and obtain the target confidence threshold indicated by the threshold prompt word based on the prompt word feature sequence; Candidate words with a matching probability greater than or equal to the target confidence threshold are identified as R undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the word indicated by the T target word prediction vectors is determined as the predicted text result corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence level corresponding to the predicted text result is generated; T is a positive integer, and the prediction confidence level is greater than or equal to the target confidence level threshold.

12. The method according to claim 1, characterized in that, include: Acquire business data, input the business data into a target large language model, and extract features from the business data through the target large language model to obtain a business feature sequence; The matching probability of each candidate word in the target large language model and the business feature sequence is obtained respectively. The business feature sequence is then used for domain identification to obtain the business domain to which the business data belongs. The target confidence threshold corresponding to the business domain in the confidence mapping table is obtained. The confidence thresholds of different business domains in the confidence mapping table are different. Candidate words with a matching probability greater than or equal to the target confidence threshold are identified as S undetermined words. The feature vector corresponding to the undetermined word with the highest matching probability among the R undetermined words is identified as the target word prediction vector; R is a positive integer. When T target word prediction vectors are obtained, the word indicated by the T target word prediction vectors is determined as the predicted text result corresponding to the business data. Based on the matching probabilities corresponding to the T target word prediction vectors, the prediction confidence level corresponding to the predicted text result is generated; T is a positive integer, and the prediction confidence level is greater than or equal to the target confidence level threshold.

13. A data processing apparatus, characterized in that, include: The prediction module is used to acquire business sample data, input the business sample data into an initial large language model, and predict the business sample data through the initial large language model to obtain M word prediction vectors; M is a positive integer. The result calculation module is used to determine the words indicated by the M word prediction vectors as the sample text results corresponding to the business sample data, and generate the sample confidence level corresponding to the sample text results based on the prediction probabilities corresponding to the M word prediction vectors respectively. The reward calculation module is used to generate a correctness reward value based on the sample label corresponding to the business sample data and the sample text result, and to generate a confidence calibration reward value based on the sample confidence, the sample label and the sample text result; The training module is used to perform reinforcement learning training on the initial large language model based on the confidence level calibration reward value and the correctness reward value to obtain the target large language model; the target large language model is used to generate predicted text results corresponding to business data and prediction confidence levels for the predicted text results.

14. A computer device, characterized in that, include: Processor, memory, and network interface; The processor is connected to the memory and the network interface, wherein the network interface is used to provide data communication functions, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes a computer program stored in a computer-readable storage medium and adapted to be read and executed by a processor so that a computer device having the processor performs the method of any one of claims 1-12.

Images