Business execution method and device based on large model, medium and equipment

Abstract

The embodiment of the invention discloses a business execution method based on a large model, and the method comprises the steps: generating a prompt text of input data through the large model, reasoning based on the prompt text and the input data to obtain an output result, and executing a business according to the output result when an external verifier verifies that the output result is correct. And meanwhile, determining a reinforcement learning reward for the prompt text to adjust model parameters of the large model. The prompt text generated in the business execution is utilized to drive reinforcement learning, so that the problems of unstable prompt project effect and high supervised fine tuning training cost are avoided, the business execution efficiency is improved, and the operation and maintenance cost is reduced.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a business execution method, apparatus, storage medium and device based on a large model. Background Technology

[0002] In recent years, Large Language Models (LLMs) have been widely applied in various fields such as natural language processing, intelligent question answering, and decision support. With technological advancements, complex reasoning tasks, such as mathematical proofs, logical derivations, and complex problem-solving, have become important application scenarios. These tasks typically require LLMs to perform multi-step reasoning, contextual understanding, and error correction, placing higher demands on the model's cognitive and reasoning capabilities. Although LLMs perform exceptionally well in generating coherent text, their performance in complex structured reasoning still needs improvement.

[0003] In existing technologies, cue engineering or supervised fine-tuning (SFT) is mainly used to improve the performance of LLMs when facing complex problems. Cue engineering involves designing specific cue inputs to the LLM, guiding it to demonstrate its thought chain of ideas (CoT) during reasoning. Because the CoT needs to be determined, the LLM is forced to activate its latent knowledge and reasoning abilities learned during training, thus improving the accuracy of its output. SFT, on the other hand, involves collecting and manually annotating samples of errors made by the LLM, using these samples to perform supervised fine-tuning of the LLM, thereby improving the accuracy of its output.

[0004] However, SFT, which relies on failure cases, heavily depends on a large amount of high-quality labeled data, which is not only costly but also makes it difficult to obtain effective training samples for tasks that LLM cannot solve. Furthermore, the effectiveness of suggestion engineering is highly dependent on the quality of the suggestions, which are based on human experience. Therefore, it is limited by the experience and skills of human designers, resulting in unstable performance and poor generalization.

[0005] Based on this, this specification provides a business execution method based on a large model to partially solve the problems existing in the prior art. Summary of the Invention

[0006] This specification provides a business execution method, apparatus, storage medium, and electronic device based on a large model to partially solve the problems existing in the prior art.

[0007] The embodiments in this specification adopt the following technical solutions: This specification provides a business execution method based on a large model, the method comprising: Obtain the business data of the business to be processed and use it as input data for the large model; The large model generates prompt text for the input data, and the output result is determined by reasoning and analysis based on the prompt text and the input data. The verification conclusion of the output result is obtained through a pre-trained verification model; When the output result is determined to be correct based on the verification conclusion, the business operation is executed based on the output result; and based on the prompt text, the reward value of the reward function for reinforcement learning of the large model is determined so as to adjust the model parameters of the large model. The reward function is used to determine the reward value for the large model to generate the prompt text under the state of the input data.

[0008] This specification provides a business execution device based on a large model, the device comprising: The acquisition module is used to acquire the business data of the business to be processed, as the initial data for inputting the large model; The reverse analysis module is used to determine the output target based on the initial data, and to generate the final steps required to obtain the output target based on the output target. The forward reasoning module is used to perform reasoning analysis on the initial data with the goal of obtaining the preconditions for executing the final step, and to deduce the reasoning boundary conditions that can be obtained based on the initial data. The docking and reconstruction module is used to construct the reasoning process from the initial data to the output target based on the final step and the reasoning boundary conditions, and to determine the output result of the large model based on the reasoning process. The execution module is used to execute the business corresponding to the business data to be processed based on the output result.

[0009] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned business execution method based on a large model.

[0010] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the aforementioned business execution method based on a large model.

[0011] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects: This specification discloses a business execution method based on a large model. This method generates prompt text for the input data using a large model, and infers the output result based on the prompt text and input data. When an external validator verifies the output result as correct, the business operation is executed according to the output result. Simultaneously, a reward for reinforcement learning is determined for the prompt text to adjust the model parameters of the large model. Utilizing prompt text generated during business execution to drive reinforcement learning avoids the problems of unstable prompt engineering effects and high supervised fine-tuning training costs, thereby improving business execution efficiency and reducing operation and maintenance costs. Attached Figure Description

[0012] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings: Figure 1 A business execution flowchart based on a large model is provided for the embodiments of this specification; Figure 2 A schematic diagram of the reinforcement learning process provided in the embodiments of this specification; Figure 3 A business execution flowchart based on a large model is provided for the embodiments of this specification; Figure 4 A flowchart of the reinforcement learning process provided in the embodiments of this specification; Figure 5 A schematic diagram of a business execution device based on a large model provided in the embodiments of this specification; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0014] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0015] Figure 1 A business execution flowchart based on a large model, provided for embodiments of this specification, specifically includes the following steps: S100: Obtain the business data of the business to be processed, and use it as input data for the large model.

[0016] In the embodiments described in this specification, the following are employed: Figure 1 The device used to perform the business operations as shown can be any electronic device, such as a computer, a server, or a server cluster consisting of multiple servers. For ease of description, the following explanation uses a server as an example only.

[0017] To address the inconsistent accuracy of Large Language Models (LLMs) outputs across different business scenarios, and to facilitate further optimization of LLMs, traditional suggestion engineering or supervised fine-tuning methods rely heavily on human experience, resulting in high costs and inconsistent performance. This specification proposes a method that utilizes data generated by the LLM itself for reinforcement learning adjustments, significantly reducing reliance on human experience and automatically adjusting the LLM's parameters. Furthermore, adjustments can be made during business execution, further reducing LLM maintenance costs. This method is particularly advantageous in business scenarios where it's easy to determine the correctness of LLM outputs but difficult to obtain large amounts of labeled data, substantially reducing model maintenance costs and the need for manual intervention.

[0018] Specifically, in business scenarios where LLM is used to execute business processes, the server can obtain the data that needs to be input into the LLM and execute the corresponding business steps based on the LLM output. Therefore, the server can first obtain the business data to be processed, which is the data that needs to be input into the LLM, and is thus also called the LLM input data.

[0019] It should be noted that in one or more embodiments of this specification, the business data can be text data, image data, audio data, video data, etc. Generally, there is no limitation on the type of data, and it can also be a combination of multiple types of data. For example, business data includes images and text; any data that can be used as input to a large model can be used as business data in the embodiments of this specification. The output result can also include text data, image data, video data, or audio data, etc. The specific type of data output can be set as needed, and this specification does not impose any limitations.

[0020] Furthermore, in one or more embodiments of this specification, the business can be a risk control business, in which case the business data requiring risk control can be the business data obtained by the server. For example, risk control can be performed on transaction business, account operation business, user-published content security and compliance risk control, insurance claims business, and so on.

[0021] Taking transaction business as an example, the acquired business data may include, but is not limited to: transaction serial number, transaction time, transaction amount, account information of both parties to the transaction, user historical behavior sequence, and associated risk tags from third-party data sources, etc. After preprocessing and vectorization, this multi-dimensional and heterogeneous data constitutes an input sequence that LLM can understand, providing an information foundation for subsequent risk identification and analysis.

[0022] Taking the security and compliance risk control of user-published content as an example, the business data obtained can include: the original content to be reviewed, such as text, images, videos, and audio; basic information of the publisher; and content metadata, such as publication time, location, device used, and network environment. In addition, it can also obtain user behavior sequences, related data such as follower / fan relationships, group / community information, and characteristics of interacting objects; external environmental information such as trending events, location risk levels, and time period characteristics; content characteristics such as object recognition results in images, audio-to-text content, video frame characteristics, black market / gray market list matching results, device fingerprints, and IP risk databases. Therefore, LLM can identify risks not only based on content but also based on other related data to identify group risks, such as online troll behavior. It can also dynamically identify risks based on changes in the external environment; for example, publishing certain information in a specific period or location may be considered sensitive and risky, while in other environments it may be insignificant.

[0023] S102: Using the large model, generate prompt text for the input data, and perform reasoning analysis based on the prompt text and the input data to determine the output result.

[0024] After acquiring the input data, the server feeds it into the large model, which then generates prompt text based on the input data. This prompt text guides the LLM (Local Management Module) to pay attention to factual information during subsequent inference, avoiding obvious errors in the output results. For example, in a risk control scenario, the generated prompt text guides the LLM to proactively identify key features related to business risk dimensions and ensure logical consistency in the inference process based on factual information. For instance, the generated prompt text might be: Please analyze the characteristics of the following transaction data in sequence: 1. Deviation from the account's historical behavior; 2. Anomalies in transaction amounts; 3. Correlation risks between the transacting parties. Based on the above analysis, provide preliminary risk conclusions.

[0025] Specifically, the server first inputs the data into the LLM and then, through preset prompts, prompts the LLM to output the corresponding text. This prompt text is information that the LLM autonomously generates based on the input data before generating the final output, guiding itself through preliminary analysis and reasoning. It is a preventative prompt text designed to avoid obvious factual errors during the LLM's reasoning process and can be considered intermediate guidance information in the LLM's reasoning process.

[0026] For example, continuing with risk control, the prompt could be designed as follows: "You are an auditor. Before formally analyzing subsequent input data, you must first develop an 'Analysis Plan.' This plan should include: 1. Listing the three key points of review that require the most attention in this data; 2. Pointing out the one pitfall that is most likely to lead to misjudgment; 3. Briefly describing the logical steps you will follow." The server can then concatenate the preset prompt with the transaction data to form a complete input, which is then fed into the LLM. The LLM will generate the following prompt text: "Analysis and Contingency Plan: The analysis of key risk dimensions needs to include the matching degree between transaction time and user habits; the anomalies of the payee's historical transaction patterns; and the deviation of this transaction amount from the account's daily flow. When analyzing risks, it is easy to overlook that although this transaction amount is small, it may be part of the entire attack. The analysis path needs to follow the steps of first comparing the user's historical behavior, then checking the payee's network, and finally making a comprehensive judgment based on intelligence data." It's important to note that since the purpose of generating prompt text doesn't change with business data, and its function is to define the output of the LLM (Local Management Module)—either the output result or the prompt text—the prompt words can be pre-set, and their content is generic text applicable to the corresponding business scenario. Prompt text, on the other hand, is used to prevent the LLM from proactively addressing problems in the reasoning process, such as avoiding traps or logical reasoning jumps. Therefore, it needs to be generated based on actual input data. Similarly, prompt words are like a work manual containing principles or rules for working, while prompt text represents the constraints on actual behavior determined by the work manual when facing real-world tasks.

[0027] Generating prompt text specifically through LLM requires analysis tailored to the specific input data to achieve the highest relevance and information content. A pre-set, general prompt cannot provide any valuable new information for subsequent in-depth analysis. The process of generating prompt text is the process by which LLM engages in substantive thinking and externalizes it into words. It forces LLM to first digest and understand the input data, extract key points, and plan action paths, rather than directly providing answers. This reasoning process itself is crucial for improving output quality and generating training data.

[0028] Then, the server can use the acquired prompt text and input data as data to re-enter the LLM, and obtain the output results for performing business operations through the LLM.

[0029] Specifically, the server can merge the preventative prompts generated by the LLM with the original input data to form an enhanced input context, which is then input into the LLM. The LLM then performs in-depth analysis based on the reconstructed context. Because the LLM now has both the original input data and its self-defined analysis strategy (i.e., the prompts), the accuracy of the LLM's output is higher.

[0030] S104: Obtain the verification conclusion of the output result through the pre-trained verification model.

[0031] In the embodiments of this specification, after the server obtains the output result, it can further verify the output result through a pre-trained verification model to obtain a verification conclusion. Based on whether the output result is correct or incorrect given by the verification conclusion, it determines how to execute the next step.

[0032] Specifically, the verification model can be a model that performs basic error verification on the output results based on factual information. This specification does not limit the type of verification model; since it is only used for factual verification of the output results, even a simple neural network model can achieve this function through training. Furthermore, because it performs factual verification, training samples and labels can be automatically generated based on real-world information from a large database for training.

[0033] Alternatively, lightweight text classification models based on Transformer (such as the BERT classifier) ​​or Natural Language Inference (NLI) may be used in the examples in this specification, and this specification does not impose any restrictions.

[0034] This verification model allows for the rapid determination of the verification conclusions of the output results, facilitating subsequent steps. Furthermore, as described in step S100, in business scenarios where it is easy to determine the correctness of the LLM output results but difficult to obtain a large amount of labeled data, this verification model is sufficient to provide accurate verification conclusions.

[0035] S106: When the output result is determined to be correct based on the verification conclusion, the business operation is executed based on the output result; and based on the prompt text, the reward value of the reward function for reinforcement learning of the large model is determined so as to adjust the model parameters of the large model.

[0036] Finally, based on the verification conclusion obtained in step S104, if the verification conclusion is that the output result is incorrect, the server will not execute the business based on the output result; if the verification conclusion is that the output result is correct, the server will determine how to execute the business based on the content of the output result.

[0037] Specifically, when the verification conclusion is incorrect, the server will block the business process to avoid business risks caused by LLM illusions or inference errors. In risk control scenarios, the server also defaults to classifying the business data as high-risk and directly performs risk control, preventing missed judgments even in the event of a false positive. Alternatively, the server can transfer the business to a manual channel for human judgment on whether to perform risk control. This ensures that critical tasks will never be interrupted due to a single point of failure in the model, meeting the system availability requirements of real-world production environments. Especially in high-risk fields such as finance and healthcare, pre-designed default strategies can control business risks caused by model errors within an acceptable range.

[0038] When the verification conclusion is correct, the server determines the specific business action to be executed based on the output. When the business to be executed is risk control, the LLM output will necessarily contain the conclusion of whether risk control is required. Therefore, the server does not need to understand complex natural language, but only needs to perform simple keyword matching or structured parsing on the output, and can map the output to the business action.

[0039] For example, parsing the JSON output: {"risk_level": "high", "action": "block"}, can be mapped to a pre-defined decision-making mapping table within the business action system. If a keyword matches a high-risk or block keyword, a pre-defined risk control action is triggered, such as: blocking the transaction, freezing the account for 24 hours, or generating a high-risk warning work order. If a low-risk or pass keyword is matched, a pass action is triggered, such as: allowing the transaction or recording the assessment in the risk control log.

[0040] In other business scenarios, the output of an LLM is unstructured text content intended for direct display. For example, in an intelligent customer service scenario, the LLM output is a response to a user's question. The server can then directly return the text generated by the LLM to the front-end interface for display to the user.

[0041] Alternatively, the server can analyze the LLM output, identify executable instructions, and execute them. For example, when the server detects that the output includes "refund application submitted," in addition to returning and displaying this message to the user, the server can also create a refund ticket record in the database in the background. In some business scenarios, the LLM output itself may be a piece of executable code, such as an SQL query or API call parameter. In this case, the server can securely verify the code, execute it to obtain the result, and integrate the result into the response. In other words, the server can either directly display the LLM output, execute business logic based on the output, or both; this specification does not impose any restrictions on this.

[0042] In addition, in one or more embodiments of this specification, when the server determines that the verification result of the output result is correct, it indicates that the prompt text has a positive effect in the process of the LLM outputting the correct result. Therefore, the concept of reinforcement learning can be used to adjust the LLM, avoiding the problems faced by prompt engineering and supervised fine-tuning. By introducing reinforcement learning, this specification constructs an autonomous learning system that does not require continuous manual intervention. By converting the prompt text generated by the LLM during business execution into training signals, the LLM can utilize the computing resources during business idle time for self-iteration, thus achieving the transformation from a static model to a growing model. This is particularly important in adversarial scenarios such as risk control and anti-fraud, because the attack methods are constantly evolving, and the model must have the ability to adapt dynamically.

[0043] Moreover, although the reinforcement learning is based on the prompt text rather than the output result, whether it is the output result or the prompt text, the LLM calls the same set of parameters and underlying knowledge representations. Therefore, when continuously training the LLM through reinforcement learning, the adjustment of the LLM will gradually be internalized into the model's parameters. Eventually, the LLM can directly output the correct answer without generating the prompt text and quickly completing similar reasoning and verification steps internally.

[0044] This reinforcement learning is still about training the LLM on how to retrieve and apply knowledge more precisely, find errors in reasoning, how to perform logical causal reasoning, analyze how an error A in reasoning leads to an incorrect result B, and plan the problem-solving steps. Similar to an experienced programmer who may no longer need to write down every step of thinking when checking code and can spot the problem at a glance.

[0045] Specifically, after the output result is verified as correct, the server can first determine the key intermediate product prompt text for generating the output result. And parse the prompt text to determine each token in the prompt text. Among them, in English, a token may be a word, such as "cat". Or, a sub-word, such as "ing". Or, even a character, such as "a". In Chinese, a token is usually a Chinese character, such as "猫". Or a word, such as "猫咪". Or a common combination of characters.

[0046] Next, for each identified word, the reward value corresponding to that word is determined through the reward function of reinforcement learning in the large model. Specifically, the reinforcement learning function used by the server can employ various strategies to determine the reward value for each word. For example, average allocation, probability weighting, attention weighting, etc., can be set according to needs. Probability weighting allocates rewards based on the initial probability when generating each word, while attention weighting allocates rewards based on the attention score of the word during the analysis process, ensuring that words that contribute more to the correct final output receive higher rewards.

[0047] Finally, based on the determined reward values ​​corresponding to each word, the total gradient of the preset loss function is determined, and the model parameters of the large model are adjusted based on the total gradient.

[0048] Specifically, a policy gradient algorithm can be used to construct a loss function based on the generation probability of each word and the determined reward value for each word. The loss function can be expressed as: in, It is the reward value for the i-th word. This represents the probability corresponding to the generation of the i-th word. The loss function increases the probability of high-reward words, transforming this into a decrease in loss through a negative sign; this is defined as the optimization objective. Subsequently, the system uses the backpropagation algorithm to calculate the gradient of this loss function relative to the LLM model parameters, i.e., the total gradient. The total gradient is a vector whose direction points towards the parameter space direction that increases the probability of high-reward words, thereby reducing the value of the loss function.

[0049] The total gradient above is expressed as Mathematically, this is equivalent to the vector sum of the independent gradient contributions of all lexical units, i.e. Each lexical unit is awarded based on its reward value. Proportionally amplify or reduce its own probability gradient. Contribution to the total gradient.

[0050] Then stochastic gradient descent or other methods can be used to proceed along... The negative direction adjusts the parameters of the LLM. This increases the probability of the LM generating high-reward words and decreases the probability of generating low-reward words when encountering similar contexts again, thereby gradually optimizing its reflection and decision-making capabilities.

[0051] based on Figure 1The business execution method based on a large model, as shown, generates prompt text for the input data through the large model, and infers the output result based on the prompt text and input data. When an external validator verifies the output result as correct, the business is executed according to the output result. At the same time, the reward for reinforcement learning is determined for the prompt text to adjust the model parameters of the large model. By using the prompt text generated during business execution to drive reinforcement learning, the unstable effect of prompt engineering and the high cost of supervised fine-tuning training are avoided, thereby improving business execution efficiency and reducing operation and maintenance costs.

[0052] The reinforcement learning method used in the embodiments of this specification differs from methods that sample multiple outputs for the same problem and perform intra-group comparisons. This specification directly utilizes data naturally generated during a single business interaction for learning. This allows for efficient completion of the reinforcement learning process from business execution to application of LLM, without excessive consumption of business computing resources. Furthermore, each successful business process directly translates into a process of fine-tuning model parameters, greatly reducing the manpower and computing costs of model maintenance.

[0053] By rewarding effective terms, LLMs not only learn how to give the correct answer, but also the thought process behind arriving at that answer. This encourages LLMs to use more rigorous and clearly defined reasoning chains in generating outputs, thereby improving their robustness and interpretability.

[0054] Furthermore, during the reinforcement learning process in step S106, the server can also determine the reward value for each word based on its context. Compared to strategies that require complete sequence analysis, local context computation has lower complexity, faster processing, and more efficient training.

[0055] Specifically, for each identified word element, the server can determine the corresponding context information based on the word element's position in the prompt text. Specifically, the server can determine the context information centered on the word element's position in the prompt text, according to a preset window size.

[0056] Alternatively, the server can enhance semantic context based on the syntactic relationships between lexical units to determine contextual information. For example, the prompt text could be: "The system should verify the user's identity before granting authorization." Based on the lexical unit "identity," the syntactic relationships are determined to be associated with "verification," "user," and "authorization."

[0057] Then, based on the lexical term and its corresponding context information, the reward value of the lexical term is determined from the preset total reward value through a preset reward function.

[0058] Specifically, the server can use a reward function to determine the corresponding score for the word element and its corresponding context information, and then determine the reward value for each word element from the total reward value based on the scores of each word element.

[0059] For example, suppose we define a lexical unit and its contextual information as v, then This represents the contextual information of the i-th lexical unit, including its position, part of speech, other lexical units within the window, and even related words under semantic enhancement. For example, [position 4, part of speech verb, context "should", "first", "user", "identity"...], which is then processed through the reward function. Determine the score Where W represents the weight matrix and b is the bias term, both of which are preset parameters in the reward function that determine which features are more important.

[0060] Finally, the scores of each word element are converted into a probability distribution with a sum of 1, which is the weight assigned to each word element. Based on the preset total reward value, the reward value for each word element is determined by assigning weights.

[0061] For example, assigning weights The total reward value can be set as needed; this manual does not impose any restrictions.

[0062] Furthermore, in step S106, based on the generation probability of each word and the determined reward value of each word, the server can first obtain the generation probability of each word when the large model generates the prompt text. This requires the LLM to also output the generation probability of each word when outputting the prompt text, and have it stored by the server. In other words, the LLM provides the probability of generating each word in the prompt text, given the preceding context.

[0063] Based on the reward value and generation probability of each word, the gradient component of each word on the loss function is determined. The independent gradient contribution, or gradient component, is calculated for each word. This can be expressed as: Gradient component = .

[0064] Finally, based on each gradient component, the total gradient of the loss function is obtained. The total gradient can be expressed as: For a word element, if >0, its gradient term ×∇ log P( The direction of ) is -∇ log P( This direction is to increase the probability P( The direction of ). If If the gradient is less than 0, the gradient direction is reversed. The larger the value, the larger the magnitude of that gradient component. The magnitude and sign of the reward value determine the direction and amplitude of each component in the total gradient.

[0065] Figure 2 This is a schematic diagram of the reinforcement learning process provided in the embodiments of this specification. Figure 2 As can be seen, after the lexical units t1 to tn are determined, the gradient components of each lexical unit are determined based on the reward of the lexical unit and the probability of the lexical unit being generated. Then, the total gradient is determined, and the parameters of LLM are updated according to the total gradient.

[0066] This specification also provides a schematic diagram of the business execution process for a large model, such as... Figure 3 As shown.

[0067] S200: Obtain the business data of the business to be processed, and use it as input data for the large model.

[0068] S202: Input the input data into the large model to obtain the initial output of the large model.

[0069] In one or more embodiments of this specification, step S200 is similar to step S100 and will not be described again, but before generating the prompt text, the server can directly input the input data into the LLM and obtain the initial output of the LLM. This initial output is generated directly by the LLM without any prompt text. That is, the LLM can output once first, and if the result is correct, there is no need to generate prompt text. Prompt text is only generated when the result is incorrect. In this case, the prompt text is a reflection text, that is, a text prompting the LLM to reflect on why the initial output was incorrect.

[0070] S204: Input the initial output into the verification model to obtain the verification conclusion of the initial output.

[0071] In one or more embodiments of this specification, after obtaining the initial output, the server can input the initial output into the same verification model in step S104, and obtain the verification conclusion of the initial output through a similar verification process.

[0072] S206: In response to the verification conclusion determining that the initial output is incorrect, based on the verification conclusion and the initial output, a prompt text is determined through the reasoning analysis of the large model, the prompt text being used to prompt the large model to correct the reasoning process that led to the incorrect verification conclusion.

[0073] In one or more embodiments of this specification, a prompt text is generated only when the initial output of the LLM is incorrect, thus correcting the initial output. When the verification conclusion is correct, step S214 is executed: in response to the verification conclusion, the initial output is determined to be correct, and the service is executed based on the initial output.

[0074] Since simple regeneration fails to address the root cause when the initial output is validated as incorrect, by using both the validation conclusion and the incorrect output as input, the model receives a clear error signal, enabling targeted analysis. This design forces the LLM to confront its own errors rather than simply trying another possibility. Essentially, this process trains the LLM to analyze its own reasoning flaws, thereby developing its self-correcting and reasoning abilities. In machine learning, error samples often contain more information than success samples. By focusing on the analysis and correction of error cases, the LLM can more efficiently identify and compensate for cognitive blind spots, achieving faster capability improvement.

[0075] Specifically, when the verification conclusion is incorrect, the verification conclusion and the initial output are used as input. Based on preset reflection prompts, the LLM is required to generate reflection text on the reasoning process related to the initial output. For example, the reflection prompts might be: "You are a risk control expert. Please conduct in-depth reflection based on the following error case: Initial input X, your incorrect judgment Y, verification result Z. Please analyze the possible causes of your reasoning flaws and provide corresponding reflection text." S208: Based on the prompt text and the input data, perform reasoning analysis to determine the output result.

[0076] S210: Obtain the verification conclusion of the output result through the pre-trained verification model.

[0077] S212: When the output result is determined to be correct based on the verification conclusion, the business operation is executed based on the output result; and based on the prompt text, the reward value of the reward function for reinforcement learning of the large model is determined so as to adjust the model parameters of the large model.

[0078] The steps S208 to S212 above can be referred to as steps S104 to S108, and the process is similar, so this instruction manual will not repeat them.

[0079] In one or more embodiments of this specification, the definition of (Reward) in the reinforcement learning method based on self-reflection of a large language model differs from that in traditional reinforcement learning. State refers to the complete context of the large language model during the generation process, including one or more of the following: original business input data, all word sequences generated by the model, or current validation conclusion information. Essentially, state is the complete information context upon which the model bases its next decision; it is a dynamically changing text sequence.

[0080] An action refers to the decision-making behavior of a large language model in selecting and generating the next lexical unit from its vocabulary. Each action is a discrete decision-making process in which the model selects a specific lexical unit from tens of thousands of possible lexical units based on its policy in the current state. The generation of the entire prompt text is a sequence of actions.

[0081] The reward is a scalar signal used to evaluate the contribution of individual lexical generation actions to the final task success (outputting the correct result). When the final output is verified to be correct, a differentiated reward value is assigned to each lexical that constitutes the success message text. The reward assignment takes into account the lexical's position, semantic importance, and role in the reasoning process. Furthermore, the reward function ensures that the total reward is conserved, meaning that the sum of all lexical reward values ​​equals the preset overall task reward.

[0082] State, action, and reward constitute a complete reinforcement learning process. LLM takes actions to generate lexical units in a specific state. The text sequence formed by these actions leads to the task result. Finally, a reward is assigned to each action based on the quality of the result. The generation strategy of LLM is optimized through the policy gradient method. This design successfully adapts the traditional reinforcement learning framework to the text generation scenario of large language models.

[0083] Furthermore, in the embodiments described in this specification, the server can also obtain verified correct output results through multiple reflection and iteration processes.

[0084] Specifically, when a verification conclusion is obtained after each round of verification model verification, the server can determine that the output result is incorrect in response to the verification conclusion, and regenerate the prompt text based on the prompt text, the initial output, and the output result using the large model; Then, based on the regenerated prompt text, the output result is obtained through re-reasoning.

[0085] Then, the output obtained from the re-reasoning is input into the verification model, and the verification conclusion is obtained through further verification. When the verification conclusion is correct, the reward value is determined and the large model is adjusted based on the prompt text regenerated in this iteration.

[0086] If the verification result is incorrect, repeat the above process until the verification result is correct or the number of iterations is reached.

[0087] In the embodiments described in this specification, the business execution process is a risk control process, and the business data is the business data to be risk controlled.

[0088] The server can then obtain the business data to be risk-controlled and use it as input data for the large model; The large model generates prompt text for the input data, and based on the prompt text and the input data, it performs reasoning analysis to determine the risk assessment result. The verification conclusion of the risk assessment result is obtained through a pre-trained verification model; When the risk assessment result is determined to be correct based on the verification conclusion, the reward value of the reward function for reinforcement learning of the large model is determined based on the prompt text, so as to adjust the model parameters of the large model. Furthermore, business operations are performed based on the risk assessment results. When the assessment result indicates that there is a risk, risk control measures are taken for the business corresponding to the business data. When the assessment result indicates that there is no risk, the business corresponding to the business data is executed. The reward function is used to determine the reward value for the large model to generate the prompt text under the state of the input data.

[0089] Figure 4 This is a flowchart illustrating the reinforcement learning process provided in the embodiments of this specification, wherein: S300: Obtain training samples; S302: Input the training samples into the large model to obtain the initial output of the large model; S304: Input the initial output into the verification model to obtain the verification conclusion of the initial output; S306: Determine whether the initial output is correct based on the verification conclusion. If it is correct, proceed to step S308; otherwise, proceed to step S310. S308: Mark the training sample as successful, determine if there are any other training samples that need to be input, if yes, execute the steps, otherwise end the model training.

[0090] S310: Based on the verification conclusion and the initial output, a prompt text is determined through the reasoning analysis of the large model. The prompt text is used to prompt the large model to correct the reasoning process that led to the erroneous verification conclusion. S312: Based on the prompt text and the input data, perform reasoning analysis to determine the output result; S314: Input the output result into the verification model to obtain the verification conclusion of the output result; S316: Determine whether the output result is correct based on the verification conclusion. If it is correct, proceed to step S318; otherwise, proceed to step S320. S318: Based on the prompt text, determine the reward value of the reward function for reinforcement learning of the large model, so as to adjust the model parameters of the large model; S320: Do not adjust the model parameters of the large model.

[0091] The above is an example of a business execution method based on a large model provided in this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0092] Figure 5 This specification provides a schematic diagram of a business execution device based on a large model, the device comprising: The acquisition module 401 is used to acquire the business data of the business to be processed, as the input data of the large model; The prompting reasoning module 402 is used to generate prompt text for the input data through the large model, and to perform reasoning analysis based on the prompt text and the input data to determine the output result; The verification module 403 is used to obtain the verification conclusion of the output result through a pre-trained verification model; The execution training module 404 is used to execute the business based on the output result when the output result is determined to be correct according to the verification conclusion; and to determine the reward value of the reward function for reinforcement learning of the large model based on the prompt text, so as to adjust the model parameters of the large model; wherein, the reward function is used to determine the reward value of the large model generating the prompt text in the state of the input data.

[0093] Optionally, the execution training module 404 is used to parse the prompt text and determine each word in the prompt text; for each determined word, the reward value corresponding to the word is determined by the reward function of reinforcement learning through the large model; based on the determined reward values ​​corresponding to each word, the total gradient of the preset loss function is determined, and the model parameters of the large model are adjusted based on the total gradient.

[0094] Optionally, the execution training module 404 is used to determine the context information corresponding to each determined word element based on its position in the prompt text; and to determine the reward value of the word element from the preset total reward value through a preset reward function based on the word element and its corresponding context information.

[0095] Optionally, the execution training module 404 is used to determine the context information corresponding to each determined word element based on its position in the prompt text; and to determine the reward value of the word element from the preset total reward value through a preset reward function based on the word element and its corresponding context information.

[0096] Optionally, the prompting reasoning module 402 is used to input the input data into the large model to obtain the initial output of the large model; input the initial output into the verification model to obtain the verification conclusion of the initial output; in response to the verification conclusion determining that the initial output is incorrect, and based on the verification conclusion and the initial output, determine prompt text through the reasoning analysis of the large model, wherein the prompt text is used to prompt the large model to correct the reasoning process that led to the incorrect verification conclusion.

[0097] Optionally, the prompting reasoning module 402 is used to determine that the initial output is correct in response to the verification conclusion, and to execute the service based on the initial output.

[0098] Optionally, the execution training module 404 is configured to, in response to the verification conclusion determining that the output result is incorrect, regenerate the prompt text and re-infer the output result using the large model based on the prompt text, the initial output, and the output result, until the verification conclusion is correct or the number of iterations is reached; wherein, when the verification conclusion is correct, the reward value is determined and the large model is adjusted based on the prompt text regenerated in this iteration.

[0099] Optionally, the business data is business data to be subject to risk control, and the output result is a risk assessment result; The execution training module 404 is used to perform risk control operations on the business corresponding to the business data when the risk assessment result indicates that there is a risk; and to execute the business corresponding to the business data when the risk assessment result indicates that there is no risk.

[0100] This specification also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can be used to perform the above-described business execution method based on a large model.

[0101] based on Figure 1 The business execution method based on the large model shown in this specification also provides the embodiments. Figure 6 The diagram shows the structure of the electronic device. Figure 6 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile storage, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to implement the aforementioned business execution method based on a large model.

[0102] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A business execution method based on a large model, the method comprising: Obtain the business data of the business to be processed and use it as input data for the large model; The large model generates prompt text for the input data, and the output result is determined by reasoning and analysis based on the prompt text and the input data. The verification conclusion of the output result is obtained through a pre-trained verification model; When the output result is determined to be correct based on the verification conclusion, the business operation is executed based on the output result; and based on the prompt text, the reward value of the reward function for reinforcement learning of the large model is determined so as to adjust the model parameters of the large model. The reward function is used to determine the reward value for the large model to generate the prompt text under the state of the input data.

2. The method as described in claim 1, wherein based on the prompt text, a reward value for the reward function of reinforcement learning on the large model is determined to adjust the model parameters of the large model, specifically including: The prompt text is parsed to determine each word element in the prompt text; For each identified word, the reward value corresponding to that word is determined by the reward function of reinforcement learning through the large model. Based on the determined reward value corresponding to each word, the total gradient of the preset loss function is determined, and the model parameters of the large model are adjusted based on the total gradient.

3. The method as described in claim 2, wherein for each determined lexical unit, a reward value corresponding to that lexical unit is determined by applying a reward function for reinforcement learning to the large model, specifically including: For each identified word element, the context information corresponding to that word element is determined based on its position in the prompt text; Based on the lexical term and its corresponding context information, the reward value of the lexical term is determined from the preset total reward value through a preset reward function.

4. The method as described in claim 2, wherein the total gradient of the preset loss function is determined based on the determined reward value corresponding to each word, specifically including: When the large model generates the prompt text, the generation probability of each word element is obtained; Based on the reward value corresponding to each word and the generation probability of each word, the gradient component of each word on the loss function is determined respectively; The total gradient of the loss function is obtained based on each gradient component.

5. The method according to any one of claims 1 to 4, wherein generating prompt text for the input data using the large model specifically includes: The input data is input into the large model to obtain the initial output of the large model; The initial output is input into the verification model to obtain the verification conclusion of the initial output; In response to the verification conclusion that the initial output is incorrect, based on the verification conclusion and the initial output, a prompt text is determined through the reasoning analysis of the large model. The prompt text is used to prompt the large model to correct the reasoning process that led to the incorrect verification conclusion.

6. The method as claimed in claim 5, further comprising: In response to the verification conclusion that the initial output is correct, the service is executed based on the initial output.

7. The method of claim 5, further comprising: In response to the verification conclusion that the output result is incorrect, the large model is used to regenerate the prompt text and re-infer the output result based on the prompt text, the initial output, and the output result, until the verification conclusion is correct or the number of iterations is reached; When the verification conclusion is correct, the reward value is determined and the large model is adjusted based on the prompt text regenerated in this iteration.

8. As in claim 1, the business data is business data to be subject to risk control, and the output result is a risk assessment result; Execute the service based on the output results, specifically including: When the risk assessment result indicates that a risk exists, risk control measures are taken for the business corresponding to the business data. When the risk assessment result indicates that there is no risk, the business corresponding to the business data is executed.

9. A business execution device based on a large model, the device comprising: The acquisition module is used to acquire the business data of the business to be processed, which is then used as input data for the large model. The prompting reasoning module is used to generate prompt text for the input data through the large model, and to perform reasoning analysis based on the prompt text and the input data to determine the output result; The verification module is used to obtain the verification conclusion of the output result through a pre-trained verification model; The execution training module is used to execute the business based on the output result when the output result is determined to be correct according to the verification conclusion; and to determine the reward value of the reward function for reinforcement learning of the large model based on the prompt text, so as to adjust the model parameters of the large model; wherein, the reward function is used to determine the reward value of the large model generating the prompt text in the state of the input data.

10. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1-8.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method according to any one of claims 1-8.

Images