Problem positioning method and related product

Abstract

The invention discloses a problem positioning method and a related product. The problem positioning method comprises the steps of obtaining a to-be-processed text; inputting the to-be-processed text into the target language model, and enabling the target language model to execute a target task on the to-be-processed text to obtain a first execution result; the training process of the target language model comprises the steps of pre-training the basic language model to obtain a pre-trained model, performing supervised fine tuning on the pre-trained model to obtain a supervised fine tuning model, obtaining a reward model based on the supervised fine tuning model, and performing reinforcement learning on the supervised fine tuning model based on the reward model to obtain the target language model; under the condition that it is determined that the first execution result is wrong, a problem model is determined from a pre-training model, a supervision fine tuning model and a reward model based on the to-be-processed text, and the problem model is a model outputting a wrong result under the condition that the input is the to-be-processed text; and determining a target reason according to the problem model. By adopting the method, the target reason of the result causing the output error of the target language model can be positioned.

Description

Technical Field

[0001] The present application relates to the technical field of large language models, and in particular to a problem localization method and related products. Background Art

[0002] With the development of large language model technology, the training process of large language model (LLM) is becoming more and more perfect, and the training involves more and more links, such as pre-training, supervised fine-tuning, reinforcement learning, etc. Based on the complex and multiple training links, when the trained large language model outputs wrong results in the application reasoning stage, how to locate the cause of the wrong output of the large language model is of great significance for the effective application of the large language model. Summary of the invention

[0003] The present application provides a problem location method and related products to locate the target cause of the target language model outputting an erroneous result, wherein the related products include a problem location device, a computer device, a computer-readable storage medium and a computer program product.

[0004] In a first aspect, the present application provides a problem localization method, the method being used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: obtaining a pre-trained model by pre-training a basic language model, performing supervised fine-tuning on the pre-trained model to obtain a supervised fine-tuning model, the supervised fine-tuning model being used to process natural language and generate results that meet preset expectations, obtaining a reward model based on the supervised fine-tuning model, the reward model being used to evaluate the quality of the results output by the language model according to preset standards, and performing reinforcement learning on the supervised fine-tuning model based on the reward model to obtain the target language model, the method comprising:

[0005] Get the text to be processed;

[0006] Inputting the text to be processed into the target language model so that the target language model performs a target task on the text to be processed to obtain a first execution result, wherein the target task is a natural language processing task;

[0007] In the case where it is determined that the first execution result is wrong, determining a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed, wherein the problem model is a model whose output result is wrong when the input is the text to be processed;

[0008] According to the problem model, the target cause is determined.

[0009] The supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, the reward model is used to evaluate the quality of the results output by the language model according to preset standards, and the target language model is obtained by reinforcing the supervised fine-tuning model based on the reward model. It can be seen that the target language model trained by the above training process can process natural language and generate results that meet preset expectations, and theoretically, the generated results can have higher quality when evaluated according to preset standards, that is, they can meet the preset standards. Using the above problem location method, the text to be processed is obtained, and the text to be processed is input into the target language model so that the target language model performs the target task on the text to be processed, and obtains the first execution result, wherein the target task is a natural language processing task. If there is no problem in the training process of the target language model, the first execution result output by the target language model should meet the preset expectations and meet the preset standards. If the first execution result is wrong, that is, it does not meet the preset expectations or does not meet the preset standards, it can indicate that there is a problem in the training process of the target language model. In order to locate the target cause of the target language model outputting the wrong result, it is necessary to trace the problem in the training process of the target language model. Based on this, when it is determined that the first execution result is wrong, the problem model is determined from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed, wherein the problem model is a model whose output result is an error when the input is the text to be processed. Specifically, the text to be processed is input into the pre-trained model, the supervised fine-tuning model, and the reward model. If any model outputs an erroneous result that does not conform to the purpose of this model, it is determined which model has a problem, that is, which model is determined to be the problem model. The pre-trained model, the supervised fine-tuning model, and the reward model are the presentation forms of the target language model during the training process. After determining which model in the training process is the problem model, the target cause of the target language model outputting the wrong result can be determined based on the problem model.

[0010] Based on this, the above problem location method can be used to determine the target cause of the target language model outputting erroneous results.

[0011] In combination with any embodiment of the present application, when it is determined that the first execution result is wrong, determining the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed includes:

[0012] In the case where it is determined that the first execution result is wrong, determining the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model using the to-be-processed text in a backtracking manner;

[0013] The backtracking refers to the sequence of generating the pre-training model, the supervised fine-tuning model, and the reward model in the process of obtaining the target language model based on training, and tracing back from the back to the front.

[0014] In combination with any embodiment of the present application, when it is determined that the first execution result is wrong, determining the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model in a backtracking manner using the text to be processed includes:

[0015] In the case where it is determined that the first execution result is wrong, inputting the to-be-processed text and the first execution result into the reward model to obtain a first score output by the reward model, wherein the first score represents the quality of the result output by the target language model evaluated according to the preset standard;

[0016] Inputting the to-be-processed text and the correct execution result of the target task into the reward model to obtain a second score output by the reward model, wherein the second score represents the quality of the correct execution result evaluated according to the preset standard;

[0017] When the first score is greater than the second score, the reward model is determined to be the problem model.

[0018] In combination with any embodiment of the present application, the reward model is obtained by training the adjusted model using the first training data set after adjusting the model structure of the supervised fine-tuning model to obtain the adjusted model;

[0019] After determining that the reward model is the problem model, the method further includes:

[0020] Determine, from the first training data set, first training data whose semantics match the semantics of the text to be processed and a label of the first training data;

[0021] Determine, according to the label of the first training data, a ranking result of n labeling results corresponding to the first training data;

[0022] Determining the target cause according to the problem model includes:

[0023] In the case where the sorting result is erroneous, the target cause is determined to be that the reward model is obtained by training the adjustment model using the first training data and the label of the first training data, resulting in the reward model obtaining erroneous information.

[0024] In combination with any implementation manner of the present application, after obtaining the first score and the second score, the method further includes:

[0025] When the first score is less than the second score, inputting the to-be-processed text into the supervised fine-tuning model to obtain a second execution result output by the supervised fine-tuning model;

[0026] In case the second execution result is erroneous, the supervised fine-tuning model is determined to be the problem model.

[0027] In combination with any implementation manner of the present application, after determining that the supervised fine-tuning model is the problem model, the method further includes:

[0028] Determining second training data whose semantics match the semantics of the text to be processed and a label of the second training data from a second training data set, wherein the second training data set is a training data set used by the supervised fine-tuning model obtained by performing supervised fine-tuning on the pre-training model;

[0029] Determining the target cause according to the problem model includes:

[0030] In the case where the label of the second training data is wrong, determining the target cause is that the label of the second training data is wrong, resulting in the supervised fine-tuning model obtaining wrong information through supervised fine-tuning.

[0031] In combination with any embodiment of the present application, after determining second training data whose semantics match the semantics of the text to be processed and a label of the second training data from the second training data set, the method further includes:

[0032] When the label of the second training data is correct, obtaining reference prompt words, the reference prompt words including reference examples and text describing the target task and the content to be determined according to a preset sentence pattern, the reference example is an example describing the reference task and the reference execution result of the reference task according to the preset sentence pattern, the position of the target task in the preset sentence pattern is the same as the position of the reference task in the preset sentence pattern, the position of the content to be determined in the preset sentence pattern is the same as the position of the reference execution result in the preset sentence pattern, and the reference example is used to guide the pre-trained model to determine the content to be determined;

[0033] Inputting the reference prompt word into the pre-training model, so that the pre-training model determines the content to be determined under the guidance of the reference example, as a third execution result output by the pre-training model;

[0034] In case the third execution result is erroneous, the pre-trained model is determined to be the problem model.

[0035] In combination with any implementation manner of the present application, after determining that the pre-trained model is the problem model, the method further includes:

[0036] Determining, from a third training data set, third training data whose semantics match the semantics of the text to be processed, wherein the third training data set is a training data set used by the pre-training model obtained by pre-training a basic language model;

[0037] Determining the target cause according to the problem model includes:

[0038] In the case where the third training data is erroneous, it is determined that the target cause is that the third training data is erroneous, causing the pre-training model to obtain erroneous information through pre-training.

[0039] In a second aspect, the present application also provides another problem location method, the method is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training a basic language model, and the pre-trained model is supervised fine-tuned to obtain a supervised fine-tuning model, the supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, a reward model is obtained based on the supervised fine-tuning model, and the reward model is used to evaluate the quality of the results output by the language model according to preset standards, and the supervised fine-tuning model is reinforced based on the reward model to obtain the target language model. The method includes:

[0040] Get the target question;

[0041] Inputting the target question into the target language model, and obtaining a first answer to the target question output by the target language model;

[0042] In the case where it is determined that the first answer is wrong, determining a question model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target question, wherein the question model is a model whose output result is an error when the input is the target question;

[0043] According to the problem model, the target cause is determined.

[0044] In a third aspect, the present application also provides a problem location device, which is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training a basic language model, and a supervised fine-tuning model is obtained by supervised fine-tuning the pre-trained model, and the supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, and a reward model is obtained based on the supervised fine-tuning model. The reward model is used to evaluate the quality of the results output by the language model according to preset standards, and the supervised fine-tuning model is reinforced based on the reward model to obtain the target language model. The device includes:

[0045] A text acquisition module is used to acquire the text to be processed;

[0046] A result generation module, used for inputting the text to be processed into the target language model, so that the target language model performs a target task on the text to be processed to obtain a first execution result, wherein the target task is a natural language processing task;

[0047] A model determination module, configured to determine a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed when it is determined that the first execution result is wrong, wherein the problem model is a model whose output result is wrong when the input is the text to be processed;

[0048] The cause determination module is used to determine the target cause according to the problem model.

[0049] In a fourth aspect, the present application further provides another problem location device, the device is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training a basic language model, and a supervised fine-tuning model is obtained by supervised fine-tuning the pre-trained model, and the supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, and a reward model is obtained based on the supervised fine-tuning model. The reward model is used to evaluate the quality of the results output by the language model according to preset standards, and the supervised fine-tuning model is reinforced based on the reward model to obtain the target language model. The device includes:

[0050] A question acquisition module is used to obtain the target question;

[0051] an answer generation module, configured to input the target question into the target language model and obtain a first answer to the target question output by the target language model;

[0052] A model positioning module, configured to determine a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target question when it is determined that the first answer is wrong, wherein the problem model is a model whose output result is wrong when the input is the target question;

[0053] The cause location module is used to determine the target cause according to the problem model.

[0054] In a fifth aspect, the present application also provides a computer device, comprising: a memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the computer device executes the method shown in the first aspect or any embodiment of the first aspect, or executes the method shown in the second aspect or any embodiment of the second aspect.

[0055] In the sixth aspect, the present application also provides another computer device, comprising: a processor, a sending device, an input device, an output device and a memory, wherein the memory stores program instructions; when the program instructions are executed by the processor, the computer device executes the method as shown in the first aspect or any embodiment of the first aspect, or executes the method as shown in the second aspect or any embodiment of the second aspect.

[0056] In the seventh aspect, the present application also provides a computer-readable storage medium, in which a computer program is stored; when the computer program runs on one or more processors, the processor executes the method as shown in the first aspect or any embodiment of the first aspect, or executes the method as shown in the second aspect or any embodiment of the second aspect.

[0057] In the eighth aspect, the present application also provides a computer program product, comprising a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to execute the method as shown in the first aspect or any embodiment of the first aspect, or execute the method as shown in the second aspect or any embodiment of the second aspect.

[0058] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0060] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0061] Figure 1 One of the flowcharts of a problem location method provided in an embodiment of the present application;

[0062] Figure 2 A schematic diagram of a process for determining a problem model from a pre-trained model, a supervised fine-tuning model, and a reward model using a text to be processed in a backtracking manner provided in an embodiment of the present application;

[0063] Figure 3 One of the interface diagrams of a full-link attribution platform provided in an embodiment of the present application;

[0064] Figure 4 A second flowchart of a problem location method provided in an embodiment of the present application;

[0065] Figure 5 A second schematic diagram of an interface of a full-link attribution platform provided in an embodiment of the present application;

[0066] Figure 6 A third flowchart of a problem location method provided in an embodiment of the present application;

[0067] Figure 7 A fourth flowchart of a problem location method provided in an embodiment of the present application;

[0068] Figure 8 A flowchart of another problem location method provided in an embodiment of the present application;

[0069] Fig. 9 A schematic diagram of the structure of a problem location device provided in an embodiment of the present application;

[0070] Fig.10 A schematic diagram of the structure of another problem locating device provided in an embodiment of the present application;

[0071] Fig.11 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0073] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units that are not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0074] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0075] The target language model is trained through the following training process: a pre-trained model is obtained by pre-training the basic language model, a supervised fine-tuning model is obtained by supervising the pre-trained model, the supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, a reward model is obtained based on the supervised fine-tuning model, the reward model is used to evaluate the quality of the results output by the language model according to preset standards, and reinforcement learning is performed on the supervised fine-tuning model based on the reward model to obtain the target language model.

[0076] The target language model is a large language model (LLM), such as a generative pre-trained model (GPT). The training process of the target language model adopts a pipeline mode. The pre-trained model (Pretrain Model) is obtained by pre-training the basic language model, the supervised fine-tuning model (Supervised Fine-Tuning Model) is obtained by fine-tuning the pre-trained model, the reward model (Reward Model) is obtained based on the supervised fine-tuning model, and the target language model is obtained by performing reinforcement learning (Reinforcement Learning) on ​​the supervised fine-tuning model based on the reward model.

[0077] The pre-trained model is obtained by pre-training the basic language model through an unsupervised training data set. The training data in this training data set can be natural language text commonly used on the Internet. The pre-trained model obtained by training has the ability to process natural language.

[0078] Furthermore, the supervised fine-tuning model can be obtained by fine-tuning the pre-trained model through a supervised training data set. Specifically, the training data set includes training data and labels corresponding to the training data. The training data is input into the pre-trained model, and the output of the pre-trained model is supervised using the labels to achieve supervised fine-tuning of the pre-trained model and obtain a supervised fine-tuning model. Among them, if the training data is a question, the label is the answer corresponding to this question that meets the preset expectations; if the training data is a task, the label is the execution result corresponding to this task that meets the preset expectations. The supervised fine-tuning model trained with the training data set, based on the pre-trained model, has the ability to process natural language and generate results that meet the preset expectations.

[0079] Further, the last layer of the supervised fine-tuning model is changed to a linear layer to obtain an adjustment model. The adjustment model is trained with a supervised training data set to obtain a reward model. The training data set includes training data and labels corresponding to the training data, and the training data includes input text input to the supervised fine-tuning model and the execution result output by the supervised fine-tuning model based on the input text. The input text and its corresponding execution result are input to the adjustment model, so that the adjustment model evaluates the quality of the execution result output by the supervised fine-tuning model based on the input text. If the supervised fine-tuning model outputs an execution result, the evaluation of the quality by the adjustment model can be presented in the form of a numerical score, and the label corresponding to the training data is a numerical score that meets the preset standard; if the supervised fine-tuning model outputs more than one execution result, the evaluation of the quality by the adjustment model can be presented in the form of a ranking of more than one execution result, and the label corresponding to the training data is a ranking of more than one execution result that meets the preset standard. The output of the adjustment model is supervised by using the label to realize the training of the adjustment model, and a reward model that can evaluate the quality of the result output by the supervised fine-tuning model according to the preset standard is obtained through training. It should be understood that the preset standard here may be a standard determined based on human feedback, that is, the reward model evaluates the quality of the results output by the supervised fine-tuning model according to human preferences.

[0080] Finally, based on the reward model's quality evaluation of the results output by the supervised fine-tuning model, the supervised fine-tuning model is reinforced for learning. This reinforcement learning can be Reinforcement Learning from Human Feedback (RLHF) or Proximal Policy Optimization (PPO). Specifically, the training data is input into the supervised fine-tuning model to output the execution result, and the training data and the execution result are used as inputs to the reward model so that the reward model evaluates the quality of the execution result based on the training data according to the preset standard. Furthermore, the supervised fine-tuning model is iterated based on the evaluation results output by the reward model to obtain the target language model. The trained target language model can process natural language and generate results that meet preset expectations and meet preset standards. When the preset standard is a standard determined based on human feedback, the target language model can generate results that meet preset expectations and meet human preferences.

[0081] Based on the complex and multiple training links in the above training process, if the trained target language model outputs an erroneous result in the application reasoning stage, it is difficult to locate the cause of the target language model outputting the erroneous result. Based on this, an embodiment of the present application provides a problem location method, which is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the above training process. The executor of the embodiment of the present application is a problem location device, wherein the problem location device can be any computer device that can execute the technical solution disclosed in the embodiment of the method of the present application. Optionally, the problem location device can be one of the following: a computer, a server.

[0082] like Figure 1 As shown, the problem location method includes the following steps 101 to 104.

[0083] 101. Get the text to be processed.

[0084] In the embodiment of the present application, the text to be processed is the input text of the target language model, which is used to instruct or guide the target language model to produce a specific output. The text to be processed can be a prompt word (prompt) or a text in other forms. When the text to be processed is a prompt word, it can represent a question, such as "Who is Liu Xiaohua's brother?", or it can represent a task, such as "Please extract the idiom contained in the following text: The mental outlook is completely renewed." It can also represent other things. The embodiment of the present application does not limit the specific information contained in the text to be processed.

[0085] 102. Input the text to be processed into the target language model so that the target language model performs a target task on the text to be processed and obtains a first execution result. The target task is a natural language processing task.

[0086] In the embodiment of the present application, the purpose of inputting the text to be processed into the target language model is to enable the target language model to perform the target task based on the text to be processed, that is, to process the natural language based on the text to be processed to generate an execution result that meets the preset expectations and meets the preset standards. If the text to be processed is "Does the apple have a core?", the execution result that meets the preset expectations can be "The apple has a core" or "The apple has no core", that is, a targeted answer is given based on the text to be processed to generate a specific output. If the execution result is "The peel of the apple can be eaten directly", then this execution result is an execution result that does not meet the preset expectations. If the preset standard is set based on an objective law, when the execution result is "The apple has no core", then it is obvious that this execution result does not meet the preset standard. When the execution result does not meet the preset expectation or does not meet the preset standard, this execution result is an incorrect execution result. It should be noted that in actual processing, whether the execution result is wrong depends on whether the execution result meets the preset expectations and preset standards set according to actual needs. Specifically, taking the above example of "Does the apple have a core" as an example, the execution result of "The apple has a core" meets both the preset expectations and the objective law. However, if the preset standard set according to actual needs also clearly requires an answer of "yes" or "no", that is, the character length of the execution result or the expression form of the execution result is specified, then the execution result of "the apple has a core" should also be regarded as an incorrect execution result. Understandably, the execution result of "yes" is the correct execution result. For another example, the text to be processed is "Which variety is better, green apples or red apples?" If the actual demand is to meet human preferences, the preset standard is set based on human preferences, and the human feedback result is that green apples are better, then the execution result of "red apples" is an incorrect execution result.

[0087] To facilitate the distinction of execution results in different situations, in this embodiment, the execution result obtained by the target language model executing the target task on the text to be processed is referred to as the first execution result.

[0088] 103. When it is determined that the first execution result is wrong, a problem model is determined from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed. The problem model is a model whose output result is wrong when the input is the text to be processed.

[0089] 104. Determine the target cause based on the problem model.

[0090] Based on the above discussion on the training process of the target language model and the purpose of the target language model, it can be known that if there is no problem in the training process of the target language model, the first execution result output by the target language model should meet the preset expectations and meet the preset standards. If the first execution result is wrong, that is, it does not meet the preset expectations or does not meet the preset standards, it may indicate that there is a problem in the training process of the target language model. In the case of determining that the first execution result is wrong, in order to locate the target cause of the target language model outputting the wrong result, it is necessary to trace the problem in the training process of the target language model. That is, full-link attribution is performed. Full-link attribution refers to the process of locating problems that occur during the training process of the target language model.

[0091] Specifically, this embodiment inputs the text to be processed into the pre-trained model, the supervised fine-tuning model, and the reward model. If any model outputs an erroneous result that does not conform to the purpose of this model, it is determined which model has a problem, that is, which model is determined to be the problem model. The pre-trained model, the supervised fine-tuning model, and the reward model are the presentation forms of the target language model during the training process. After determining which model in the training process is the problem model, the target cause of the target language model outputting the wrong result can be determined based on the problem model.

[0092] In an embodiment of the present application, when the target language model outputs an erroneous first execution result based on the text to be processed, it indicates that a problem occurred during the training process of the target language model. Based on this, the problem location method in this embodiment is adopted to input the text to be processed into the pre-training model, the supervised fine-tuning model, and the reward model, and determine which model in the training process is the problem model based on the results output by each model, so as to trace the problem in the training process of the target language model. Furthermore, based on the problem model, the target cause of the target language model outputting the wrong result can be located.

[0093] As an optional implementation, in the above step 103, when it is determined that the first execution result is wrong, determining the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed includes the following steps:

[0094] When it is determined that the first execution result is wrong, the problem model is determined from the pre-trained model, the supervised fine-tuning model, and the reward model using the text to be processed in a backtracking manner.

[0095] Among them, backtracking refers to the sequence of generating the pre-training model, supervised fine-tuning model, and reward model in the process of obtaining the target language model based on training, tracing back from back to front.

[0096] Specifically, the training process of the target language model adopts a pipeline mode, from the pre-training model to the supervised fine-tuning model, and then to the reward model, and finally trains to obtain the target language model. In the case where the first execution result is determined to be wrong, in this embodiment, the text to be processed is input into the reward model, the supervised fine-tuning model, and the pre-training model in a backtracking manner in sequence to trace the problem in the training process of the target language model until the problem model that outputs the wrong result is determined, and the target cause is further determined based on the problem model. For example, in the case where the reward model outputs an incorrect result, the reward model is determined to be a problem model, and the target cause is further determined based on the problem with the reward model. If the result output by the reward model is correct, the text to be processed will continue to be input into the supervised fine-tuning model. In the case where the supervised fine-tuning model outputs an incorrect result, the supervised fine-tuning model is determined to be a problem model, and the target cause is further determined based on the problem with the supervised fine-tuning model.

[0097] In the embodiment of the present application, the problem model is determined by backtracking, which can avoid missing problems that may occur during the training process. In addition, starting from the wrong first execution result, the cause of the error in the first execution result is gradually and in-depth and targetedly deduced, which is conducive to quickly and accurately locating the cause of the error in the first execution result, that is, quickly and accurately locating the target cause of the wrong result output by the target language model.

[0098] As an optional implementation, Figure 2 As shown, the above steps: when it is determined that the first execution result is wrong, the problem model is determined from the pre-trained model, the supervised fine-tuning model, and the reward model using the text to be processed in a backtracking manner, including the following steps 201 to 203.

[0099] 201. When it is determined that the first execution result is wrong, the text to be processed and the first execution result are input into a reward model to obtain a first score output by the reward model. The first score represents the quality of the result output by the target language model evaluated according to a preset standard.

[0100] When it is determined that the first execution result is wrong, the text to be processed is first input into the reward model in a backtracking manner, starting from the wrong first execution result, to trace the problem in the training process of the target language model. Specifically, the text to be processed and the first execution result are input into the reward model, so that the reward model is based on the text to be processed and evaluates the quality of the first execution result according to the preset standard to obtain a first score. In layman's terms, the reward model needs to evaluate the quality of the first execution result based on the specific information contained in the text to be processed, and the specific quality evaluation mechanism needs to meet the preset standards.

[0101] 202. Input the to-be-processed text and the correct execution result of the target task into the reward model to obtain a second score output by the reward model. The second score represents the quality of the correct execution result evaluated according to a preset standard.

[0102] The correct execution result of the target task refers to the expected execution result that meets the preset expectations and the preset standards when the target language model executes the target task based on the text to be processed. For example, if the text to be processed is "Does the apple have a core?", if the preset standard is set based on objective laws, when the execution result is "the apple has no core", it indicates that the execution result is wrong; when the execution result is "the apple has a core", this execution result is the correct execution result of the target task. The text to be processed and the correct execution result of the target task are input into the reward model, so that the reward model evaluates the quality of the correct execution result of the target task based on the text to be processed and according to the preset standard to obtain a second score.

[0103] 203. When the first score is greater than the second score, determine that the reward model is a problem model.

[0104] It can be understood that the execution result output by the target language model is evaluated based on the text to be processed. When there is no problem with the reward model, the score corresponding to the quality of the erroneous execution result should be lower than the score corresponding to the quality of the correct execution result. That is, the first score should be lower than the second score. In the embodiment of the present application, when the first score is greater than the second score, it indicates that the reward model outputs an erroneous result, that is, there is a problem with the reward model. At this time, it can be determined that the reward model is a problem model. Based on this, the reason why the reward model has a problem can be further determined based on the fact that the reward model is a problem model, that is, the target reason that causes the target language model to output an erroneous result can be quickly and accurately located.

[0105] Exemplarily, the problem locating device runs a full-link attribution platform (hereinafter referred to as the "platform"), and performs full-link attribution through the platform, that is, tracing problems in the training process of the target language model. Figure 3 A schematic diagram of the interface of a full-link attribution platform provided for an embodiment of the present application. Through the interface of the platform, the user can be shown the relevant indicators in each training link during the problem tracing process of the training process of the target language model when the target language model outputs an erroneous first execution result based on the text to be processed. The relevant indicators in each training link are the expression of the platform's conclusion on whether there is a problem in the corresponding training link, whether the model of the corresponding training link is a problem model, and the target cause of the error in the first execution result. By consulting the relevant indicators in each training link, the user can quickly know the problem model that causes the error in the target language model, as well as the target cause. The specific analysis is as follows.

[0106] like Figure 3 As shown in (a), Bad Case refers to an error example, that is, an example in which the target language model outputs an incorrect first execution result based on the text to be processed. Among them, Prompt is the prompt word, that is, it represents the text to be processed; Response is the response, that is, it represents the first execution result. Key_info is the key information of the prompt word or response, and error_info is the error information of the response. There is currently a Bad Case, the Prompt is "Who is Liu Xiaohua's brother?", and the Response of the target language model is "Hello, Liu Xiaohua's brother is Zhu Xiaohua.". It should be noted that the correct execution result of the target task corresponding to Prompt is "Liu Dahua.". "Hello, Liu Xiaohua's brother is Zhu Xiaohua." This Response is the incorrect first execution result of the target language model. The user inputs this Bad Case into the platform, and the interface information presented is as follows Figure 3 As shown in the "Bad Case" section (a) in the figure, the Key_info of Prompt is Liu Xiaohua and Brother, the Key_info of Response is Liu Xiaohua and Brother, and the error_info of Response is Zhu Xiaohua.

[0107] When a Bad Case is input into the platform, the platform will trace the problem in the training process of the target language model and display the relevant indicators of the training process corresponding to the target language model, reward model, supervised fine-tuning model, and pre-trained model in sequence through the interface. It should be noted that the target language model is obtained through reinforcement learning of the supervised fine-tuning model. Figure 3 The related indicators shown in the "Reinforcement Learning (RL)" part (a) refer to the related indicators of the training phase of obtaining the target language model through reinforcement learning, and RL Model is used to refer to the target language model.

[0108] In the process of obtaining the target language model through reinforcement learning of the supervised fine-tuning model, in addition to iterating the supervised fine-tuning model based on the evaluation results output by the reward model, the supervised fine-tuning model is also iterated based on the execution results output by the reference model. Specifically, the reference model is a model with similar learning prediction capabilities to the supervised fine-tuning model. The text to be processed is input into the reference model, and the reference model can output an execution result that meets the learning prediction capabilities of the supervised fine-tuning model. In the actual processing process, the reference model and the supervised fine-tuning model can be regarded as the same model according to actual needs. In order to prevent the execution results output by the supervised fine-tuning model from being too consistent with the preset standards based on the reward model in the process of iterating the supervised fine-tuning model based on the evaluation results output by the reward model, thereby reducing the learning prediction capabilities that the supervised fine-tuning model itself needs to have. In the process of iterating the supervised fine-tuning model based on the evaluation results output by the reward model, the supervised fine-tuning model is also iterated based on the execution results output by the reference model, that is, the supervised fine-tuning model is iterated based on the evaluation results output by the reward model and the execution results output by the reference model at the same time. The target language model obtained by this iterative method generates an execution result that meets both the preset standards based on the reward model and the preset expectations required by the supervised fine-tuning model. It should be understood that the target language model iteratively obtained by the above method, based on the execution result output by the to-be-processed text, should have a high similarity with the execution result output by the reference model based on the to-be-processed text.

[0109] like Figure 3As shown in (a), the first relevant metric in the Reinforcement Learning (RL) training session is Token Metric(s), which represents the metric of each word or term in the Response output by the target language model based on the prompt appearing at the specified position. The metric of each word or term appearing at the specified position includes prob, ref_prob, logp, and ref_logp. Among them, prob refers to probability, logp refers to the logarithm of probability, ref_prob refers to the probability of the same word or term appearing at the same specified position in the execution result output by the reference model based on the prompt, and ref_logp refers to the logarithm of the probability of the same word or term appearing at the same specified position in the execution result output by the reference model. The same word or term appearing at the same specified position means the same word or term in the execution result of the target language model appears at the same specified position. For example, 0.424351 refers to the probability that the target language model outputs the word "您" at the 0th specified position; 0.179505 refers to the probability that the reference model outputs the word "您" at the 0th specified position. By comparing the prob and ref_prob, logp and ref_logp of each word or term appearing at the specified position, it can be seen that there are significant differences between the metrics of each word or term in the Response output by the target language model appearing at the specified position and the metrics of the same word or term in the execution result output by the reference model appearing at the same specified position. That is, at this time, there are significant differences between the Response output by the target language model based on the prompt and the execution result output by the reference model based on the prompt. KL(response-level) and KL(token-level) represent the difference level of the execution results output by the target language model and the reference model and the difference level of each word or term in the execution results, respectively. KL(response-level) reaches 2.38 and KL(token-level) reaches 0.07, indicating that there are significant differences between the execution results output by the target language model and the execution results output by the reference model, and it is difficult for the target language model to demonstrate the learning and prediction ability of the supervised fine-tuning model.

[0110] The second relevant indicator of the Reinforcement Learning (RL) training phase is Top K Next Token(s), which is used to represent the top K characters or words that may appear in the next position of the specified position and their probabilities in the execution result. By comparing the top K characters or words that may appear in the next position of the specified position and their probabilities of the RL Model and the Reference Mode, it can be seen that there are significant differences between the two models. In other words, the second relevant indicator also shows that the response output by the target language model based on the prompt is significantly different from the execution result output by the reference model based on the prompt.

[0111] The third relevant indicator of the Reinforcement Learning (RL) training phase is the Training Dataset. This training dataset refers to the training dataset used to perform reinforcement learning on the supervised fine-tuning model to obtain the target language model. The platform determines the top N prompts that are most similar to the prompt in this Bad Case from this training dataset, that is, determines the training data. N is a positive integer. And trace back the execution results (RL Response) corresponding to the reinforcement learning and the execution results (ReferenceResponse) corresponding to the reference model based on the most similar prompts in the reinforcement learning process. Then calculate the embedding similarity (EmbeddingSimilarity) between the execution results based on the two types of execution results. Among them, if Figure 3 As shown in the "Training Dataset" section (a), the platform displays the above data information as indicator information, and the Embedding Similarity between the prompt and the execution result corresponds one to one. By analyzing the EmbeddingSimilarity between the first five prompts that are most similar to the prompt in this Bad Case, it can be seen that in the process of obtaining the target language model through reinforcement learning, the similarity between the execution results corresponding to the reinforcement learning and the execution results corresponding to the reference model is generally low. In other words, the third related indicator can also be used to analyze that the Response output by the target language model based on the prompt will be significantly different from the execution result output by the reference model based on the prompt.

[0112] Through the related indicators of the Reinforcement Learning (RL) training phase displayed on the platform interface, it can be clearly known that the response output by the target language model based on the prompt will be significantly different from the execution result output by the reference model based on the prompt. In other words, it is difficult for the target language model to reflect the learning and prediction capabilities of the supervised fine-tuning model, and the execution result output by the target language model does not meet the preset expectations, and the execution result output by the target language model is wrong. The above related indicators confirm that the Response in this Bad Case is wrong.

[0113] Based on this, the training process of the target language model is traced back to the relevant indicators of the training link corresponding to the reward model displayed on the platform interface, such as Figure 3 As shown in (b) in the figure. The first relevant indicator of the Reward Model (RM) training phase is the numerical score (Reward). Specifically, the user inputs the Bad Case Prompt "Who is Liu Xiaohua's brother?" and the wrong first execution result (Bad Case Response) "Hello, Liu Xiaohua's brother is Zhu Xiaohua.", as well as the correct execution result (Custom Response) "Liu Dahua." that conforms to objective laws into the platform. The platform analyzes and displays the Prompt and Response1, Response2, and displays the Reward that the reward model evaluates the quality of the two execution results based on the prompt according to the preset standards. As shown in Figure 3 As shown in (b), the Reward corresponding to the wrong first execution result is higher than the Reward corresponding to the correct execution result. Among them, Win Rate indicates a higher confidence level for the corresponding Reward, which is an automatic scoring of the numerical score of the evaluation by the platform. The higher confidence level of the Reward for the wrong first execution result reached 0.53. This indicates that the reward model outputted an incorrect result, that is, there was a problem with the reward model. Based on this, in the process of tracing the problem in the training process of the target language model, it can be determined that the reward model is a problem model.

[0114] Optionally, the problem location method, problem location device, and platform in this embodiment can support the custom addition of multiple reward models. That is, it supports reinforcement learning of the supervised fine-tuning model based on multiple reward models to obtain the target language model, supports multiple reward models to simultaneously evaluate the quality of the execution results output by the supervised fine-tuning model, the target language model, and other language models, and supports specific analysis of whether multiple reward models have errors.

[0115] In an embodiment of the present application, in the process of tracing problems in the training process of the target language model, if the score corresponding to the quality of the incorrect execution result evaluated by the reward model is higher than the score corresponding to the quality of the correct execution result evaluated, that is, the first score is greater than the second score, it indicates that there is a problem with the reward model. At this time, the reward model can be determined to be a problem model. Based on this, the cause of the problem with the reward model can be further determined based on the fact that the reward model is a problem model, which is conducive to quickly and accurately locating the target cause that causes the target language model to output an erroneous result. In addition, in the problem location device operation platform, the platform interface is used to display to the user the relevant indicators in each training link during the process of tracing problems in the training process of the target language model. In this case, the problem location method in this embodiment can more intuitively display the target language model's ability to understand text and its working principle through the visualization of probability; through the visualization of each relevant indicator, it is conducive to enabling users to quickly know the problem model and the target cause that caused the error in the target language model.

[0116] As an optional implementation, Figure 4 As shown, the problem location method includes the following steps 401 to 408. Among them, steps 401 to 405 correspond to steps 101 to 102 and 301 to 303 in the above-mentioned embodiment. The specific discussion of steps 401 to 405 in this embodiment can refer to the specific discussion of steps 101 to 102 and 301 to 303 in the above-mentioned embodiment, which will not be repeated in this embodiment.

[0117] 401. Get the text to be processed.

[0118] 402. Input the text to be processed into the target language model so that the target language model performs a target task on the text to be processed and obtains a first execution result. The target task is a natural language processing task.

[0119] 403. When it is determined that the first execution result is wrong, the text to be processed and the first execution result are input into the reward model to obtain a first score output by the reward model. The first score represents the quality of the result output by the target language model evaluated according to a preset standard. The reward model is obtained by training the adjusted model using the first training data set after adjusting the model structure of the supervised fine-tuning model to obtain an adjusted model.

[0120] 404. Input the to-be-processed text and the correct execution result of the target task into the reward model to obtain a second score output by the reward model. The second score represents the quality of the correct execution result evaluated according to a preset standard.

[0121] 405. When the first score is greater than the second score, determine that the reward model is a problem model. A problem model is a model whose output result is an error when the input is the text to be processed.

[0122] 406 . Determine, from the first training data set, first training data whose semantics match the semantics of the text to be processed and a label of the first training data.

[0123] In the case where the reward model is determined to be a problem model, the target cause can be determined based on the reward model. Referring to the above description of the training process of the reward model, it can be seen that the first training data in the first training data set includes the input text input to the supervised fine-tuning model and the execution result output by the supervised fine-tuning model based on the input text. The input text and its corresponding execution result in the first training data are input into the adjustment model, so that the adjustment model evaluates the quality of the execution result output by the supervised fine-tuning model based on the input text. If the supervised fine-tuning model outputs more than one execution result, the evaluation of the quality by the adjustment model is presented in the form of a ranking of more than one execution result. At this time, the label corresponding to the first training data in the first training data set is the ranking of more than one execution result. Using the label of the first training data to supervise the output of the adjustment model is conducive to the training of the adjustment model, so as to train a reward model that can evaluate the quality of the results output by the supervised fine-tuning model. In the case where the semantics of the text to be processed matches the semantics of the first training data, if the label of the first training data is correct, then the reward model obtained by training the first training data and the label of the first training data, when evaluating the quality of the execution result output based on the text to be processed, the evaluation result will be similar to the result shown by the label of the first training data and meet the preset standard. If the label of the first training data is wrong, the reward model obtained by training the first training data and the label of the first training data will have problems in predicting the text to be processed that matches the semantics of the first training data. At this time, the evaluation result of evaluating the quality of the execution result output based on the text to be processed will also have problems. It should be noted that the semantics of the text to be processed matches the semantics of the first training data, specifically refers to the semantics of the text to be processed and the input text contained in the first training data that is input to the supervised fine-tuning model; matching can be understood as similarity, that is, semantic similarity, and the higher the similarity, the better the match. For example, the first training data whose semantics matches the semantics of "Who is Liu Xiaohua's brother?" should have semantics related to Liu Xiaohua's brother.

[0124] Specifically, if there is a problem with the reward model, it may be that there is a problem with the first training data set for training the reward model. If there is a problem with the first training data set, it may be that there is a problem with the label of the first training data whose semantics match the semantics of the text to be processed. If the label is wrong, when the adjustment model is optimized based on the label and the output of the adjustment model, it will affect the optimization parameter adjustment range of the adjustment model, etc., and the knowledge learned by the reward model through parameter adjustment will be wrong. Furthermore, it will cause the optimized reward model to have deviations in its learning and prediction ability for the text to be processed, and then cause the reward model to output an erroneous evaluation result. Based on this, in order to determine the target cause according to the reward model, the first training data whose semantics match the semantics of the text to be processed and the label of the first training data can be first determined from the first training data set.

[0125] Optionally, an embedding of the text to be processed and an embedding of each training data in the first training data set are generated. In layman's terms, embedding can be understood as a semantic feature vector. Among them, the embedding of the training data in the present embodiment refers to the semantic feature vector of the input text contained in the training data. If the distance between the semantic feature vector of the training data and the semantic feature vector of the text to be processed is short, it indicates that the semantic feature vectors are similar, and also indicates that the semantics of the training data and the semantics of the text to be processed are similar, that is, the semantics of the training data and the semantics of the text to be processed match. Based on this, the first training data whose semantics match the semantics of the text to be processed can be determined by the semantic feature vector of the text to be processed and the semantic feature vector of each training data in the first training data set. Automatic and fast search of training data is achieved.

[0126] 407. Determine, according to the label of the first training data, a ranking result of the n labeling results corresponding to the first training data.

[0127] According to the above discussion, if the supervised fine-tuning model outputs more than one execution result, the evaluation of the quality of the adjustment model is presented in the form of a ranking of more than one execution result. At this time, the label corresponding to the first training data in the first training data set is the ranking of more than one execution result. Based on this, in order to determine whether there is a problem with the label of the first training data that matches the semantics of the text to be processed, the ranking result in the label of the first training data can be determined first, that is, the ranking result of the n labeled results corresponding to the first training data can be determined first. Among them, n means more than one, and the labeled result refers to the execution result labeled manually or by other means.

[0128] 408. When the sorting result is wrong, the target cause is determined to be that the reward model is obtained by training the adjustment model using the first training data and the label of the first training data, resulting in the reward model obtaining wrong information.

[0129] In the case of an error in the sorting result, it indicates that there is a problem with the label of the first training data whose semantics match the semantics of the text to be processed. Based on this, it can be deduced that the error in the label of the first training data causes the optimized reward model to have a deviation in its learning prediction ability for the text to be processed, which in turn causes the reward model to output an erroneous evaluation result. In other words, it can be determined that the reward model is obtained by training the adjustment model using the first training data and the label of the first training data, which causes the reward model to obtain erroneous information, which in turn causes the reward model to become a problem model, which in turn causes the target language model to output an erroneous first execution result.

[0130] Exemplarily, when a Bad Case is input into the platform, the platform traces the problem in the training process of the target language model, and displays the relevant indicators in the training links corresponding to the target language model, reward model, supervised fine-tuning model, and pre-training model in sequence through the interface. Figure 5 This is a schematic diagram of the interface of a full-link attribution platform provided in an embodiment of the present application. Figure 5 As shown in (a), the second relevant indicator of the training phase corresponding to the reward model displayed by the platform through the interface is the training dataset (Training Dataset). When it is determined that the reward model is a problem model, the platform determines the first training data whose semantics match the semantics of the text to be processed from the first training dataset. Referring to the above discussion, semantic matching means semantic similarity; the first training data includes input text input into the supervised fine-tuning model and one or more execution results output by the supervised fine-tuning model based on the input text; determining the first training data whose semantics match the semantics of the text to be processed, in this example, refers to determining the first training data corresponding to the input text whose semantics are similar to the semantics of the text to be processed. According to Figure 5 As shown in (a), the platform determines the first N input texts with the most similar semantics to the text to be processed from the first training data set, corresponding to the first training data. Each first training data contains an input text (prompt) and three execution results: Response1, Response2 and Response3. And the label (Labeld) of each first training data is determined. In addition, the platform compares the sorting results of the three annotation results presented by the label of each first training data with the correct sorting results, and obtains the embedding similarity (EmbeddingSimilarity) of the two types of sorting results of each first training data. Figure 5As shown in (a), the platform displays the above data information as indicator information, and the Embedding Similarity between the prompt and the sorting result corresponds one to one. According to the data displayed by the embedding similarity, the embedding similarity is generally low, and the sorting results of the three annotation results presented by the labels of the first training data do not meet the preset standards as a whole. There is a problem with the label of the first training data. In other words, based on the second relevant indicator of the reward model corresponding to the training link, it can be analyzed that it is due to the error in the label of the first training data that the optimized reward model has a deviation in its learning and prediction ability for the processed text, causing the reward model to become a problem model, which in turn causes the target language model to output an erroneous first execution result.

[0131] In this embodiment, in the process of tracing problems in the training process of the target language model, when the reward model is determined to be a problem model, the first training data whose semantics match the semantics of the text to be processed and the label of the first training data are determined from the first training data set obtained by training the reward model. Furthermore, by determining that the label of the first training data is wrong, it is determined that the label error of the first training data is the reason why the reward model becomes a problem model and the reason why the target language model outputs an erroneous first execution result. Through the problem locating method in this embodiment, it is achieved to quickly and accurately locate the target cause of the result that the target language model outputs an erroneous result. Furthermore, it is conducive to quickly and accurately eliminating the target cause and to reduce the performance problems of each model caused by the training data error. At the same time, by determining the first training data whose semantics match the semantics of the text to be processed and the label of the first training data, an automatic and rapid search for training data is achieved. In the case of displaying the first training data through the platform, the visualization of the training data is also achieved, which is convenient for users to observe and process the training data.

[0132] As an optional implementation, Figure 6 As shown, the problem location method includes the following steps 601 to 608. Among them, steps 601 to 605 and 608 correspond to steps 101 to 102, 301 to 303, and 104 in the aforementioned embodiment. The specific discussion of steps 601 to 604 and 607 to 608 in this embodiment can refer to the specific discussion of steps 101 to 102, 301 to 303, and 104 in the aforementioned embodiment, which will not be repeated in this embodiment.

[0133] 601. Get the text to be processed.

[0134] 602. Input the text to be processed into the target language model so that the target language model performs a target task on the text to be processed, and obtains a first execution result. The target task is a natural language processing task.

[0135] 603. When it is determined that the first execution result is wrong, the text to be processed and the first execution result are input into the reward model to obtain a first score output by the reward model. The first score represents the quality of the result output by the target language model evaluated according to a preset standard.

[0136] 604. Input the to-be-processed text and the correct execution result of the target task into the reward model to obtain a second score output by the reward model. The second score represents the quality of the correct execution result evaluated according to a preset standard.

[0137] 605. When the first score is greater than the second score, determine that the reward model is a problem model. A problem model is a model whose output result is an error when the input is the text to be processed.

[0138] Or 606, when the first score is less than the second score, input the text to be processed into the supervised fine-tuning model to obtain a second execution result output by the supervised fine-tuning model.

[0139] As discussed in the above embodiments, when the first score is less than the second score, it indicates that the reward model outputs the correct result, that is, there is no problem with the reward model. Based on this, the problem location device continues to trace the problem in the training process of the target language model, inputs the text to be processed into the supervised fine-tuning model, and obtains the second execution result output by the supervised fine-tuning model. Then, based on whether the second execution result is correct or wrong, it is determined whether there is a problem with the supervised fine-tuning model.

[0140] 607. When the second execution result is erroneous, determine that the supervised fine-tuning model is a problem model.

[0141] 608. Determine the target cause based on the problem model.

[0142] It should be understood that in the embodiment of the present application, after the problem location device completes step 604 and obtains the first score and the second score, based on the magnitude relationship between the first score and the second score, it selects to execute step 605, or executes steps 606 to 607. Furthermore, after determining the problem model, step 608 is executed.

[0143] In the case where the problem location device chooses to execute step 605, after step 605, the problem location method further includes the above steps 406 to 407. In the case where the above steps 406 to 407 are included, step 608 includes the above step 408.

[0144] In the case where the problem location device chooses to execute steps 606 to 607, the second execution result is wrong, indicating that the supervised fine-tuning model does not output the correct result based on the text to be processed. It can be determined that there is a problem with the supervised fine-tuning model, that is, the supervised fine-tuning model is a problem model. Based on this, the reason why the supervised fine-tuning model has a problem can be further determined based on the fact that the supervised fine-tuning model is a problem model, that is, the target reason that causes the target language model to output an erroneous result can be quickly and accurately located.

[0145] It should be noted that if the second execution result is correct, it means that the supervised fine-tuning model has output the correct result based on the text to be processed, and there is no problem with the supervised fine-tuning model. Based on this, it can be reasonably deduced that there is a problem in the training process of the target language model obtained by iterating the supervised fine-tuning model based on the reward model. Due to the problem in this training process, the target language model becomes a problematic model, which in turn causes the target language model to output an incorrect result.

[0146] For example, when a Bad Case is input into the platform, the platform traces the problem in the training process of the target language model and displays the relevant indicators of the training links corresponding to the target language model, reward model, supervised fine-tuning model, and pre-trained model in sequence through the interface. The platform displays the relevant indicators of the supervised fine-tuning model corresponding to the training link through the interface, such as Figure 5 The first relevant indicator of the Supervised Fine-Tuning (SFT) training phase is the numerical score (Reward) of the result (Generation Answer) obtained by inputting the text to be processed (prompt) into the supervised fine-tuning model for testing (Generation Test). Figure 5 From the information displayed in the interface shown in (b), we can see that the platform inputs the prompt "Who is Liu Xiaohua's brother?" into the supervised fine-tuning model for multiple times to test and obtain the corresponding test results, that is, the corresponding second execution results. This approach can be used to check the exploration ability of the supervised fine-tuning model. Then, the quality of multiple second execution results is evaluated through the reward model to obtain corresponding numerical scores. Figure 5 As shown in (b), the platform displays the above data information as indicator information. Reward Mean refers to the average numerical score of multiple second execution results, and Reward Std refers to the variance of the numerical scores of multiple second execution results. It should be noted that Figure 5The interface shown in (b) only shows the first five tests, but in fact the number of tests is greater than five. It can be seen from the Reward Mean of 0.48 and the Reward Std of 0.19 that the overall quality of the multiple second execution results output by the supervised fine-tuning model obtained from multiple tests is low. It can be reasonably deduced that the correctness of the second execution result output by the supervised fine-tuning model is low, that is, the second execution result output by the supervised fine-tuning model is wrong. It should be noted that the platform can only perform one test, input the text to be processed into the supervised fine-tuning model to obtain the second execution result, and then evaluate the quality of the second execution result through the reward model, and determine whether the second execution result is correct or wrong through the numerical score obtained by the evaluation. The platform determines whether the second execution result is correct or wrong by the average numerical score and numerical score variance of multiple test results, which is conducive to analyzing the changing trend of the numerical score and improving the accuracy of the conclusions obtained. Based on the first relevant indicator of the corresponding training link of the supervised fine-tuning model, it can be analyzed that there is a problem with the supervised fine-tuning model. In the process of tracing the problem of the training process of the target language model, the supervised fine-tuning model can be determined as a problem model.

[0147] In this embodiment, when the first score is less than the second score, it indicates that there is no problem with the reward model. Based on this, the problem location device continues to trace the problem in the training process of the target language model, and inputs the text to be processed into the supervised fine-tuning model to obtain the second execution result output by the supervised fine-tuning model. Furthermore, based on the error in the second execution result, it is determined that the supervised fine-tuning model is a problem model. Based on this, the cause of the problem with the supervised fine-tuning model can be further determined based on the fact that the supervised fine-tuning model is a problem model, which is conducive to quickly and accurately locating the target cause of the error in the output of the target language model.

[0148] As an optional implementation, in Figure 6 In the problem localization method embodiment shown, when the problem localization device chooses to execute steps 606 to 607, after step 607, the problem localization method further includes the following steps: determining second training data whose semantics match the semantics of the text to be processed and the label of the second training data from the second training data set. The second training data set is a training data set used by the supervised fine-tuning model obtained by performing supervised fine-tuning on the pre-trained model.

[0149] In the case where supervised fine-tuning is determined to be a problem model, the target cause can be determined based on supervised fine-tuning. Referring to the aforementioned description of the training process of supervised fine-tuning and the specific discussion on "problems with the reward model may be caused by problems with the first training data set for training the reward model. Problems with the first training data set may be caused by problems with the labels of the first training data whose semantics match the semantics of the text to be processed." It can be seen that problems with the supervised fine-tuning model may be caused by problems with the second training data set for training the supervised fine-tuning model. Problems with the second training data set may be caused by problems with the labels of the second training data whose semantics match the semantics of the text to be processed in the second training data set. Similarly, semantic matching can be understood as semantic similarity, and the higher the similarity, the better the match. Label errors will affect the optimization parameter adjustment range of the pre-training model when optimizing the pre-training model based on the labels and the output of the pre-training model, and the knowledge learned by the supervised fine-tuning model through parameter adjustment will be wrong. In turn, the optimized supervised fine-tuning model will have deviations in its learning and prediction ability for the text to be processed, which will lead to the output of the supervised fine-tuning model with incorrect execution results. Based on this, in order to determine the target cause according to the supervised fine-tuning model, the second training data whose semantics match the semantics of the text to be processed and the label of the second training data can be first determined from the second training data set. Optionally, the embedding of the text to be processed and the embedding of each training data in the second training data set can be generated to achieve the second training data set to determine the semantics match the semantics of the text to be processed. The specific implementation process can refer to the specific discussion on determining the first training data whose semantics match the semantics of the text to be processed in the aforementioned embodiment, which will not be repeated here. By determining the second training data, an automatic and fast search for training data is achieved.

[0150] In the case where the problem localization method includes the above steps, step 608 includes the following steps: in the case where the label of the second training data is wrong, determining that the target cause is that the label of the second training data is wrong, resulting in the supervised fine-tuning model obtaining wrong information through supervised fine-tuning.

[0151] In the case where the label of the second training data is wrong, it can be deduced that the label of the second training data is wrong, which causes the optimized supervised fine-tuning model to have a deviation in the learning prediction ability of the processed text, which in turn causes the supervised fine-tuning model to output an erroneous second execution result. In other words, it can be determined that the supervised fine-tuning model obtains erroneous information due to the label error of the second training data, which in turn causes the supervised fine-tuning model to become a problem model, which in turn causes the target language model to output an erroneous first execution result.

[0152] For example, when a Bad Case is input into the platform, the platform traces the problem in the training process of the target language model and displays the relevant indicators in the training links corresponding to the target language model, reward model, supervised fine-tuning model, and pre-training model in sequence through the interface. Figure 5 As shown in (c), the second relevant indicator of the training phase corresponding to the supervised fine-tuning model displayed by the platform through the interface is the training dataset (Training Dataset). When the supervised fine-tuning model is determined to be the problem model, the platform determines the second training data whose top N semantics match the semantics of the text to be processed from the second training dataset, that is, determines the prompts of the top N semantic matches. And determines the label (Labeled Response) of each second training data. In addition, the platform compares the annotation results presented by the labels of each second training data with the correct execution results of the target task corresponding to each second training data, and obtains the embedding similarity (Embedding Similarity) of the two types of results of each second training data. As shown Figure 5 As shown in (c), the platform displays the above data information as indicator information, and the Embedding Similarity between prompt and result corresponds one to one. According to the data displayed by the embedding similarity, the embedding similarity is overall low, the annotation results presented by the labels of the second training data are generally less correct, and there is a problem with the labels of the second training data. In other words, based on the second relevant indicator of the corresponding training link of the supervised fine-tuning model, it can be analyzed that it is due to the error in the label of the second training data that the optimized supervised fine-tuning model has a deviation in its learning and prediction ability for the processed text, causing the supervised fine-tuning model to become a problematic model, which in turn causes the target language model to output an erroneous first execution result.

[0153] In the embodiment of the present application, in the process of tracing the problem of the training process of the target language model, when the supervised fine-tuning model is determined to be a problem model, the second training data whose semantics matches the semantics of the text to be processed and the label of the second training data are determined from the second training data set of the supervised fine-tuning model trained. Furthermore, by determining that the label of the second training data is wrong, it is determined that the label error of the second training data is the reason why the supervised fine-tuning model becomes a problem model, and the reason why the target language model outputs an erroneous first execution result. Through the problem locating method in this embodiment, it is achieved to quickly and accurately locate the target cause of the result that causes the target language model to output an erroneous result. Furthermore, it is conducive to quickly and accurately eliminating the target cause, and it is conducive to reducing the performance problems of each model caused by the training data error. At the same time, by determining the second training data whose semantics matches the semantics of the text to be processed and the label of the second training data, an automatic and rapid search for training data is achieved. In the case of displaying the second training data through the platform, the visualization of the training data is also achieved, which is convenient for users to observe and process the training data.

[0154] As an optional implementation, in Figure 6 In the problem locating method embodiment shown, when the problem locating device chooses to execute steps 606 to 607, after step 607, the problem locating method further includes the following steps 6071 to 6074.

[0155] 6071. Determine second training data whose semantics match the semantics of the text to be processed and a label of the second training data from the second training data set. The second training data set is a training data set used by a supervised fine-tuning model obtained by performing supervised fine-tuning on a pre-trained model.

[0156] As discussed in the foregoing embodiments, in order to determine the target cause according to the supervised fine-tuning model, second training data whose semantics match the semantics of the text to be processed and the label of the second training data may be first determined from the second training data set.

[0157] 6072. If the label of the second training data is correct, obtain reference prompt words. The reference prompt words include reference examples and text that describes the target task and the content to be determined according to a preset sentence pattern. The reference example is an example that describes the reference task and the reference execution result of the reference task according to the preset sentence pattern. The position of the target task in the preset sentence pattern is the same as the position of the reference task in the preset sentence pattern. The position of the content to be determined in the preset sentence pattern is the same as the position of the reference execution result in the preset sentence pattern. The reference example is used to guide the pre-trained model to determine the content to be determined.

[0158] When the label of the second training data is correct, it indicates that it is not due to the error in the label of the second training data that the optimized supervised fine-tuning model has a deviation in the learning prediction ability of the processed text, which in turn causes the supervised fine-tuning model to output an erroneous second execution result. Based on this, another possible reason for the problem with the supervised fine-tuning model can be further considered, that is, due to a problem with the pre-training model, the supervised fine-tuning model obtained by training based on the pre-training model has a problem. If there is a problem with the pre-training model, the pre-training model should output an erroneous execution result. Based on this, the pre-training model can be made to output an execution result, and the correctness or error of the execution result can be used to determine whether there is a problem with the pre-training model, and then determine whether the above speculation about the cause of the problem with the supervised fine-tuning model is correct.

[0159] Since the pre-trained model has the ability to process natural language, but does not have the ability to generate specific results that meet preset expectations. Therefore, if the text to be processed is directly input into the pre-trained model, the pre-trained model cannot output a specific execution result according to the guidance of the text to be processed. Based on this, the pre-trained model can be guided by in-context learning (ICL) to guide the pre-trained model to output a specific execution result that meets preset expectations. Specifically, a reference prompt word is obtained, and the reference prompt word includes a reference example that describes the reference task and the reference execution result of the reference task according to a preset sentence pattern, and a text that describes the target task and the content to be determined according to a preset sentence pattern. For example, the reference prompt word is "Xiao Ming's brother is Daming; Zhang Xiaojie's brother is Zhang Dajie; Liu Xiaohua's brother is". "Xiao Ming's brother is Daming; Zhang Xiaojie's brother is Zhang Dajie" is a reference example that describes the reference task and the reference execution result of the reference task according to a preset sentence pattern. "Liu Xiaohua's brother is" is a text that describes the target task and the content to be determined according to a preset sentence pattern, that is, the text to be processed. Among them, the position of the target task "Liu Xiaohua's brother is" in the preset sentence is the same as the position of the reference tasks "Xiao Ming's brother is" and "Zhang Xiaojie's brother is" in the preset sentence. The current content to be determined is a null value, and its position in the preset sentence should be after "is" in "Liu Xiaohua's brother is", which is the same as the position of the reference execution results "Da Ming" and "Zhang Dajie" in the preset sentence. It can be understood that the reference examples can be one or more, and the more reference examples there are, the more conducive it is for the pre-trained model to learn the rules in the reference examples, and the better the guiding effect on the pre-trained model.

[0160] Optionally, the user may directly provide reference prompt words to the problem locating device. The problem locating device may also call an application programming interface (API) such as GPT-4 to obtain reference prompt words based on the information written by GPT-4. Optionally, if the obtained reference prompt words do not fully meet the preset sentence pattern or the need to guide the pre-trained model, the reference prompt words may be fine-tuned so that the final reference prompt words meet the corresponding needs.

[0161] 6073. Input the reference prompt word into the pre-training model, so that the pre-training model determines the content to be determined under the guidance of the reference example, as the third execution result output by the pre-training model.

[0162] The reference prompt word is input into the pre-trained model, and the reference example in the reference prompt word can guide the pre-trained model with the same preset sentence pattern and the corresponding field at the same position in the preset sentence pattern, so that the pre-trained model outputs a non-empty value of the content to be determined based on the target task and the text of the content to be determined described in the preset sentence pattern. The output non-empty value of the content to be determined is a specific third execution result output by the pre-trained model that meets the preset expectations.

[0163] 6074. When the third execution result is erroneous, determine that the pre-trained model is a problem model.

[0164] In the case of an error in the third execution result, it indicates that the content to be determined determined by the pre-trained model under the guidance of the reference example is wrong, indicating that there is a problem with the prediction ability learned by the pre-trained model during the training process, or the pre-trained model has learned wrong knowledge during the training process. At this time, it can be determined that the pre-trained model is also a problem model. So far, it is demonstrated that the supervised fine-tuning model obtained by training based on the pre-trained model has a problem due to the problem of the pre-trained model, which in turn leads to the first execution result in which the target language model outputs an error.

[0165] It can be understood that in the embodiment of the present application, after executing step 607, the problem locating device continues to execute steps 6071 to 6074. After executing step 6074, step 608 is executed.

[0166] In an embodiment of the present application, in the process of tracing problems in the training process of the target language model, when it is determined that the supervised fine-tuning model is a problem model, the second training data whose semantics match the semantics of the text to be processed and the label of the second training data are determined from the second training data set in which the supervised fine-tuning model is trained. When it is determined that the label of the second training data is correct, the pre-trained model is guided by means of ICL to guide the pre-trained model to output a specific third execution result that meets the preset expectations. Furthermore, in the case where the third execution result is wrong, it is determined that the pre-trained model is also a problem model. That is, it is determined that the problem is caused by the pre-training model, which causes the supervised fine-tuning model obtained by training based on the pre-training model to have a problem, and then causes the target language model to output an erroneous first execution result. Based on this, the cause of the problem with the pre-training model can be further determined based on the fact that the pre-training model is a problem model, which is conducive to quickly and accurately locating the target cause of the target language model outputting an erroneous result.

[0167] As an optional implementation, in the embodiment where the problem locating method further includes steps 6071 to 6074, when the problem locating device selects to execute steps 606 to 607, and continues to execute steps 6071 to 6074 after completing step 607, after step 6074, the problem locating method further includes the following steps:

[0168] Determine the third training data set from the third training data set whose semantics matches the semantics of the text to be processed. The third training data set is a training data set used by the pre-training model obtained by pre-training the basic language model.

[0169] In the case where the pre-trained model is determined to be a problem model, the target cause can be determined based on the pre-trained model. Referring to the above specific discussion on "there is a problem with the reward model, which may be a problem with the first training data set used to train the reward model. There is a problem with the first training data set, which may be a problem with the label of the first training data whose semantics matches the semantics of the text to be processed", it can be seen that there is a problem with the pre-trained model, which may be a problem with the third training data set used to train the pre-trained model. There is a problem with the third training data set, which may be a problem with the third training data whose semantics match the semantics of the text to be processed in the third training data set. Similarly, semantic matching can be understood as semantic similarity, and the higher the similarity, the better the match. It should be noted that the pre-trained model is obtained by pre-training the basic language model with an unsupervised third training data set, so there is no label supervision during the training process of the pre-trained model. In the case of an error in the third training data set, the knowledge learned by the pre-trained model based on the third training data set is wrong. In turn, it will cause the pre-trained model to have a deviation in its learning and prediction ability for the text to be processed, and then cause the pre-trained model to output an erroneous third execution result based on the text to be processed under the guidance of the reference example. Based on this, in order to determine the target cause according to the pre-training model, third training data whose semantics matches the semantics of the text to be processed can be first determined from the third training data set.

[0170] In the case where the problem localization method includes the above steps, step 608 includes the following steps: in the case where the third training data is wrong, determining that the target cause is that the third training data is wrong, causing the pre-training model to obtain wrong information through pre-training.

[0171] It can be understood that in the embodiment of the present application, after executing step 607, the problem location device continues to execute steps 6071 to 6074. After executing step 6074, the step of determining the third training data set whose semantics matches the semantics of the text to be processed is continued. Then, the step of determining the target cause as the error in the third training data causing the pre-training model to obtain erroneous information through pre-training is executed again in the case where the third training data is wrong.

[0172] In the case where the third training data is wrong, it can be deduced that the error in the third training data causes the pre-trained model to have a deviation in its learning and prediction ability for the text to be processed, which in turn causes the pre-trained model to output an erroneous third execution result based on the text to be processed under the guidance of the reference example. In other words, it can be determined that the error in the third training data causes the pre-trained model to obtain erroneous information, which in turn causes the pre-trained model to become a problem model, which in turn causes the target language model to output an erroneous first execution result.

[0173] For example, when a Bad Case is input into the platform, the platform traces the problem in the training process of the target language model and displays the relevant indicators in the training links corresponding to the target language model, reward model, supervised fine-tuning model, and pre-training model in sequence through the interface. Figure 5 As shown in (d), the first relevant indicator of the training link corresponding to the pre-training model (Pretraining Model) displayed by the platform through the interface is the result (Ouput) output by the pre-training model based on the ICL reference prompt word (ICL Prompt) and the probability (prob) of each character or word in the result. This result is also the third execution result mentioned above. Specifically, the user inputs the reference prompt word into the platform, or the platform calls APIs such as GPT-4 to obtain the reference prompt word based on the information written by GPT-4. Among them, the user can fine-tune the obtained reference prompt word so that the final reference prompt word meets the corresponding needs. As Figure 5 As shown in (d), the third execution result output by the pre-trained model based on the ICL reference prompt word is "Liu Dahua.", and the probability of each character or word appearing in the third execution result is generally high, and this third execution result is correct. When the platform displays the above indicator information, it indicates that there is no problem with the pre-trained model, and the pre-trained model is not the cause of the problem with the supervised fine-tuning model.

[0174] like Figure 5 As shown in (d), the second relevant indicator of the training link corresponding to the pre-trained model displayed by the platform through the interface is Top K Next Token(s), which is used to represent the first K characters or words and their probabilities that may appear in the next position of the specified position in the third execution result output by the pre-trained model based on the ICL reference prompt word. By observing the data information in this relevant indicator, it can be seen that the probability that the third execution result is "Liu Dahua." is low as a whole. In other words, it can be deduced that the third execution result output by the pre-trained model is not "Liu Dahua.". When the platform displays the above indicator information, it indicates that the pre-trained model outputs an erroneous third execution result, there is a problem with the pre-trained model, and the pre-trained model is a problematic model. On this basis, the platform determines the third training data whose semantics match the semantics of the text to be processed from the third training data set. For details, please refer to the third relevant indicator of the training link corresponding to the pre-trained model displayed by the platform through the interface: Training Dataset. According to Figure 5As shown in (d), the platform determines the first N corpus paragraphs (Corpus Paragraph) with the most similar semantics to the text to be processed from the third training data set based on the keywords, i.e., based on the key information (key_info) Liu Xiaohua, Brother, Zhu Xiaohua, i.e., the third training data. Among them, the embedding similarity (Embedding Similarity) refers to the similarity between the semantics of the third training data and the text to be processed. According to Figure 5 It can be seen from the third training data shown in (d) that none of the third training data describes the correct information that Liu Xiaohua's brother is Liu Dahua. On the contrary, based on the description of the relationship between Liu Xiaohua and Zhu Xiaohua in the third training data, the pre-trained model learns the wrong information that Liu Xiaohua's brother is Zhu Xiaohua. When the platform displays the above indicator information, it indicates that the third training data in the third training data set of the pre-trained model is wrong. Therefore, it can be quickly and accurately known that it is due to the error in the third training data that the pre-trained model obtains wrong information, causing the pre-trained model to output an erroneous third execution result, causing the pre-trained model to become a problem model, and then causing the target language model to output an erroneous first execution result.

[0175] In this embodiment, in the process of tracing problems in the training process of the target language model, when the pre-trained model is determined to be a problem model, the third training data whose semantics matches the semantics of the text to be processed is determined from the third training data set obtained by training the pre-trained model. Furthermore, by determining that the third training data is wrong, it is determined that the error in the third training data is the reason why the pre-trained model becomes a problem model, and the reason why the target language model outputs an erroneous first execution result. Through the problem locating method in this embodiment, it is achieved to quickly and accurately locate the target cause of the result that causes the target language model to output an erroneous result. Furthermore, it is conducive to quickly and accurately eliminating the target cause, and it is conducive to reducing the performance problems of each model caused by the training data error. At the same time, by determining the third training data whose semantics matches the semantics of the text to be processed, an automatic and rapid search for training data is achieved. When the third training data is displayed through the platform, the visualization of the training data is also achieved, which is convenient for users to observe and process the training data.

[0176] As an optional implementation, the problem location method is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training the basic language model, a supervised fine-tuning model is obtained by fine-tuning the pre-trained model, the supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, a reward model is obtained based on the supervised fine-tuning model, the reward model is used to evaluate the quality of the results output by the language model according to preset standards, and the target language model is obtained by reinforcing the supervised fine-tuning model based on the reward model. Figure 7 As shown, the problem location method includes the following steps 701 to 717.

[0177] 701. Get the text to be processed.

[0178] 702. Input the text to be processed into the target language model so that the target language model performs a target task on the text to be processed and obtains a first execution result. The target task is a natural language processing task.

[0179] 703. When it is determined that the first execution result is wrong, the text to be processed and the first execution result are input into the reward model to obtain a first score output by the reward model. The first score represents the quality of the result output by the target language model evaluated according to a preset standard. The reward model is obtained by training the adjusted model using the first training data set after adjusting the model structure of the supervised fine-tuning model to obtain an adjusted model.

[0180] 704. Input the to-be-processed text and the correct execution result of the target task into the reward model to obtain a second score output by the reward model. The second score represents the quality of the correct execution result evaluated according to a preset standard.

[0181] It should be noted that the problem locating device chooses to execute the following steps 705 to 708, or execute steps 709 to 717 based on the magnitude relationship between the first score and the second score.

[0182] 705. When the first score is greater than the second score, determine that the reward model is a problem model. A problem model is a model whose output result is an error when the input is the text to be processed.

[0183] 706 . Determine, from the first training data set, first training data whose semantics match the semantics of the text to be processed and a label of the first training data.

[0184] 707. Determine, according to the label of the first training data, a ranking result of the n labeling results corresponding to the first training data.

[0185] 708. When the sorting result is wrong, the target cause is determined to be that the reward model is obtained by training the adjustment model using the first training data and the label of the first training data, resulting in the reward model obtaining wrong information.

[0186] Or 709, when the first score is less than the second score, input the text to be processed into the supervised fine-tuning model to obtain a second execution result output by the supervised fine-tuning model.

[0187] 710. When the second execution result is erroneous, determine that the supervised fine-tuning model is a problem model.

[0188] 711. Determine second training data whose semantics match the semantics of the text to be processed and a label of the second training data from the second training data set. The second training data set is a training data set used by a supervised fine-tuning model obtained by performing supervised fine-tuning on a pre-trained model.

[0189] It should be noted that the problem locating device chooses to execute the following step 712 or execute steps 713 to 717 based on whether the label of the second training data is wrong or correct.

[0190] 712. When the label of the second training data is wrong, determine that the target cause is that the label of the second training data is wrong, resulting in the supervised fine-tuning model obtaining wrong information through supervised fine-tuning.

[0191] Or 713, if the label of the second training data is correct, obtain reference prompt words. The reference prompt words include reference examples and text that describes the target task and the content to be determined according to a preset sentence pattern. The reference example is an example that describes the reference task and the reference execution result of the reference task according to the preset sentence pattern. The position of the target task in the preset sentence pattern is the same as the position of the reference task in the preset sentence pattern. The position of the content to be determined in the preset sentence pattern is the same as the position of the reference execution result in the preset sentence pattern. The reference example is used to guide the pre-trained model to determine the content to be determined.

[0192] 714. Input the reference prompt word into the pre-training model, so that the pre-training model determines the content to be determined under the guidance of the reference example, as a third execution result output by the pre-training model.

[0193] 715. When the third execution result is erroneous, determine that the pre-trained model is a problem model.

[0194] 716. Determine from the third training data set third training data whose semantics match the semantics of the text to be processed. The third training data set is a training data set used by the pre-training model obtained by pre-training the basic language model.

[0195] 717. In the case where the third training data is wrong, determining that the target cause is that the third training data is wrong, causing the pre-training model to obtain wrong information through pre-training.

[0196] The problem location method in this embodiment can quickly and accurately locate the target cause of the target language model outputting erroneous results. The problem location method in this embodiment embodies the calculation logic of the full-link attribution platform discussed in the previous embodiment. When the Bad Case is input into the platform, the platform traces the problem in the training process of the target language model, and displays the relevant indicators in the training links corresponding to the target language model, reward model, supervised fine-tuning model, and pre-training model in sequence through the interface. Figure 3 and Figure 5 . When the platform displays the relevant indicators in each training link, users can quickly, accurately and clearly know the corresponding performance of the model in each training link by consulting the indicator information displayed on the platform. In addition, the target cause that causes the target language model to output incorrect results can be quickly, accurately and clearly determined, and the target cause can be eliminated in a timely and rapid manner, as well as the model that is affected by the target cause and has problems can be repaired and iteratively optimized. This is conducive to saving time and communication costs in finding the target cause.

[0197] Other technical effects that can be achieved by this embodiment can refer to the technical effects that can be achieved by any of the aforementioned problem locating method embodiments, and will not be repeated in this embodiment.

[0198] The embodiment of the present application also provides another problem location method, which is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training the basic language model, a supervised fine-tuning model is obtained by supervising the pre-trained model, the supervised fine-tuning model is used to process natural language and generate results that meet preset expectations, a reward model is obtained based on the supervised fine-tuning model, the reward model is used to evaluate the quality of the results output by the language model according to preset standards, and the target language model is obtained by reinforcing the supervised fine-tuning model based on the reward model. Figure 8 As shown, the problem location method includes the following steps 801 to 804.

[0199] 801. Get the target problem.

[0200] 802. Input the target question into the target language model, and obtain a first answer to the target question output by the target language model.

[0201] Among them, when the text to be processed input to the target language model is a specific target question, the target language model performs the target task on the text to be processed, and the first execution result obtained is the first answer to the target question.

[0202] 803. When it is determined that the first answer is wrong, a problem model is determined from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target problem. The problem model is a model whose output result is an error when the input is the target problem.

[0203] 804. Determine the target cause based on the problem model.

[0204] The problem localization method provided in the embodiment of the present application is based on the method described in any of the aforementioned problem localization method embodiments, and limits the text to be processed to a specific target problem, and then locates the target cause of the target language model outputting an erroneous result when the first answer output by the target language model based on the target problem is wrong. Based on this, the specific description of each step of the problem localization method in the embodiment of the present application, feasible examples, and achievable technical effects can all refer to the corresponding discussion of the method described in any of the aforementioned problem localization method embodiments. It will not be repeated in the embodiment of the present application.

[0205] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0206] like Fig. 9As shown, the embodiment of the present application also provides a problem location device 900, which is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training the basic language model, a supervised fine-tuning model is obtained by supervising the pre-trained model, the supervised fine-tuning model is used to process the natural language and generate a result that meets the preset expectations, a reward model is obtained based on the supervised fine-tuning model, the reward model is used to evaluate the quality of the result output by the language model according to the preset standard, and the supervised fine-tuning model is reinforced based on the reward model to obtain the target language model. The problem location device 900 includes a text acquisition module 901, a result generation module 902, a model determination module 903 and a cause determination module 904. The text acquisition module 901 is used to acquire the text to be processed. The result generation module 902 is used to input the text to be processed into the target language model so that the target language model performs the target task on the text to be processed and obtains a first execution result, and the target task is a natural language processing task. The model determination module 903 is used to determine the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed when the first execution result is determined to be wrong. The problem model is a model whose output result is wrong when the input is the text to be processed. The cause determination module 904 is used to determine the target cause according to the problem model.

[0207] In combination with any embodiment of the present application, the above-mentioned model determination module 903 is also used to determine the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model in a backtracking manner using the text to be processed when it is determined that the first execution result is wrong; wherein, backtracking refers to the process of obtaining the target language model based on training, generating the pre-trained model, the supervised fine-tuning model, and the reward model in sequence, and tracing back from the back to the front.

[0208] In combination with any embodiment of the present application, the above-mentioned model determination module 903 is also used to input the text to be processed and the first execution result into the reward model to obtain a first score output by the reward model when it is determined that the first execution result is wrong, and the first score represents the quality of the result output by the target language model evaluated according to the preset standard; input the text to be processed and the correct execution result of the target task into the reward model to obtain a second score output by the reward model, and the second score represents the quality of the correct execution result evaluated according to the preset standard; when the first score is greater than the second score, determine that the reward model is a problem model.

[0209] In combination with any embodiment of the present application, the reward model is obtained by training the adjusted model using the first training data set after adjusting the model structure of the supervised fine-tuning model to obtain the adjusted model. The above-mentioned model determination module 903 is also used to determine the first training data whose semantics match the semantics of the text to be processed and the label of the first training data from the first training data set; according to the label of the first training data, determine the sorting result of the n annotation results corresponding to the first training data. The above-mentioned cause determination module 904 is also used to determine that the target cause is that the reward model is obtained by training the adjustment model using the first training data and the label of the first training data, resulting in the reward model obtaining wrong information when the sorting result is wrong.

[0210] In combination with any embodiment of the present application, the above-mentioned model determination module 903 is also used to input the text to be processed into the supervised fine-tuning model when the first score is less than the second score, and obtain a second execution result output by the supervised fine-tuning model; when the second execution result is wrong, determine that the supervised fine-tuning model is a problem model.

[0211] In combination with any embodiment of the present application, the above-mentioned model determination module 903 is also used to determine the second training data whose semantics match the semantics of the text to be processed and the label of the second training data from the second training data set, and the second training data set is a training data set used by the supervised fine-tuning model obtained by fine-tuning the pre-trained model. The above-mentioned cause determination module 904 is also used to determine that the target cause is that the label of the second training data is wrong, resulting in the supervised fine-tuning of the supervised fine-tuning model to obtain wrong information.

[0212] In combination with any embodiment of the present application, the above-mentioned model determination module 903 is also used to obtain reference prompt words when the label of the second training data is correct. The reference prompt words include a reference example and a text that describes the target task and the content to be determined according to a preset sentence pattern. The reference example is an example of describing the reference task and the reference execution result of the reference task according to the preset sentence pattern. The position of the target task in the preset sentence pattern is the same as the position of the reference task in the preset sentence pattern, and the position of the content to be determined in the preset sentence pattern is the same as the position of the reference execution result in the preset sentence pattern. The reference example is used to guide the pre-trained model to determine the content to be determined; the reference prompt words are input into the pre-trained model so that the pre-trained model determines the content to be determined under the guidance of the reference example as the third execution result output by the pre-trained model; when the third execution result is wrong, the pre-trained model is determined to be a problem model.

[0213] In conjunction with any embodiment of the present application, the model determination module 903 is further used to determine from a third training data set third training data whose semantics match the semantics of the text to be processed, the third training data set being a training data set used by a pre-trained model obtained by pre-training a basic language model. The cause determination module 904 is further used to, in the case where the third training data is erroneous, determine that the target cause is the erroneous third training data causing the pre-trained model to obtain erroneous information through pre-training.

[0214] For the explanation of each term, please refer to the relevant description in the aforementioned problem location method embodiment, which will not be elaborated here.

[0215] It should be noted that the specific execution process of the above-mentioned problem locating device 900 can be found in Figures 1 to 7 The specific description of the illustrated embodiment will not be repeated here.

[0216] Each module in the problem location device 900 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0217] like Fig.10 As shown, the embodiment of the present application also provides another problem location device 1000, which is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training the basic language model, and a supervised fine-tuning model is obtained by supervising the pre-trained model, and the supervised fine-tuning model is used to process the natural language and generate a result that meets the preset expectations, and a reward model is obtained based on the supervised fine-tuning model. The reward model is used to evaluate the quality of the result output by the language model according to the preset standard, and the supervised fine-tuning model is reinforced based on the reward model to obtain the target language model. The problem location device 1000 includes a problem acquisition module 1001, an answer generation module 1002, a model location module 1003 and a cause location module 1004. The problem acquisition module 1001 is used to obtain the target problem. The answer generation module 1002 is used to input the target problem into the target language model and obtain the first answer to the target problem output by the target language model. The model localization module 1003 is used to determine the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target question when the first answer is determined to be wrong. The problem model is a model whose output result is wrong when the input is the target question. The cause localization module 1004 is used to determine the target cause based on the problem model.

[0218] For the explanation of each term, please refer to the relevant description in the aforementioned problem location method embodiment, which will not be elaborated here.

[0219] It should be noted that the specific execution process of the above-mentioned problem location device 1000 can be found in Figure 8 The specific description of the illustrated embodiment will not be repeated here.

[0220] Each module in the problem location device 1000 can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0221] like Fig.11 As shown, the embodiment of the present application further provides a computer device 1100. Exemplarily, the computer device 1100 may include a processor 1101, a communication interface 1102, a communication bus 1103, and a memory 1104. Specifically, the computer device 1100 may include:

[0222] At least one processor 1101, such as a CPU, at least one communication interface 1102, a memory 1104, and at least one communication bus 1103. The communication bus 1103 is used to realize the connection and communication between these components. The communication interface 1102 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface or a Bluetooth interface, etc.). The memory 1104 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 1104 may optionally be at least one storage device located away from the aforementioned processor 1101. Fig.11 As shown, the memory 1104 as a computer storage medium may include an operating system and program instructions.

[0223] Exemplarily, the processor 1101 may be used to implement the above Fig. 9 The steps or methods executed by the text acquisition module 901, the result generation module 902, the model determination module 903 and the cause determination module 904 in the embodiment.

[0224] It is understandable that the above method is only an example, and the processor 1101 and other modules in the above computer device 1100 may also cooperate to perform the above method. Fig. 9 The steps or methods executed by the text acquisition module 901, the result generation module 902, the model determination module 903 and the cause determination module 904 are not limited in this document.

[0225] exist Fig.11 In the computer device 1100 shown, the processor 1101 may be used to load program instructions stored in the memory 1104 and specifically perform the following operations:

[0226] Get the text to be processed;

[0227] Inputting the text to be processed into the target language model so that the target language model performs a target task on the text to be processed, and obtaining a first execution result, wherein the target task is a natural language processing task;

[0228] In the case where it is determined that the first execution result is wrong, determining a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed, where the problem model is a model whose output result is an error when the input is the text to be processed;

[0229] Based on the problem model, determine the target cause.

[0230] For the explanation of each term, please refer to the relevant description in the aforementioned problem location method embodiment, which will not be elaborated here.

[0231] It should be noted that the specific implementation process can be found in Figures 1 to 7 The specific description of the illustrated embodiment will not be repeated here.

[0232] Exemplarily, the processor 1101 may be used to implement the above Fig.10 The steps or methods executed by the question acquisition module 1001, the answer generation module 1002, the model positioning module 1003 and the cause positioning module 1004.

[0233] It is understandable that the above method is only an example, and the processor 1101 and other modules in the above computer device 1100 may also cooperate to perform the above method. Fig.10 The steps or methods performed by the question acquisition module 1001, the answer generation module 1002, the model positioning module 1003 and the cause positioning module 1004 are not limited in this document.

[0234] exist Fig.11 In the computer device 1100 shown, the processor 1101 may be used to load program instructions stored in the memory 1104 and specifically perform the following operations:

[0235] Get the target question;

[0236] Input the target question into the target language model, and obtain the first answer to the target question output by the target language model;

[0237] When it is determined that the first answer is wrong, a problem model is determined from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target problem. The problem model is a model whose output result is an error when the input is the target problem.

[0238] Based on the problem model, determine the target cause.

[0239] For the explanation of each term, please refer to the relevant description in the aforementioned problem location method embodiment, which will not be elaborated here.

[0240] It should be noted that the specific implementation process can be found in Figure 8 The specific description of the illustrated embodiment will not be repeated here.

[0241] Understandably, Fig.11 Only a simplified design of a computer device is shown. In practical applications, the computer device may also include other necessary components, including but not limited to any number of sending devices, input / output devices, processors, memories, etc., and all computer devices that can implement the embodiments of the present application are within the protection scope of the present application.

[0242] The present application also provides a computer-readable storage medium, which can store a plurality of instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figures 1 to 8 The method steps of the embodiment shown in the figure can be found in the specific execution process. Figures 1 to 8 The specific description of the illustrated embodiment will not be repeated here.

[0243] As used in the above embodiments, the term "when..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrases "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0244] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any fusion thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a server or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integration. The available medium can be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk), etc.

[0245] Those skilled in the art can understand that to implement all or part of the processes in the above-mentioned embodiments, the processes can be completed by computer programs to instruct related hardware, and the programs can be stored in computer-readable storage media. When the programs are executed, they can include the processes of the above-mentioned method embodiments. The aforementioned storage media include: ROM or random access memory RAM, magnetic disk or optical disk and other media that can store program codes.

[0246] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0247] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A problem location method, characterized in that: The method is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: obtaining a pre-trained model by pre-training a basic language model, performing supervised fine-tuning on the pre-trained model to obtain a supervised fine-tuning model, the supervised fine-tuning model is used to process natural language and generate a result that meets preset expectations, obtaining a reward model based on the supervised fine-tuning model, the reward model is used to evaluate the quality of the result output by the language model according to a preset standard, and performing reinforcement learning on the supervised fine-tuning model based on the reward model to obtain the target language model. The method includes: Get the text to be processed; Inputting the text to be processed into the target language model so that the target language model performs a target task on the text to be processed to obtain a first execution result, wherein the target task is a natural language processing task; In the case where it is determined that the first execution result is wrong, determining a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed, wherein the problem model is a model whose output result is wrong when the input is the text to be processed; According to the problem model, the target cause is determined.

2. The method according to claim 1, characterized in that: In the case where it is determined that the first execution result is wrong, determining a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed, comprises: In the case where it is determined that the first execution result is wrong, determining the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model using the to-be-processed text in a backtracking manner; The backtracking refers to the sequence of generating the pre-training model, the supervised fine-tuning model, and the reward model in the process of obtaining the target language model based on training, and tracing back from the back to the front.

3. The method according to claim 2, characterized in that In the case where it is determined that the first execution result is wrong, determining the problem model from the pre-trained model, the supervised fine-tuning model, and the reward model by backtracking using the to-be-processed text, comprises: In the case where it is determined that the first execution result is wrong, inputting the to-be-processed text and the first execution result into the reward model to obtain a first score output by the reward model, wherein the first score represents the quality of the result output by the target language model evaluated according to the preset standard; Inputting the to-be-processed text and the correct execution result of the target task into the reward model to obtain a second score output by the reward model, wherein the second score represents the quality of the correct execution result evaluated according to the preset standard; When the first score is greater than the second score, the reward model is determined to be the problem model.

4. The method according to claim 3, characterized in that The reward model is obtained by training the adjusted model using the first training data set after adjusting the model structure of the supervised fine-tuning model to obtain the adjusted model; After determining that the reward model is the problem model, the method further includes: Determine, from the first training data set, first training data whose semantics match the semantics of the text to be processed and a label of the first training data; Determine, according to the label of the first training data, a ranking result of n labeling results corresponding to the first training data; Determining the target cause according to the problem model includes: In the case where the sorting result is erroneous, the target cause is determined to be that the reward model is obtained by training the adjustment model using the first training data and the label of the first training data, resulting in the reward model obtaining erroneous information.

5. The method according to claim 3 or 4, characterized in that: After obtaining the first score and the second score, the method further includes: When the first score is less than the second score, inputting the to-be-processed text into the supervised fine-tuning model to obtain a second execution result output by the supervised fine-tuning model; In case the second execution result is erroneous, the supervised fine-tuning model is determined to be the problem model.

6. The method according to claim 5, characterized in that After determining that the supervised fine-tuning model is the problem model, the method further includes: Determining second training data whose semantics match the semantics of the text to be processed and a label of the second training data from a second training data set, wherein the second training data set is a training data set used by the supervised fine-tuning model obtained by performing supervised fine-tuning on the pre-training model; Determining the target cause according to the problem model includes: In the case where the label of the second training data is wrong, determining the target cause is that the label of the second training data is wrong, resulting in the supervised fine-tuning model obtaining wrong information through supervised fine-tuning.

7. The method according to claim 6, characterized in that After determining, from the second training data set, second training data whose semantics match the semantics of the text to be processed and a label of the second training data, the method further includes: When the label of the second training data is correct, obtaining reference prompt words, the reference prompt words including reference examples and text describing the target task and the content to be determined according to a preset sentence pattern, the reference example is an example describing the reference task and the reference execution result of the reference task according to the preset sentence pattern, the position of the target task in the preset sentence pattern is the same as the position of the reference task in the preset sentence pattern, the position of the content to be determined in the preset sentence pattern is the same as the position of the reference execution result in the preset sentence pattern, and the reference example is used to guide the pre-trained model to determine the content to be determined; Inputting the reference prompt word into the pre-training model, so that the pre-training model determines the content to be determined under the guidance of the reference example, as a third execution result output by the pre-training model; In case the third execution result is erroneous, the pre-trained model is determined to be the problem model.

8. The method according to claim 7, characterized in that After determining that the pre-trained model is the problem model, the method further includes: Determining, from a third training data set, third training data whose semantics match the semantics of the text to be processed, wherein the third training data set is a training data set used by the pre-training model obtained by pre-training a basic language model; Determining the target cause according to the problem model includes: In the case where the third training data is erroneous, it is determined that the target cause is that the third training data is erroneous, causing the pre-training model to obtain erroneous information through pre-training.

9. A problem location method, characterized in that: The method is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: obtaining a pre-trained model by pre-training a basic language model, performing supervised fine-tuning on the pre-trained model to obtain a supervised fine-tuning model, the supervised fine-tuning model is used to process natural language and generate a result that meets preset expectations, obtaining a reward model based on the supervised fine-tuning model, the reward model is used to evaluate the quality of the result output by the language model according to a preset standard, and performing reinforcement learning on the supervised fine-tuning model based on the reward model to obtain the target language model. The method includes: Get the target question; Inputting the target question into the target language model, and obtaining a first answer to the target question output by the target language model; In the case where it is determined that the first answer is wrong, determining a question model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target question, wherein the question model is a model whose output result is wrong when the input is the target question; According to the problem model, the target cause is determined.

10. A problem location device, characterized in that: The device is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training a basic language model, and a supervised fine-tuning model is obtained by supervised fine-tuning the pre-trained model, wherein the supervised fine-tuning model is used to process natural language and generate a result that meets preset expectations, a reward model is obtained based on the supervised fine-tuning model, and the reward model is used to evaluate the quality of the result output by the language model according to a preset standard, and reinforcement learning is performed on the supervised fine-tuning model based on the reward model to obtain the target language model. The device includes: A text acquisition module is used to acquire the text to be processed; A result generation module, used for inputting the text to be processed into the target language model, so that the target language model performs a target task on the text to be processed to obtain a first execution result, wherein the target task is a natural language processing task; A model determination module, configured to determine a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the text to be processed when it is determined that the first execution result is wrong, wherein the problem model is a model whose output result is wrong when the input is the text to be processed; The cause determination module is used to determine the target cause according to the problem model.

11. A problem location device, characterized in that: The device is used to determine the target cause of the target language model outputting an erroneous result from the training process when the target language model is trained through the following training process: a pre-trained model is obtained by pre-training a basic language model, and a supervised fine-tuning model is obtained by supervised fine-tuning the pre-trained model, wherein the supervised fine-tuning model is used to process natural language and generate a result that meets preset expectations, a reward model is obtained based on the supervised fine-tuning model, and the reward model is used to evaluate the quality of the result output by the language model according to a preset standard, and reinforcement learning is performed on the supervised fine-tuning model based on the reward model to obtain the target language model. The device includes: A question acquisition module is used to obtain the target question; an answer generation module, configured to input the target question into the target language model and obtain a first answer to the target question output by the target language model; A model positioning module, configured to determine a problem model from the pre-trained model, the supervised fine-tuning model, and the reward model based on the target question when it is determined that the first answer is wrong, wherein the problem model is a model whose output result is wrong when the input is the target question; The cause location module is used to determine the target cause according to the problem model.

12. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions; when the program instructions are executed by the processor, the computer device executes the method according to any one of claims 1 to 8, or executes the method according to claim 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program; when the computer program runs on one or more processors, the processor is caused to execute the method according to any one of claims 1 to 8, or execute the method according to claim 9.

14. A computer program product, characterized in that The computer program product comprises a computer program or instructions; when the computer program or instructions are run on a computer, the computer is caused to execute the method according to any one of claims 1 to 8, or execute the method according to claim 9.

Images