Task execution method, device, storage medium and electronic device

Abstract

This specification discloses a task execution method, device, storage medium and electronic device. The server can redefine the question text input by the user and the original answer text output by the large language model into propositions after logical conversion, and use the feedback between multiple intelligent agents as the observation state to verify the propositions after logical conversion. The verification result can then be used as a prompt statement for the next round of task execution by the large language model, and then spliced ​​with the question text and input into the large language model again to improve the accuracy of the reply text output again by the large language model.

Description

Technical Field

[0001] The present invention relates to the field of large model technology, and in particular to a task execution method, device, storage medium and electronic device. Background Art

[0002] With the rapid development of artificial intelligence technology, generative large language models (LLMs) are increasingly used in vertical fields, especially in question-answering systems (QA) and dialogue tasks, showing great potential. These large language models can learn and generate human language-style texts from massive data through deep learning technology, thus showing extremely high application value in content generation, knowledge processing and refinement in professional fields.

[0003] However, when performing dialogue tasks through a large language model, the content generated by the large language model may be inaccurate, untrue, conflicting with input instructions, or contrary to known facts, which may cause users to obtain incorrect information and mislead their decisions and judgments, thereby reducing the user's experience.

[0004] Therefore, how to improve the accuracy of content generated by large language models is an urgent problem to be solved. Summary of the invention

[0005] This specification provides a task execution method, device, storage medium and electronic device to partially solve the above-mentioned problems existing in the prior art.

[0006] This manual adopts the following technical solutions:

[0007] This specification provides a task execution method, including:

[0008] Get the question text entered by the user;

[0009] Inputting the question text into a predetermined target large language model, so as to generate an answer text as an original answer text according to the question text through the target large language model;

[0010] Recombining the question text and the original answer text to generate a logical conversion text corresponding to the question text, and determining an authenticity verification result of the logical conversion text, wherein the logical conversion text is a text obtained by logically converting a proposition text composed of the question text and the original answer text, and the logical conversion text is logically associated with the proposition text;

[0011] The question text and the authenticity verification result are re-input into the target large language model to obtain a target answer text corresponding to the question text through the target large language model, and the task is executed according to the target answer text.

[0012] Optionally, determining a target large language model specifically includes:

[0013] Obtain a predetermined model interaction relationship graph, in which different preset large language models correspond to different nodes, and for each node, an edge between the node and other nodes is used to represent a data transmission relationship between the large language model corresponding to the node and the large language models corresponding to other nodes;

[0014] According to the model interaction relationship diagram, a target large language model is determined.

[0015] Optionally, determining a target large language model according to the model interaction relationship diagram specifically includes:

[0016] For each preset large language model, according to the model interaction relationship diagram, the number of other large language models having data transmission relationships with the large language model is determined, and the importance of the large language model is determined according to the number of other large language models having data transmission relationships with the large language model, wherein the number of other large language models having data transmission relationships with the large language model is positively correlated with the importance of the large language model;

[0017] According to the importance of each large language model, a target large language model is determined from the large language models.

[0018] Optionally, determine a model interaction diagram, specifically including:

[0019] Obtaining an initial model interaction relationship diagram and performance data of each preset large language model, wherein for each large language model, the performance data of the large language model includes: performance data for reflecting a network environment of a device on which the large language model is deployed, and performance data for reflecting task execution efficiency and task execution effect of the large language model;

[0020] Determine the data transmission relationship between the major language models based on the performance data of each major language model;

[0021] According to the data transmission relationship, the initial model interaction relationship diagram is adjusted to determine the model interaction relationship diagram.

[0022] Optionally, the question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and the authenticity verification result of the logical conversion text is determined, specifically including:

[0023] In the case where it is determined that the proposition text composed of the question text and the original answer text is a compound proposition text, splitting the proposition text composed of the question text and the original answer text into sub-proposition texts;

[0024] For each sub-proposition text, the sub-proposition text is reorganized to generate a basic logic conversion text corresponding to the sub-proposition text, and a basic authenticity verification result of the basic logic conversion text corresponding to the sub-proposition text is determined.

[0025] Optionally, determining the authenticity verification result of the logical conversion text specifically includes:

[0026] According to the model interaction relationship diagram, determining other large language models that have a data transmission relationship with the target large language model as verification large language models;

[0027] The logic conversion text is input into the verification large language model to obtain the authenticity verification result of the logic conversion text through the verification large language model.

[0028] Optionally, the question text is input into a predetermined target large language model, so as to generate an answer text as an original answer text according to the question text through the target large language model, specifically including:

[0029] In the case where there are multiple predetermined target large language models, for each predetermined target large language model, the question text is input into the target large language model, so as to generate an answer text according to the question text through the target large language model as the original answer text;

[0030] The question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and the authenticity verification result of the logical conversion text is determined, specifically including:

[0031] Determining the alignment of each target large language model with respect to the question text according to the difference between the original answer texts output by each target large language model;

[0032] Determining alignment results of each target large language model according to the alignment degree;

[0033] If it is determined based on the alignment result that the target large language models are not aligned, the question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and an authenticity verification result of the logical conversion text is determined.

[0034] This specification provides a task execution device, including:

[0035] The acquisition module is used to obtain the question text input by the user;

[0036] A generating module, used for inputting the question text into a predetermined target large language model, so as to generate an answer text as an original answer text according to the question text through the target large language model;

[0037] A verification module, used to reorganize the question text and the original answer text, generate a logical conversion text corresponding to the question text, and determine the authenticity verification result of the logical conversion text, wherein the logical conversion text is a text obtained by logically converting a proposition text composed of the question text and the original answer text, and the logical conversion text is logically associated with the proposition text;

[0038] An execution module is used to re-input the question text and the authenticity verification result into the target large language model to obtain a target answer text corresponding to the question text through the target large language model, and perform a task according to the target answer text.

[0039] At least one of the above technical solutions adopted in this specification can achieve the following beneficial effects:

[0040] In the task execution method provided in this specification, the question text input by the user is first obtained, and the question text is input into a predetermined target large language model, so that the target large language model can generate an answer text according to the question text as the original answer text, and the question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and the authenticity verification result of the logical conversion text is determined. The logical conversion text is the text after the proposition text composed of the question text and the original answer text is logically converted, and the logical conversion text is logically associated with the proposition text. The question text and the authenticity verification result are re-input into the target large language model to obtain the target answer text corresponding to the question text through the target large language model, and the task is executed according to the target answer text.

[0041] It can be seen from the above method that the server can redefine the question text input by the user and the original answer text output by the large language model into propositions after logical transformation, verify the propositions after logical transformation, and then use the verification result as the prompt statement for the next round of task execution by the large language model, spliced ​​with the question text and input into the large language model again, so as to improve the accuracy of the reply text output again by the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0043] Figure 1 A flowchart of a task execution method provided in this specification;

[0044] Figure 2 A schematic diagram of test results of different large language models provided in this specification;

[0045] Figure 3 A schematic diagram of the model interaction relationship diagram provided in this specification;

[0046] Figure 4 A schematic diagram of the task execution process provided in this manual;

[0047] Figure 5 It is an overall schematic diagram of the task execution process provided in this manual;

[0048] Figure 6 A schematic diagram of a task execution device provided in this specification;

[0049] Figure 7 A method corresponding to the Figure 1 Schematic diagram of electronic equipment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0051] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0052] Figure 1 A flowchart of a task execution method provided in this specification includes the following steps:

[0053] S101: Obtain the question text input by the user.

[0054] In this specification, the business platform can provide users with services such as information retrieval, knowledge acquisition, customer service, decision support, emotional communication, intelligent services and personalized recommendations. Prior to this, the business platform needs to first obtain the question text sent by the user to the business platform through the application provided by the business platform, and then generate an answer text based on the received question text, and display the answer text to the user and provide services to the user.

[0055] For example, in the knowledge acquisition business scenario, the business platform can receive question texts sent by users in the fields of education, medical care, scientific research, law, etc., so that it can help users quickly find the corresponding answer texts based on the obtained question texts, and then display the answer texts to users.

[0056] Another example: in the smart home service business scenario, the business platform can receive question texts sent by users regarding smart homes, and then, based on the obtained question texts, help users perform language control, information query and other functions, and generate corresponding answer texts, which can then be displayed to users.

[0057] Among them, the above-mentioned business platform can provide a variety of applications, such as: client application APP, Web application and the like.

[0058] In this specification, the execution subject used to implement the task execution method can refer to a designated device such as a server set up on a business platform, or it can refer to a terminal device such as a desktop computer or a laptop computer. For the sake of ease of description, the task execution method provided in this specification is explained below using the server as the execution subject as an example.

[0059] S102: Input the question text into a predetermined target large language model, so as to generate an answer text as an original answer text according to the question text through the target large language model.

[0060] In actual application scenarios, for each type of question text, the accuracy of the answer text output by different large language models for that type of question text is often different. For example Figure 2 shown.

[0061] Figure 2 A schematic diagram of the test results of different large language models provided in this specification.

[0062] Combination Figure 2 It can be seen that when a large language model performs dialogue tasks, the content generated by the large language model may be inaccurate, which may cause users to obtain wrong information and mislead their decisions and judgments.

[0063] Based on this, in this specification, after obtaining the question text input by the user, the server can input the obtained question text into a predetermined target large language model to generate an answer text as the original answer text based on the question text through the target large language model.

[0064] There are many methods for the server to determine the target large language model. For example, the server can determine the target large language model from the various language models based on the matching relationship between each preset large language model and each type of question text.

[0065] For another example: the server can obtain a predetermined model interaction relationship diagram, and then, for each preset large language model, determine the number of other large language models that have data transmission relationships with the large language model based on the model interaction relationship diagram, and determine the importance of the large language model based on the number of other large language models that have data transmission relationships with the large language model.

[0066] In the above-mentioned model interaction relationship diagram, different preset large language models correspond to different nodes. For each node, the edge between the node and other nodes is used to characterize the data transmission relationship between the large language model corresponding to the node and the large language models corresponding to other nodes. It should be noted that the edge here can be a unidirectional edge (i.e., a directed edge). If it is determined that the logical conversion text corresponding to a large language model needs to be transmitted to another large language model for authenticity verification, it can be considered that the large language model data depends on the large language model used to verify the authenticity of the logical conversion text corresponding to the large language model. At this time, there is an edge connecting the node corresponding to the large language model and the node corresponding to the large language model used to verify the authenticity of the logical conversion text corresponding to the large language model, and the direction of the edge is from the node corresponding to the large language model to the node corresponding to the large language model used to verify the authenticity of the logical conversion text corresponding to the large language model.

[0067] For each large language model, the number of other large language models that have a data transmission relationship with the large language model is positively correlated with the importance of the large language model.

[0068] S103: Reorganize the question text and the original answer text to generate a logical conversion text corresponding to the question text, and determine the authenticity verification result of the logical conversion text. The logical conversion text is a text obtained by logically converting a proposition text composed of the question text and the original answer text. The logical conversion text is logically associated with the proposition text.

[0069] In this specification, after obtaining the original answer text, the server can reorganize the question text entered by the user and the original answer text to obtain the logical conversion text corresponding to the question text entered by the user, and then can verify the authenticity of the logical conversion text corresponding to the question text entered by the user to obtain the authenticity verification result of the logical conversion text corresponding to the question text entered by the user.

[0070] Among them, the above-mentioned logical conversion text may refer to the text after the proposition text composed of the question text input by the user and the original answer text is logically converted, and the logical conversion text is logically associated with the proposition text composed of the question text input by the user and the original answer text.

[0071] The above-mentioned logical conversion text may refer to the contrapositive text corresponding to the proposition text composed of the question text input by the user and the original answer text. For example, if the question text input by the user is "What has the greatest impact on the directional stability of the aircraft?", and the corresponding answer text is "vertical tail", then it can be determined that the proposition text composed of the question text input by the user and the original answer text is "The vertical tail has the greatest impact on the directional stability of the aircraft", and the contrapositive text obtained after logical conversion is "If a certain part is not the vertical tail, then its impact on the directional stability of the aircraft is not the greatest".

[0072] Of course, the above-mentioned logical conversion text can also be directly a proposition text composed of the question text input by the user and the original answer text, or such as: the inverse proposition text corresponding to the proposition text composed of the question text input by the user and the original answer text, the negative proposition text corresponding to the proposition text composed of the question text input by the user and the original answer text, etc., which are texts that have a logical relationship with the proposition text composed of the question text input by the user and the original answer text. This manual will not list them one by one here.

[0073] It should be noted that in the above content, there are also multiple methods for the server to reorganize the question text and the original answer text input by the user to obtain the logical conversion text corresponding to the question text input by the user. For example, the server can determine the proposition text composed of the question text and the original answer text input by the user, and then perform semantic analysis on the proposition text composed of the question text and the original answer text input by the user to determine the various semantic keywords contained in the proposition text composed of the question text and the original answer text input by the user, and then reorganize the various semantic keywords according to the preset logical conversion template to obtain the logical conversion text corresponding to the question text.

[0074] For another example: the server may input the question text and original answer text input by the user into a preset logical conversion model to reorganize the question text and original answer text input by the user through the preset logical conversion model to generate a logical conversion text corresponding to the question text.

[0075] It is worth noting that in the above content, the method for the server to determine the authenticity verification result of the logical conversion text can be to input the logical conversion text into other large language models to verify the authenticity of the logical conversion text through other large language models and output the authenticity verification result of the logical conversion text.

[0076] Specifically, the method for the server to determine other large language models for authenticity verification of the above-mentioned logical conversion text can be, based on a predetermined model interaction relationship diagram, determining other large language models that have a data transmission relationship with the target large language model as verification large language models, and then the logical conversion text can be input into the verification large language model to obtain the authenticity verification result of the logical conversion text through the verification large language model.

[0077] Among them, the method for the server to determine the above-mentioned model interaction relationship diagram can be to obtain the initial model interaction relationship diagram and the performance data of each large language model that is preset, and then determine the data transmission relationship between the large language models based on the performance data of each large language model, and adjust the initial model interaction relationship diagram based on the data transmission relationship between the large language models to determine the model interaction relationship diagram.

[0078] For each large language model, the performance data of the large language model includes: performance data used to reflect the network environment of the device on which the large language model is deployed, and performance data used to reflect the task execution efficiency and task execution effect of the large language model (for example: the accuracy of the output results, the grammatical standardization of the output results, the timeliness of the output results, etc.).

[0079] For ease of understanding, the following combination Figure 3 The method for determining the above-mentioned model interaction relationship diagram is described in detail.

[0080] Figure 3 A schematic diagram of the model interaction relationship diagram provided in this manual.

[0081] Combination Figure 3 It can be seen that there can be multiple types of the above initial model interaction relationship diagrams. For example, when it is determined based on the performance data of each large language model that the network environment of each large language model is good, the task execution efficiency and task execution effect of each large language model are good, and the computing resources are abundant, the preset full model interaction relationship diagram (such as Figure 3As shown in (a) in the figure, as the initial model interaction diagram mentioned above, combined with Figure 3 As can be seen from (a) in the above-mentioned full model interaction relationship diagram, for each node, there are edges connecting the node with itself, and the node with each other node. In other words, if the model interaction relationship diagram used in the above-mentioned task execution method is the full model interaction relationship diagram here, the server can use the target large language model itself and each other large language model as the above-mentioned verification large language model to verify the authenticity of the logical conversion text corresponding to the question text.

[0082] For another example: if it is determined based on the performance data of each large language model that the quality of the network environment of one of the large language models is higher than that of other large language models, and that the task execution efficiency and task execution effect of one of the large language models are higher than those of other large language models, then the preset centralized model interaction relationship diagram (such as Figure 3 As shown in (b) in the figure, as the initial model interaction diagram mentioned above, combined with Figure 3 As can be seen from (b) in the above-mentioned centralized model interaction relationship diagram, the large language model whose network environment quality is higher than other large language models, and whose task execution efficiency and task execution effect are higher than other large language models can be used as the central large language model, and then the node corresponding to each other large language model can be connected to the node corresponding to the central large language model. In other words, if the model interaction relationship diagram used in the above-mentioned task execution method is the central model interaction relationship diagram here, the server can use the central large language model as the verification large language model of each large language model to verify the authenticity of the logical conversion text corresponding to the question text.

[0083] For another example: if the quality of the network environment of each large language model is determined to be similar based on the performance data of each large language model, and the task execution efficiency and task execution effect of each large language model are also similar, in order to avoid the situation where a single node is blocked due to excessive access, the preset decentralized model interaction relationship diagram (such as Figure 3 As shown in (c) in the figure, as the initial model interaction diagram mentioned above, combined with Figure 3It can be seen from (c) that in the above-mentioned decentralized model interaction relationship diagram, each node can be connected end to end. In other words, if the model interaction relationship diagram used in the above-mentioned task execution method is the decentralized model interaction relationship diagram here, for each large language model, the server can determine a large language model for authenticity verification of the logical conversion text corresponding to the large language model according to the decentralized model interaction relationship diagram, and the large language model will also serve as a verification large language model for other large language models.

[0084] From the above content, it can be seen that the server can determine the network environment in which each large language model is located and the task execution efficiency and task execution effect of each large language model based on the performance data of each large language model. Based on this, the server can determine an initial model interaction relationship diagram, and then for each large language model, according to the performance data of the large language model, determine the data transmission relationship between the large language model and other large language models, and adjust the initial model interaction relationship diagram according to the data transmission relationship to determine the model interaction relationship diagram.

[0085] It should be noted that the model interaction relationship diagram used by the server can be adjusted in real time, that is, at every specified time period, or before each verification large language model that needs to determine the target large language model based on the model interaction relationship diagram, the performance data of each large language model can be obtained, and the model interaction relationship diagram can be adjusted based on the performance data of each large language model.

[0086] For example, for each large language model, if it is determined based on the performance data of the large language model that the quality of the network environment in which the large language model is located has deteriorated, it can be determined that there is a data transmission relationship between the large language model and a smaller number of other large language models.

[0087] For another example: for each large language model, if it is determined based on the performance data of the large language model that the task execution efficiency of the large language model is high, it can be determined that there is a data transmission relationship between the large language model and a large number of other large language models.

[0088] For another example: for each large language model, if it is determined based on the performance data of the large language model that the task execution effect of the large language model is better (for example, the accuracy of the output result of the large language model is higher), it can be determined that there is a data transmission relationship between the large language model and more other large language models.

[0089] Of course, in the above content, the method for the server to determine the authenticity verification result of the logical conversion text can also be to determine the historical proposition text that matches the logical conversion text from the historical proposition texts saved in the preset expert knowledge base, and then determine the authenticity verification result corresponding to the logical conversion text based on the authenticity verification result corresponding to the historical proposition text that matches the logical conversion text.

[0090] In actual application scenarios, the question text input by the user may be used to represent more complex questions, resulting in the proposition text composed of the question text and the original answer text being a compound proposition text (that is, using logical conjunctions to connect multiple simple propositions to form a proposition text with a more complex structure).

[0091] For example: "If my order is delayed and I don't receive any notification, can I get compensation?" In this example, the compound proposition contains two conditions (the order is delayed and I didn't receive any notification) and a result (I can get compensation). This proposition will only be true if both conditions are true at the same time.

[0092] Another example is: "If the server load is too high, or the network traffic increases abnormally, it may be a DDoS attack." In this example, the compound proposition contains two conditions (the server load is too high, and the network traffic increases abnormally) and a result (it may be a DDoS attack). If at least one of the two conditions is true, the proposition will be true.

[0093] At this time, the server can split the proposition text composed of the question text and the original answer text into sub-proposition texts, reorganize each sub-proposition text, generate the basic logical conversion text corresponding to the sub-proposition text, and determine the basic authenticity verification result of the basic logical conversion text corresponding to the sub-proposition text.

[0094] Furthermore, the server can determine the truth table of the proposition text composed of the question text and the original answer text according to the basic authenticity verification result of the basic logic conversion text of each sub-proposition text corresponding to the question text, and then re-input the question text and the truth table of the proposition text composed of the question text and the original answer text into the target large language model to obtain the target answer text corresponding to the question text through the target large language model.

[0095] In actual application scenarios, in order to further improve the accuracy of the reply text generated by the target large language model, the target large language model determined above can also be multiple, specifically as follows: Figure 4 shown.

[0096] Figure 4A schematic diagram of the task execution process provided in this manual.

[0097] Combination Figure 4 It can be seen that when there are multiple predetermined target large language models, the server can input the question text into the target large language model for each predetermined target large language model, so as to generate an answer text as the original answer text according to the question text through the target large language model, and then determine the alignment of each target large language model with respect to the question text according to the differences between the original answer texts output by each target large language model, and determine the alignment results of each target large language model according to the alignment of each target large language model with respect to the question text.

[0098] Furthermore, if it is determined based on the alignment result that the target large language models are not aligned, the question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and the authenticity verification result of the logical conversion text is determined.

[0099] If the alignment between the target large language models is determined based on the alignment result, the server can determine the target answer text based on the original answer text output by each target large language model, and perform the task based on the target answer text.

[0100] In the above content, the greater the difference between the original answer texts output by each target large language model, the smaller the alignment of each target large language model with respect to the question text.

[0101] In the above content, if the alignment degree of each target large language model for the question text is higher than a preset threshold, an alignment result between the target large language models can be obtained; otherwise, an alignment result between the target large language models that is not aligned can be obtained.

[0102] In the above content, the method for the server to determine the target answer text based on the original answer text output by each target large language model can be to determine the target answer text from the original answer text output by each target large language model according to a predetermined weight of each target large language model, wherein for each target large language model, the higher the accuracy of the reply text historically output by the target large language model, the greater the weight of the target large language model.

[0103] S104: re-input the question text and the authenticity verification result into the target large language model to obtain a target answer text corresponding to the question text through the target large language model, and perform a task according to the target answer text.

[0104] In the present specification, the server can re-input the user's question text and at least one authenticity verification result obtained by the above method into the target large language model, and verify the answer text obtained by the target large language model by the above method again until the preset termination condition is met. At this time, the target answer text corresponding to the question text can be obtained by the target large language model, and the task can be executed according to the target answer text.

[0105] Among them, the above-mentioned termination condition can be set according to actual needs. For example, if the above-mentioned target large language model is a single one, it can be considered that the preset termination condition is met when it is determined that the number of iteration rounds reaches a preset threshold.

[0106] For another example, if there are multiple target large language models, when the alignment between the target large language models is determined based on the alignment results between the target large language models, it can be considered that the preset termination condition is met.

[0107] Furthermore, if there are multiple target large language models as mentioned above, if after determining that the iteration rounds have reached the preset threshold, based on the alignment results between the target large language models, it is still determined that the target large language models are not aligned, then the large language model with the highest accuracy of the answer text historically outputted from the preset large language models can be determined as the decision large language model, and then the question text can be input into the decision large language model, and the reply text output by the decision large language model can be used as the target reply text.

[0108] Of course, after determining that the number of iterations has reached a preset threshold, the server can also display the question text to the designated user based on the alignment results between the target large language models. If it is still determined that the target large language models are not aligned, the server can enable the designated user to enter a manual response text for the question text as the target response text.

[0109] From the above content, it can be seen that the server can effectively improve the accuracy of the answer text output by the large language model by redefining the question text into the proposition text after logical transformation and submitting it to other large language models for cross-validation, such as Figure 2 As shown in , the accuracy for test set 1 is improved to 90.00% and the accuracy for test set 2 is improved to 86.67%.

[0110] Furthermore, in order to facilitate the control and management of access to each large language model, the server can also call each large language model through multiple agents, thereby further improving the efficiency and security of task execution. Figure 5 shown.

[0111] Figure 5This is an overall schematic diagram of the task execution process provided in this manual.

[0112] Combination Figure 5 It can be seen that after the server obtains the question text input by the user, it can input the question text into the preset application front end, so that the application front end can determine each target large language model according to the model interaction relationship diagram, and then the question text can be input into each target intelligent agent corresponding to each target large language model, and then the question text can be preprocessed by the target intelligent agent to obtain the processed question text, and then the processed question text can be input into the target large language model to generate an answer text according to the question text through the target large language model as the original answer text.

[0113] Furthermore, each target agent can return each original answer text to the application front end, so that the application front end can determine the alignment of each target large language model with respect to the question text based on each original answer text, and obtain the alignment result. If it is determined based on the alignment result that the target large language models are not aligned, each target agent can reorganize the question text and the original answer text to generate a logical conversion text corresponding to the question text, and determine the verification agent where the verification large language model is located based on the model interaction relationship diagram, and then send a verification request to the verification agent, so that the verification agent returns the authenticity verification result based on the verification request.

[0114] Furthermore, the target intelligent agent can re-input the question text and the authenticity verification result into the target large language model to obtain the target answer text corresponding to the question text through the target large language model.

[0115] From the above content, it can be seen that the server can redefine the question text input by the user and the original answer text output by the large language model into propositions after logical conversion, and use the feedback between multiple agents as the observation state to verify the propositions after logical conversion. Then, the verification result can be used as the prompt statement for the next round of task execution by the large language model, and then spliced ​​with the question text and input into the large language model again to improve the accuracy of the reply text output by the large language model again. The loop is stopped when the predetermined number of times is reached, and the interaction relationship between agents is defined through a directed graph, thereby reducing communication overhead and context processing costs.

[0116] The above are one or more implementation task execution methods of this specification. Based on the same idea, this specification also provides corresponding task execution devices, such as Figure 6 shown.

[0117] Figure 6 A schematic diagram of a task execution device provided for this specification includes:

[0118] The acquisition module 601 is used to acquire the question text input by the user;

[0119] A generating module 602 is used to input the question text into a predetermined target large language model, so as to generate an answer text as an original answer text according to the question text through the target large language model;

[0120] Verification module 603, used for reorganizing the question text and the original answer text, generating a logical conversion text corresponding to the question text, and determining the authenticity verification result of the logical conversion text, wherein the logical conversion text is a text obtained by logically converting a proposition text composed of the question text and the original answer text, and the logical conversion text is logically associated with the proposition text;

[0121] The execution module 604 is used to re-input the question text and the authenticity verification result into the target large language model to obtain the target answer text corresponding to the question text through the target large language model, and perform the task according to the target answer text.

[0122] Optionally, the acquisition module 601 is specifically used to obtain a predetermined model interaction relationship diagram, in which different preset large language models correspond to different nodes, and for each node, the edge between the node and other nodes is used to characterize the data transmission relationship between the large language model corresponding to the node and the large language models corresponding to other nodes; according to the model interaction relationship diagram, the target large language model is determined.

[0123] Optionally, the acquisition module 601 is specifically configured to, for each preset large language model, determine, according to the model interaction relationship diagram, the number of other large language models that have a data transmission relationship with the large language model, and determine the importance of the large language model according to the number of other large language models that have a data transmission relationship with the large language model, wherein the number of other large language models that have a data transmission relationship with the large language model is positively correlated with the importance of the large language model; and determine a target large language model from the large language models according to the importance of each large language model.

[0124] Optionally, the acquisition module 601 is specifically used to obtain an initial model interaction relationship diagram and performance data of each preset large language model, wherein, for each large language model, the performance data of the large language model includes: performance data for reflecting the network environment of the device on which the large language model is deployed, and performance data for reflecting the task execution efficiency and task execution effect of the large language model; based on the performance data of each large language model, determining the data transmission relationship between the large language models; based on the data transmission relationship, adjusting the initial model interaction relationship diagram to determine the model interaction relationship diagram.

[0125] Optionally, the verification module 603 is specifically used to, when it is determined that the proposition text composed of the question text and the original answer text is a compound proposition text, split the proposition text composed of the question text and the original answer text into sub-proposition texts; for each sub-proposition text, reorganize the sub-proposition text to generate a basic logical conversion text corresponding to the sub-proposition text, and determine a basic authenticity verification result of the basic logical conversion text corresponding to the sub-proposition text.

[0126] Optionally, the verification module 603 is specifically used to determine, based on the model interaction relationship diagram, other large language models that have a data transmission relationship with the target large language model as verification large language models; input the logical conversion text into the verification large language model to obtain the authenticity verification result of the logical conversion text through the verification large language model.

[0127] Optionally, the generating module 602 is specifically configured to, when there are multiple predetermined target large language models, input the question text into each predetermined target large language model, so as to generate an answer text according to the question text through the target large language model as the original answer text;

[0128] The verification module 603 is specifically used to determine the alignment of each target large language model with respect to the question text according to the difference between the original answer texts output by each target large language model; determine the alignment results of each target large language model according to the alignment; if it is determined according to the alignment result that the target large language models are not aligned, then reorganize the question text and the original answer text to generate a logical conversion text corresponding to the question text, and determine the authenticity verification result of the logical conversion text.

[0129] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 Provides a task execution method.

[0130] This manual also provides Figure 7 The one shown corresponds to Figure 1 A schematic diagram of the electronic device. Figure 7 As mentioned above, at the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 Of course, in addition to the software implementation, this specification does not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0131] For the improvement of a technology, it can be clearly distinguished whether it is a hardware improvement (for example, improvement of the circuit structure of diodes, transistors, switches, etc.) or a software improvement (improvement of the method flow). However, with the development of technology, many improvements of the method flow today can be regarded as direct improvements of the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be implemented with a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming themselves, without having to ask chip manufacturers to design and make dedicated integrated circuit chips. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one kind of HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0132] The controller may be implemented in any suitable manner, for example, the controller may take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (e.g., software or firmware) executable by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320, and the memory controller may also be implemented as part of the control logic of the memory. It is also known to those skilled in the art that, in addition to implementing the controller in a purely computer-readable program code manner, the controller may be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, such a controller may be considered as a hardware component, and the devices for implementing various functions included therein may also be considered as structures within the hardware component. Or even, the devices for implementing various functions may be considered as both software modules for implementing the method and structures within the hardware component.

[0133] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0134] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0135] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0140] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0141] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0142] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0143] It should be understood by those skilled in the art that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0144] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0145] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0146] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A task execution method, characterized in that: include: Get the question text entered by the user; The question text is input into a predetermined target large language model, so as to generate an answer text as an original answer text according to the question text through the target large language model, wherein the target large language model is determined according to a predetermined model interaction relationship diagram, in which different preset large language models correspond to different nodes, and for each node, the edge between the node and other nodes is used to represent the data transmission relationship between the large language model corresponding to the node and the large language models corresponding to other nodes; Recombining the question text and the original answer text to generate a logical conversion text corresponding to the question text, and determining an authenticity verification result of the logical conversion text, wherein the logical conversion text is a text obtained by logically converting a proposition text composed of the question text and the original answer text, and the logical conversion text is logically associated with the proposition text; The question text and the authenticity verification result are re-input into the target large language model to obtain a target answer text corresponding to the question text through the target large language model, and the task is executed according to the target answer text.

2. The method according to claim 1, characterized in that According to the model interaction relationship diagram, determining the target large language model specifically includes: For each preset large language model, according to the model interaction relationship diagram, the number of other large language models having data transmission relationships with the large language model is determined, and the importance of the large language model is determined according to the number of other large language models having data transmission relationships with the large language model, wherein the number of other large language models having data transmission relationships with the large language model is positively correlated with the importance of the large language model; According to the importance of each large language model, a target large language model is determined from the large language models.

3. The method according to claim 1, characterized in that Determine the model interaction diagram, including: Obtaining an initial model interaction relationship diagram and performance data of each preset large language model, wherein for each large language model, the performance data of the large language model includes: performance data for reflecting a network environment of a device on which the large language model is deployed, and performance data for reflecting task execution efficiency and task execution effect of the large language model; Determine the data transmission relationship between the major language models based on the performance data of each major language model; According to the data transmission relationship, the initial model interaction relationship diagram is adjusted to determine the model interaction relationship diagram.

4. The method according to claim 1, characterized in that The question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and the authenticity verification result of the logical conversion text is determined, specifically including: In the case where it is determined that the proposition text composed of the question text and the original answer text is a compound proposition text, splitting the proposition text composed of the question text and the original answer text into sub-proposition texts; For each sub-proposition text, the sub-proposition text is reorganized to generate a basic logic conversion text corresponding to the sub-proposition text, and a basic authenticity verification result of the basic logic conversion text corresponding to the sub-proposition text is determined.

5. The method according to claim 1, characterized in that Determining the authenticity verification result of the logical conversion text specifically includes: According to the model interaction relationship diagram, determining other large language models that have a data transmission relationship with the target large language model as verification large language models; The logic conversion text is input into the verification large language model to obtain the authenticity verification result of the logic conversion text through the verification large language model.

6. The method according to claim 1, characterized in that Inputting the question text into a predetermined target large language model to generate an answer text as an original answer text according to the question text through the target large language model, specifically including: In the case where there are multiple predetermined target large language models, for each predetermined target large language model, the question text is input into the target large language model, so as to generate an answer text as the original answer text according to the question text through the target large language model; The question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and the authenticity verification result of the logical conversion text is determined, specifically including: Determining the alignment of each target large language model with respect to the question text according to the difference between the original answer texts output by each target large language model; Determining alignment results of each target large language model according to the alignment degree; If it is determined based on the alignment result that the target large language models are not aligned, the question text and the original answer text are reorganized to generate a logical conversion text corresponding to the question text, and an authenticity verification result of the logical conversion text is determined.

7. A task execution device, characterized in that: include: The acquisition module is used to obtain the question text input by the user; A generation module, used for inputting the question text into a predetermined target large language model, so as to generate an answer text according to the question text through the target large language model as an original answer text, wherein the target large language model is determined according to a predetermined model interaction relationship diagram, in which different preset large language models correspond to different nodes, and for each node, the edge between the node and other nodes is used to represent the data transmission relationship between the large language model corresponding to the node and the large language models corresponding to other nodes; A verification module, used to reorganize the question text and the original answer text, generate a logical conversion text corresponding to the question text, and determine the authenticity verification result of the logical conversion text, wherein the logical conversion text is a text obtained by logically converting a proposition text composed of the question text and the original answer text, and the logical conversion text is logically associated with the proposition text; An execution module is used to re-input the question text and the authenticity verification result into the target large language model to obtain a target answer text corresponding to the question text through the target large language model, and perform a task according to the target answer text.

8. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 6 is implemented.

Images