Method and device for correcting knowledge ontology belief of agent model based on reinforcement learning

Abstract

The application relates to the technical field of artificial intelligence, and discloses a knowledge ontology belief correction method and device for an agent model based on reinforcement learning, which are used for solving the technical problem that the existing belief correction mode for the knowledge ontology of an agent model is not intelligent and reasonable enough, and that the corrected knowledge system is prone to contradictions or deviates from the fact. The method comprises the following steps: acquiring newly-added knowledge assertions found in the runtime of the agent model; detecting whether the newly-added knowledge assertions conflict with original knowledge assertions in the knowledge ontology of the agent model; if yes, generating a minimum conflict assertion set based on the newly-added knowledge assertions and the original knowledge assertions in conflict; solving a correction strategy set based on the minimum conflict assertion set; calling a strategy selection model constructed based on a reinforcement learning algorithm to select a target correction strategy in the correction strategy set; correcting the knowledge ontology based on the target correction strategy, and injecting the knowledge ontology after belief correction into the agent model.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a knowledge ontology belief correction method and device for an agent model based on reinforcement learning, an electronic device, a computer storage medium, and a computer program product. BACKGROUND

[0002] In the prior art, when an agent model based on a large language model performs a multi-round and long-time sequence task, the agent model usually relies on a shared world model or knowledge ontology to maintain environment state information. This is because the background information of a task may change over time. By using a shared world model or knowledge ontology, the agent model can record and update observed facts (such as object positions, user intentions, and task progress) in a structured manner, thereby making up for the lack of explicit long-term memory of the large language model itself, avoiding starting reasoning from scratch in each round of dialogue, and preventing contradictions between task decisions, and thus being more suitable for improving reasoning ability and task planning quality.

[0003] However, when the agent model dynamically introduces new factual knowledge during task execution, the introduced knowledge content may conflict with the axiomatic logic in the existing knowledge ontology. In this case, if the agent model based on the large language model is forced to reason and make decisions based on axiomatic logic that contradicts each other, the risk of the model producing "hallucinations" may be exacerbated, thereby causing reasoning errors, task interruptions, or distorted output results.

[0004] To alleviate the problem of knowledge conflict, the prior art usually adopts a deterministic correction strategy based on the "principle of minimum change": that is, to change the existing information in the world model or knowledge ontology as little as possible on the premise that the new knowledge does not cause logical conflicts. Although this method can maintain the consistency within the system to some extent, it also has limitations: only pursuing the minimum change may result in the updated knowledge ontology being unable to accurately reflect the state of the real world, causing information bias; at the same time, such methods lack an evaluation mechanism for different correction paths, which may reduce the overall success rate of the task.

[0005] Therefore, there is an urgent need for a more intelligent knowledge ontology belief correction method that can dynamically and reasonably adjust the knowledge system of the agent model in combination with newly acquired factual information. SUMMARY

[0006] The main purpose of the present application is to solve the technical problem that the prior art is not intelligent and reasonable in correcting the knowledge ontology of the agent model, resulting in contradictions or deviations from the facts in the corrected knowledge system.

[0007] The first aspect of the present application provides a knowledge ontology belief correction method for an agent model based on reinforcement learning, comprising:

[0008] acquiring a new knowledge assertion discovered in a runtime of the agent model;

[0009] detecting whether the new knowledge assertion conflicts with an original knowledge assertion in a knowledge ontology of the agent model;

[0010] if yes, generating a minimal conflict assertion set based on the new knowledge assertion and the original knowledge assertion in conflict;

[0011] solving a correction strategy set based on the minimal conflict assertion set;

[0012] calling a strategy selection model constructed based on a reinforcement learning algorithm to select a target correction strategy from the correction strategy set;

[0013] correcting the knowledge ontology based on the target correction strategy, and injecting the corrected knowledge ontology into the agent model.

[0014] Optionally, in the first implementation manner of the first aspect, the solving of the correction strategy set based on the minimal conflict assertion set comprises:

[0015] assigning a correction strategy variable to each knowledge assertion in the minimal conflict assertion set;

[0016] constructing a correction constraint condition according to a solving requirement, finding values of the correction strategy variables when the correction constraint condition is satisfied, and generating a correction strategy;

[0017] constructing the correction strategy set based on all the correction strategies when the constraint condition is satisfied.

[0018] Optionally, in the second implementation manner of the first aspect, the selection of the target correction strategy from the correction strategy set comprises:

[0019] acquiring text information and corresponding metadata of the knowledge assertions in the minimal conflict assertion set, and generating a corresponding state representation vector of each knowledge assertion in the minimal conflict assertion set;

[0020] calling a strategy selection model constructed based on a reinforcement learning algorithm based on the state representation vectors, and calculating an action probability of each correction strategy in the correction strategy set;

[0021] taking the correction strategy with the maximum action probability value as the target correction strategy.

[0022] Optionally, in the third implementation manner of the first aspect, the detection of whether the new knowledge assertion conflicts with the original knowledge assertion in the knowledge ontology of the agent model comprises:

[0023] invoke the inference engine to perform consistency checking on the new knowledge assertion and the original knowledge assertions in the knowledge ontology of the agent model;

[0024] If the new knowledge assertion and any original knowledge assertion contained in the knowledge ontology do not pass the consistency checking, a conflict exists.

[0025] Optionally, in a fourth implementation form of the first aspect of the present application, before the knowledge ontology is modified based on the target modification strategy, the method further comprises:

[0026] based on the target modification strategy, simulating the modification of the knowledge ontology to obtain a simulated knowledge ontology after the simulated modification;

[0027] invoke the inference engine to perform simulated consistency checking on the new knowledge assertion and the knowledge assertions contained in the simulated knowledge ontology;

[0028] If the new knowledge assertion and all the knowledge assertions contained in the simulated knowledge ontology pass the simulated consistency checking, the formal modification can be performed, and the target modification strategy can be submitted.

[0029] Optionally, in a fifth implementation form of the first aspect of the present application, after the knowledge ontology after the belief modification is injected into the agent model, the method further comprises:

[0030] invoke the agent model to perform a task based on the knowledge ontology after the belief modification, and give a task execution result;

[0031] collect feedback information on the task execution result;

[0032] based on the feedback information and a preset reward function, optimize the processing parameters of the policy selection model.

[0033] The second aspect of the present application provides a knowledge ontology belief modification device of an agent model based on reinforcement learning, comprising:

[0034] an ontology management module configured to acquire a new knowledge assertion found in the runtime of the agent model;

[0035] a conflict detection module configured to detect whether the new knowledge assertion conflicts with an original knowledge assertion in the knowledge ontology of the agent model; if so, generate a minimal conflict assertion set based on the new knowledge assertion and the original knowledge assertion in conflict;

[0036] a policy generation module configured to solve a modification strategy set based on the minimal conflict assertion set;

[0037] The policy selection module is configured to call a policy selection model constructed based on a reinforcement learning algorithm, and select a target correction policy from the correction policy set;

[0038] The correction execution module is configured to correct the knowledge ontology based on the target correction policy, and inject the knowledge ontology after belief correction into the agent model.

[0039] The third aspect of the present application provides an agent model knowledge ontology belief correction device based on reinforcement learning, comprising a memory and at least one processor, the memory stores instructions; the at least one processor calls the instructions in the memory, so that the agent model knowledge ontology belief correction device based on reinforcement learning executes the steps of the agent model knowledge ontology belief correction method based on reinforcement learning described above.

[0040] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium stores instructions, when it runs on the computer, makes the computer execute the steps of the agent model knowledge ontology belief correction method based on reinforcement learning described above.

[0041] The fifth aspect of the present application provides a computer program product, comprising computer programs / instructions, when the computer programs / instructions are executed by the processor, the steps of the agent model knowledge ontology belief correction method based on reinforcement learning described above are realized.

[0042] In the technical solution provided by the present application, the newly added knowledge assertion found in the running of the agent model is acquired; it is detected whether the newly added knowledge assertion conflicts with the original knowledge assertion in the knowledge ontology of the agent model; if so, the minimum conflict assertion set is generated based on the newly added knowledge assertion and the original knowledge assertion in conflict, and the correction policy set is solved based on the minimum conflict assertion set; the policy selection model constructed based on the reinforcement learning algorithm is called, and the target correction policy is selected from the correction policy set; the knowledge ontology is corrected based on the target correction policy, and the knowledge ontology after belief correction is injected into the agent model. This method can combine the specific content of the newly acquired factual information and the conflict situation with the knowledge contained in the knowledge ontology, and more intelligently and reasonably correct the knowledge ontology of the agent model. In addition, the device, electronic equipment, computer readable storage medium and computer program product provided by the present application also solve the corresponding technical problems. BRIEF DESCRIPTION OF DRAWINGS

[0043] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0044] Figure 1 A first flowchart of a first embodiment of the knowledge ontology belief revision method for the agent model based on reinforcement learning in the embodiments of the present application;

[0045] Figure 2 A second flowchart of the first embodiment of the knowledge ontology belief revision method for the agent model based on reinforcement learning in the embodiments of the present application;

[0046] Figure 3 A flowchart of a second embodiment of the knowledge ontology belief revision method for the agent model based on reinforcement learning in the embodiments of the present application;

[0047] Figure 4 A working flowchart of the policy selection model in the second embodiment of the knowledge ontology belief revision method for the agent model based on reinforcement learning in the embodiments of the present application;

[0048] Figure 5 A schematic diagram of one embodiment of the knowledge ontology belief revision device for the agent model based on reinforcement learning in the embodiments of the present application;

[0049] Figure 6 A schematic diagram of another embodiment of the knowledge ontology belief revision device for the agent model based on reinforcement learning in the embodiments of the present application;

[0050] Figure 7 A schematic diagram of one embodiment of the knowledge ontology belief revision device for the agent model based on reinforcement learning in the embodiments of the present application;

[0051] Figure 8 A schematic diagram of one embodiment of the knowledge ontology belief revision device for the agent model based on reinforcement learning in the embodiments of the present application; DETAILED DESCRIPTION

[0052] Exemplary embodiments of the present application will now be described more fully with reference to the accompanying drawings. The exemplary embodiments of the present application, however, can be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these exemplary embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the inventive concept to those skilled in the art. Like reference numerals refer to like elements throughout the specification.

[0053] Features, structures, characteristics or other details described in relation to a certain embodiment can be incorporated into other embodiments in a suitable manner without departing from the technical concept of the present application.

[0054] In the description of specific embodiments, the features, structures, characteristics or other details described in the present application are to enable those skilled in the art to fully understand the embodiments. However, one or more of the specific features, structures, characteristics or other details can not be excluded from the technical solution of the present application.

[0055] The flowchart shown in the drawing is only an exemplary illustration, and does not necessarily include all contents and operations / steps, nor does it necessarily execute in the order described. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation.

[0056] The block diagram shown in the drawing is only a functional entity, and does not necessarily correspond to a physically independent entity. That is, the functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0057] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0058] Please refer to Figures 1-2 The first embodiment of the knowledge ontology belief revision method of the agent model based on reinforcement learning in the embodiment of the present application is as follows:

[0059] S101, acquiring a new knowledge assertion found in the runtime of the agent model;

[0060] It can be understood that the execution subject of the present application can be a knowledge ontology belief revision device of an agent model based on reinforcement learning, and can also be a terminal or a server, and the specific implementation is not limited herein. The server is taken as an example for description in the embodiment of the present application.

[0061] In the specific working process, the server can acquire a new knowledge assertion found in the runtime of the agent model, at this time, the processing of the task is suspended, the new knowledge assertion is temporarily stored and the processing flow is suspended, and the subsequent step of judging whether the knowledge ontology needs to be revised is entered.

[0062] S102, detecting whether the new knowledge assertion conflicts with the original knowledge assertion in the knowledge ontology of the agent model;

[0063] The inference engine is called to perform consistency checking on the new knowledge assertion and the original knowledge assertion in the knowledge ontology of the agent model. If the new knowledge assertion and any one of the original knowledge assertions contained in the knowledge ontology do not pass the consistency checking, there is a conflict.

[0064] S103, if yes, generating a minimum conflict assertion set based on the new knowledge assertion and the original knowledge assertion in conflict;

[0065] If there is no conflict between the original knowledge assertion and the new knowledge assertion in the knowledge ontology, the new knowledge assertion can be directly added to the knowledge ontology. If there is any conflict between the original knowledge assertion and the new knowledge assertion in the knowledge ontology, a minimum conflict assertion set is generated based on the new knowledge assertion and the original knowledge assertion in conflict.

[0066] S104, solving a correction strategy set based on the minimum conflict assertion set;

[0067] Assigning a correction strategy variable to each knowledge assertion in the minimum conflict assertion set; constructing a correction constraint condition according to the solving requirement, finding the value of each correction strategy variable when the correction constraint condition is satisfied, and generating a correction strategy; based on all the correction strategies that satisfy the constraint condition, a correction strategy set is constructed.

[0068] S105, calling a strategy selection model constructed based on a reinforcement learning algorithm to select a target correction strategy in the correction strategy set;

[0069] Obtaining the text information and corresponding metadata of the knowledge assertion in the minimum conflict assertion set, and generating the corresponding state representation vector of each knowledge assertion in the minimum conflict assertion set; based on each state representation vector, calling a strategy selection model constructed based on a reinforcement learning algorithm to calculate the action probability of each correction strategy in the correction strategy set; the correction strategy with the largest action probability value is taken as the target correction strategy.

[0070] In a preferred embodiment, after obtaining the target correction strategy, it further includes: based on the target correction strategy, simulating the correction of the knowledge ontology to obtain a simulated knowledge ontology after simulation correction; calling a reasoning machine to perform simulation consistency checking on the new knowledge assertion and the knowledge assertion contained in the simulated knowledge ontology; if the new knowledge assertion and all the knowledge assertions contained in the simulated knowledge ontology pass the simulation consistency checking, formal correction can be performed, and the target correction strategy is submitted.

[0071] S106, correcting the knowledge ontology based on the target correction strategy, and injecting the belief-corrected knowledge ontology into the agent model.

[0072] After completing the belief correction, a correction execution module can be called to inject the belief-corrected knowledge ontology into the agent model, so that the agent model can continue to perform the tasks that should have been performed and the subsequent main tasks.

[0073] In a preferred implementation, further comprising invoking the agent model to perform a task based on the belief-corrected knowledge ontology, and giving a task execution result; collecting feedback information on the task execution result; and based on the feedback information and a preset reward function, optimizing the processing parameters of the policy selection model.

[0074] Based on the description in the present embodiment, please continue to refer to Figure 2 For a specific example, a specific flow of the knowledge ontology belief correction method of the agent model of the reinforcement learning in the present embodiment can be as follows:

[0075] S201: The agent model proposes to add a new knowledge assertion;

[0076] S202: Temporarily suspend the new knowledge assertion;

[0077] S203: Perform conflict detection: determine whether there is a conflict; if there is a conflict, go to S204, and if there is no conflict, go to S210;

[0078] S204: Generate a minimum conflict assertion set;

[0079] S205: Enumerate all valid correction strategies;

[0080] S206: Select the optimal action based on the metadata;

[0081] S207: Accurately perform the correction operation;

[0082] S208: Verify ontology consistency, if consistent, go to S209;

[0083] S209: Update the ontology and inject;

[0084] S210: When the result obtained in S203 is that there is no conflict, directly add the new knowledge assertion to the ontology;

[0085] S211: The agent model continues to perform the main task: inject the corrected knowledge ontology into the agent model, and continue to perform the main task using the agent model;

[0086] S212: Task completion evaluation and strategy update;

[0087] S213: Wait for the next belief correction.

[0088] The method provided in the present embodiment can combine the specific content of the newly acquired fact information and the conflict situation with the knowledge contained in the knowledge ontology, to more intelligently and reasonably correct the knowledge ontology of the agent model, improve the accuracy of the knowledge ontology, and further optimize the task execution effect of the agent model.

[0089] Please refer to Figures 3-4 The second embodiment of the knowledge ontology belief revision method of the agent model based on reinforcement learning in the embodiment of the application is specifically as follows:

[0090] The knowledge ontology belief revision method of the agent model based on reinforcement learning in the embodiment is mainly used in a multi-agent model agent system. Multiple agent models can use a shared world model or knowledge ontology to perform reasoning and execute specific tasks, such as code generation, task planning, intelligent question answering, and the like. The multi-agent model agent system can be constructed based on an LLM (Large Language Model). In the embodiment, an ontology management module is used to maintain a knowledge ontology (or world model), which can save multiple knowledge assertions in the form of a directed graph. In the embodiment, the knowledge assertion refers to a structured knowledge unit used to describe facts, rules, or constraints, which is stored in the knowledge ontology and participates in logical reasoning and consistency verification. The knowledge assertion can be dynamically added, deleted, or modified according to the task execution needs. In addition, the knowledge assertion is often referred to as an “axiom” in the field, and the scope of the axioms and Axioms mentioned in the specific examples in the latter part of the embodiment is the same as that of the knowledge assertion.

[0091] In a specific implementation, when multiple knowledge assertions are saved in the form of a directed graph, the directed graph can be constructed by Owlready2 and NetworkX, and the knowledge ontology can be saved in the OWL (Web Ontology Language) format, so that the directed graph in the embodiment can support fast addition and deletion of knowledge assertions or axioms. The Owlready2 is a Python library for ontology programming, and the NetworkX is a Python library for creating, operating, and researching complex network structures, dynamics, and functions.

[0092]

[0093] S301, acquiring a newly added knowledge assertion found in the agent model runtime;

[0094] In the specific working process, the ontology management module can accept a request to “add a knowledge assertion”, thereby acquiring a newly added knowledge assertion found in the agent model runtime. After obtaining the newly added knowledge assertion, it is temporarily stored and the processing flow is suspended, and the complete metadata of the newly added knowledge assertion is recorded.

[0095] S302, detecting whether the newly added knowledge assertion conflicts with an original knowledge assertion in the knowledge ontology of the agent model;

[0096] S303, if yes, generating a minimum conflict assertion set based on the new knowledge assertion and the original knowledge assertion in conflict;

[0097] Next, the conflict detection module detects whether the new knowledge assertion is in conflict with the original knowledge assertion in the knowledge ontology of the agent model. If there is no conflict between the original knowledge assertion and the new knowledge assertion in the knowledge ontology, the new knowledge assertion can be directly added to the knowledge ontology. If there is any original knowledge assertion and the new knowledge assertion in conflict, the specific belief revision scheme is started; at the same time, the conflict detection module generates a minimum conflict assertion set based on the new knowledge assertion and the original knowledge assertion in conflict. That is, all original knowledge assertion entries in conflict with the new knowledge assertion are found in the knowledge ontology, and the new knowledge assertion is composed into a minimum conflict assertion set, and the metadata corresponding to the knowledge assertion contained in the minimum conflict assertion set is obtained. Specifically, the conflict detection module can call HermiT inference engine to perform consistency check on the new knowledge assertion and the original knowledge assertion in the knowledge ontology to determine whether the original knowledge assertion and the new knowledge assertion are in conflict; wherein the HermiT inference engine is a logical inference engine for OWL (Web Ontology Language) knowledge ontology.

[0098] Next, taking the agent model executing a trip planning task as an example, a specific example is given for illustration. When the agent model queries the flight, it finds a major news: "All flights from Paris to Berlin on A month B day (user departure day) are canceled due to air traffic control personnel strike." At this time, the agent model immediately converts this new observation into an OWL axiom: "All flights from Paris to Berlin on A month B day in 2025 are canceled", and tries to add the ontology. After receiving the request of the agent model to add the axiom, the ontology management module temporarily suspends the addition of the new axiom, and records the complete metadata. Next, the conflict detection module calls HermiT inference engine to perform consistency check; the inference engine feedback result is "inconsistent", and the minimum conflict assertion set is given, which includes:

[0099] Axiom_New: "All flights from Paris to Berlin on A month B day in 2025 are canceled";

[0100] Axiom_Old_1: "There are direct flights from Paris to Berlin every day";

[0101] Axiom_Old_2: "Flights of C company and D airline belong to reliable transportation mode".

[0102] S304, based on the minimum conflict assertion set to solve the revision strategy set;

[0103] After obtaining the minimum conflict assertion set, a strategy generation module can be called to solve the corresponding modification strategy. The strategy generation module can be constructed based on an SMT (Satisfiability Modulo Theories) solver, which receives the minimum conflict assertion set and outputs all logically valid and minimum modification strategy sets (or action spaces) through logical calculation. In an embodiment, the Z3 solver can be used for implementation.

[0104] The specific manner of solving the modification strategy set based on the minimum conflict assertion set is described in the following steps:

[0105] (1) The minimum conflict assertion set is converted into a list or array form for subsequent processing.

[0106] (2) Each knowledge assertion contained therein is assigned a modification strategy variable.

[0107] In this step, a Boolean variable can be used as the modification strategy variable, and the value of each modification strategy variable can only be true (True) or false (False). When the value of the modification strategy variable is true, the corresponding knowledge assertion is retained, and when the value of the modification strategy variable is false, the corresponding knowledge assertion is deleted. Thus, the corresponding modification scheme is represented by the sequence of modification strategy variables.

[0108] (3) Modification constraint conditions are constructed according to the solving requirements, the values of the modification strategy variables when the modification constraint conditions are satisfied are found, and the modification strategy is generated. Based on all the modification strategies that satisfy the constraint conditions, a modification strategy set is constructed.

[0109] Since there are conflicting knowledge assertions in the minimum conflict assertion set described in this step, if the conflicts are to be modified, all knowledge assertions in the minimum conflict assertion set cannot be deleted. When solving the modification strategy, the effect desired is that the deleted conflicting knowledge assertions are as few as possible. When modification is performed only by deletion, it is also necessary to limit that all knowledge assertions cannot be retained (if there are other ways of proof in addition to deletion, then it can not be limited to retain all knowledge assertions). Next, the solver is used to solve in a loop, and the knowledge assertions that are assigned true in the result obtained each time are used as a modification strategy result that satisfies the above constraints.

[0110] Further, a blocking clause of "prohibit obtaining the same model again" is added to the solver when solving, so that the next modification with smaller changes is found in the loop until there is no solution. In addition, solving restrictions can be added to this step, such as when the solving time exceeds the preset setting solving time limit, the solving is stopped, and all the modification strategies obtained are used as the modification strategy result.

[0111] In a preferred embodiment, in addition to the above-mentioned modification by deleting knowledge assertions, a preset modification template library is included, and the modified knowledge assertions are generated according to the knowledge assertions contained in the minimum conflict assertion set according to the preset modification template library. The specific semantics of the knowledge assertions in conflict in the minimum conflict assertion set are understood by calling a large language model, and more modification strategy results are given by combining the existing modification ways of the modification template library. The specific modification ways include but are not limited to the following ways:

[0112] (1) Concept splitting:

[0113] Split the concepts contained in one or more knowledge assertions in the minimum conflict assertion set to obtain lower concepts, and obtain the strategy for modifying the conflict by distinguishing the knowledge assertions of different lower concepts;

[0114] (2) Template weakening:

[0115] Weaken the content of one or more knowledge assertions contained in the minimum conflict assertion set, such as increasing the range limitation of the knowledge assertion;

[0116] (3) Adding exceptions:

[0117] Retain the conflict knowledge assertions in the minimum conflict assertion set, but add an exception knowledge assertion that can make the two conflicting knowledge assertions exist at the same time.

[0118] Taking the minimum conflict assertion set in S303 as an example, one of the modification strategy sets can be as follows:

[0119] Strategy 1, delete Axiom_New;

[0120] Strategy 2, delete Axiom_Old_1 (i.e. delete the old flight schedule);

[0121] Strategy 3, delete Axiom_Old_1 + Axiom_Old_2 (i.e. combined deletion);

[0122] Strategy 4, concept splitting: split the concept of "flight" into two sub-classes "normal flight" and "flight during strike", and obtain Axiom_New only associated to the latter "flight during strike" sub-class concept;

[0123] Strategy 5, template weakening: weaken Axiom_Old_1 to "Paris-Berlin usually has direct flights (unless on strike)";

[0124] Strategy 6: Add an exception-qualified knowledge assertion: Retain Axiom_New, Axiom_Old_1, and Axiom_Old_2, but add a new knowledge assertion: "Strike events belong to the subclass relationship that causes flight cancellations."

[0125] After obtaining the minimum conflict assertion set and the set of corrective policies, this embodiment can call the policy selection module and, based on the policy selection model built using the reinforcement learning algorithm, select the target corrective policy from the set of corrective policies. Please continue reading... Figure 3 intermediate steps S305-S308 and Figure 4 The content of this document will be explained in detail below.

[0126] S305. Obtain the text information and corresponding metadata of the knowledge assertions contained in the minimum conflict assertion set, and generate the corresponding state representation vector for each knowledge assertion in the minimum conflict assertion set.

[0127] Specifically, in step S305, firstly, text features are extracted by calling a natural language processing model to encode the text content of the conflicting knowledge assertions, generating a text vector conflict assertion vector; secondly, structured metadata of the conflicting knowledge assertions is extracted to construct a metadata vector; the metadata vector is input into a multilayer perceptron to map it into high-dimensional features, and the text embedding vector and the metadata feature vector are concatenated to obtain a state representation vector. See also... Figure 4 The content of the state space construction and vectorization section shown in S410.

[0128] based on Figure 4 The process shown in S410 is described in detail below for step S305 of this embodiment:

[0129] S411: Input the minimum set of conflicting assertions and metadata;

[0130] After obtaining the minimum conflict assertion set, it is also necessary to extract the metadata of the knowledge assertions contained in the minimum conflict assertion set.

[0131] Next, features will be extracted from text paths and metadata paths.

[0132] S412: Text Feature Extraction

[0133] For text paths, pre-trained natural language processing models (such as sentence-transformers or all-MiniLM-L6-v2) are used to encode the text content of knowledge assertions contained in the minimum conflict assertion set. Each knowledge assertion is converted into a text vector, and the average of all axiom vectors is taken (such as through Mean Pooling) to generate a text embedding vector, i.e., Text Embedding.

[0134] S413: Metadata Feature Extraction:

[0135] For the metadata path, the structured metadata of the knowledge assertions contained in the minimum conflict assertion set is extracted, including Source, Timestamp, Confidence Score, Dependency Count, Historical Task Success Rate, and Current Main Task Type. These numerical features are normalized to construct the metadata feature vector, i.e., Metadata Vector.

[0136] S414: Feature Concatenation and Fusion:

[0137] The metadata vector is input into a multilayer perceptron (MLP) containing a linear layer (Linear) and an activation function (Linear Rectification Function, ReLU), which is mapped to a high-dimensional feature (e.g., 128 dimensions). The text embedding vector (e.g., 384 dimensions) is concatenated with the processed metadata feature (128 dimensions).

[0138] S415: Full Connection Layer Mapping, Generating State Representation Vector:

[0139] The concatenated vector is mapped to the final state representation vector (StateRepresentation Vector, denoted as ) through a full connection layer (nn.Linear). In a specific example, the dimension of the state representation vector can be set to 256 dimensions.

[0140] S306, based on each state representation vector, calling a policy selection model constructed based on a reinforcement learning algorithm to calculate the action probability of each correction strategy in the correction strategy set;

[0141] In step S306 of the present embodiment, the state representation vector is input into a policy selection model constructed based on a reinforcement learning algorithm to generate the non-normalized log probability of each valid correction action; and the Softmax function is called to normalize the non-normalized log probability to output the probability of each correction strategy being executed. The policy selection model can also be constructed based on the Transformer architecture. For details, please refer to the content of the probability distribution generation algorithm part shown in S420 of Figure 4 .

[0142] Based on Figure 4The following describes step S306 in detail by taking a policy selection model based on a Transformer architecture as an example.

[0143] S421: Forward propagation of the policy network:

[0144] The state representation vector is input into the policy network head (Policy Head) to perform action scoring (LogitsCalculation) and output non-normalized log probabilities (Logits) corresponding to each valid revision action in the current finite action space, denoted as . .

[0145] S422: Normalization:

[0146] To obtain a reasonable probability distribution, the Logits of all candidate valid revision actions in S422 are normalized by applying a Softmax function, and the specific expression is as follows:

[0147] .

[0148] wherein represents the probability of each revision action being executed after normalization, is the number of valid actions generated by the revision policy generation module (usually 20), represents a valid revision action, and

[0149] S423: Output of the action probability distribution:

[0150] The normalized result in S422 is output to obtain an action probability distribution, which represents the probability of the corresponding revision policy (such as “delete old axiom”, “weaken new axiom”, etc.) being executed. For example, action 1 has a probability of 10%, action 2 has a probability of 89%, action 3 has a probability of 1%, and so on.

[0151] S307: The revision policy with the largest action probability value is taken as the target revision policy.

[0152] S308: The knowledge ontology is revised based on the target revision policy, and the revised knowledge ontology is injected into the agent model.

[0153] After obtaining the action probability of each revision policy in the revision policy set, the revision policies are sorted according to the size of the action probability, and the revision policy with the largest action probability value is taken as the target revision policy.

[0154] As a specific example, in the use case of S303, if the action probability highest modification strategy is actually "delete Axiom_Old_1 (old flight schedule) + weaken Axiom_Old_2 to 'airlines are generally reliable but affected by strikes'", then the current modification strategy is taken as the target modification strategy.

[0155] In this embodiment, after obtaining the target modification strategy, it further includes: based on the target modification strategy, simulating the modification of the knowledge ontology to obtain the simulated knowledge ontology after the simulation modification; calling the reasoning machine to perform simulation consistency checking according to the newly added knowledge assertion and the knowledge assertion contained in the simulated knowledge ontology; if all the knowledge assertions in the newly added knowledge assertion and the simulated knowledge ontology pass the simulation consistency checking, the formal modification can be performed, and the target modification strategy is submitted. For details, please refer to the content of the probability distribution generation algorithm part shown in S430 of Figure 4

[0156] Based on Figure 4 The steps S307 and S308 of this embodiment are specifically explained as follows based on the flow shown in S430:

[0157] S431: select a sampling strategy;

[0158] S432: random sampling;

[0159] S433: select the action with the maximum probability;

[0160] In S431-S433, in the training stage, the probability distribution obtained above is used for random sampling to maintain the exploration of the model, but in the actual reasoning / application stage, the action with the maximum probability (Argmax) is selected to ensure the optimality.

[0161] S434: select a modification action;

[0162] S435: send to the deterministic analysis module for execution;

[0163] In S434 and S435, the selected action instruction is sent to the deterministic analysis module for accurate execution, thereby modifying the knowledge ontology and completing the specific belief modification.

[0164] S436: the agent model executes the downstream main task;

[0165] After completing the belief modification, the modified knowledge ontology after the belief modification can be injected into the agent model, and the agent model is called to execute the downstream main task.

[0166] ​​In a preferred embodiment, after obtaining the target revision strategy, a deterministic revision module is further called to revise the knowledge ontology based on the target revision strategy to obtain a simulated revised simulated knowledge ontology; a reasoning machine is called to perform simulated consistency checking on the new knowledge assertion and the knowledge assertion contained in the simulated knowledge ontology; if the new knowledge assertion and all the knowledge assertions contained in the simulated knowledge ontology pass the simulated consistency checking, formal revision can be performed, and the target revision strategy is submitted. When the reasoning machine is called to perform consistency checking, the specific method is similar to that in S303 of the present embodiment, and will not be described here.

[0167] S309, calling the agent model to execute a task based on the knowledge ontology after belief revision, and giving a task execution result;

[0168] After completing belief revision, a revision execution module can be called to inject the knowledge ontology after belief revision into the agent model, so that the agent model can continue to execute the task that should have been executed and give a task execution result.

[0169] Taking the use scenario in S303 and the target revision strategy obtained in S308 as examples, after injecting the repaired ontology into the agent model, the agent model obtains more comprehensive and accurate knowledge. When the agent model learns that "there is no flight from Paris to Berlin on the same day", it will give a new travel plan based on the revised knowledge ontology: "recommend TGV high-speed rail, 6 hours direct …".

[0170] Please continue to refer to Figure 3 Step S310 and Figure 4 the contents in the present embodiment, after completing belief revision of the knowledge ontology, the agent model is further called to execute a task based on the knowledge ontology after belief revision, and the policy selection model is optimized according to the feedback of the task execution result, so that the effect of subsequent policy selection is better.

[0171] S310, collecting feedback information on the task execution result, and optimizing the processing parameters of the policy selection model based on the feedback information and a preset reward function.

[0172] In the present embodiment, after belief revision, a feedback optimization module is further called to optimize the policy selection model through back propagation. For details, please refer to the related content of S440 in Figure 4 The contents of S309 and S310 in the present embodiment will be described in the following steps. The specific policy optimization steps include:

[0173] S441: task result determination;

[0174] After the end of the downstream main task (such as code generation, itinerary planning), the feedback signal from the agent model integration and reward module is received. Different reward parameters can be set for different task processing results, and the policy selection model is optimized according to different reward parameters; in a specific example:

[0175] S442: success, reward +1; S443: failure or manual intervention, reward -1; S444: timeout, reward -0.5.

[0176] S445: the optimization algorithm calculates the loss to update the policy network parameters;

[0177] In this embodiment, the proximal policy optimization algorithm (PPO) can be used to calculate the loss function, and the parameters of the policy network are updated according to the reward signal. This process enables the network to assign a higher probability to the action that leads to task success when encountering similar metadata characteristics (such as "real-time data vs. outdated data") in the future. For example, the next time a similar "real-time strike / weather vs. old flight schedule" conflict is encountered, the probability of the policy selection model choosing "believing real-time information" will be improved, so that more accurate recommendations can be given next time.

[0178] The method provided by the embodiment of the application can combine the specific content of the newly acquired factual information and the conflict with the knowledge contained in the knowledge ontology, and more intelligently and reasonably correct the knowledge ontology of the agent model, thereby improving the accuracy of the knowledge ontology. Moreover, in the embodiment, the steps of generating a strategy and selecting a strategy are respectively assigned to different processing steps, which realizes decoupling and controllability of the scheme, improves the automation and intelligence of belief correction, improves the generation and selection effect of the strategy, and further optimizes the execution effect of the agent model task.

[0179] The above describes the knowledge ontology belief correction method of the agent model based on reinforcement learning in the embodiment of the application, and the following describes the knowledge ontology belief correction device of the agent model based on reinforcement learning in the embodiment of the application. Please refer to Figure 5 The embodiment of the knowledge ontology belief correction device of the agent model based on reinforcement learning in the embodiment of the application is as follows:

[0180] The ontology management module 501 is configured to acquire the newly added knowledge assertion found in the agent model runtime;

[0181] The conflict detection module 502 is configured to detect whether the new knowledge assertion conflicts with an original knowledge assertion in the knowledge ontology of the agent model, and generate a minimum conflict assertion set based on the new knowledge assertion and the original knowledge assertion in conflict if the new knowledge assertion conflicts with the original knowledge assertion;

[0182] The strategy generation module 503 is configured to solve a correction strategy set based on the minimum conflict assertion set.

[0183] The strategy selection module 504 is configured to call a strategy selection model constructed based on a reinforcement learning algorithm to select a target correction strategy from the correction strategy set.

[0184] The correction execution module 505 is configured to correct the knowledge ontology based on the target correction strategy, and inject the knowledge ontology after belief correction into the agent model.

[0185] The device provided in the embodiment of the application can intelligently and reasonably correct the knowledge ontology of the agent model in combination with the specific content of the newly acquired factual information and the conflict situation with the knowledge contained in the knowledge ontology, improve the accuracy of the knowledge ontology, and further optimize the execution effect of the agent model task.

[0186] In another embodiment of the application, the strategy generation module 503 is specifically configured to:

[0187] assign a correction strategy variable to each knowledge assertion included in the minimum conflict assertion set;

[0188] construct a correction constraint condition according to a solving requirement, find values of the correction strategy variables when the correction constraint condition is satisfied, and generate a correction strategy;

[0189] construct a correction strategy set based on all the correction strategies when the constraint condition is satisfied.

[0190] In another embodiment of the application, the strategy selection module 504 is specifically configured to:

[0191] a vector generation unit configured to acquire text information and corresponding metadata of the knowledge assertions included in the minimum conflict assertion set, and generate a corresponding state representation vector of each knowledge assertion in the minimum conflict assertion set;

[0192] a probability calculation unit configured to call a strategy selection model constructed based on a reinforcement learning algorithm based on the state representation vectors, and calculate an action probability of each correction strategy in the correction strategy set;

[0193] a strategy output unit configured to take the correction strategy with the maximum action probability value as the target correction strategy.

[0194] In another embodiment of the present application, the conflict detection module 502 is specifically configured to:

[0195] invoke a reasoning machine to perform consistency checking on the new knowledge assertion and the original knowledge assertions in the knowledge ontology of the agent model;

[0196] If the new knowledge assertion and any of the original knowledge assertions contained in the knowledge ontology fail the consistency checking, a conflict exists.

[0197] Please continue to refer to Figure 6 In another embodiment of the present application, the knowledge ontology belief revision apparatus of the agent model based on reinforcement learning further comprises a deterministic analysis module 506, configured to:

[0198] based on the target revision strategy, simulate revision on the knowledge ontology to obtain a simulated knowledge ontology after simulated revision;

[0199] invoke a reasoning machine to perform simulated consistency checking on the new knowledge assertion and the knowledge assertions contained in the simulated knowledge ontology;

[0200] If the new knowledge assertion and all the knowledge assertions contained in the simulated knowledge ontology pass the simulated consistency checking, formal revision can be performed and the target revision strategy can be submitted.

[0201] Please continue to refer to Figure 6 In another embodiment of the present application, the knowledge ontology belief revision apparatus of the agent model based on reinforcement learning further comprises a feedback optimization module 507, configured to:

[0202] invoke the agent model to perform a task based on the knowledge ontology after belief revision and give a task execution result;

[0203] collect feedback information on the task execution result;

[0204] based on the feedback information and a preset reward function, optimize the processing parameters of the policy selection model.

[0205] Please continue to refer to Figure 6In another embodiment of the present application, the knowledge ontology belief revision device of the reinforcement learning-based agent model can also be connected with an LLM agent model multi-agent set system, which can perform downstream main tasks such as code generation, planning, and question and answer tasks according to the revised knowledge ontology, and the LLM agent model multi-agent set system can be connected with a user interaction module, which can be used to submit feedback to the feedback optimization module 507 according to the task execution of the LLM agent model multi-agent set system, so that the feedback optimization module 507 can provide a reward signal to the strategy selection module 504.

[0206] The device provided by the embodiment of the present application can combine the specific content of the newly acquired factual information and the conflict with the knowledge contained in the knowledge ontology to more intelligently and reasonably revise the knowledge ontology of the agent model. In the embodiment, the steps of generating and selecting strategies are respectively given to different modules when performing belief revision, which realizes decoupling and controllability of the scheme, improves the automation and intelligence of belief revision, improves the generation and selection effect of the strategy, and further optimizes the execution effect of the agent model task.

[0207] Based on the same inventive concept, the present specification also provides a knowledge ontology belief revision system of a reinforcement learning-based agent model provided in the present application. The corresponding description of the knowledge ontology belief revision system of the reinforcement learning-based agent model can be referred to the above-mentioned embodiments, which will not be described here.

[0208] Based on the same inventive concept, the present specification also provides a knowledge ontology belief revision system of a reinforcement learning-based agent model provided in the present application. The corresponding description of the knowledge ontology belief revision system of the reinforcement learning-based agent model can be referred to the above-mentioned embodiments, which will not be described here.

[0209] Figure 7 The structure schematic diagram of an electronic device provided by the embodiment of the present specification is shown in FIG. 7. Figure 7 The electronic device 700 shown in FIG. 7 is only an example, and should not bring any limitation to the function and use range of the embodiment of the present application. Figure 7 The electronic device 700 shown in FIG. 7 is only an example, and should not bring any limitation to the function and use range of the embodiment of the present application.

[0210] As Figure 7As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 can include, but are not limited to, at least one processing unit 710, at least one storage unit 720, a bus 730 that connects the various system components, including the storage unit 720 and the processing unit 710, a display unit 740, and the like.

[0211] The storage unit stores program code that can be executed by the processing unit 710 such that the processing unit 710 performs the steps of the methods described above in the processing method section of the specification in accordance with various exemplary embodiments of the present application. For example, the processing unit 710 can execute the steps of the methods shown above. Figures 1-4

[0212] The storage unit 720 can include a readable medium in the form of volatile storage such as a random access memory (RAM) 7201 and / or cache memory 7202, and can further include a read-only memory (ROM) 7203.

[0213] The storage unit 720 can also include program / utility 7204 having a set of programs / modules 7205, including an operating system, one or more application programs, other programs / modules, and program / data, each or some combination thereof, used by the processing unit 710 to analyze and respond to user requests and perform other functions.

[0214] The bus 730 can represent one or more of several types of bus structures, including a storage bus or bus controller, a peripheral bus, a graphics bus (e.g., an Accelerated Graphics Port, or AGP bus) and a local bus using any of a variety of bus architectures.

[0215] The electronic device 700 can also communicate with one or more external devices 100 such as a keyboard or pointing device, a Bluetooth device, and / or other devices via the I / O interface 750. The electronic device 700 can also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or one or more devices that enable the electronic device 700 to communicate with one or more other computing devices. Such communication can be via an I / O interface 750. The electronic device 700 can also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or the Internet) via a network adapter 760. The network adapter 760 can be any of a variety of modems, including cable modems, telephone modems, and fiber optics. Utilizing a modem, the electronic device 700 can communicate with the outside world via a telephone line or cable line depending on the type of modem utilized. It will be appreciated that the electronic device 700 can be connected to more than one network, and that the electronic device 700 can communicate with the more than one network via the network adapter 760. Figure 7 ​Other hardware and / or software modules can be used in conjunction with the electronic device 700, as desired, including, but not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0216] Through the above description of the embodiments, those skilled in the art can easily understand that the exemplary embodiments described in the present application can be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a computer readable storage medium (which can be a CD-ROM, an U disk, a mobile hard disk, etc.) or a network, and includes a number of instructions to make a computing device (which can be a personal computer, a server, or a network device, etc.) execute the above-mentioned method according to the present application. When the computer program is executed by a data processing device, the computer readable medium can realize the above-mentioned method of the present application, i.e., the method shown in Figures 1-4 .

[0217] Figure 8 A schematic diagram of a computer readable medium according to an embodiment of the present application.

[0218] A computer program for implementing the method shown in Figures 1-4 may be stored in one or more computer readable media. The computer readable medium can be a readable signal medium or a readable storage medium. The readable storage medium, for example, can be, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any suitable combination of the above. More specific examples (a non-exhaustive list) of the readable storage medium include an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0219] The computer readable storage medium can include a data signal carried in a baseband or propagated as a carrier wave in a propagated data signal, in which the readable program code is carried. Such a propagated data signal can take on many forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium can also be any readable medium that can send, propagate or transmit the program code for use by or in connection with an instruction execution system, apparatus or device. The program code contained on the readable storage medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0220] Furthermore, the application also provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the steps of the knowledge ontology belief revision method of the reinforcement learning-based agent model as described in any of the above embodiments.

[0221] The program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider.

[0222] In light of the above, the present application can be implemented in hardware, or as software modules running on one or more processors, or in a combination thereof. Those skilled in the art will appreciate that the various means for implementing some or all of the components according to embodiments of the present application can be realized in the practice using general purpose data processors, microprocessors, or digital signal processors (DSPs) and the like. The present application can also be realized as a device or apparatus program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein. Such an implementation of the present application can be stored on a computer readable medium, or transmitted from one place to another over a network, or a carrier signal. Such a computer program product can be downloaded and / or otherwise obtained from, for example, an Internet site, a server, or a cloud.

[0223] The above described embodiments are intended to be illustrative of the present application, and should not be construed as limiting the present application. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application should be included in the scope of the present application.

[0224] Each of the embodiments in the specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments.

[0225] The above embodiments are only used to illustrate the present application, but not to limit it. Various modifications and changes can be made by those skilled in the art. Any modification, equivalent replacement, improvement, and the like made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. A method for revising knowledge ontology beliefs based on a reinforcement learning-based agent model, characterized in that, include: Obtain assertions of newly discovered knowledge during the runtime of the intelligent agent model; Detect whether the newly added knowledge assertion conflicts with the existing knowledge assertions in the knowledge ontology of the agent model; If so, then a minimum set of conflicting assertions is generated based on the newly added knowledge assertions and the existing knowledge assertions that conflict. The modified strategy set is obtained based on the minimum conflict assertion set; Obtain the text information and corresponding metadata of the knowledge assertions contained in the minimum conflict assertion set, generate the state representation vector corresponding to each knowledge assertion in the minimum conflict assertion set, call the policy selection model built based on the reinforcement learning algorithm, and select the target correction policy in the correction policy set based on the state representation vector; The knowledge ontology is modified based on the target modification strategy, and the modified knowledge ontology is injected into the agent model.

2. The knowledge ontology belief correction method based on reinforcement learning agent model according to claim 1, characterized in that, The process of solving for the modified strategy set based on the minimum conflict assertion set includes: Assign a modified policy variable to each knowledge assertion contained in the minimum conflict assertion set; Based on the solution requirements, construct modified constraints, find the values ​​of each modified strategy variable that satisfy the modified constraints, and generate a modified strategy. A set of correction strategies is constructed based on all correction strategies that satisfy the aforementioned constraints.

3. The knowledge ontology belief correction method based on reinforcement learning agent model according to claim 1, characterized in that, The invocation of the policy selection model built based on the reinforcement learning algorithm, and the selection of the target correction policy based on the state representation vector in the correction policy set, includes: Based on the state representation vectors, the policy selection model built based on the reinforcement learning algorithm is invoked to calculate the action probability of each corrected policy in the corrected policy set; The correction strategy that maximizes the probability value of the action is taken as the target correction strategy.

4. The knowledge ontology belief correction method based on reinforcement learning agent model according to claim 1, characterized in that, The step of detecting whether the newly added knowledge assertion conflicts with the existing knowledge assertions in the knowledge ontology of the agent model includes: Invoke the inference engine to perform a consistency check between the newly added knowledge assertions and the existing knowledge assertions in the knowledge ontology of the agent model. If the newly added knowledge assertion and any existing knowledge assertion contained in the knowledge ontology fail the consistency check, a conflict exists.

5. The knowledge ontology belief correction method based on reinforcement learning agent model according to claim 1, characterized in that, Before modifying the knowledge ontology based on the target modification strategy, the method further includes: Based on the target correction strategy, the knowledge ontology is simulated and corrected to obtain the simulated knowledge ontology after simulation correction. Invoke the inference engine to perform a simulation consistency check based on the newly added knowledge assertions and the knowledge assertions contained in the simulated knowledge ontology; If the newly added knowledge assertions and all knowledge assertions contained in the simulated knowledge ontology pass the simulation consistency check, then a formal correction can be performed and the target correction strategy can be submitted.

6. The knowledge ontology belief correction method for an agent model based on reinforcement learning according to any one of claims 1-5, characterized in that, After injecting the knowledge ontology with corrected beliefs into the agent model, the process also includes: The intelligent agent model is invoked to perform tasks based on the knowledge ontology after belief correction, and the task execution results are given. Collect feedback information on the results of the task execution; Based on the feedback information and the preset reward function, the processing parameters of the strategy selection model are optimized.

7. A knowledge ontology belief correction device based on a reinforcement learning-based agent model, characterized in that, include: The ontology management module is used to acquire new knowledge assertions discovered during the runtime of the intelligent agent model; The conflict detection module is used to detect whether the newly added knowledge assertion conflicts with the existing knowledge assertions in the knowledge ontology of the agent model. If so, generate a minimum set of conflicting assertions based on the newly added knowledge assertions and the existing knowledge assertions that conflict; The strategy generation module is used to solve for the modified strategy set based on the minimum conflict assertion set; The strategy selection module is used to obtain the text information and corresponding metadata of the knowledge assertions contained in the minimum conflict assertion set, generate the state representation vector corresponding to each knowledge assertion in the minimum conflict assertion set, call the strategy selection model built based on the reinforcement learning algorithm, and select the target correction strategy in the correction strategy set based on the state representation vector. The correction execution module is used to correct the knowledge ontology based on the target correction strategy and inject the corrected knowledge ontology into the agent model.

8. A knowledge ontology belief correction device based on a reinforcement learning-based intelligent agent model, characterized in that, The knowledge ontology belief correction device based on the reinforcement learning-based agent model includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the knowledge ontology belief correction device based on the reinforcement learning agent model to perform the steps of the knowledge ontology belief correction method based on the reinforcement learning agent model as described in any one of claims 1-6.

9. A computer-readable storage medium storing a computer program / instructions thereon, characterized in that, When the program / instruction is executed by the processor, it implements the steps of the knowledge ontology belief correction method for a reinforcement learning-based agent model as described in any one of claims 1-6.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, the steps of the knowledge ontology belief correction method based on the intelligent agent model of reinforcement learning as described in any one of claims 1-6 are implemented.

Images