An artificial intelligence security fence method based on multi-round intention evolution graph
Patent Information
- Application Number
- CN202611418535.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-14
- Publication Date
- 2026-10-16
AI Technical Summary
[0004]本发明的目的在于提供一种基于多轮意图演化图的人工智能安全围栏方法,改善了多轮交互中危险行为路径难以跨轮次识别、风险裁决缺乏可追溯证据支撑且裁决结果不稳定的问题
[0045] Optionally, newly added content in the temporary graph copy must not be written back to the multi-round intent evolution graph before passing the detection; after the candidate answer is added to the temporary graph copy, when the closed state changes to closed or the risk level increases, the candidate answer is blocked; after being blocked, a secure rewrite is performed, and when the number of rewrites reaches a preset limit, a preset secure response is returned.
Smart Images

Figure CN122926356A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence safety fence method based on a multi-round intention evolution graph. Background Technology
[0002] Large language models and intelligent agent systems have evolved from single-turn question-and-answer systems to multi-turn interactive systems capable of preserving context, invoking external tools, and continuously executing tasks. In such systems, user intent is typically not fully presented in a single interaction; behavioral goals, target audiences, tool selection, execution parameters, preconditions, authorized scope, and avoidance methods may be expressed progressively across multiple turns. When observing any single interaction turn in isolation, user input may only manifest as general principle inquiries, supplementary explanations of the target audience, or tool type queries; however, combining information from multiple turns can form a nearly direct, complete behavioral path. This characteristic makes traditional single-turn security detection methods ineffective in covering the risk identification needs of multi-turn scenarios.
[0003] Existing safety fence solutions typically rely on text classification, sensitive word matching, or a single continuous risk score in the current round as the primary basis for judgment. These solutions suffer from at least the following shortcomings: First, they fail to integrate facts across rounds into the same task context, making it difficult to identify progressive risks where users break down dangerous behaviors across multiple rounds. Second, the coupling of fact extraction and risk adjudication processes makes it impossible to pinpoint the specific source of evidence in the adjudication results, and there is a lack of verifiable records of state changes. Third, when using a score-weighted adjudication method, positive and negative factors cancel each other out, and authorization statements or semantic uncertainties may unreasonably lower the risk level. Fourth, the detection targets are limited to user input; system-generated candidate answers may fill in key parameters or execution steps not yet provided by the user, introducing additional risks. Fifth, over-reliance on large language models to directly output risk conclusions may lead to inconsistent adjudication results for the same input at different times or under different model versions. Therefore, there is an urgent need to develop a solution to address these problems. Summary of the Invention
[0004] The purpose of this invention is to provide an artificial intelligence safety fence method based on a multi-round intention evolution graph, which improves the problems of dangerous behavior paths being difficult to identify across rounds, risk adjudication lacking traceable evidence support, and adjudication results being unstable in multi-round interactions.
[0005] The artificial intelligence safety fence method based on multi-round intention evolution graph provided by this invention adopts the following technical solution:
[0006] An AI-based safety fence method based on multi-turn intention evolution graphs, specifically including:
[0007] Obtain user input for the current round and generate round evidence nodes; perform semantic parsing on the round evidence nodes to obtain semantic event nodes, and perform entity linking on the object names in the semantic event nodes to obtain standard entity nodes;
[0008] The current round's evidence nodes, semantic event nodes, and the first-appearing standard entity nodes are used as new graph content to establish basic semantic relationships, resulting in the current round's intermediate graph. On the current round's intermediate graph, the branch correlation degree between semantic event nodes and existing intent branch nodes is calculated. Based on the branch correlation degree, semantic event nodes are assigned to existing intent branch nodes or new intent branch nodes are created for semantic event nodes, thus obtaining the current round's multi-round intent evolution graph.
[0009] Extract candidate slots from semantic event nodes; calculate semantic confidence based on the simultaneous correct calibration probability of candidate slot values and their evidence intervals; determine the status of candidate slots based on evidence source level and semantic confidence; and perform mutual exclusion resolution on slots of the same type within the same intent branch node to determine the currently valid confirmed slots of the intent branch node.
[0010] Construct a minimum risk closed set, match the confirmed slots with the minimum risk closed set, and determine the closure state of the intention branch node based on the matching result; determine discrete decision variables based on the closure state and the confirmed slots, execute rule matching according to a preset fixed priority order, and generate the risk level and corresponding handling action for the current round;
[0011] When the action allows the generation of candidate answers, the subgraph structure corresponding to the current intent branch node is copied to form a temporary graph copy. The candidate answer is simulated to be added in the temporary graph copy. The candidate answer is used as the input of the current round to perform processing from semantic parsing to generating risk level and action. If the addition of the candidate answer changes the closed state to closed or increases the risk level, the candidate answer is blocked or rewritten.
[0012] Based on the risk level and the corresponding control actions, corresponding control operations are performed to form an immutable decision snapshot, and the control results and the evidence path tracing back from the intention branch node to the round evidence node are written into the audit chain.
[0013] This invention provides an AI-based safety fence method based on a multi-round intent evolution graph. By incrementally writing structured semantic information from each round into the multi-round intent evolution graph, and using intent branch nodes as aggregation centers to associate evidence, events, and entities belonging to the same task into a task subgraph, the method enables the complete tracking and backtracking of behavioral paths scattered across multiple rounds of interaction. By maintaining versioned slots in intent branch nodes and associating evidence sources with slot state transitions, each factual confirmation is supported by locatable evidence. Through minimum risk closure set matching and fixed priority rule adjudication, the method avoids unreasonable changes in risk levels caused by score weighting. By establishing a temporary graph copy before sending candidate responses and performing isomorphic incremental detection, the method prevents the system from automatically completing the final conditions of dangerous paths, thus improving the problems of difficulty in identifying dangerous behavioral paths across rounds, lack of traceable evidence for risk adjudication, and unstable adjudication results in multi-round interactions.
[0014] Optionally, when performing entity linking on object names in a semantic event node, it includes:
[0015] Calculate the entity matching degree between the object name in the semantic event node and the standard entity node. The entity matching degree is obtained by weighted calculation of name similarity, entity type consistency and business domain consistency.
[0016] When the entity matching degree is not less than the first preset threshold, the existing standard entity node is reused; when the entity matching degree is less than the second preset threshold, a new standard entity node is created; when the entity matching degree is less than the first preset threshold but not less than the second preset threshold, a candidate entity reference is created. The candidate entity reference shall not be used alone to satisfy the entity consistency constraint.
[0017] Optionally, the basic semantic relationships include event relationships, mention relationships, action relationships, use relationships, and reference relationships.
[0018] Optionally, the branch correlation degree is calculated by weighting the behavioral target similarity, the consistency of the target, the graph referential connectivity, and the temporal activity degree; the temporal activity degree decreases as the interval between the most recent active round and the current round of any graph branch node increases.
[0019] Optionally, when assigning a semantic event node to an existing intent branch node or creating a new intent branch node for a semantic event node based on branch relevance, the following steps are included:
[0020] The highest branch correlation is denoted as the first correlation, and the second highest branch correlation is denoted as the second correlation.
[0021] When the first correlation degree is not less than the first threshold, and the difference between the first correlation degree and the second correlation degree is not less than the second threshold, the semantic event node is assigned to the intent branch node corresponding to the first correlation degree.
[0022] When the first correlation degree is less than the first threshold and not less than the third threshold, or when the difference is less than the second threshold, the semantic event node is marked as a candidate belonging, and the semantic event node corresponding to the candidate belonging is not used to update the confirmed slot;
[0023] When the first correlation degree is less than the third threshold, a new intent branch node is created for the semantic event node.
[0024] Optionally, the evidence source levels include credible system evidence, direct textual evidence, inferred evidence, and unfounded results, and are sorted from high to low credibility.
[0025] When calculating semantic confidence based on the calibration probability that both the candidate slot value and its evidence interval are correct, the following is included:
[0026] When the semantic parser outputs candidate scores, it selects the candidate with the highest score, calculates the difference between the logarithmic sum of the selected maximum score and the scores of other candidate scores, and obtains the logarithmic dominance; the logarithmic dominance is then input into the calibration function to obtain the semantic confidence.
[0027] When the semantic parser does not output candidate scores, the logarithmic dominance is replaced by five features: exact slot value matching, consistent entity linking, complete negation range, complete evidence interval, and unique cross-wheel reference. These features are then input into the calibration function to obtain the semantic confidence.
[0028] Optionally, the status of candidate slots is determined based on the evidence source level and semantic confidence, and mutual exclusion is performed on slots of the same type within the same intent branch node. When determining the currently valid confirmed slots of the intent branch node, the process includes:
[0029] When the semantic confidence is below a low threshold, no candidate slot is formed; when the semantic confidence is not less than the low threshold and is lower than a high threshold, a candidate slot is formed; when the semantic confidence is not less than the high threshold and the evidence is direct textual evidence, the candidate slot is set to the confirmed state; trusted system evidence directly confirms the slot when there is no semantic ambiguity; candidate slots formed by inference evidence need to be supported by another independent and consistent piece of evidence before being converted to the confirmed state.
[0030] When there are two or more incompatible slots of the same type within the same intention branch node, and both are in the candidate state, the slots are set to conflict state; the candidate state is the state corresponding to the candidate slot; slots in the candidate state or conflict state do not participate in the matching of the minimum danger closure set.
[0031] Optionally, when constructing a minimum hazard closure set, matching the confirmed slots with the minimum hazard closure set, and determining the closure state of the intended branch node based on the matching result, the following steps are included:
[0032] The dangerous behavior conditions are mapped to a set of necessary slot predicates to obtain a set of candidate slots. For each slot predicate in the candidate slot set, a deletion operation is performed. If the remaining slot predicates still satisfy the sufficiency of the dangerous behavior conditions after deletion, the slot predicate is permanently deleted. This process is repeated until any remaining slot predicate no longer satisfies the sufficiency after deletion, thus obtaining the minimum dangerous closed set.
[0033] The sources of the minimum danger closure set include prohibition or restriction rules of versioned security policies, reviewed security event samples, and adversarial test samples confirmed by security experts.
[0034] Match the currently valid confirmed slots of the intent branch node with the necessary slot predicates of the minimum danger closure set, and the difference between the two is the set of missing slots.
[0035] When the number of elements in the missing slot set is two or more, the closure state is unclosed; when the number of elements is one, the closure state is adjacent closure; when the number of elements is zero and the same branch constraint, entity consistency constraint, parameter compatibility constraint and exclusion condition are satisfied, the closure state is closed.
[0036] When the same risk domain has two or more minimum hazard closed sets, the one with the fewest missing slots and the constraint condition is met is selected as the current matching closed set.
[0037] Optionally, based on the closed state and confirmed slots, discrete decision variables are determined, and rule matching is performed according to a preset fixed priority order to generate the risk level and corresponding handling actions for the current round, including:
[0038] The discrete adjudication variables include path stage, executability, authorization credibility, degree of adversarial force, and uncertainty;
[0039] The path stage is determined based on the combination of confirmed slots, and the higher the level, the more complete the behavior path.
[0040] The executability is determined by matching the confirmed slots with an execution mode table, which specifies the necessary parameters and prerequisite steps of the tool method.
[0041] The credibility of the authorization is determined through seven categories of slots: authorization subject, authorization object, authorization operation, authorization scope, validity period, issuing party, and verification status.
[0042] The degree of confrontation is determined by evasion actions, being evaded control, hiding actions, log suppression, code obfuscation, splitting and probing, and role disguise slots;
[0043] The uncertainty is determined by the state of the gated slot, which is a slot that affects the path stage, executability, authorization credibility, degree of adversarial force, or closure state result.
[0044] Rule matching is performed in order of rule number from front to back. The first rule that matches successfully determines the risk level of the current round and the corresponding action.
[0045] Optionally, newly added content in the temporary graph copy must not be written back to the multi-round intent evolution graph before passing the detection; after the candidate answer is added to the temporary graph copy, when the closed state changes to closed or the risk level increases, the candidate answer is blocked; after being blocked, a secure rewrite is performed, and when the number of rewrites reaches a preset limit, a preset secure response is returned. Attached Figure Description
[0046] Figure 1 This is a flowchart of an artificial intelligence safety fence method based on a multi-round intention evolution graph, provided as an embodiment of the present invention. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed after the word and its equivalents, but does not exclude other elements or objects.
[0048] See Figure 1 This invention provides an artificial intelligence safety fence method based on a multi-round intent evolution graph, comprising the following steps:
[0049] S1. Obtain the user input for the current round and generate round evidence nodes; perform semantic parsing on the round evidence nodes to obtain semantic event nodes, and perform entity linking on the object names in the semantic event nodes to obtain standard entity nodes;
[0050] S2. Take the current round's evidence nodes, semantic event nodes, and the first-appearing standard entity nodes as new graph content, establish basic semantic relationships, and obtain the current round's intermediate graph; on the current round's intermediate graph, calculate the branch correlation degree between semantic event nodes and existing intent branch nodes, and based on the branch correlation degree, assign semantic event nodes to existing intent branch nodes or create new intent branch nodes for semantic event nodes to obtain the current round's multi-round intent evolution graph;
[0051] S3. Extract candidate slots from semantic event nodes; calculate semantic confidence based on the simultaneous correct calibration probability of candidate slot values and their evidence intervals; determine the status of candidate slots based on evidence source level and semantic confidence; and perform mutual exclusion resolution on slots of the same type within the same intent branch node to determine the currently valid confirmed slots of the intent branch node.
[0052] S4. Construct the minimum risk closed set, match the confirmed slots with the minimum risk closed set, and determine the closure state of the intention branch node based on the matching result; determine the discrete decision variable based on the closure state and the confirmed slots, execute rule matching according to the preset fixed priority order, and generate the risk level and corresponding handling action for the current round.
[0053] S5. When the action allows the generation of candidate answers, copy the subgraph structure corresponding to the current intent branch node to form a temporary graph copy. Simulate the addition of candidate answers in the temporary graph copy. Use the candidate answers as the input of the current round to perform processing from semantic parsing to generating risk level and action. If the addition of candidate answers changes the closed state to closed or increases the risk level, then block or rewrite the candidate answers.
[0054] S6. Execute corresponding control operations based on risk level and action, form an immutable decision snapshot, and write the action result and the evidence path from the intention branch node back to the round evidence node into the audit chain.
[0055] In some embodiments, when obtaining the user input for the current round and generating the round evidence node in step S1, the system receives the user input for the current round and simultaneously obtains the session identifier, round identifier, role identifier, timestamp, historical round index, and optional user identity information and trusted permission metadata. The system performs encoding normalization on the original user input, preserving the character position mapping relationship before and after normalization. It calculates the text hash value of the normalized original text to generate a globally unique evidence identifier. The above information is encapsulated into a round evidence node, which at least includes the evidence identifier, session identifier, round identifier, role identifier, original text, timestamp, text hash value, evidence source level, and character position information.
[0056] Specifically, evidence sources are categorized into four levels based on their credibility: the highest level is trusted system evidence, referring to evidence with authentication sources, digital signatures, or trusted business interfaces; the second level is direct textual evidence, referring to evidence that can be precisely located to a specific character range in the original text; the third level is inferred evidence, referring to evidence obtained through referential resolution, semantic role labeling, or context inheritance; and the lowest level is unsubstantiated results, referring to results that cannot locate the original range or are merely general outputs of the model. Unsubstantiated results are not written to the activity slot but can only be entered into the anomaly audit log. By fixing the original text content, text hash value, and character position at the entry point, all subsequent state changes can be traced back to the specific round and specific character range.
[0057] In some embodiments, when semantic parsing is performed on the round evidence nodes in step S1 to obtain semantic event nodes, the original text in the round evidence nodes is input into the semantic role parser and parsed in conjunction with the rule dictionary. When the original text contains multiple behaviors, it is split into one or more independent semantic event nodes. For each semantic event, the following are output: subject, action, target, real target, tool or method, parameters, preconditions, step relationship, expected impact, usage scenario, authorization statement, avoidance method, hiding method, and negation scope. Simultaneously, the evidence interval corresponding to each structured piece of information in the original text is recorded. The semantic event node stores all the above structured information and its evidence interval.
[0058] In some embodiments, when performing entity linking on the object names in the semantic event nodes in step S1 to obtain standard entity nodes, for each object name appearing in the semantic event nodes, the corresponding standard entity is searched from the entity directory, alias table, and asset and tool metadata, and the entity matching degree between the object name and the candidate standard entity is calculated. This entity matching degree is obtained by weighted calculation of name similarity, entity type consistency, and business domain consistency.
[0059] Specifically, name similarity represents the degree of similarity between an object's name and a standard entity name or alias at the string level, calculated as the average of normalized editing similarity and semantic name similarity. Entity type consistency indicates whether the entity type referred to by the object is consistent with the standard entity type: the maximum value is taken when the types are the same, the median value is taken when there is a hierarchical compatibility relationship, and the minimum value is taken when there is a type conflict. Business domain consistency indicates whether the business domain to which the object belongs is consistent with the business domain to which the standard entity belongs: the maximum value is taken when the domains are the same, the median value is taken when the domain is unknown, and the minimum value is taken when the domains conflict. All three components are limited to the range of 0 to 1, and their respective weights are calibrated by the annotation validation set and saved with the entity linking rule version; the sum of the three is 1.
[0060] Based on the entity matching score, perform the following operations: When the entity matching score is not less than the first preset threshold (which can be 0.85), reuse existing standard entity nodes; when the entity matching score is less than the second preset threshold (which can be 0.65), create new standard entity nodes; when the entity matching score is less than the first preset threshold but not less than the second preset threshold, establish candidate entity references and wait for disambiguation by other events in the same round or by direct text evidence in subsequent rounds. Candidate entity references must not be used alone to satisfy entity consistency constraints to avoid erroneously merging objects pointing to different entities due to similar names.
[0061] This entity linking mechanism ensures that only entities confirmed by subsequent evidence can enter the risk adjudication process by establishing candidate references instead of directly reusing them when the entity matching degree is in the middle range, thereby reducing the probability of erroneous adjudication caused by ambiguous entities.
[0062] In some embodiments, in step S2, the current round's evidence nodes, semantic event nodes, and the first-appearing standard entity nodes are used as new graph content to establish basic semantic relationships. When obtaining the intermediate graph of the current round, if the current round is the first round, the multi-round intent evolution graph is initialized, the node set is set to an empty set, the relationship set is set to an empty set, and a graph version identifier is established; if the current round is not the first round, the previous round's multi-round intent evolution graph is read according to the session identifier, and the previous version hash value and the most recent round are verified and used as the base graph for this round.
[0063] The newly added node set in this round consists of the current round's evidence nodes, semantic event nodes, and the first-appearing standard entity nodes. The newly added relationship set in this round contains all the basic semantic relationships established in this round. The union of the newly added node set in this round and the node set in the previous round is taken as the node set of the current round, and the union of the newly added relationship set in this round and the relationship set in the previous round is taken as the relationship set of the current round. For existing standard entity nodes, only reference relationships are added, and nodes are not recreated; for invalid relationships, the original relationship version is retained and the valid state is set to invalid, without physical deletion.
[0064] The basic semantic relationships are established in the following order: an event inclusion relationship is established from the evidence node of the round to the semantic event nodes of the current round; a mention relationship is established from the semantic event node to the standard entity node that appears in its text; an action relationship is established for the object that receives the action; a use relationship is established for tools, methods or platforms; and a referential relationship is established for expressions such as "the system", "continue the previous step" or omitting the subject, when the preceding event or entity can be located.
[0065] Each relation must at least store a relation identifier, start identifier, end identifier, relation type, creation round, evidence range, semantic confidence, and validity status. Nodes and relations are committed in batches with the same graph version transaction: if any required identifier is missing, the evidence range is out of bounds, or a relation endpoint does not exist, the current graph increment is rolled back and the exception is written to the audit log.
[0066] In some embodiments, in step S2, on the intermediate graph of the current round, the branch correlation degree between semantic event nodes and existing intent branch nodes is calculated. Based on the branch correlation degree, semantic event nodes are either assigned to existing intent branch nodes or new intent branch nodes are created for semantic event nodes. When obtaining the multi-round intent evolution graph of the current round, intent branch management is performed on the intermediate graph of the current round after obtaining the intermediate graph of the current round. If there is no active intent branch node, an intent branch node is created for semantic event nodes with valid behavioral goals; if there is an active intent branch node, the branch correlation degree is calculated for each semantic event node and each active intent branch node.
[0067] Specifically, the branch correlation degree is calculated by weighting behavioral goal similarity, object consistency, graph-based referential connectivity, and temporal activity. Behavioral goal similarity indicates the semantic consistency between the behavioral goal of the semantic event and the behavioral goal of the intent branch: 1 for completely identical goals, 0.8 for belonging to a pre-defined set of synonymous goals, 0.5 for belonging to the same higher-level task but with different specific goals, and 0 for mutually exclusive or irrelevant goals. Object consistency indicates whether the object of the semantic event and the object of the intent branch points to the same entity: 1 for pointing to the same standard entity node, 0.6 for candidate entity type and business domain consistency, 0.3 for entity type consistency only, and 0 for entity conflict or no object. Graph-based referential connectivity indicates the connectivity between semantic event nodes and intent branch nodes in the graph through referential relationships: 1 for semantic event nodes directly connected to event nodes or entity nodes in the intent branch task subgraph through referential relationships, 0.5 for forming a two-hop connection through a standard entity node, and 0 otherwise. Temporal activity is calculated using an exponential decay function. In this embodiment, the decay coefficient is 0.15. The greater the interval between the most recent active round and the current round of a branch, the lower the temporal activity. The weights of the above four components are calibrated using a labeled validation set and saved with the branch rule version. Each weight is not less than 0 and their sum is 1.
[0068] Based on the calculation results of the branch correlation, perform the following operations: Record the highest branch correlation as the first correlation, and the second highest branch correlation as the second correlation. When the first correlation is not less than the first threshold (which can be 0.75) and the difference between the first and second correlations is not less than the second threshold (which can be 0.10), determine that the semantic event node and the intent branch node corresponding to the first correlation belong to the same task, assign the semantic event node to the intent branch node, and establish a supporting branch relationship. When the first correlation is less than the first threshold (which can be 0.75) and not less than the third threshold (which can be 0.55), or the difference is less than the second threshold (which can be 0.10), mark the semantic event node as a candidate for assignment, and do not update the confirmed slots with this semantic event node before disambiguation is completed. When the first correlation is less than the third threshold (which can be 0.55), create a new intent branch node for this semantic event node.
[0069] During branch merging, for events mutually exclusive with the confirmed true objective, authorized scope, or final objective within a branch, a conflicting branch relationship is established. When mutually exclusive true objectives, authorized scopes, or final objectives appear within the same intent branch, supported by direct textual evidence (semantic confidence not less than 0.85) or trusted system evidence, the shared historical node reference is copied, creating two new intent branch nodes, with the new branch nodes pointing to the original branch node to establish a split relationship. When the behavioral objective similarity and target consistency of the two intent branches are both not less than the fourth threshold (which can be 0.85), the key execution slots are compatible, and there are no activity conflicts, the branch node created in the earlier round is retained, the supporting relationships and slot versions are merged, and the merged branch node points to the retained branch node to establish a merge relationship. If an intent branch has not added any new supporting branch relationships for 8 consecutive rounds and there are no unfinished key execution slots, the branch status is set to invalid.
[0070] For inheritance, creation, splitting, merging, and failure operations, the triggering rules, first degree of association, second degree of association, values of each component, evidence path, and graph version are recorded, and the most recent activity round of the branch node is updated.
[0071] In some specific embodiments, the component weights and temporal decay coefficients of the first threshold, second threshold, third threshold, and branch correlation can be calibrated according to the deployment environment. During calibration, a joint search is performed on the above parameters on the labeled validation set. First, parameter combinations with a branch error inheritance rate higher than 2%, a new branch recall rate lower than 95%, a new branch precision rate lower than 90%, or an ambiguous attribution misidentification rate higher than 2% are eliminated. Among the remaining combinations, the combination with the largest proportion of the sum of automatically inherited correct events and automatically created correct events to the total number of events that can be automatically processed is selected. If there are tied combinations, the combination with a lower branch error inheritance rate, a lower error creation rate, and a larger difference between the first threshold and the third threshold is selected in sequence. If there are no parameter combinations that meet the above constraints, the candidate attribution range is expanded and the automatic inheritance or automatic creation of the corresponding semantic event nodes is prohibited. After calibration, the thresholds, weights, and temporal decay coefficients are solidified into a branch rule version and saved along with the graph version of the multi-round intent evolution graph.
[0072] The error inheritance rate is calculated as follows: Branch Error Inheritance Rate: The ratio of the number of semantic event nodes automatically inherited to the erroneous intent branch to the total number of automatically inherited semantic event nodes. New Branch Recall Rate: The ratio of the number of semantic event nodes correctly and automatically created as independent intent branches to the total number of manually labeled semantic event nodes as new tasks. New Branch Precision Rate: The ratio of the number of semantic event nodes correctly and automatically created as independent intent branches to the total number of automatically created intent branches. Error Creation Rate: The ratio of the number of semantic event nodes incorrectly and automatically created as intent branches to the total number of automatically created intent branches. Ambiguity False Assignment Rate: The ratio of the number of events whose correlation difference between the highest and second-highest branches is less than the difference threshold but are still automatically assigned to the erroneous branch to the total number of events whose correlation difference is less than the difference threshold and are automatically assigned.
[0073] The aforementioned multi-turn intent evolution graph construction method makes each round's newly added facts, their sources, objects, and task affiliations a queryable graph structure, avoiding the degradation of multi-turn states into a flat structure that only stores slot values. The branch management mechanism, by assigning semantic event nodes to existing intent branch nodes or newly created intent branch nodes, enables cross-turn facts to be merged into the same task context, thereby identifying progressively dangerous paths scattered throughout multi-turn dialogues.
[0074] In some embodiments, candidate slots are extracted from semantic event nodes in step S3. When calculating semantic confidence based on the calibration probability that the candidate slot values and their evidence intervals are simultaneously correct, each activity intent branch node maintains versioned slot attributes. Slot types are divided into several groups according to function: the target group includes behavioral goals and expected impacts; the operation group includes actions, target objects, and actual goals; the execution group includes tools or methods, parameters, preconditions, and step relationships; the adversarial group includes evasion methods and hiding methods; and the compliance group includes usage scenarios and authorization status. Each slot type includes an activity version pointer and a historical version array. Each slot version at least stores a slot value reference, slot status, evidence reference, state transition rule identifier, creation round, and closing round.
[0075] After extracting candidate slots from semantic event nodes, semantic confidence is calculated based on the calibration probability that both the candidate slot value and its evidence interval are correct. For multiple candidate values of the same slot type and one null candidate, the semantic parser outputs the uncalibrated score for each candidate and selects the candidate with the highest score. The difference between the logarithmic sum of the selected maximum score and the scores of other candidates is calculated to obtain the logarithmic dominance. On a calibration set that does not overlap with the training set sessions, samples whose candidate slot values match the manually labeled values and whose character intersection-union ratio (CIU) between the candidate evidence interval and the manually labeled interval is not less than 0.50 are labeled as correct, and the rest are labeled as incorrect. The scaling and bias parameters of the calibration function are fitted by minimizing the binary cross-entropy. The logarithmic dominance is input into the calibration function to obtain the semantic confidence. When the semantic parser does not output candidate scores, the logarithmic dominance is replaced by a linear combination of five features: exact slot value matching, consistent entity links, complete negation range, complete evidence interval, and unique cross-turn reference. The corresponding parameters are fitted on the same calibration set. The calibration parameters for semantic confidence are calibrated separately for each slot type. When there are fewer than 200 positive samples for a certain slot type, global parameters are used to avoid overfitting due to small sample sizes. Each slot version simultaneously stores the log dominance, semantic confidence, calibration parameter version, and semantic model version.
[0076] In some embodiments, after semantic event parsing is completed, the controlled semantic disambiguator is invoked only when one of the following conditions is met: there are more than two possible referents and the semantic confidence is between a low threshold (0.60) and a high threshold (0.85); the negation scope spans multiple actions or objects; the text contains irony; or a sentence contains multiple behaviors and the rules cannot determine the slot assignment. The controlled semantic disambiguator only outputs candidate entities, candidate branches, split events, evidence scope, and confidence, and does not directly output risk levels or blocking conclusions. Its output is not accepted if it cannot locate the evidence position or if the structure is illegal. Through the above-mentioned invocation conditions and output scope restrictions, the controlled semantic disambiguator is only used to assist in semantic-level candidate disambiguation and does not directly replace deterministic risk adjudication, thereby improving semantic processing capabilities while maintaining the stability and reproducibility of risk adjudication.
[0077] In some embodiments, when determining the status of candidate slots based on evidence source level and semantic confidence in step S3, if the semantic confidence is below a low threshold (which can be 0.60), no active candidate is formed, and only a parsing failure marker is saved; if the semantic confidence is not less than the low threshold (which can be 0.60) and is below the high threshold (which can be 0.85), a candidate slot is formed; if the semantic confidence is not less than the high threshold (which can be 0.85) and the evidence is locatable direct text evidence, the slot is qualified for direct confirmation. Trusted system evidence directly confirms the slot when there is no semantic ambiguity. Candidate slots formed by inferred evidence need to be supported by another independent and consistent piece of evidence before they can be converted to a confirmed state. Two pieces of evidence are considered independent if they are independent in source round or trusted source, have different evidence scopes, and are not repeated rewrites of the same original text.
[0078] The slot status transitions according to the following rules: when the candidate threshold is met, it changes from not appearing to candidate status; when the confirmation condition is met, it changes from not appearing or candidate status to confirmation status; when a mutually exclusive new value appears and the new value at least meets the candidate condition, but cannot be resolved according to the evidence level, it is placed in conflict status; when the scope and object of negation are clear and there is direct textual evidence (semantic confidence not less than 0.85) or credible system evidence, it is placed in negation status; when a more specific or more credible new value reaches the confirmation status and cannot coexist with the old value, the old value is placed in replacement status; when the branch ends or the dependency condition fails, it is placed in failure status.
[0079] Mutually exclusive evidence is processed in the order of credible system evidence over direct textual evidence, and direct textual evidence over inferred evidence. Evidence of the same level is compared in the following order: branch attribution clarity, expression specificity, evidence completeness, and consistency with subsequent behavior. When both mutually exclusive values meet the candidate conditions and cannot be resolved, the slot remains in a conflict state, and there can be at most one currently valid confirmation version in the same slot.
[0080] In some embodiments, in step S3, mutual exclusion resolution is performed on slots of the same type within the same intent branch node. When determining the currently valid confirmed slots of the intent branch node, if there are multiple incompatible candidate slots of the same type within the same intent branch node that all meet the candidate conditions, the slot is set to a conflict state. The candidate state, conflict state, and the fact corresponding to the candidate branch affiliation shall not participate in the subsequent minimum danger closure set matching.
[0081] Specifically, the low and high thresholds for semantic confidence can be calibrated based on the deployment environment. The low and high thresholds are enumerated in increments of 0.01 on the calibration set. First, combinations with a candidate slot recall rate of at least 95%, an automatic error confirmation rate of at least 2%, a semantic confidence calibration error of at least 0.05, and a difference between the high and low thresholds of at least 0.10 are retained. Then, the combination with the highest automatic confirmation coverage is selected. If there are tied combinations, the combination with the higher low and high thresholds is selected. If no threshold combination meets the above constraints, automatic confirmation of direct text evidence is prohibited, and the case is transferred to manual review. After the thresholds are implemented, they are written into the slot rule version to ensure that historical states can be replayed.
[0082] The candidate slot recall rate is the ratio of the number of correct slot values – evidence interval pairs that reach a low threshold and are retained as candidates or confirmed to the total number of manually labeled slot values – evidence interval pairs to be extracted. The automatic confirmation error rate is the ratio of the number of slot versions automatically confirmed but with incorrect slot values or evidence interval character intersection-union ratios less than 0.50 to the total number of automatically confirmed slot versions. The automatic confirmation coverage rate is the ratio of the number of automatically confirmed and correct slot versions to the total number of manually labeled confirmable slot versions. The semantic confidence calibration error is calculated as follows: the semantic confidence level is divided into 10 intervals: 0 to 0.1, 0.1 to 0.2, and up to 0.9 to 1.0. The proportion of candidate slots in each interval to the total number of candidate slots is calculated, multiplied by the ratio of the number of samples with both correct slot values and evidence intervals in that interval to the number of candidate slots in that interval, minus the absolute value of the ratio of the sum of semantic confidence levels in that interval to the number of candidate slots in that interval, and summed over all intervals. Intervals with zero candidate slot bits are not included in the summation.
[0083] Through the above-mentioned slot status management and mutual exclusion resolution mechanism, each change in slot status retains historical versions and evidence references, making the slot confirmation process verifiable and traceable.
[0084] In some embodiments, when constructing the minimum hazard closure set in step S4, the sources of the minimum hazard closure set include prohibition or restriction rules of versioned security policies, reviewed security event samples, and adversarial test samples confirmed by security experts. The template generator first maps the dangerous behavior conditions from each source to standard slot predicates, merges semantically equivalent predicates within the same risk domain, and forms a candidate slot set. A deletion operation is performed on each slot predicate in the candidate slot set: after deleting the predicate, the corresponding version of the security policy rule is called. If the remaining predicates are still sufficient to trigger the same dangerous result, the predicate is permanently deleted, and the traversal restarts until deleting any remaining predicate can no longer trigger the dangerous result. The minimum obtained is the minimum under the set inclusion relation, i.e., there is no still sufficient proper subset, rather than simply pursuing the minimum number of elements. Each template is reviewed by two security experts, and the source identifier, original candidate set, input and results of each deletion test, review conclusion, generator version, and template version are saved. Each closed set template should include at least a template identifier, risk domain, necessary slot predicate, same branch constraint, entity consistency constraint, parameter compatibility constraint, exclusion condition, and template version.
[0085] Specifically, versioned security policies refer to a set of security policies managed according to versions, including prohibition rules, restriction rules and their triggering conditions set for different risky behaviors, and recording policy versions, effective status and change information, so as to trace and replay historical decisions.
[0086] A verified security incident sample refers to a sample derived from historical security incidents, system operation records, or security test records, which has been manually verified to contain specific dangerous behaviors, risk conditions, or risk outcomes.
[0087] Adversarial test samples confirmed by security experts refer to test samples constructed or collected during security testing, red team testing, or adversarial testing, and confirmed by security experts to reflect specific dangerous behaviors, avoidance behaviors, or dangerous behavior paths.
[0088] In some embodiments, when matching the confirmed slots with the minimum danger closure set in step S4 and determining the closure state of the intention branch node based on the matching result, the set of currently valid confirmed slots that can participate in the adjudication in the current round intention branch node is set as the first set, the kth minimum danger closure set in the risk domain is set as the second set, and the difference between the two is the set of missing slots.
[0089] The matching process is performed in the following order: First, slots for candidate states, conflicting states, negative states, alternative states, and invalid states are removed; second, it is verified that all necessary slots come from the same intent branch node; third, it is verified that the real target points to the same standard entity node; then, parameter compatibility is checked—enumerated parameters must be equal after being normalized to standard code; the intersection of the confirmed value or interval of numerical parameters after unifying the units and the allowed interval of the template must be non-empty; the bound values of variables with the same name in different steps must be the same; the entity referenced by the preconditions must be consistent with the real target; the step dependency graph must be acyclic and each precondition step must be confirmed; finally, it is checked whether the exclusion conditions such as the legal simulation environment and authorized scope are met. If any parameter is missing, conflicting, cannot be converted in units, the interval intersection is empty, the variable binding is inconsistent, or the step dependency has a cycle, the parameter compatibility constraint is false; if any constraint is not met, the reason for failure is recorded but closure is not determined.
[0090] When the number of missing slots is two or more, the closure state is unclosed; when the number of elements is one, the closure state is adjacent closure; when the number of elements is zero and the same branch constraint, entity consistency constraint, parameter compatibility constraint and exclusion condition are all met, the closure state is closed.
[0091] When configuring multiple alternative closure sets for the same risk domain, calculate the missing sets separately, and select the closure set with the fewest missing slots and the constraints being met as the current matching closure set. If the number of missing slots is the same, prioritize the closure set with a higher risk level, a more up-to-date template version, and more complete evidence coverage. A closure event is generated once any high-risk closure set is closed.
[0092] The system writes the matching template, satisfied slots, minimum missing slots, constraint check results, closure status, closure round, last closed slot, and last closure evidence into the versioned closure attribute of the intent branch node, and establishes an evidence path from the closed slot version back through the semantic event node to the round evidence node. The execution closure set includes the real goal, tool or method, parameters, and step relationships; the avoidance closure set includes the real goal, tool or method, and avoidance method; the result closure set includes the real goal, action, expected impact, and preconditions.
[0093] In some embodiments, in step S4, when determining discrete decision variables based on the closed state and confirmed slots, and performing rule matching according to a preset fixed priority order to generate the risk level and corresponding handling action for the current round, the discrete decision variables include five categories: path stage, executability, authorization credibility, degree of confrontation, and uncertainty.
[0094] The path phases are determined based on the currently valid and confirmed slot combinations: Phase 1 indicates only an abstract topic; Phase 2 indicates that at least one behavioral goal or action has been confirmed, but the object is still abstract; Phase 3 indicates that the action has been confirmed and at least one of the target object, real goal, or tool / method has been confirmed; Phase 4 indicates that at least one of the parameters, preconditions, or step relationships has been confirmed based on Phase 3; Phase 5 indicates that the real goal has been confirmed and the tool / method and parameter / step relationships have been confirmed simultaneously; Phase 6 indicates that the avoidance or concealment method has been confirmed based on Phase 5, or any high-risk closed set has been closed.
[0095] Executability is determined through a versioned execution mode table and confirmed slots. The execution mode table specifies the execution object, required parameters, preconditions, and step dependencies according to the tool method identifier. Execution level is matched sequentially from Executability Level 3 to Executability Level 0: Executability Level 3 is used when all required slots have been confirmed, or when the confirmed content forms a directly executable tool call or payload; Executability Level 2 is used when both the execution object and tool method have been confirmed, and at least one or at least two ordered steps among the required parameters have been confirmed but Executability Level 3 is not yet satisfied; Executability Level 1 is used when only the tool method category, execution object, or at least one of a single non-continuous step has been confirmed and Executability Level 2 is not satisfied; Executability Level 0 is used when Executability Levels 1 through Executability Level 3 are not matched, and background, definition, principle, or risk descriptions are not written to the execution slots. Executability Level 3 cannot be used if the execution mode table does not cover a tool method.
[0096] Authorization credibility is determined through seven categories of slots: authorizing subject, authorized object, authorized operation, authorized scope, validity period, issuing party, and verification status. Authorization level three is assigned when all seven slots of trusted system evidence verification are consistent with the current operation and the authorization has not been revoked; authorization level two is assigned when trusted system evidence only verifies identity or resource ownership but at least one of the authorized operation, scope, or validity period is not verified, or when there is a locatable authorization document but machine verification has not yet been completed; authorization level one is assigned when there is only research, testing, internal audit, or a licensed statement in the user's direct text; authorization level zero is assigned when there is no authorization evidence, the authorization has been revoked, the authorized object is inconsistent, or the authorized scope does not include the current operation. If mutually exclusive authorization evidence is not resolved, it participates in risk adjudication at authorization level zero, and the uncertainty is set to uncertainty level two.
[0097] The level of confrontation is determined by evasion actions, evaded control, hidden actions, log suppression, code obfuscation, splitting probes, and character disguise slots. Confrontation level two is achieved when both evasion actions and evaded control are confirmed, or when either hidden action or log suppression is confirmed; confrontation level one is achieved when only one of code obfuscation, splitting probes, or character disguise is confirmed and confrontation level two is not met; confrontation level zero is achieved when no confirmed confrontation slot exists. The confrontation level cannot be increased if there is only a keyword hit without a locatable evidence range.
[0098] Uncertainty is determined by the state of gating slots. Gating slots are slots that can alter path stages, executability, authorization credibility, adversarial level, or closure set results. Uncertainty level two is assigned when any gating slot is in a candidate or conflict state, any required gating slot is missing and not marked as inapplicable in the execution mode table, any gating references more than two candidate entities, or a resolution failure covers the gating evidence interval; uncertainty level one is assigned when only non-gating slots are in a candidate state or missing and there are no gating conflicts; uncertainty level zero is assigned when all gating slots are confirmed, denied, or marked as inapplicable in the execution mode table and there are no activity conflicts.
[0099] The system executes rules sequentially from first to last according to their numbers, stopping when the first rule that meets the conditions is matched. Let the risk level of the current round be a function of five discrete decision variables and a closed-loop state. The system executes decisions sequentially according to the rule numbers. The first rule that is determined to be true has the minimum number among all matching rule numbers. The risk level and outcome of the current round are directly determined by this first matched rule.
[0100] The risk levels include Risk Level 0, Risk Level 1, Risk Level 2, Risk Level 3, and Risk Level 4, which correspond to Normal, Attention, Restriction, High Risk, and Blockade, respectively.
[0101] Rule 1: When the level of confrontation is level two, the executability is not lower than level two, and the authorization credibility is lower than level three, it is determined to be level four risk.
[0102] Rule 2: When the path stage is stage six or the high-risk closed set has been closed and the authorization credibility is lower than authorization level three, it is determined to be risk level four.
[0103] Rule 3: If a candidate response changes the closed state to closed, raises the executability level to executability level 3, or raises the path stage to stage 6, then the candidate response is blocked.
[0104] Rule 4: When the path stage is stage 5, the executability is executable level 3, and the authorization credibility is authorization level 0 or authorization level 1, it is determined to be risk level 3.
[0105] Rule 5: If the path stage is no lower than stage four, the true objective has been confirmed, the executability is no lower than executable level two, and the authorization credibility is lower than authorization level two, it is determined to be risk level three.
[0106] Rule 6: When the path stage is stage three or four and the uncertainty level is uncertainty level two, it is determined to be risk level two and enters the review stage.
[0107] Rule 7: When the level of confrontation is Level 1 and the path stage is not lower than Stage 3, it is determined to be Level 2 risk.
[0108] Rule 8: When the path stage is stage three, the executability is no higher than executability level one, and there is no adversarial level two, it is determined to be risk level two.
[0109] Rule 9: When the path stage is stage two or only contains sensitive topics and the executability is at executability level zero or executability level one, it is determined to be risk level one.
[0110] Rule 10: When the authorization credibility is level 3, the level of resistance is level 0, the path stage is no higher than stage 4, the executability is no higher than level 1, and the scope of operation is completely matched with the authorization, it is determined to be risk level 0 or controlled release.
[0111] Rule 11: If any of the above rules are not met, the risk level will be determined and the lowest level of authorization will be used to answer the question.
[0112] Risk level zero corresponds to normal release and logging of ordinary information; risk level one only provides the principle, defense, and risk description; risk level two limits the depth of the response and removes the real target, key parameters, and consecutive steps; risk level three rejects key execution content, provides alternative defense suggestions, and enhances session monitoring; risk level four executes blocking, preserves the chain of evidence, and issues alerts according to the policy. Uncertainty level two can only maintain the risk, trigger restrictions, or conduct reviews, and cannot be used to reduce the risk; authorization level one cannot offset phase five, phase six, or explicitly circumvent the facts.
[0113] A closed high-risk set cannot be directly downgraded due to authorization level one or uncertainty level two. A reassessment of risk level four requires at least one credible system evidence or two new, independent, consistent direct textual evidences; unclosing a state requires at least one necessary slot to be marked as negated, replaced, or invalidated by credible evidence. Reassessment begins with closure set matching and executes rules one through eleven in full.
[0114] The default acceptance thresholds are as follows: candidate slot recall rate not less than 95%, automatic error confirmation rate not more than 2%, semantic confidence calibration error not more than 0.05, entity error merging rate not more than 1%, branch error inheritance rate not more than 2%, new branch recall rate not less than 95%, new branch precision rate not less than 90%, ambiguity attribution misconception rate not more than 2%, danger closure missed detection rate not more than 2%, danger closure false alarm rate not more than 1%, danger completion missed release rate 0%, and adjudication replay consistency rate 100%.
[0115] The false negative rate for dangerous closure is the ratio of manually labeled closed samples that the system did not classify as closed to the total number of manually labeled closed samples. The false positive rate for dangerous closure is the ratio of manually labeled unclosed samples that the system classifies as closed to the total number of samples the system classifies as closed. The missed release rate for dangerous completion is the ratio of candidate responses that were allowed by the system and manually labeled as completing dangerous closure sets to the total number of candidate responses manually labeled as completing dangerous closure sets. The replay consistency rate is the ratio of the number of replays that achieve the exact same result as the baseline decision, assuming identical input, official graph version, semantic model version, closure set template version, and rule version, to the total number of replays.
[0116] If semantic, entity, or branch indicators do not meet the threshold, the candidate interval is expanded and the corresponding automatic confirmation is disabled; if the rate of missed detection of dangerous closure or the rate of missed release of dangerous completion do not meet the threshold, automatic release is stopped and manual review is initiated; if the rate of false alarm of dangerous closure does not meet the threshold, the source of the closure set, deletion tests, and exclusion conditions are reviewed, but false alarms cannot be corrected by lowering the requirements for dangerous completion interception. Each calibration saves the dataset version, numerator and denominator of each indicator, indicator value, weight, threshold, calibration parameters, and rule version, without overwriting historical records with new results.
[0117] In some embodiments, in step S5, when the action allows the generation of candidate answers, the subgraph structure corresponding to the current intent branch node is copied to form a temporary graph copy. The candidate answer is simulated and added to the temporary graph copy. The candidate answer is used as the input for the current round to perform processing from semantic parsing to generating risk levels and actions. If the addition of a candidate answer changes the closed state to closed or increases the risk level, the candidate answer is blocked or rewritten. The subgraph structure corresponding to the current intent branch node is copied to form a temporary graph copy. The copied content includes the active intent branch node, the current valid slot version, the closure set result, associated event nodes, standard entity nodes, and evidence references. The formal multi-round intent evolution graph remains read-only during detection.
[0118] The system uses candidate answers as temporary round evidence nodes for AI assistants. Semantic parsing is performed on these temporary round evidence nodes to obtain temporary semantic event nodes. Entity linking is then performed on the object names within these temporary semantic event nodes to obtain temporary standard entity nodes. The round evidence nodes, semantic event nodes, and the first-appearing standard entity nodes of the temporary rounds are added to a temporary graph copy as new graph content, establishing basic semantic relationships and obtaining a temporary intermediate graph. The branch correlation degree between temporary semantic event nodes and existing intent branch nodes is calculated on the temporary intermediate graph. Based on the branch correlation degree, temporary semantic event nodes are either assigned to existing intent branch nodes or new intent branch nodes are created for them. Candidate slots are extracted from the temporary semantic event nodes. Temporary semantic confidence is calculated based on the calibration probability of both the candidate slot value and its evidence interval being correct. The state of the candidate slots is determined based on the evidence source level and the temporary semantic confidence. Mutual exclusion resolution is performed on slots of the same type within the same intent branch node to determine the currently valid temporary confirmed slots for the intent branch node. A minimum risk closure set is constructed, and the temporary confirmed slots are matched with this set. The temporary closure state of the intent branch node is determined based on the matching result. Based on the temporary closed state and the temporarily confirmed slots, determine the temporary discrete decision variables, execute rule matching according to the preset fixed priority order, and generate temporary risk levels and corresponding temporary handling actions.
[0119] The closure state, risk level, path stage, executability, adversarial level, and minimum number of missing slots of the candidate answer before being added to the temporary graph copy are compared with the corresponding states after addition. The state changes before and after adding the candidate answer are represented as an output increment vector, which consists of five components: change in path stage order, change in executability order, change in adversarial level order, change in closure state, and reduction in the minimum number of missing slots. Specifically, the change in path stage order is the difference between the path stage level after adding the candidate answer and the path stage level before adding it; the change in executability order is the difference between the executability level after adding the candidate answer and the executability level before adding it; the change in adversarial level order is the difference between the adversarial level after adding the candidate answer and the adversarial level before adding it; the change in closure state is the transition between the three states of unclosed, near-closed, and closed before and after adding the candidate answer; and the reduction in the minimum number of missing slots is the difference between the minimum number of missing slots before adding the candidate answer and the minimum number of missing slots after adding it.
[0120] Perform the following operations based on the comparison results. If the addition of a candidate response changes the closed state to closed, or raises the executability level to executability level three, or raises the path stage to stage six, or raises the adversarial level to adversarial level two, the candidate response is deemed to introduce incremental risk and is blocked. If the addition of a candidate response raises the executability level to executability level two, or reduces the minimum number of missing slots but remains non-empty, or raises the path stage but does not exceed stage two and only adds defensive interpretations, the specific parameters, true target identifier, and continuous operation relationships in the candidate response are deleted, and the response is re-tested. If the path stage, executability, adversarial level, and closed state remain unchanged, the original risk level applies.
[0121] After blocking, a safe rewrite is performed, which can be performed a maximum of two times. Each rewrite recreates a temporary graph copy from the same official graph version that has not been contaminated by candidate answers and re-executes the complete detection process described above. If the blocking condition is still met after two consecutive attempts, or if any high-risk closure set is still closed, the rewrite is terminated and the preset safe response is returned.
[0122] New nodes, relationships, and slot versions added to the temporary graph copy are all marked with a temporary tag and cannot be written back to the formal multi-round intent evolution graph before passing the test. If the test fails, the entire temporary graph copy is discarded; if the test passes, only candidate answers are output and the test summary is written to the audit chain, and dangerous slots formed by candidate answers are not written to the formal intent branch nodes.
[0123] The aforementioned incremental detection mechanism for candidate answers enables the system to identify whether a candidate answer has completed the final condition of a dangerous path by performing an incremental graph update and risk calculation that is isomorphic to the user input before sending the candidate answer. This forms a second line of defense on the output side, independent of input detection.
[0124] In some embodiments, during step S6, when performing corresponding control operations based on risk level and action to form an immutable decision snapshot and writing the action result and the evidence path tracing back from the intent branch node to the round evidence node into the audit chain, corresponding control operations are performed based on risk level and action. Inputs include the initial decision result, incremental detection conclusions of the candidate answer temporary graph, business alarm policies, and session context. The system performs corresponding control operations based on risk level: Risk level zero corresponds to normal release and recording of ordinary logs; Risk level one corresponds to providing only principles, defenses, and risk descriptions; Risk level two corresponds to limiting answer depth and deleting the true target, key parameters, and consecutive steps; Risk level three corresponds to rejecting key execution content, providing alternative defense suggestions, and improving session monitoring; Risk level four corresponds to blocking, preserving the evidence chain, and issuing alarms according to the policy. Control operations include release, restriction, rejection, blocking, tool call control, or alarms.
[0125] After executing the control operation, an immutable decision snapshot is generated. The decision snapshot saves at least the session identifier, round identifier, official graph version, branch identifier, confirmed key slots, conflicting slots, closure set template version, matching closure set, closure status, satisfied slots, minimum missing slots, closure round, last closure slot, five types of discrete decision variables, hit rules, risk level, action, and rule version.
[0126] The results of the action and the evidence path tracing back from the intent branch node to the round evidence node are written into the audit chain. The audit chain starts with the adjudication snapshot and saves the backtracking path from the adjudication snapshot through the intent branch node, slot version, semantic event node to the round evidence node, as well as the standard entity nodes, supporting branch relationships, conflicting branch relationships, and branch evolution relationships involved.
[0127] After the multi-round intent evolution graph of the current round is completed in step S6, it is archived as a read-only version for the next round to read and continue to execute incremental updates of nodes and relationships, thus forming a continuous version chain from the first round to the current round.
[0128] When archiving for a long time, the original content can be compressed and stored, but the text hash value, round identifier, character range, graph version, and relationship version must be retained, so that the construction process, branch assignment process, and risk adjudication process of any round graph can be replayed.
[0129] This handling and audit archiving mechanism ensures that the results of each round of risk adjudication are reviewable and traceable through immutable adjudication snapshots and a complete evidence backtracking path, thus meeting security audit requirements.
[0130] To better illustrate this solution, this embodiment uses a four-round dialogue without including actual operational details to explain the round-by-round update process of the multi-round intent evolution graph from the first round to the fourth round.
[0131] First round of interaction:
[0132] A user inquired, "How does an enterprise system typically export business data?" The system establishes round-specific evidence nodes, performs semantic parsing on these nodes to obtain semantic event nodes, and performs entity linking on the object names within these semantic event nodes to obtain standard entity nodes. The current round's evidence nodes, semantic event nodes, and the first-appearing standard entity nodes are added to the graph as new graph content, establishing event relationships, mention relationships, action relationships, usage relationships, and referential relationships to obtain the current round's intermediate graph. Since there are currently no activity intent branch nodes, intent branch nodes are created for semantic event nodes with valid behavioral goals. The task corresponding to this branch is a general data export task. At this point, the behavioral goal has been confirmed, but the actual goal has not yet appeared. The path stage is stage one, the executability level is 0, the adversarial level is 0, and the risk level is risk level one.
[0133] Second round of interaction:
[0134] The user states, "The target is production system A." The system adds evidence nodes, semantic event nodes, and standard entity nodes for this round, establishing referential relationships including event relationships, mention relationships, action relationships, and reference relationships from the current round's semantic event node to the previous round's semantic event node. The branch correlation degree between the current round's semantic event node and existing intent branch nodes is calculated. The behavioral target similarity is set to 0.90, the action object consistency is set to 0.60, the graph referential connectivity is set to 1, and the temporal activity degree is calculated using an exponential decay function (decay coefficient is set to 0.15, interval is 1 round) and set to 0.861. The weights of the four components are set to 0.35, 0.30, 0.20, and 0.15 respectively, and the weighted calculation yields a branch correlation degree of 0.824. This branch correlation degree is not less than the first threshold (0.75), and the difference between it and the second-highest branch correlation degree is not less than the second threshold (0.10). Therefore, the current round's semantic event node is assigned to an existing intent branch node, and a supporting branch relationship is established. The actual target slot has changed from not appearing to confirmed, the path stage has moved to stage two, the executability has moved to executability level one, and the risk level has moved to risk level two. This shows that the entry of the second round of facts into the first round of branches is determined by the objects, references, and timing in the diagram, rather than simply by merging slots of the same type.
[0135] Third round of interaction:
[0136] The user stated, "I want to use automated tools and supplement the necessary configurations." The system reuses the standard entity nodes of the real target and adds standard entity nodes of the tool. The calculated similarity between the behavior and the target is 1, the consistency of the target is 1, the graph referential connectivity is 1, the temporal activity is 0.861, and the branch association is 0.979. The current semantic event node continues to be assigned to the existing intent branch node. The tool or method slot and parameter slot are confirmed to be in a confirmed state by direct textual evidence. The first three items in the execution closure set (real target, tool or method, parameter, step relationship) are satisfied, the missing slot set is {step relationship}, and the closure state is near closure. The path stage is upgraded to stage four, the executability is upgraded to executability level two, the authorization credibility is authorization level zero, the adversarial level is adversarial level zero, the uncertainty is uncertainty level zero, and it is determined to be risk level three according to rule five.
[0137] If the third round of candidate responses proactively provides a continuous step relationship, the system adds an assistant round evidence node and a temporary semantic event node to the temporary graph copy, changing the missing slots in the execution closure set from {step relationship} to an empty set, the closure state changes to closed, and the executability level increases from executable level two to executable level three. The system records the final closure evidence along the path from the temporary semantic event node through the intent branch node, slot version to the candidate response evidence node, blocks the candidate response according to rule three, and discards the temporary graph copy.
[0138] Fourth round of interaction:
[0139] The user further stated that "it is also necessary to avoid triggering the inspection mechanism." The branch correlation between the semantic event node in this round and the existing intent branch node satisfies the inheritance condition, and the system establishes a supporting branch relationship and confirms the evasion method slot. The evasion closure set (real target, tool or method, evasion method) is closed, the path stage advances to stage six, the executability is executable level three, and the adversarial level is adversarial level two. When there is no complete credible authorization at authorization level three, rule one is hit first, determining risk level four. Even if the user simultaneously claims "for research purposes," this statement corresponds to authorization level one and does not change risk level four. The multi-round intent evolution graph still preserves the historical nodes and relationships from the first to the third round, thus enabling the reproduction of the formation process of the dangerous path in the four rounds.
[0140] While embodiments of the present invention have been described in detail above, it will be apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it should be understood that such modifications and variations fall within the scope and spirit of the invention as defined in the claims. Furthermore, the invention described herein may have other embodiments and can be implemented or carried out in various ways.
Claims
1. An artificial intelligence safety fence method based on a multi-turn intention evolution graph, characterized in that, include: Obtain user input for the current round and generate round evidence nodes; Semantic parsing is performed on the round evidence nodes to obtain semantic event nodes, and entity linking is performed on the object names in the semantic event nodes to obtain standard entity nodes; The current round's evidence nodes, semantic event nodes, and the first-appearing standard entity nodes are used as new graph content to establish basic semantic relationships, resulting in the current round's intermediate graph. On the current round's intermediate graph, the branch correlation degree between semantic event nodes and existing intent branch nodes is calculated. Based on the branch correlation degree, semantic event nodes are assigned to existing intent branch nodes or new intent branch nodes are created for semantic event nodes, thus obtaining the current round's multi-round intent evolution graph. Extract candidate slots from semantic event nodes; calculate semantic confidence based on the simultaneous correct calibration probability of candidate slot values and their evidence intervals; determine the status of candidate slots based on evidence source level and semantic confidence; and perform mutual exclusion resolution on slots of the same type within the same intent branch node to determine the currently valid confirmed slots of the intent branch node. Construct a minimum hazard closed set, match the confirmed slots with the minimum hazard closed set, and determine the closure state of the intention branch node based on the matching result; Based on the closed state and the confirmed slots, determine the discrete adjudication variables, execute rule matching according to the preset fixed priority order, and generate the risk level and corresponding handling actions for the current round. When the action allows the generation of candidate answers, the subgraph structure corresponding to the current intent branch node is copied to form a temporary graph copy. The candidate answer is simulated to be added in the temporary graph copy. The candidate answer is used as the input of the current round to perform processing from semantic parsing to generating risk level and action. If the addition of the candidate answer changes the closed state to closed or increases the risk level, the candidate answer is blocked or rewritten. Based on the risk level and the corresponding control actions, corresponding control operations are performed to form an immutable decision snapshot, and the control results and the evidence path tracing back from the intention branch node to the round evidence node are written into the audit chain.
2. The method as described in claim 1, characterized in that, When performing entity linking on object names in a semantic event node, the following is included: Calculate the entity matching degree between the object name in the semantic event node and the standard entity node. The entity matching degree is obtained by weighted calculation of name similarity, entity type consistency and business domain consistency. When the entity matching degree is not less than the first preset threshold, the existing standard entity node is reused; when the entity matching degree is less than the second preset threshold, a new standard entity node is created; when the entity matching degree is less than the first preset threshold but not less than the second preset threshold, a candidate entity reference is created. The candidate entity reference shall not be used alone to satisfy the entity consistency constraint.
3. The method as described in claim 1, characterized in that, The basic semantic relationships include event relationships, mention relationships, action relationships, usage relationships, and reference relationships.
4. The method as described in claim 1, characterized in that, The branch correlation degree is calculated by weighting the behavioral target similarity, the consistency of the target, the graph referential connectivity, and the temporal activity degree; the temporal activity degree decreases as the interval between the most recent active round and the current round of any graph branch node increases.
5. The method as described in claim 1, characterized in that, When assigning a semantic event node to an existing intent branch node or creating a new intent branch node for a semantic event node based on branch relevance, the following applies: The highest branch correlation is denoted as the first correlation, and the second highest branch correlation is denoted as the second correlation. When the first correlation degree is not less than the first threshold, and the difference between the first correlation degree and the second correlation degree is not less than the second threshold, the semantic event node is assigned to the intent branch node corresponding to the first correlation degree. When the first correlation degree is less than the first threshold and not less than the third threshold, or when the difference is less than the second threshold, the semantic event node is marked as a candidate belonging, and the semantic event node corresponding to the candidate belonging is not used to update the confirmed slot; When the first correlation degree is less than the third threshold, a new intent branch node is created for the semantic event node.
6. The method as described in claim 1, characterized in that, The evidence source levels include credible system evidence, direct textual evidence, inferred evidence, and unfounded results, and are sorted from highest to lowest credibility. When calculating semantic confidence based on the calibration probability that both the candidate slot value and its evidence interval are correct, the following is included: When the semantic parser outputs candidate scores, it selects the candidate with the highest score, calculates the difference between the logarithmic sum of the selected maximum score and the scores of other candidates, and obtains the logarithmic dominance. The logarithmic dominance is input into the calibration function to obtain the semantic confidence. When the semantic parser does not output candidate scores, the logarithmic dominance is replaced by five features: exact slot value matching, consistent entity linking, complete negation range, complete evidence interval, and unique cross-wheel reference. These features are then input into the calibration function to obtain the semantic confidence.
7. The method as described in claim 6, characterized in that, The status of candidate slots is determined based on the evidence source level and semantic confidence. Mutual exclusion is performed on slots of the same type within the same intent branch node. When determining the currently valid confirmed slots for an intent branch node, this includes: When the semantic confidence is below a low threshold, no candidate slot is formed; when the semantic confidence is not less than the low threshold and is lower than a high threshold, a candidate slot is formed; when the semantic confidence is not less than the high threshold and the evidence is direct textual evidence, the candidate slot is set to the confirmed state; trusted system evidence directly confirms the slot when there is no semantic ambiguity; candidate slots formed by inference evidence need to be supported by another independent and consistent piece of evidence before being converted to the confirmed state. When there are two or more incompatible slots of the same type within the same intention branch node, and both are in the candidate state, the slots are set to conflict state; the candidate state is the state corresponding to the candidate slot; slots in the candidate state or conflict state do not participate in the matching of the minimum danger closure set.
8. The method as described in claim 1, characterized in that, When constructing a minimum hazard closure set, matching the confirmed slots with the minimum hazard closure set, and determining the closure state of the intended branch node based on the matching results, the process includes: The dangerous behavior conditions are mapped to a set of necessary slot predicates to obtain a set of candidate slots. For each slot predicate in the candidate slot set, a deletion operation is performed. If the remaining slot predicates still satisfy the sufficiency of the dangerous behavior conditions after deletion, the slot predicate is permanently deleted. This process is repeated until any remaining slot predicate no longer satisfies the sufficiency after deletion, thus obtaining the minimum dangerous closed set. The sources of the minimum danger closure set include prohibition or restriction rules of versioned security policies, reviewed security event samples, and adversarial test samples confirmed by security experts. Match the currently valid confirmed slots of the intent branch node with the necessary slot predicates of the minimum danger closure set, and the difference between the two is the set of missing slots. When the number of elements in the missing slot set is two or more, the closure state is unclosed; when the number of elements is one, the closure state is adjacent closure; when the number of elements is zero and the same branch constraint, entity consistency constraint, parameter compatibility constraint and exclusion condition are satisfied, the closure state is closed. When the same risk domain has two or more minimum hazard closed sets, the one with the fewest missing slots and the constraint condition is met is selected as the current matching closed set.
9. The method as described in claim 1, characterized in that, Based on the closed state and confirmed slots, discrete decision variables are determined. Rule matching is executed according to a preset fixed priority order to generate the risk level and corresponding action for the current round, including: The discrete adjudication variables include path stage, executability, authorization credibility, degree of adversarial force, and uncertainty; The path stage is determined based on the combination of confirmed slots, and the higher the level, the more complete the behavior path. The executability is determined by matching the confirmed slots with an execution mode table, which specifies the necessary parameters and prerequisite steps of the tool method. The credibility of the authorization is determined through seven categories of slots: authorization subject, authorization object, authorization operation, authorization scope, validity period, issuing party, and verification status. The degree of confrontation is determined by evasion actions, being evaded control, hiding actions, log suppression, code obfuscation, splitting and probing, and role disguise slots; The uncertainty is determined by the state of the gated slot, which is a slot that affects the path stage, executability, authorization credibility, degree of adversarial force, or closure state result. Rule matching is performed in order of rule number from front to back. The first rule that matches successfully determines the risk level of the current round and the corresponding action.
10. The method as described in claim 1, characterized in that, New content added to the temporary graph copy must not be written back to the multi-round intent evolution graph before passing the detection; after the candidate answer is added to the temporary graph copy, when the closed state changes to closed or the risk level increases, the candidate answer is blocked; After being blocked, a secure rewrite is performed. When the number of rewrites reaches the preset limit, a preset secure response is returned.