Industrial knowledge graph driven large model evaluation agent

Abstract

This invention relates to the field of intelligent evaluation technology, specifically to an intelligent agent for evaluating large models driven by industrial knowledge graphs. The agent includes: an evolutionary scenario generation module, a topology deviation calculation module, an interaction granularity control module, a closure graph construction module, and a homogeneous logic verification module. In this invention, by analyzing the closed-loop path gain attributes of the knowledge graph, an evaluation scenario containing dynamic evolutionary logic is constructed to simulate divergent or convergent states. The topology deviation index is used to monitor the degree of reasoning focus in real time and adaptively adjust the interaction granularity. Strong constraint questions are implemented when there is a logical deviation to lock the reasoning boundary. Transitive closure technology is introduced to transform text responses into graph structures for isomorphic mapping verification, solving the matching problem caused by skip-step reasoning, verifying the logical completeness of key control attributes, and achieving a deep quantitative evaluation of the complex causal reasoning ability and dynamic scenario adaptability of large models.

Description

Technical Field

[0001] This invention relates to the field of intelligent evaluation technology, and in particular to a large-scale model evaluation intelligent agent driven by industrial knowledge graphs. Background Technology

[0002] The field of intelligent evaluation technology involves the use of computer algorithms and data analysis techniques to quantify and grade the performance indicators of specific objects. It mainly includes core aspects such as constructing evaluation indicator systems, executing automated test scripts, data comparison and analysis, and generating multi-dimensional scoring reports. By establishing standardized test benchmarks and automated execution processes, it achieves objective evaluation and quality verification of software systems, algorithm models, or complex engineering objects. Traditional large-scale model evaluation agents refer to automated execution systems that verify the accuracy and logical ability of artificial intelligence language models in specific application scenarios. Traditional methods typically use public datasets or manually compiled question-answer pairs as static test sets. During implementation, test scripts are written to input test questions into the model under test, and the generated text responses are obtained. The n-gram syntactic overlap between the generated text and the standard reference answer is calculated using BLEU or ROUGE algorithms based on word frequency statistics, or regular expressions are used to extract key options from the output content and perform character-level matching with the standard answer to determine the correctness of the model output and calculate the overall pass rate.

[0003] Existing technologies rely on static question-answer pairs as testing benchmarks, which cannot simulate the complex processes of dynamic evolution of physical parameters and causal chain transmission in industrial scenarios. Evaluation methods are limited to text overlap calculation or keyword matching, making it difficult to capture the correctness of deep logical structures. When dealing with complex industrial logic, they exhibit the skip-step reasoning characteristics of human experts (i.e., directly outputting high-dimensional conclusions while omitting basic common sense steps). Traditional evaluation mechanisms based on literal matching or fixed link comparison will crudely judge them as errors or logical incoherence, leading to misjudgments of skip-step reasoning. In addition, the lack of dynamic interactive guidance mechanisms for the reasoning process makes it difficult to explore the true cognitive boundaries through follow-up questions when logic diverges, resulting in serious biases in the evaluation of the model's long-chain reasoning ability and logical stability. Summary of the Invention

[0004] To address the technical problems existing in the prior art, embodiments of the present invention provide a large-model evaluation intelligent agent driven by industrial knowledge graphs. The technical solution is as follows:

[0005] On the one hand, it provides an intelligent agent for evaluating large models driven by industrial knowledge graphs, which includes: The evolution scenario generation module analyzes the knowledge graph to identify closed-loop paths, extracts the gain attributes and key control attributes of the closed-loop paths, accumulates the gain attributes to determine the evolution state, combines the evolution state, key control attributes and perturbations to construct test text, and outputs evolution evaluation scenario data packages. The topology deviation calculation module receives the natural language response stream of the large model for the evolution evaluation scenario data packet and maps it into a set of response nodes. It calculates the shortest path length between the set of response nodes and the closed loop path, and generates a topology deviation index based on the average shortest path length. The interaction granularity control module compares the topology deviation index with the preset deviation tolerance range. When the deviation is within the tolerance range, it generates open follow-up questions. When the deviation exceeds the tolerance range, it constructs closed questions based on the untraversed nodes of the evolution evaluation scenario data packet and encapsulates them into an adaptive interaction instruction set. The closure graph construction module calls the adaptive interaction instruction set to parse the multi-round response text generated by the large model, constructs the answer graph structure, combines the evolutionary evaluation scenario data package, shrinks the non-critical segments of the closed-loop path and retains the mapping to generate a transitive closure graph, and combines them into a logical verification graph pair. The isomorphic logic verification module maps the logic verification graph to the answer graph and then to the transitive closure graph. It verifies the edge correspondence and verifies the existence of key control attributes based on the original path mapping information. Based on the edge correspondence and node existence, it generates a dynamic evolution logic evaluation result.

[0006] As a further aspect of the present invention, the evolutionary evaluation scenario data package specifically includes a closed-loop topology sequence, physical gain parameters, and evolutionary trend labels. The topology deviation indicators include node mapping distance, mean path deviation, and divergence of thought. The adaptive interaction instruction set includes open guidance text, attribute constraint problems, and granularity adjustment strategies. The logic verification graph pair includes a response reasoning subgraph, a transitive closure baseline graph, and a virtual edge mapping table. The dynamic evolutionary logic evaluation results include structural isomorphism score, key node coverage, and logic validity information.

[0007] As a further aspect of the present invention, the evolution scene generation module includes: The closed-loop attribute extraction submodule acquires node connection relationship data in the industrial knowledge graph, performs path traversal, retrieves the path topology where the start node and end node coincide, extracts the physical quantity transmission multiple corresponding to each connected edge in the path as the gain attribute, and uses the identification information of the safety veto right defined on the geometric path node as the key control attribute, and establishes a closed-loop feature mapping table. The evolution state determination submodule calls the closed-loop feature mapping table, extracts the gain attribute of each connected edge for cumulative calculation, obtains the total loop gain coefficient, and determines the evolution state, including divergence and convergence, by comparing it with the unit benchmark value, and generates dynamic evolution trend labels. The scenario data encapsulation submodule sets initial perturbation parameters based on the dynamic evolution trend labels, combines the key control attributes in the closed-loop feature mapping table, constructs evolution prediction test text, and integrates the text with the closed-loop path structure to establish an evolution evaluation scenario data package containing test context and standard topology reference.

[0008] As a further aspect of the present invention, the process of determining the evolution state by comparing it with a unit reference value, including divergence and convergence, specifically includes: Call the historical steady-state operating interval data of each node in the closed-loop path stored in the industrial knowledge graph, calculate the maximum boundary value and minimum boundary value of the historical steady-state operating interval data, set the difference obtained by subtracting the value 1 from the maximum boundary value as the positive tolerance threshold, and set the difference obtained by subtracting the value 1 from the minimum boundary value as the negative tolerance threshold. Set the value 1 as the unit reference value characterizing the conservation of closed-loop energy input and output; Calculate the gain deviation value obtained by subtracting the unit reference value from the total loop gain coefficient; Determine whether the gain deviation value is positive. If the gain deviation value is positive, compare the gain deviation value with the positive tolerance threshold. If the gain deviation value is greater than the positive tolerance threshold, determine that the evolution state is a divergent state. Determine whether the gain deviation value is negative. If the gain deviation value is negative, calculate the absolute value of the gain deviation value and compare the absolute value with the negative tolerance threshold. If the absolute value is greater than the negative tolerance threshold, then determine that the evolution state is a convergent state.

[0009] As a further aspect of the present invention, the topology deviation calculation module includes: The response stream parsing and mapping submodule receives the natural language response stream fed back by the large model for the evolution evaluation scenario data packet, performs semantic decomposition and key entity extraction on the natural language response stream, identifies the industrial domain terms and entity object names contained in the text, compares and matches them with the standard node index in the industrial knowledge graph database, maps and locks the corresponding node unique identifier in the graph according to the matching result, and establishes a response node set. The path distance calculation submodule calls the response node set, unpacks and extracts the closed-loop path topology from the evolution evaluation scenario data packet as a spatial reference benchmark, traverses each independent node element in the response node set, searches for the shortest connected path from each independent node element to the reference benchmark in the graph connection network, and generates a shortest path length sequence. The deviation mean aggregation submodule obtains the shortest path length sequence, quantifies the dispersion of node distribution by performing an arithmetic mean operation on the path lengths, and outputs a topology deviation index.

[0010] As a further aspect of the present invention, the process of comparing and matching with the standard node index in the industrial knowledge graph database, and mapping and locking the corresponding unique identifier of the node in the graph based on the matching result, specifically involves: The thesaurus and standard terminology table preset in the industrial knowledge graph database are retrieved, and the mapping entries in the thesaurus and the standard entries in the standard terminology table are combined to construct the standard node index. Traverse all word pairs with established mapping relationships in the thesaurus, calculate the baseline edit distance similarity between the two word strings in each word pair, perform an arithmetic mean operation on all calculated baseline edit distance similarities, and set the average value as the matching confidence threshold; The industrial terms and entity names are decomposed into character serialization, and the edit distance algorithm is used to calculate the text similarity coefficient between the decomposed character sequence and each index item in the standard node index. The text similarity coefficients corresponding to each character sequence are sorted in descending order, and the coefficient with the first value is selected as the target coefficient. The index item corresponding to the target coefficient is marked as a candidate standard node. Determine whether the target coefficient is not less than the matching confidence threshold, and if the determination result is yes, determine that the industrial domain term or entity object name and the candidate standard node point to the same entity concept; In response to the identification result pointing to the same entity concept, the underlying storage field is accessed through the index pointer of the candidate standard node in the database, the primary key code bound to the candidate standard node is read and returned, and the primary key code is used as the unique identifier of the corresponding node.

[0011] As a further aspect of the present invention, the interactive granularity control module includes: The state logic determination submodule compares the topology deviation index with the preset deviation tolerance range, determines whether the position of the index value on the coordinate axis is within the defined range, determines the current inference focus state, determines the logic focus state when the deviation is within the tolerance range, and determines the logic divergence state when the deviation exceeds the tolerance range, and generates an inference focus state label. The constraint content generation submodule generates open-ended follow-up questions based on the inference focus state label. When the deviation is within the tolerance range, it extracts the closed-loop path from the evolution evaluation scenario data package and identifies untraversed nodes when the deviation exceeds the tolerance range. It then obtains the attribute information of the untraversed nodes and constructs closed-ended strongly constrained question content to generate granular adaptive text. The instruction encapsulation configuration submodule receives the granular adaptive text, performs standardized format verification and special character cleaning on the text, assigns an interaction round sequence number and corresponding priority weight parameters to the cleaned text, encapsulates the text into an instruction template that conforms to the large model input interface specification, and establishes an adaptive interaction instruction set by combining the sequence number, weight parameters and instruction template.

[0012] As a further aspect of the present invention, the process of comparing the topology deviation index with a preset deviation tolerance range, determining whether the index value is located within the predetermined range on the coordinate axis, and determining the current inference focus state, wherein the deviation is within the tolerance range and the process is determined to be a logic focus state, and the deviation is determined to be outside the tolerance range and the process is determined to be a logic divergence state, specifically as follows: The node topology of the closed-loop path is parsed from the evolution evaluation scenario data packet, and an undirected adjacency matrix of the connectivity between nodes in the closed-loop path is constructed. Based on the undirected adjacency matrix, traverse all non-overlapping node pairs in the closed-loop path, calculate the shortest topological path length between each pair of nodes, perform an arithmetic mean operation on all calculated shortest topological path lengths, and generate a structural cohesion coefficient that reflects the tightness of the reference path's own structure. The structural cohesion coefficient is set as a positive critical threshold that allows deviation. Using zero as the starting point, the deviation tolerance range consisting of zero to the positive critical threshold is constructed. The topological deviation index is projected as a one-dimensional coordinate point onto a positive ray coordinate axis with the value of zero as the origin, and the cutoff point of the positive critical threshold on the coordinate axis is determined. Determine whether the coordinate point is located within the closed interval formed by the origin and the cutoff point; If the coordinate point is located within the closed interval, the reasoning process is considered to be in the effective control domain, and the reasoning focus state is determined to be the logical focus state. If the coordinate point is located on the positive outer side of the closed interval, it is determined that a structural drift has occurred in the reasoning process, and the reasoning focus state is determined to be a logical divergence state.

[0013] As a further aspect of the present invention, the closure graph construction module includes: The answer graph construction submodule calls the adaptive interaction instruction set as the context constraint benchmark for semantic parsing, obtains multi-turn response text data generated by the large model under test, performs entity relationship extraction operation, identifies causal logic chains and entity objects in the text, converts the identified entity objects into graph nodes and causal logic into directed connection edges, and combines nodes and edges according to the dialogue sequence to establish the answer reasoning graph structure. The transitive closure generation submodule extracts closed-loop path topology data from the evolution evaluation scenario data package, traverses each node in the path, identifies non-critical connection segments that do not contain key control attributes, performs topology shrinkage operation, removes intermediate nodes and establishes direct connection relationships between the first and last nodes of the segments, embeds the index information of the original path into the new connection relationships, and generates a transitive closure standard graph. The logic verification combination submodule obtains the answer reasoning graph structure and the transitive closure standard graph, aligns the data formats, unifies the node encoding rules and coordinate space reference system of the two, constructs a logic verification container containing two-layer graph data, fills the aligned reasoning graph structure into the actual layer of the container and fills the standard graph structure into the base layer of the container, and establishes a logic verification graph pair.

[0014] As a further aspect of the present invention, the isomorphic logic verification module includes: The structural feature mapping submodule deconstructs the logical verification graph pair and extracts the answer reasoning graph structure and the transitive closure standard graph. It traverses the bilateral graph data, obtains the node attribute identifier and topological adjacency matrix features, performs feature vector comparison, establishes a one-way mapping link from the answer node to the standard node, verifies the topological connection relationship between nodes and corrects the positional deviation, and generates a graph isomorphic mapping index that records the alignment relationship between the corresponding state and structure of the node. The path attribute verification submodule, based on the graph isomorphic mapping index, traverses and retrieves the connecting edges in the answer reasoning graph structure, performs type matching between the retrieved connecting edges and the original edges and contracted edges in the transitive closure standard graph, and retrieves the key control attribute data in the original path mapping information when a contracted edge is matched, detects whether the corresponding logical path in the answer graph contains key control attributes, and establishes a path attribute integrity vector representing the coverage status of logical nodes based on the detection results. The logic evaluation generation submodule calls the path attribute integrity vector, counts the total number of structural matches of the connecting edges and the total number of key attribute verifications passed, calculates the structural matching rate and attribute coverage rate respectively, performs a weighted summation of the structural matching rate and attribute coverage rate, obtains the comprehensive logic score, and generates the dynamic evolution logic evaluation result.

[0015] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: By analyzing the closed-loop path gain attributes of the knowledge graph, an evaluation scenario containing dynamic evolution logic is constructed to simulate divergent or convergent states. The topology deviation index is used to monitor the degree of inference focus in real time and adaptively adjust the interaction granularity. Strong constraint questions are implemented when there is a logical deviation to lock the inference boundary. By performing a topology contraction operation in the closure graph construction module, non-critical connection segments in the industrial knowledge graph that do not contain key control attributes are automatically removed, generating a standard transitive closure graph that focuses on the core causality. This allows large models to be accurately judged as logically valid when reasonable skipping steps occur in reasoning (such as directly inferring the final disaster by crossing intermediate redundant equipment), as long as the key logical endpoint and control attribute match. This improves the quantitative accuracy of the evaluation results for the real complex reasoning ability of large models and realizes a deep quantitative evaluation of the complex causal reasoning ability and dynamic scenario adaptability of large models. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the industrial knowledge graph-driven large model evaluation intelligent agent provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the intelligent agent framework of the present invention; Figure 3 This is a flowchart of the evolution scene generation module in this invention; Figure 4 This is a flowchart of the topology deviation calculation module in this invention; Figure 5 This is a flowchart of the interactive granularity control module in this invention; Figure 6 This is a flowchart of the closure graph construction module in this invention; Figure 7 This is a flowchart of the isomorphic logic verification module in this invention. Detailed Implementation

[0018] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0019] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0020] This invention provides an intelligent agent for evaluating large models driven by industrial knowledge graphs, such as... Figure 1-2The diagram shown illustrates a large-scale model evaluation agent driven by an industrial knowledge graph. This agent includes: The evolution scenario generation module analyzes the knowledge graph to identify closed-loop paths, extracts the gain attributes and key control attributes of the closed-loop paths, accumulates the gain attributes to determine the evolution state, combines the evolution state, key control attributes and perturbations to construct test text, and outputs evolution evaluation scenario data packages. The topology deviation calculation module receives the natural language response stream of the large model for the evolution evaluation scenario data packet and maps it into a set of response nodes. It calculates the shortest path length between the set of response nodes and the closed loop path, and generates the topology deviation index based on the average shortest path length. The interaction granularity control module compares the topology deviation index with the preset deviation tolerance range. When the deviation is within the tolerance range, it generates open follow-up questions. When the deviation exceeds the tolerance range, it constructs closed questions based on the untraversed nodes of the evolution evaluation scenario data package and encapsulates them into an adaptive interaction instruction set. The closure graph construction module calls the adaptive interaction instruction set to parse the multi-round response text generated by the large model, constructs the answer graph structure, combines the evolutionary evaluation scenario data package, shrinks the non-critical segments of the closed-loop path and retains the mapping to generate the transitive closure graph, and combines them into a logical verification graph pair. The isomorphic logic verification module maps the logic verification graph to the answer graph and then to the transitive closure graph. It verifies the edge correspondence and verifies the existence of key control attributes based on the original path mapping information. Based on the edge correspondence and node existence, it generates dynamic evolution logic evaluation results.

[0021] The evolutionary evaluation scenario data package specifically includes a closed-loop topology sequence, physical gain parameters, and evolutionary trend labels. Topology deviation indicators include node mapping distance, mean path deviation, and divergence of thought. The adaptive interaction instruction set includes open guiding text, attribute constraint problems, and granularity adjustment strategies. The logic verification graph pair includes response reasoning subgraph, transitive closure baseline graph, and virtual edge mapping table. The dynamic evolutionary logic evaluation results include structural isomorphism score, key node coverage, and logic validity information.

[0022] Specifically, such as Figure 2 , 3 As shown, the evolution scene generation module includes: The closed-loop attribute extraction submodule acquires node connection relationship data in the industrial knowledge graph, performs path traversal, retrieves the path topology where the start node and end node coincide, extracts the physical quantity transmission multiple corresponding to each connected edge in the path as the gain attribute, and uses the identification information of the safety veto right defined on the geometric path node as the key control attribute, and establishes a closed-loop feature mapping table. First, a low-level connection channel is established with the industrial knowledge graph database. This channel supports high-concurrency graph query operations, using predefined Cypher query statements or Gremlin traversal scripts to obtain all nodes and their corresponding connection relationships through a full graph scan. A depth-first search-based path traversal algorithm is initialized. This algorithm is configured with a stack structure for recording visited paths and a hash set for marking visited nodes. Starting from each node in the graph marked with a "start" attribute, it explores downstream nodes level by level along directed edges. During each exploration, it monitors in real time whether the end node of the current path overlaps with the starting node. Once an overlap event of node unique identifiers is detected, a closed-loop topology is determined to have been found, and the path is stored as a candidate in a temporary cache queue. Subsequently, a deep attribute analysis is performed on each closed-loop path in the cache queue. For each connected edge in the path that connects two adjacent nodes, its attribute list is accessed, and the value of the field named "Physical Quantity Transmission Multiple" is retrieved. This value reflects the degree of amplification or attenuation of energy or signal flowing through the edge. For example, in a hydraulic transmission circuit, a connected edge represents the pipeline from the hydraulic pump to the cylinder, and its transmission multiple may be labeled as 0.98 to reflect pressure loss. This value is extracted and defined as a gain attribute. At the same time, each geometric node on the closed-loop path is traversed, and its metadata is checked to see if it contains the definition of a safety veto right, that is, whether there is identification information such as "emergency cut-off", "overload protection" or "logic interlock". Once such identification is identified, the ID of the node and the corresponding control logic description are extracted and defined as a key control attribute. Finally, a multi-dimensional closed-loop feature mapping table is constructed. This table uses the hash ID of the closed-loop path as the primary key and stores the topological sequence of the path, the gain attribute sequence of each connected edge, and the control attribute set of key nodes in separate columns. For example, in a closed loop containing "boost pump - check valve - accumulator - boost pump", the gain of the boost pump (1.2), the gain of the pipeline (0.95), and the key control attribute of the check valve ("backflow cut-off") are extracted. These data are then structured and written into a mapping table to provide an accurate data basis for subsequent evolution state determination.

[0023] The evolution state determination submodule calls the closed-loop feature mapping table, extracts the gain attribute of each connected edge for cumulative calculation, obtains the total loop gain coefficient, and determines the evolution state, including divergence and convergence, by comparing it with the unit benchmark value, and generates dynamic evolution trend labels. The system retrieves steady-state operating range data for each node in the closed-loop path from the historical database of the industrial knowledge graph. This data typically originates from long-term historical monitoring records of SCADA or distributed control systems, and is stored after cleaning and standardization. For each node, its time-series data under normal operating conditions is extracted, and the upper and lower edges of the data distribution are identified using statistical algorithms, which are then determined as the maximum and minimum boundary values, respectively. For example, if the normalized historical steady-state pressure value of a pressure node is distributed between 0.85 and 1.15, then the maximum boundary value is 1.15, and the minimum boundary value is 0.85. Next, a threshold setting operation is performed: the maximum boundary value of 1.15 is subtracted from the value of 1, resulting in a difference of 0.15, which is set as the positive tolerance threshold; simultaneously, the value of 1 is subtracted from the minimum boundary value of 0.85, resulting in a difference of 0.15, which is set as the negative tolerance threshold. Here, the value of 1 is set as the unit reference value characterizing the conservation of closed-loop energy input and output, representing an ideal state of zero gain and zero loss equilibrium. Subsequently, the cumulative product of the gain attributes of all connected edges of this path in the closed-loop feature mapping table is read, i.e., the total loop gain coefficient. Assuming this coefficient is 1.18, a subtraction operation is performed, calculating 1.18 minus the unit reference value of 1, resulting in a gain deviation value of 0.18. Then, the branch judgment logic is entered: first, it is determined whether the gain deviation value of 0.18 is positive. If yes, it is further compared with the positive tolerance threshold of 0.15. Since 0.18 is greater than 0.15, it indicates that the energy amplification effect within the loop exceeds the historically allowed fluctuation limit. Based on this, the evolution state of the closed loop is determined to be divergent, and a corresponding red warning label is generated. Conversely, if the calculated gain deviation value is negative, such as -0.2, its absolute value of 0.2 is calculated and compared with the negative tolerance threshold of 0.15. If the absolute value is greater than the negative tolerance threshold, it is determined to be convergent, meaning that the energy decay is too rapid, which may lead to an infinite loop or shutdown. This process accurately maps complex physical evolution trends into computable discrete state labels through rigorous numerical comparison.

[0024] The scenario data encapsulation submodule, based on dynamic evolution trend labels, sets initial perturbation parameters, combines key control attributes in the closed-loop feature mapping table, constructs evolution prediction test text, associates and integrates the text with the closed-loop path structure, and establishes an evolution evaluation scenario data package containing test context and standard topology reference. Based on the dynamic evolution trend labels, the test text construction process is initiated. If the label shows "divergent state," high-intensity initial disturbance parameters will be automatically matched, such as setting a step signal or pulse interference to simulate instability causes under extreme conditions. If the label shows "convergent state," continuous small noise interference will be set to test sensitivity. Key control attributes, such as "open the pressure relief valve when the pressure exceeds the threshold," are retrieved from the closed-loop feature mapping table. The initial disturbance condition "the inlet pressure suddenly increases to 1.5 times the rated value" is combined with this control attribute, and an evolution prediction test text is constructed using a natural language generation template, such as: "How should the pressure relief valve act to maintain loop stability when the inlet pressure suddenly increases to 1.5 times the rated value?" To ensure the comprehensiveness of the evaluation, the test text is associated and integrated with the complete topology of the closed-loop path. The test context (including disturbance conditions and expected evolution direction) and standard topology references (node ​​ID sequence, standard gain value, and key attribute description) are packaged and encapsulated in JSON or XML format. For example, a data package is created containing fields such as "ScenarioID", "PromptText", "ReferenceTopology", and "KeyAttributes". The "ReferenceTopology" field fully records the node connection order of the standard closed loop. This process ensures that what is sent to the large model is not just an isolated problem, but an evolutionary evaluation scenario data package containing a complete physical background and logical constraints, providing a standardized input carrier for subsequent bias calculation and logical verification.

[0025] Specifically, such as Figure 2 , 4 As shown, the topology deviation calculation module includes: The response stream parsing and mapping submodule receives the natural language response stream from the large model in response to the data packet of the evolution evaluation scenario. It performs semantic decomposition and key entity extraction on the natural language response stream, identifies the industrial terms and entity object names contained in the text, and compares and matches them with the standard node index in the industrial knowledge graph database. Based on the matching results, it maps and locks the corresponding node unique identifier in the graph and establishes a set of response nodes. First, the text preprocessing engine is initialized, receiving the natural language response stream from the large model's data packets for the evolution evaluation scenario. A pre-built industry thesaurus and standard terminology table from the industrial knowledge graph database are loaded. This thesaurus covers common mapping relationships such as "centrifugal pump - water pump" and "one-way valve - check valve". Set operations are performed, unifying all mapped terms in the thesaurus with the standardized terms in the standard terminology table to build a full standard node index. To determine the strictness of the matching, all pairs of terms with established mapping relationships in the thesaurus are traversed, and the normalized edit distance similarity between the two strings is calculated. The similarity calculation here uses the logic of "1 - (edit distance ÷ maximum length of the two strings)". The arithmetic mean of the similarity of each pair of synonyms in the thesaurus is calculated, and based on engineering experience, the calculated average is adjusted to a global matching confidence threshold, for example, set to 0.55 (to be compatible with common abbreviations and shorthand terms in industrial scenarios). Subsequently, each identified industrial term (e.g., "thermostat") in the response stream is decomposed into character serialization, and the same normalized edit distance algorithm is used to calculate its text similarity coefficient with each index entry in the standard node index (e.g., "temperature controller"). All calculated coefficients are sorted in descending order, and the coefficient with the highest value is selected as the target coefficient. For example, the edit distance between "thermostat" (length 3) and "temperature controller" (length 5) is 2 (the difference characters are "degree" and "system"), and the calculated similarity is 1−(2 / 5)=0.6. This value is greater than the preset matching confidence threshold of 0.55, and the judgment result is yes, that is, the two names are determined to refer to the same entity concept. Based on this determination, the index pointer of the candidate standard node "temperature controller" in the database is used to directly access the underlying storage field, read its bound primary key code (e.g., "DEV_T_001"), and use this code as the unique identifier of the corresponding node in the response node set. Through this series of precise calculations and comparisons, a deterministic mapping from fuzzy natural language to precise graph nodes is achieved.

[0026] The path distance calculation submodule calls the response node set, unpacks the evolution evaluation scenario data package and extracts the closed-loop path topology as a spatial reference benchmark, traverses each independent node element in the response node set, searches for the shortest connected path from each independent node element to the reference benchmark in the graph connection network, and generates a shortest path length sequence. The system invokes the mapped set of response nodes and extracts the closed-loop path topology from the evolutionary evaluation scenario data packet as a spatial reference. The closed-loop path is treated as a standard directed graph structure containing the correct node sequence and connection directions. For each independent node element in the response node set, a shortest path search algorithm (such as Dijkstra's algorithm or Breadth-First Search) is initiated in the global connectivity network of the graph. Starting from this independent node element and considering all nodes in the closed-loop path as potential endpoints, the shortest connected path length (i.e., minimum number of hops) from the starting point to any point in the closed-loop structure is calculated. For example, if the response node is "A" and the closed-loop path contains node "A", the shortest path length is 0; if the response node is "A'" (A's direct upstream), the path length is 1. All nodes in the set are traversed to generate a sequence of shortest path lengths containing multiple integer values. As shown in Table 1, if the response node set contains 5 nodes, the calculated distance sequence is [0, 1, 0, 2, 0]. This indicates that 3 nodes accurately hit the closed-loop path, while the other 2 nodes deviate by 1 hop and 2 hops, respectively. This sequence precisely quantifies the spatial position deviation of each entity output by the model relative to the standard answer, and can sensitively capture any tiny drifts in the model's inference process, providing fine-grained measurement data for subsequent overall deviation calculation.

[0027] Table 1 Example of Response Node Distance Calculation

[0028] As shown in Table 1, the topological deviation of each response node relative to the standard closed-loop path is recorded in detail, where non-zero values ​​directly reflect the inference deviation.

[0029] The deviation mean aggregation submodule obtains the shortest path length sequence, quantifies the dispersion of node distribution by performing an arithmetic mean operation on the path lengths, and outputs a topology deviation index. Obtain the shortest path length sequence, for example, [0, 1, 0, 2, 0]. Initialize the accumulator variable by summing all values ​​in the sequence (0+1+0+2+0) to obtain a total distance value of 3. Simultaneously, count the total number of elements in the sequence (5). Then, perform an arithmetic mean operation, dividing the total distance value of 3 by the total number of elements (5), resulting in 0.6. This value of 0.6 is the final generated topology deviation index. To ensure the robustness of the index, if the sequence is empty, the index is set to the preset maximum penalty value. This index intuitively reflects the dispersion of the response node distribution. The closer the value is to 0, the more tightly the large model's response revolves around the standard closed-loop path; the larger the value, the more irrelevant or erroneous entities are included in the response, and the more divergent the reasoning process. This quantification process compresses the complex graph spatial relationships into a single, comparable scalar value, making the response quality of different models or different rounds directly comparable, providing a core decision-making basis for subsequent interaction granularity control.

[0030] Specifically, such as Figure 2 , 5 As shown, the interactive granularity control module includes: The state logic determination submodule compares the topology deviation index with the preset deviation tolerance range, determines whether the index value is located within the specified range on the coordinate axis, determines the current inference focus state, determines the logic focus state when the deviation is within the tolerance range, and determines the logic divergence state when the deviation exceeds the tolerance range, and generates an inference focus state label. The node topology of the reference closed-loop path is parsed from the evolutionary evaluation scenario data package. An undirected adjacency matrix describing the connectivity between nodes within the closed-loop path is constructed using graph theory algorithms. Assuming the closed loop contains four nodes A, B, C, and D, the matrix dimension is 4x4, and the matrix element M[i][j] indicates whether node i and node j are directly connected (1 for connected, 0 for otherwise). Based on this undirected adjacency matrix, a full-source shortest path algorithm (such as the Floyd-Warshall algorithm) is used to traverse all non-overlapping node pairs (such as AB, AC, AD, etc.) in the closed-loop path, calculating the shortest topological path length between each pair. Taking a four-node ring as an example, the distance between adjacent nodes is 1, and the distance between relative nodes is 2. All calculated shortest topological path lengths are summed (e.g., the sum is 8), divided by the total number of node pairs (e.g., 6 pairs), and the arithmetic mean is calculated, yielding a result of 1.33. This value is used as the structural cohesion coefficient, reflecting the tightness of the reference path's structure. The cohesion coefficient of the structure, 1.33, was then set as the positive critical threshold for allowable deviation. Using zero as the starting point, a deviation tolerance range [0, 1.33] was constructed, ranging from zero to 1.33. Next, the topological deviation index (assumed to be 0.6) calculated in the previous steps was projected as a one-dimensional coordinate point onto the positive ray coordinate axis with zero as the origin, and the cutoff point of the positive critical threshold 1.33 on the coordinate axis was marked. It was then determined whether the coordinate point 0.6 was located within the closed interval [0, 1.33] formed by the origin 0 and the cutoff point 1.33. In this example, 0.6 was within the interval, so the determination was yes, the reasoning process was considered to be in the effective control domain, the current reasoning focus state was determined to be logical focus, and a corresponding state label was generated. If the deviation index was 1.5, located outside the positive range, it was determined to be a logical divergence state. This process achieves adaptive threshold determination based on the difficulty of the problem itself.

[0031] The constraint content generation submodule generates open-ended follow-up questions based on the inference focus state label when the deviation is within the tolerance range. When the deviation exceeds the tolerance range, it extracts the closed-loop path from the evolution evaluation scenario data package and identifies untraversed nodes, obtains the attribute information of the untraversed nodes, constructs closed-ended strongly constrained question content, and generates granular adaptive text. When the label indicates "Logically Focused State," it means the model's current reasoning path is within a controllable range. An open-ended question template is invoked to generate follow-up questions such as, "Based on the above analysis, please further deduce the possible secondary disasters that may occur," aiming to guide the model to think more deeply and uncover more potential logic. When the label indicates "Logically Divergent State," it means the model has deviated from the preset reasoning track. A strong constraint mechanism is immediately activated, extracting the closed-loop path again from the evolutionary evaluation scenario data package and comparing it with the nodes already covered by the model to identify nodes that have not yet been traversed (e.g., the "safety valve" node in the closed loop is missed). The attribute information of this untraversed node (e.g., "opening pressure: 10MPa") is obtained, and closed-ended strong constraint questions are constructed, such as, "Please explain specifically what state the safety valve will be in when the pressure reaches 10MPa—open or closed?" In this way, the model's attention is forcibly pulled back to the key nodes of the standard path. Ultimately, the generated open-ended follow-up questions or closed-ended strongly constrained questions are used as core text to generate granular adaptive text. This ensures that every sentence in the interaction can be dynamically adjusted according to the model's real-time performance, neither limiting the performance of excellent models nor failing models.

[0032] The instruction encapsulation configuration submodule receives granular adaptive text, performs standardized format validation and special character cleanup on the text, assigns an interaction round sequence number and corresponding priority weight parameters to the cleaned text, encapsulates the text into an instruction template that conforms to the large model input interface specification, and establishes an adaptive interactive instruction set by combining the sequence number, weight parameters, and instruction template. The text standardization processor is activated to perform format validation and cleaning operations on the text. Regular expressions are used to remove any illegal control characters, extra spaces, or garbled characters, and full-width characters are converted to half-width characters to ensure the text encoding conforms to the UTF-8 standard. Then, based on the current interaction round, an incrementing sequence number (e.g., Sequence_ID: 3) is assigned to the cleaned text, and priority weight parameters are assigned according to the text type. For example, a higher weight (e.g., 0.9) is given to "closed-ended, strongly constrained questions," while a normal weight (e.g., 0.5) is given to "open-ended follow-up questions." A pre-built instruction template conforming to the large model API input interface specification is invoked. This template contains standard fields such as "Role," "Context," and "Instruction." The sequence number, weight parameters, and cleaned text content are filled into the corresponding slots in the template, combined into a structured JSON object, for example: `{"id": 3, "role": "user", "content": "Please explain the safety valve...", "weight": 0.9, "timestamp": "..."}`. Finally, the object is encapsulated into an adaptive interactive instruction set, which not only contains natural language text but also carries metadata for scheduling and context management. It can be directly called by the large model inference interface to achieve efficient and orderly automated evaluation interaction.

[0033] Specifically, such as Figure 2 , 6 As shown, the closure graph construction module includes: The answer graph construction submodule calls the adaptive interaction instruction set as the context constraint benchmark for semantic parsing, obtains multi-turn response text data generated by the large model under test, performs entity relationship extraction operation, identifies causal logic chains and entity objects in the text, transforms the identified entity objects into graph nodes and causal logic into directed connection edges, and combines nodes and edges according to the dialogue sequence to establish the answer reasoning graph structure. An adaptive interaction instruction set is used as the contextual constraint benchmark for semantic parsing to obtain multi-turn response text data generated by the large model under test. Entity relation extraction models based on BERT or BiLSTM-CRF are used to scan the response text sentence by sentence. First, causal logic chains within the text are identified, capturing connectives such as "cause," "trigger," and "because...therefore...", while simultaneously identifying entity objects in the text (such as "pressure sensor" and "controller"). The identified entity objects are converted into graph nodes, and causal logic is transformed into directed edges, for example, from the "pressure increases" node to the "alarm triggers" node. To maintain the temporal order of the logic, based on the time stamps generated by the dialogue, newly extracted nodes and edges are added to the graph structure in chronological order, combining to build the answer reasoning graph structure. For example, the subgraph generated in the first round of dialogue and the subgraph generated in the second round of dialogue are concatenated through shared entity nodes. The final answer reasoning graph structure is a directed cyclic or acyclic graph, completely replicating the thought process of the large model throughout the evaluation process. This process transforms unstructured natural language into a structured graph that can be processed by computers, laying the data foundation for subsequent graph isomorphism verification.

[0034] The transitive closure generation submodule extracts closed-loop path topology data from the evolutionary evaluation scenario data package, traverses each node in the path, identifies non-critical connection segments that do not contain key control attributes, performs topology shrinkage operation, removes intermediate nodes and establishes direct connection relationships between the first and last nodes of the segments, embeds the index information of the original path into the new connection relationships, and generates a transitive closure standard graph. Standard topology data for closed-loop paths is extracted from the evolutionary evaluation scenario data package. This data defines the ideal physical causal chain. Each node in the path is traversed, and its attribute list is examined. Nodes that do not contain critical control attributes (such as pipes or wires that only serve a connecting or transitive function) are identified and marked as non-critical connection segments. For these segments, a topology shrinkage operation is performed: assuming a path A->B->C, where B is a non-critical node, node B and its associated edges A->B and B->C are removed, and a directed connection edge is directly established from node A to node C. While establishing the new connection, the index information of the original path (A->B->C) is embedded into the attributes of the newly established edge (A->C) as metadata. By performing this shrinkage on all non-critical nodes, a simplified standard graph of transitive closures is generated. This graph contains only critical control nodes and direct causal transitions, eliminating interference from intermediate redundant links. For example, a physical loop with 10 nodes may be shrunk to a closure graph containing only 3 critical nodes. In highly abstract industrial scenarios, large models often exhibit a "skip-step reasoning" phenomenon, directly skipping basic steps (e.g., directly determining 'A leads to C' while omitting the common-sense propagation node B). This invention, through the design of a transitive closure standard graph, ensures that as long as the logical endpoint of the large model is correct and its implicit key attributes are not lost, it can still perfectly match the shrunk standard graph. This mechanism effectively overcomes the technical bottleneck of "skip-step reasoning misjudgment" commonly found in traditional linear evaluation, accommodating the model's leaps in thinking, and maintaining the bottom line of logical completeness by preserving key nodes.

[0035] The logic verification and combination submodule obtains the answer reasoning graph structure and the transitive closure standard graph, aligns the data formats, unifies the node encoding rules and coordinate space reference system of the two, constructs a logic verification container containing two-layer graph data, fills the actual layer of the container with the aligned reasoning graph structure and fills the base layer of the container with the standard graph structure, and establishes a logic verification graph pair. First, data format alignment is performed, checking the consistency of the storage formats (e.g., RDF, Adjacency List) of the two graphs and unifying the node encoding rules to an internal unique identifier system. Simultaneously, the coordinate space reference system is calibrated to ensure the two graphs are in the same dimension when performing visualizations or matrix operations. Next, a logic verification container containing two-layer graph data is constructed, resembling a three-dimensional data cube or layered graph object. The aligned answer reasoning graph structure is filled into the container's measured layer (Layer_Measured), and the transitive closure standard graph structure is filled into the container's reference layer (Layer_Reference). During the filling process, inter-layer indexes are established, allowing nodes in the measured layer to quickly index into their corresponding standard nodes in the reference layer. Finally, a logic verification graph pair is created. This is a well-encapsulated data object that contains not only the actual reasoning logic of the large model but also simplified standard logical truth values, with both spatially and semantically aligned, awaiting isomorphic mapping analysis in subsequent modules.

[0036] Specifically, such as Figure 2 , 7 As shown, the isomorphic logic verification module includes: The structural feature mapping submodule deconstructs the logic verification graph pair and extracts the answer reasoning graph structure and the transitive closure standard graph. It traverses the bilateral graph data, obtains the node attribute identifier and topological adjacency matrix features, performs feature vector comparison, establishes a one-way mapping link from the answer node to the standard node, verifies the topological connection relationship between nodes and corrects the positional deviation, and generates a graph isomorphic mapping index that records the alignment relationship between the corresponding state and structure of the node. The logic verification graph pairs are deconstructed to extract the answer reasoning graph structure and the transitive closure standard graph at different levels. The bilateral graph data is traversed, and for each node, its attribute identifiers (such as device type and function label) and topological adjacency matrix features (such as in-degree, out-degree, and PageRank value) are obtained. A feature vector comparison algorithm is executed to calculate the cosine similarity of nodes in the answer graph and nodes in the standard graph in terms of attributes and structure. Based on the similarity matrix, a one-way mapping link is established from the answer node to the standard node, that is, determining that "node X" in the answer graph corresponds to "node Y" in the standard graph. After establishing the mapping, the topological connection relationship between nodes is verified, for example, checking whether the connection X->Z in the answer graph corresponds to the connection Y->W in the standard graph. If a positional deviation is found (such as opposite connection directions), an attempt is made to correct it or record the deviation type. Finally, a graph isomorphism mapping index is generated, recording the alignment relationship of all nodes' corresponding states and structures. This index details which nodes matched successfully, which edge relationships are consistent, and which are missing. For example, the index might show that "Answer Node ID_101" is precisely mapped to "Standard Node ID_Ref_05", and the similarity of their neighbor structures is as high as 0.95. This provides an accurate navigation map for subsequent specific attribute verification.

[0037] The path attribute verification submodule, based on the graph isomorphic mapping index, traverses and retrieves the connecting edges in the answer reasoning graph structure, performs type matching between the retrieved connecting edges and the original edges and contracted edges in the transitive closure standard graph, and retrieves the key control attribute data in the original path mapping information when a contracted edge is matched, detects whether the corresponding logical path in the answer graph contains key control attributes, and establishes a path attribute integrity vector representing the coverage of logical nodes based on the detection results. Based on the generated graph isomorphic mapping index, each connecting edge in the answer reasoning graph structure is traversed and retrieved. The retrieved connecting edges are then matched against the original edges and shrunk edges in the transitive closure standard graph. When a shrunk edge is matched (i.e., the edge represents a step connection in the standard graph where intermediate links are omitted), the original path mapping information embedded in that shrunk edge is immediately retrieved, and the omitted intermediate nodes and their key control attributes are read. For example, edge A->C in the standard graph is shrunk from A->B->C, and B has the key attribute "must be grounded." The contextual attributes or auxiliary descriptions of the corresponding logical path (A->C) in the answer graph are checked to see if they contain the semantic representation of the key control attribute "grounded." If the large model, in answering "A leads to C," skips B but mentions "under grounded conditions," the detection is considered successful. Based on the detection results of all edges, a path attribute integrity vector composed of 0s and 1s is established. Each bit of the vector corresponds to a key control point; 1 indicates the attribute exists or is logically implicitly correct, and 0 indicates the attribute is missing. For example, the vector [1, 1, 0, 1] indicates that three of the four key attributes were verified, but the third attribute was missed during the reasoning.

[0038] The logic evaluation generation submodule calls the path attribute integrity vector, counts the total number of structural matching of connecting edges and the total number of key attribute verifications passed, calculates the structural matching rate and attribute coverage respectively, performs a weighted summation of the structural matching rate and attribute coverage, obtains the comprehensive logic score, and generates the dynamic evolution logic evaluation result. The path attribute integrity vector is invoked. First, the total number of structurally matched edges is counted, i.e., the number of edges in the answer graph that can be correctly mapped to the standard graph. Simultaneously, the total number of key attributes that pass validation is counted, i.e., the number of elements in the vector with a value of 1. The structural matching rate (number of matched edges / total number of edges) and attribute coverage rate (number of validations passed / total number of key attributes) are calculated separately. For example, if 7 out of 8 edges are matched, the matching rate is 0.875; if 4 out of 5 attributes pass validation, the coverage rate is 0.8. Preset weight parameters (e.g., structural weight 0.4, attribute weight 0.6) are introduced to perform a weighted summation of the structural matching rate and attribute coverage rate. The calculation process is: 0.875 * 0.4 + 0.8 * 0.6 = 0.35 + 0.48 = 0.83. The calculated 0.83 is taken as the comprehensive logic score, and a dynamic evolutionary logic evaluation result (e.g., "Excellent," "Good," or a specific numerical score) is generated based on this score. As shown in Table 2, the final output of each indicator not only provides a total score, but also details the specific performance of the two dimensions of structure and attributes, making the evaluation results both macroscopic reference value and microscopic diagnostic significance.

[0039] Table 2 Example of Logical Evaluation Results

[0040] As shown in Table 2, the evaluation results clearly demonstrate the model's specific performance scores in topology restoration and key logical attribute preservation. The weighted score of 0.83 intuitively quantifies the model's overall performance in this evolution evaluation.

[0041] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An industrial knowledge graph-driven large-scale model evaluation intelligent agent, characterized in that, The intelligent agent includes: The evolution scenario generation module analyzes the knowledge graph to identify closed-loop paths, extracts the gain attributes and key control attributes of the closed-loop paths, accumulates the gain attributes to determine the evolution state, combines the evolution state, key control attributes and perturbations to construct test text, and outputs evolution evaluation scenario data packages. The topology deviation calculation module receives the natural language response stream of the large model for the evolution evaluation scenario data packet and maps it into a set of response nodes. It calculates the shortest path length between the set of response nodes and the closed loop path, and generates a topology deviation index based on the average shortest path length. The interaction granularity control module compares the topology deviation index with the preset deviation tolerance range. When the deviation is within the tolerance range, it generates open follow-up questions. When the deviation exceeds the tolerance range, it constructs closed questions based on the untraversed nodes of the evolution evaluation scenario data packet and encapsulates them into an adaptive interaction instruction set. The closure graph construction module calls the adaptive interaction instruction set to parse the multi-round response text generated by the large model, constructs the answer graph structure, combines the evolutionary evaluation scenario data package, shrinks the non-critical segments of the closed-loop path and retains the mapping to generate a transitive closure graph, and combines them into a logical verification graph pair. The isomorphic logic verification module maps the logic verification graph to the answer graph and then to the transitive closure graph. It verifies the edge correspondence and verifies the existence of key control attributes based on the original path mapping information. Based on the edge correspondence and node existence, it generates a dynamic evolution logic evaluation result.

2. The industrial knowledge graph-driven large model evaluation agent according to claim 1, characterized in that, The evolutionary evaluation scenario data package specifically includes a closed-loop topology sequence, physical gain parameters, and evolutionary trend labels. The topology deviation indicators include node mapping distance, mean path deviation, and divergence of thought. The adaptive interaction instruction set includes open guiding text, attribute constraint problems, and granularity adjustment strategies. The logic verification graph pair includes a response reasoning subgraph, a transitive closure baseline graph, and a virtual edge mapping table. The dynamic evolutionary logic evaluation results include structural isomorphism score, key node coverage, and logic validity information.

3. The industrial knowledge graph-driven large model evaluation agent according to claim 1, characterized in that, The evolution scene generation module includes: The closed-loop attribute extraction submodule acquires node connection relationship data in the industrial knowledge graph, performs path traversal, retrieves the path topology where the start node and end node coincide, extracts the physical quantity transmission multiple corresponding to each connected edge in the path as the gain attribute, and uses the identification information of the safety veto right defined on the geometric path node as the key control attribute, and establishes a closed-loop feature mapping table. The evolution state determination submodule calls the closed-loop feature mapping table, extracts the gain attribute of each connected edge for cumulative calculation, obtains the total loop gain coefficient, and determines the evolution state, including divergence and convergence, by comparing it with the unit benchmark value, and generates dynamic evolution trend labels. The scenario data encapsulation submodule sets initial perturbation parameters based on the dynamic evolution trend labels, combines the key control attributes in the closed-loop feature mapping table, constructs evolution prediction test text, and integrates the text with the closed-loop path structure to establish an evolution evaluation scenario data package containing test context and standard topology reference.

4. The industrial knowledge graph-driven large model evaluation agent according to claim 3, characterized in that, The process of determining the evolution state by comparing it with a unit benchmark value, including divergence and convergence, is specifically as follows: Call the historical steady-state operating interval data of each node in the closed-loop path stored in the industrial knowledge graph, calculate the maximum boundary value and minimum boundary value of the historical steady-state operating interval data, set the difference obtained by subtracting the value 1 from the maximum boundary value as the positive tolerance threshold, and set the difference obtained by subtracting the value 1 from the minimum boundary value as the negative tolerance threshold. Set the value 1 as the unit reference value characterizing the conservation of closed-loop energy input and output; Calculate the gain deviation value obtained by subtracting the unit reference value from the total loop gain coefficient; Determine whether the gain deviation value is positive. If the gain deviation value is positive, compare the gain deviation value with the positive tolerance threshold. If the gain deviation value is greater than the positive tolerance threshold, determine that the evolution state is a divergent state. Determine whether the gain deviation value is negative. If the gain deviation value is negative, calculate the absolute value of the gain deviation value and compare the absolute value with the negative tolerance threshold. If the absolute value is greater than the negative tolerance threshold, then determine that the evolution state is a convergent state.

5. The industrial knowledge graph-driven large model evaluation agent according to claim 3, characterized in that, The topology deviation calculation module includes: The response stream parsing and mapping submodule receives the natural language response stream fed back by the large model for the evolution evaluation scenario data packet, performs semantic decomposition and key entity extraction on the natural language response stream, identifies the industrial domain terms and entity object names contained in the text, compares and matches them with the standard node index in the industrial knowledge graph database, maps and locks the corresponding node unique identifier in the graph according to the matching result, and establishes a response node set. The path distance calculation submodule calls the response node set, unpacks and extracts the closed-loop path topology from the evolution evaluation scenario data packet as a spatial reference benchmark, traverses each independent node element in the response node set, searches for the shortest connected path from each independent node element to the reference benchmark in the graph connection network, and generates a shortest path length sequence. The deviation mean aggregation submodule obtains the shortest path length sequence, quantifies the dispersion of node distribution by performing an arithmetic mean operation on the path lengths, and outputs a topology deviation index.

6. The industrial knowledge graph-driven large model evaluation agent according to claim 5, characterized in that, The process of comparing and matching the standard node index with the industrial knowledge graph database, and mapping and locking the corresponding node's unique identifier in the graph based on the matching result, specifically involves: The thesaurus and standard terminology table preset in the industrial knowledge graph database are retrieved, and the mapping entries in the thesaurus and the standard entries in the standard terminology table are combined to construct the standard node index. Traverse all word pairs with established mapping relationships in the thesaurus, calculate the baseline edit distance similarity between the two word strings in each word pair, perform an arithmetic mean operation on all calculated baseline edit distance similarities, and set the average value as the matching confidence threshold; The industrial terms and entity names are decomposed into character serialization, and the edit distance algorithm is used to calculate the text similarity coefficient between the decomposed character sequence and each index item in the standard node index. The text similarity coefficients corresponding to each character sequence are sorted in descending order, and the coefficient with the first value is selected as the target coefficient. The index item corresponding to the target coefficient is marked as a candidate standard node. Determine whether the target coefficient is not less than the matching confidence threshold, and if the determination result is yes, determine that the industrial domain term or entity object name and the candidate standard node point to the same entity concept; In response to the identification result pointing to the same entity concept, the underlying storage field is accessed through the index pointer of the candidate standard node in the database, the primary key code bound to the candidate standard node is read and returned, and the primary key code is used as the unique identifier of the corresponding node.

7. The industrial knowledge graph-driven large model evaluation agent according to claim 5, characterized in that, The interaction granularity control module includes: The state logic determination submodule compares the topology deviation index with the preset deviation tolerance range, determines whether the position of the index value on the coordinate axis is within the defined range, determines the current inference focus state, determines the logic focus state when the deviation is within the tolerance range, and determines the logic divergence state when the deviation exceeds the tolerance range, and generates an inference focus state label. The constraint content generation submodule generates open-ended follow-up questions based on the inference focus state label. When the deviation is within the tolerance range, it extracts the closed-loop path from the evolution evaluation scenario data package and identifies untraversed nodes when the deviation exceeds the tolerance range. It then obtains the attribute information of the untraversed nodes and constructs closed-ended strongly constrained question content to generate granular adaptive text. The instruction encapsulation configuration submodule receives the granular adaptive text, performs standardized format verification and special character cleaning on the text, assigns an interaction round sequence number and corresponding priority weight parameters to the cleaned text, encapsulates the text into an instruction template that conforms to the large model input interface specification, and establishes an adaptive interaction instruction set by combining the sequence number, weight parameters and instruction template.

8. The industrial knowledge graph-driven large model evaluation agent according to claim 7, characterized in that, The process of comparing the topology deviation index with a preset deviation tolerance range, determining whether the index value is located within the defined range on the coordinate axis, and determining the current inference focus state, is as follows: If the deviation is within the tolerance range, it is determined to be a logic-focused state; if the deviation exceeds the tolerance range, it is determined to be a logic-divergent state. The node topology of the closed-loop path is parsed from the evolution evaluation scenario data packet, and an undirected adjacency matrix of the connectivity between nodes in the closed-loop path is constructed. Based on the undirected adjacency matrix, traverse all non-overlapping node pairs in the closed-loop path, calculate the shortest topological path length between each pair of nodes, perform an arithmetic mean operation on all calculated shortest topological path lengths, and generate a structural cohesion coefficient that reflects the tightness of the reference path's own structure. The structural cohesion coefficient is set as a positive critical threshold that allows deviation. Using zero as the starting point, the deviation tolerance range consisting of zero to the positive critical threshold is constructed. The topological deviation index is projected as a one-dimensional coordinate point onto a positive ray coordinate axis with the value of zero as the origin, and the cutoff point of the positive critical threshold on the coordinate axis is determined. Determine whether the coordinate point is located within the closed interval formed by the origin and the cutoff point; If the coordinate point is located within the closed interval, the reasoning process is considered to be in the effective control domain, and the reasoning focus state is determined to be the logical focus state. If the coordinate point is located on the positive outer side of the closed interval, it is determined that a structural drift has occurred in the reasoning process, and the reasoning focus state is determined to be a logical divergence state.

9. The industrial knowledge graph-driven large model evaluation agent according to claim 7, characterized in that, The closure graph construction module includes: The answer graph construction submodule calls the adaptive interaction instruction set as the context constraint benchmark for semantic parsing, obtains multi-turn response text data generated by the large model under test, performs entity relationship extraction operation, identifies causal logic chains and entity objects in the text, converts the identified entity objects into graph nodes and causal logic into directed connection edges, and combines nodes and edges according to the dialogue sequence to establish the answer reasoning graph structure. The transitive closure generation submodule extracts closed-loop path topology data from the evolution evaluation scenario data package, traverses each node in the path, identifies non-critical connection segments that do not contain key control attributes, performs topology shrinkage operation, removes intermediate nodes and establishes direct connection relationships between the first and last nodes of the segments, embeds the index information of the original path into the new connection relationships, and generates a transitive closure standard graph. The logic verification combination submodule obtains the answer reasoning graph structure and the transitive closure standard graph, aligns the data formats, unifies the node encoding rules and coordinate space reference system of the two, constructs a logic verification container containing two-layer graph data, fills the aligned reasoning graph structure into the actual layer of the container and fills the standard graph structure into the base layer of the container, and establishes a logic verification graph pair.

10. The industrial knowledge graph-driven large model evaluation agent according to claim 9, characterized in that, The isomorphic logic verification module includes: The structural feature mapping submodule deconstructs the logical verification graph pair and extracts the answer reasoning graph structure and the transitive closure standard graph. It traverses the bilateral graph data, obtains the node attribute identifier and topological adjacency matrix features, performs feature vector comparison, establishes a one-way mapping link from the answer node to the standard node, verifies the topological connection relationship between nodes and corrects the positional deviation, and generates a graph isomorphic mapping index that records the alignment relationship between the corresponding state and structure of the node. The path attribute verification submodule, based on the graph isomorphic mapping index, traverses and retrieves the connecting edges in the answer reasoning graph structure, performs type matching between the retrieved connecting edges and the original edges and contracted edges in the transitive closure standard graph, and retrieves the key control attribute data in the original path mapping information when a contracted edge is matched, detects whether the corresponding logical path in the answer graph contains key control attributes, and establishes a path attribute integrity vector representing the coverage status of logical nodes based on the detection results. The logic evaluation generation submodule calls the path attribute integrity vector, counts the total number of structural matches of the connecting edges and the total number of key attribute verifications passed, calculates the structural matching rate and attribute coverage rate respectively, performs a weighted summation of the structural matching rate and attribute coverage rate, obtains the comprehensive logic score, and generates the dynamic evolution logic evaluation result.

Images