Model defect knowledge detection method, control device, and storage medium

Abstract

The application provides a model defect knowledge detection method, a control device and a storage medium, and belongs to the technical field of large models. The method comprises the following steps: determining initial Monte Carlo trees based on a knowledge graph; starting from a root node of each initial Monte Carlo tree, selecting a next node based on the relationship between a current node and a neighbor node and a preset tree strategy; in the selection process, if the type of the current node is a leaf node, the current node is expanded; using one-dimensional structure entropy as an internal reward mechanism of an evaluation function, the expanded new node is evaluated; when the termination state is reached, the statistical data of each node is updated, and the processes of selecting the next node, expanding the new node and evaluating the new node are cyclically executed, so as to determine a knowledge path in which the target model has defects through the preset knowledge graph. The model defect knowledge detection method provides a collaborative verification framework based on knowledge structure entropy, and detects the knowledge defects of the large model through structured knowledge detection.

Description

Technical Field

[0001] This application belongs to the field of large model technology, and specifically relates to a model defect knowledge detection method, control device, and storage medium. Background Technology

[0002] Large Language Models (LLMs) have revolutionized the field of natural language processing by utilizing massive pre-trained corpora (which can be considered the "fossil fuel" of modern artificial intelligence), demonstrating superior performance in many areas. However, with the increase in model size, data volume, and computational resources, the performance improvement of LLMs follows scaling laws. At the same time, the supply of high-quality human-annotated data, especially in specialized domains, is gradually decreasing. Therefore, synthetic data is needed as a new source of energy to drive further development.

[0003] To address the scarcity of training data in the current large-scale modeling field, related technologies focus on LLM-driven synthetic data generation as an alternative to manual data annotation methods. However, existing data synthesis methods mainly rely on knowledge distillation, i.e., the teacher model generates training data for a lightweight student model. However, this one-way distillation paradigm has fundamental limitations. For example, synthetic data is not explicitly designed to address the knowledge gaps in the teacher model itself; while it may improve the performance of the student model, it does not enhance the performance of the teacher model itself, but merely replicates the existing knowledge distribution of the teacher model. Summary of the Invention

[0004] This application provides a method, control device, and storage medium for detecting knowledge defects in large language models.

[0005] According to a first aspect of this application, a method for detecting model defects is provided, comprising: determining an initial Monte Carlo tree based on a preset knowledge graph; starting from the root node of each initial Monte Carlo tree, selecting the next node based on the relationship between the current node and its neighboring nodes and a preset tree strategy; during the selection of the next node, if the current node is a leaf node, expanding the current node and adding the expanded new node to the corresponding Monte Carlo tree; evaluating the expanded new node using an intrinsic reward mechanism with one-dimensional structural entropy as the evaluation function; and when reaching a termination state, updating the statistical data of each node and cyclically executing the process of selecting the next node, expanding the new node, and evaluating the new node, so as to determine the knowledge path in which the target model has defects through the preset knowledge graph.

[0006] In an optional implementation, entities in the knowledge graph are defined as nodes or states of the corresponding Monte Carlo tree. In each state, an action set is used to represent the relationship between the current entity and its neighboring nodes. The step of selecting the next node based on the relationship between the current node and its neighboring nodes and a preset tree strategy includes: determining the next node by using the cumulative value of taking action a in the current state and the access frequency of the current state.

[0007] In an optional implementation, before evaluating the expanded new nodes using the intrinsic reward mechanism of one-dimensional structural entropy as the evaluation function, the model defect knowledge detection method further includes: for a knowledge graph G = (V, E) consisting of the set of all nodes V and the set of all edges E and its triples τ = ... <subject e i ,relationρ,subject e j Each edge ρ∈E is assigned a weight based on its self-information to determine the entity node e. i The weighted degree of edge ρ∈V is used to determine the entity node e by assigning weights to each edge ρ∈E based on its self-information. i The weighted degree of ∈V; and the one-dimensional structural entropy of the knowledge graph G based on the weighted degree of each entity node.

[0008] In an optional implementation, the one-dimensional structural entropy H of the knowledge graph G is determined by the following formula. 1 (G):

[0009]

[0010] Where, d i Represents entity node e i The weighted degree vol(G) represents the total weighted degree of G, and V represents the set of all nodes.

[0011] In an optional implementation, when expanding the current node, starting from the newly expanded node, multiple knowledge paths with the new node as the starting node are simulated by taking random actions. The evaluation of the newly expanded node using the intrinsic reward mechanism of one-dimensional structural entropy as the evaluation function includes: calculating the one-dimensional structural entropy of each simulated knowledge path and selecting the knowledge path with the highest one-dimensional structural entropy as the target path.

[0012] In an optional implementation, determining the knowledge path where the target model has defects through the preset knowledge graph includes: identifying the target path with the highest one-dimensional structural entropy or the target path where the one-dimensional structural entropy exceeds a preset threshold as the knowledge path where the target model has defects.

[0013] In an optional implementation, the model defect knowledge detection method further includes: extracting the corresponding trajectory for the determined defective knowledge path; and synthesizing data by prompting the target model to generate questions about key entities and answers that logically explain the relationship between adjacent entities of the key entities in the trajectory, in order to repair the defective knowledge path.

[0014] In an optional implementation, the model defect knowledge detection method further includes: using a predefined prompt template to prompt questions and answers for generating key entities, so as to ensure that the structure, punctuation and length of the synthesized data meet the requirements.

[0015] According to a second aspect of this application, a control device is provided, the control device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the computer program to implement the above-described model defect knowledge detection method.

[0016] According to a third aspect of this application, a machine-readable storage medium is provided, on which instructions are stored, causing a machine to execute the above-described model defect knowledge detection method.

[0017] Through the above technical solution, this application provides a model defect knowledge detection method, including: determining an initial Monte Carlo tree based on a preset knowledge graph; starting from the root node of each initial Monte Carlo tree, selecting the next node based on the relationship between the current node and its neighboring nodes and a preset tree strategy; during the selection of the next node, if the current node is a leaf node, expanding the current node and adding the expanded new node to the corresponding Monte Carlo tree; evaluating the expanded new node using a one-dimensional structural entropy as the intrinsic reward mechanism of the evaluation function; and when reaching the termination state, updating the statistical data of each node and cyclically executing the process of selecting the next node, expanding the new node, and evaluating the new node, so as to determine the knowledge path of the target model with defects through the preset knowledge graph. This model defect knowledge detection method provides a collaborative verification framework based on knowledge structure entropy, which detects knowledge defects in large models through structured knowledge detection. For knowledge defect detection, this application introduces the knowledge structure entropy index to measure the knowledge uncertainty of large language models, modeling uncertainty as structural entropy on the knowledge graph path. By using Monte Carlo tree search to efficiently explore the region of maximum uncertainty, this application can identify key knowledge defects.

[0018] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures and processes shown in the description and the accompanying drawings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating the model defect knowledge detection method provided in an exemplary embodiment of this application.

[0021] Figure 2 This is a schematic diagram illustrating the principle of model defect knowledge detection provided in an exemplary embodiment of this application.

[0022] Figure 3 This is a schematic diagram illustrating an example prompt of the synthetic data generation stage of an exemplary embodiment of this application.

[0023] Figure 4 This is a schematic diagram of data distribution analysis of an exemplary embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] To address the aforementioned problems, this application provides a self-generation paradigm, whereby the model generates data solely based on its own capabilities to achieve continuous self-improvement. However, self-generation faces key challenges: 1) Pre-training bias: The content generated by large language models (LLMs) is affected by pre-training bias, making it difficult to cover missing knowledge outside the pre-training corpus, especially in specialized domains. Even with cue engineering, the generated output tends to favor high-frequency distributions appearing in the pre-training data, leading to a systematic omission of low-frequency domain knowledge. 2) Knowledge illusion: When generating knowledge that the model itself lacks, LLMs lack an inherent fact verification mechanism, increasing the risk of generating "pseudo-knowledge" that contradicts reality.

[0026] Based on this, embodiments of this application provide a model defect knowledge detection method, which provides a collaborative verification framework SENATOR based on knowledge structure entropy (SE) to detect knowledge defects in large language models through structured knowledge exploration.

[0027] Figure 1 This is a flowchart illustrating a model defect knowledge detection method provided in an exemplary embodiment of this application. The model defect knowledge detection method may include the following steps:

[0028] Step S110: Determine the initial Monte Carlo tree based on the preset knowledge graph.

[0029] Please refer to Figure 2 For example, this application embodiment, based on the standard Monte Carlo Tree Search (MCTS) paradigm, implements the process of detecting model defect knowledge, which may include: Selection, Expansion, Simulation, and Backup. MCTS starts from an initial state s0, which corresponds to the seed entity (i.e., the root node of the initial Monte Carlo tree), and defines entities in the knowledge graph as nodes or states of the Monte Carlo tree. In each state, the possible action set A{a1, a2, ..., a... m} represents the connection relationship between the current entity and its neighboring entities. By selecting an action, the algorithm will transition to a new node (i.e., a new state). t+1 That is, moving along the selected edge (relation) ρ to another entity e. j Based on the above exploration, it is possible to identify paths in knowledge graphs that can reveal knowledge deficiencies in large language models.

[0030] Step S120: Starting from the root node of each initial Monte Carlo tree, select the next node based on the relationship between the current node and its neighboring nodes and the preset tree strategy.

[0031] In this embodiment of the application, for the selection process, a node that balances search quality and computational efficiency can be identified as the next node.

[0032] In a preferred embodiment of this application, selecting the next node based on the relationship between the current node and its neighboring nodes and a preset tree strategy may include: determining the next node by utilizing the cumulative value of action a taken in the current state and the access frequency of the current state.

[0033] For example, two key variables can be used to guide the selection process: the key variable Q(s) t , a), can represent the state s t (Please refer to) Figure 2 For example, the cumulative value of taking action 'a' in the current entity in the knowledge graph; key variable N(s) t ), can represent state s t The frequency of visits to nodes. To balance exploring new nodes with utilizing already visited nodes, the frequency of visits to node s... t At this point, the following formula can be used to select the next node.

[0034]

[0035] Among them, c puct P(a|s) represents the coefficients of the PUCT algorithm. t ) indicates that in state s t The prior probability of choosing action a, N(s) t a) indicates that from state s t The number of times to choose action a.

[0036] Step S130: During the process of selecting the next node, if the current node is a leaf node, expand the current node and add the expanded new node to the corresponding Monte Carlo tree.

[0037] Continuing with the example above, the expansion process occurs at the leaf nodes of the selection process, and can be used to integrate new nodes and evaluate rewards using the following formula:

[0038]

[0039] Where, r t+1 Let r(s) be the instantaneous reward function at step (t+1). t a) indicates that the quantization is used in state s t For each available action a, I(s) t ,a,s t+1 ) represents the self-information of the corresponding triple. Represents state s t+1The corresponding weighting degree, vol(G) represents the total weighting degree of G, and γ represents the future state value V(s). t The discount factor is T, where T represents the depth of the Monte Carlo tree search space, and k represents the number of states from s. t The simulation begins at a certain number of steps, and stops and backpropagates when the knowledge path length reaches the total search depth T.

[0040] Step S140: Utilize the inherent reward mechanism of one-dimensional structural entropy as the evaluation function to evaluate the newly expanded nodes.

[0041] This application provides a knowledge defect measurement based on structural entropy. The reward function based on structural entropy combines the uncertainty of a large language model regarding knowledge graph triples with the topological structure information of the knowledge graph. The reward function based on structural entropy then guides a large language model-driven agent to search the knowledge graph using Monte Carlo tree search to find knowledge paths with critical defects. A higher structural entropy value indicates greater uncertainty and a lower level of model mastery of the knowledge.

[0042] Preferably, before step S140, the model defect knowledge detection method may further include: for a knowledge graph G = (V, E) consisting of the set of all nodes V and the set of all edges E and its triples τ = <subject e i ,relationρ,subject e j Each edge ρ∈E is assigned a weight based on its self-information to determine the entity node e. i The weighted degree of ∈V; and the one-dimensional structural entropy of the knowledge graph G based on the weighted degree of each entity node.

[0043] In this embodiment, for the self-information used to measure the uncertainty of triples, the self-information quantifies the amount of information contained in a "fact" under a given probability distribution. In a knowledge graph, a "fact" can be represented as a triple τ = <subjecte i ,relationρ,subject e j To measure the uncertainty of large language models regarding such "facts," embodiments of this application convert τ into a cloze test format. The context of the cloze test can be provided by entity e. i Combined with the relation ρ, it forms a cue for predicting entities with missing tails. The self-information of τ can be expressed by the following formula:

[0044] I(e i ,ρ,e j )=-log2P(e j |e i ,ρ) (4)

[0045] Wherein, P(e j |e i ρ) represents the output probability in the context of the cloze test. Since the relations in the knowledge graph are directed, the self-information calculated by equation (4) can be regarded as a measure of the confidence of factual knowledge of the entire triple.

[0046] In this embodiment of the application, for a given knowledge path, in order to integrate the uncertainty of all triples on the path and consider the structural importance of all triples, this embodiment of the application preferably uses structural entropy to measure the knowledge confidence of large language models, such as... Figure 2 As shown in 1(e), structural importance captures the topological importance of triples in a knowledge graph. Triples involving highly connected entities are considered more central because these entities participate in more relational paths and exert a wider influence throughout the graph. Unlike self-information or Shannon entropy, structural entropy describes the topological structure of a knowledge graph and the interactions between knowledge elements. Since each triple is not an isolated piece of information but part of a structured network, the relationships between entities contribute to the overall knowledge representation.

[0047] For example, for a given knowledge graph G = (V, E), each edge ρ ∈ E can be weighted according to the self-information in equation (4). The entity node e can be represented by the following equation. i The weighted degree d of ∈V i :

[0048]

[0049] Among them, Nei(e i ) represents entity node e i The set of neighboring nodes, d i This can measure the overall uncertainty contained within a node. To quantify the average information content of a knowledge graph G, this application embodiment preferably determines the one-dimensional structural entropy H of the knowledge graph G using the following formula. 1 (G):

[0050]

[0051] Where vol(G) represents the total weighting degree of G.

[0052] In this embodiment of the application, H 1 (G) Higher values ​​can indicate more complex and lower-confidence regions within the knowledge graph. By using one-dimensional structural entropy as the exploration reward in MCTS, embodiments of this application enable the search algorithm to prioritize paths traversing knowledge structures with the greatest uncertainty, thereby efficiently exposing the systematic weaknesses of the model.

[0053] In a preferred embodiment of this application, when expanding the current node, starting from the expanded new node, multiple knowledge paths with the new node as the starting node are simulated by taking random actions. Preferably, step S140 may include: calculating the one-dimensional structural entropy of each simulated knowledge path, and selecting the knowledge path with the highest one-dimensional structural entropy as the target path.

[0054] Please refer to Figure 2 For example, newly expanded nodes (e.g., node s) t The process begins by simulating a knowledge search path by taking random actions until a termination state is reached. Next, an evaluation function is used to evaluate the newly expanded nodes. This embodiment preferably utilizes one-dimensional structural entropy as the intrinsic reward mechanism of the evaluation function, so that the evaluation function integrates future rewards, state relevance, and actual results. This intrinsic reward mechanism solves the problem of imbalanced exploration of low-value states in the Shannon entropy strategy. The following equation represents using one-dimensional structural entropy as the intrinsic reward to achieve effective exploration:

[0055]

[0056] Wherein, the subgraphs of the knowledge graph G are S = {s0, s1, ..., s...} t} represents the selection trajectory (simulated knowledge path) in the t-th iteration, which terminates in state s after a complete simulation. T Finish, Let H(S) represent the expected value, and let H(S) represent the structural entropy of the knowledge path S.

[0057] Step S150: When the termination state is reached, update the statistical data of each node, and repeatedly execute the process of selecting the next node, expanding the new node, and evaluating the new node, so as to determine the knowledge path where the target model has defects through the preset knowledge graph.

[0058] Please refer to Figure 2 During backpropagation, this embodiment updates the statistical data for each state in the tree traversed during the selection phase. During backpropagation, the cumulative value and visit count of all root nodes on the trajectory (i.e., the target path) are updated to ensure that the evaluation results of the leaf nodes can provide information for higher-level decision-making. The update rules are as follows:

[0059]

[0060] in, Indicates that from state s t In the i-th simulation run, whether action a was selected, the value can be 1 if yes, otherwise it can be 0.

[0061] In a preferred embodiment of this application, determining the knowledge path where the target model has defects through the preset knowledge graph may include: identifying the target path with the highest one-dimensional structural entropy or the target path with one-dimensional structural entropy exceeding a preset threshold as the knowledge path where the target model has defects.

[0062] Please refer to Figure 2 Based on the preset number of iterations (Repeated X times), one or more target paths can be obtained. The target path with the highest one-dimensional structural entropy can be identified as a knowledge path with defects in the knowledge graph. Alternatively, the target path with one-dimensional structural entropy exceeding a preset threshold can be identified as a knowledge path with defects in the target model.

[0063] The model defect knowledge detection method provided in this application provides a collaborative verification framework SENATOR based on knowledge structure entropy, which detects knowledge defects in large models through structured knowledge detection. Furthermore, this application embodiment can also selectively synthesize closed loops formed by data to repair knowledge gaps in large models.

[0064] In a preferred embodiment of this application, the model defect knowledge detection method may further include: extracting the corresponding trajectory for the determined defective knowledge path; and synthesizing data by prompting the target model to generate questions about key entities and answers that logically explain the relationship between adjacent entities of the key entities in the trajectory, in order to repair the defective knowledge path.

[0065] like Figure 2 As shown in (f) to (h), embodiments of this application also provide a method for guiding the generation of synthetic data by utilizing the trajectory with the highest structural entropy value obtained through MCTS. For example, by prompting a large language model agent to synthesize one or more sets of question-answer pairs based on determined knowledge paths, where the large language model has high uncertainty, the knowledge deficiencies of the large language model can be compensated by training on these question-answer pairs.

[0066] like Figure 3 As shown, for a given trajectory e = {e0, e1, ..., e...} r} or S = {s0, s1, ..., s t The instructions and prompts indicate that the large language model generation focuses on the key entity e. i The question, and the logical explanation of e i Its neighboring entity Nei(e) in e i The answers are derived from the relationships between the two. This synthesized question-and-answer pairs follow the foundational knowledge of knowledge paths, compensating for the knowledge gaps in large language models. Furthermore, to maintain high-quality data, embodiments of this application provide a multi-level evaluation mechanism including heuristic rules and judgments based on large language models.

[0067] In a preferred embodiment of this application, the model defect knowledge detection method may further include: using a predefined prompt template to prompt questions and answers for generating key entities, so as to ensure that the structure, punctuation and length of the synthesized data meet the requirements.

[0068] like Figure 3 As shown, a multi-level evaluation mechanism can include: 1) Format consistency: The generated question-and-answer pairs adhere to predefined prompt templates, ensuring that the structure, punctuation, and length conform to specifications to guarantee a uniform format of the synthesized data, facilitating downstream processing. 2) Logical coherence: The question-and-answer pairs can express clear and reasonable reasoning. The answers should provide logically consistent explanations, i.e., relationships and context derived from the knowledge path, ensuring that the data effectively captures and addresses identified knowledge gaps. 3) Avoiding illusions: The generated content is based on the input trajectory. For example, all entities and facts mentioned in the question-and-answer pairs come from the given trajectory, preventing the introduction of foreign or unfounded information that may compromise the reliability of the model. Data samples that do not meet the above criteria can be filtered out through the evaluation mechanism, thereby ensuring that only high-quality synthetic data is used to fill knowledge gaps in large language models.

[0069] This application also provides embodiments for verifying the effectiveness of a large-scale language model agent based on knowledge structure entropy in detecting model knowledge deficiencies. For example, all data samples are projected onto a unified semantic space, and kernel density estimation (KDE) maps of the original pre-training data and the synthesized data are calculated in a two-dimensional UMAP space. Data distribution analysis shows that the synthesized data generated by the framework covers the knowledge missing in the pre-training corpus (such as...). Figure 4 As shown, red represents the distribution of synthetic data, and blue represents the distribution of original pre-training data. MedQA represents a multiple-choice dataset used to assess medical understanding and reasoning abilities. MedMCQA also represents a dataset that can include over 194,000 exam questions covering 2,400 topics across 21 disciplines. PubMedQA represents a biomedical question-and-answer dataset built based on PubMed summaries, including 1,000 expert annotations and 211,000 automatically generated question-and-answer pairs. The results demonstrate that the synthetic data used in this application effectively expands the knowledge coverage of existing pre-training corpora, injecting domain knowledge previously missing from large-scale language models, indirectly proving the efficiency of this application's embodiments as a "new energy source" for artificial intelligence data annotation.

[0070] Furthermore, these data were used to supervise and fine-tune models such as Llama-3 and Qwen2. Validation showed that, within the SENATOR framework, the average performance of Llama-3 and Qwen2 was improved by 5.07 and 4.42 percentage points, respectively, across four benchmark tests. This significantly improved model performance. This demonstrates that the targeted injection of missing knowledge in the embodiments of this application can significantly enhance the overall performance of the model, and that this targeted strategy for missing knowledge is more efficient than traditional, large-scale, undirected knowledge supervision methods.

[0071] Accordingly, this application provides a model defect knowledge detection method, including: determining an initial Monte Carlo tree based on a preset knowledge graph; starting from the root node of each initial Monte Carlo tree, selecting the next node based on the relationship between the current node and its neighboring nodes and a preset tree strategy; during the selection of the next node, if the current node is a leaf node, expanding the current node and adding the expanded new node to the corresponding Monte Carlo tree; evaluating the expanded new node using a one-dimensional structural entropy as the intrinsic reward mechanism of the evaluation function; and when reaching the termination state, updating the statistical data of each node and cyclically executing the process of selecting the next node, expanding the new node, and evaluating the new node, so as to determine the knowledge path of the target model with defects through the preset knowledge graph. This model defect knowledge detection method provides a collaborative verification framework based on knowledge structure entropy, which detects knowledge defects in large models through structured knowledge exploration. For knowledge defect detection, this application introduces the knowledge structure entropy index to measure the knowledge uncertainty of large language models, modeling uncertainty as structural entropy on the knowledge graph path. By using Monte Carlo tree search to efficiently explore the region of maximum uncertainty, this application can identify key knowledge defects.

[0072] Furthermore, embodiments of this application also provide a closed loop for targeted synthetic data formation to address knowledge gaps in large models. For knowledge synthesis and repair, synthetic data is generated using defective knowledge paths identified by MCTS, while a knowledge graph is employed to ensure the credibility of the data source and the synthesized content. Then, this synthetic data is used to fine-tune the model through supervised learning SFT, achieving continuous self-improvement and effective knowledge defect repair.

[0073] This application embodiment also provides a control device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the above-described model defect knowledge detection method.

[0074] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the above-described model defect knowledge detection method.

[0075] It should be noted that the control device and machine-readable storage medium described above can implement the model defect knowledge detection method provided in the above embodiments. For specific implementation methods, please refer to the description of the model defect knowledge detection method in the above embodiments, which will not be repeated here.

[0076] It is understood that the circuit structures, names, and parameters described in the above embodiments are merely examples. Those skilled in the art can also make readily conceived combinations and adjustments to the structural features of the above embodiments according to their needs, and the concept of this application should not be limited to the specific details of the above examples.

[0077] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for detecting model defect knowledge, characterized in that, The model defect knowledge detection method includes: The initial Monte Carlo tree is determined based on the pre-defined knowledge graph; Starting from the root node of each initial Monte Carlo tree, the next node is selected based on the relationship between the current node and its neighboring nodes and the preset tree strategy; During the process of selecting the next node, if the current node is a leaf node, expand the current node and add the expanded new node to the corresponding Monte Carlo tree; The newly expanded nodes are evaluated using the intrinsic reward mechanism of one-dimensional structural entropy as the evaluation function; and When the termination state is reached, the statistics of each node are updated, and the process of selecting the next node, expanding the new node, and evaluating the new node is executed in a loop to determine the knowledge path where the target model has defects through the preset knowledge graph. The knowledge path for determining the defects in the target model through the preset knowledge graph includes: The target path with the highest one-dimensional structural entropy or the target path with one-dimensional structural entropy exceeding a preset threshold is identified as a knowledge path in which the target model has defects. The knowledge graph is determined by the following formula. One-dimensional structural entropy : ； in, Represents entity nodes The weighting degree, express Total weighted degree V This represents the set of all nodes.

2. The model defect knowledge detection method according to claim 1, characterized in that, Entities in the knowledge graph are defined as nodes or states in the corresponding Monte Carlo tree. In each state, an action set represents the relationship between the current entity and its neighboring nodes. The selection of the next node based on the relationship between the current node and its neighboring nodes and a preset tree strategy includes: The next node is determined by using the cumulative value of action 'a' taken in the current state and the access frequency of the current state.

3. The model defect knowledge detection method according to claim 1, characterized in that, Before evaluating the expanded new nodes using the intrinsic reward mechanism of one-dimensional structural entropy as the evaluation function, the model defect knowledge detection method further includes: For a set of all nodes V and the set of all edges E knowledge graph and its triples Each edge Entity nodes are determined by assigning weights based on self-information. The weighted degree of each entity node; and the knowledge graph is determined based on the weighted degree of each entity node. The one-dimensional structural entropy.

4. The model defect knowledge detection method according to claim 1, characterized in that, When expanding the current node, starting from the newly expanded node, multiple knowledge paths originating from the new node are simulated by taking random actions. The evaluation of the newly expanded node using an intrinsic reward mechanism with one-dimensional structural entropy as the evaluation function includes: Calculate the one-dimensional structural entropy of each simulated knowledge path, and select the knowledge path with the highest one-dimensional structural entropy as the target path.

5. The model defect knowledge detection method according to claim 1, characterized in that, The model defect knowledge detection method also includes: For the identified knowledge paths with defects, extract the corresponding trajectories; By providing prompts to the target model to generate questions about key entities and answers that logically explain the relationships between the key entities and adjacent entities in the trajectory, data is synthesized to repair the defective knowledge path.

6. The model defect knowledge detection method according to claim 5, characterized in that, The model defect knowledge detection method also includes: Using predefined prompt templates, questions and answers are provided to generate key entities, ensuring that the structure, punctuation, and length of the synthesized data meet the requirements.

7. A control device, characterized in that, The control device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the model defect knowledge detection method according to any one of claims 1-6.

8. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions that cause the machine to execute the model defect knowledge detection method according to any one of claims 1-6.

Images