A model detection method, device, medium and electronic equipment

Abstract

This specification discloses a model detection method, apparatus, medium, and electronic device, comprising: acquiring a statement to be detected of a target model; determining, based on a preset knowledge graph, the entity relations corresponding to at least some of the entity words contained in the statement to be detected, and generating supplementary statements to describe the entity relations; determining the corresponding question statement for each anomaly category corresponding to the statement to be detected; inputting the question statement, the statement to be detected, and the supplementary statements for each anomaly category into a preset decision model to obtain a decision result and criterion text for whether the target model possesses the anomaly category for each anomaly category; determining the comprehensive question statement corresponding to the statement to be detected; inputting the comprehensive question statement, the statement to be detected, the supplementary statements, the decision result and criterion text for each anomaly category into the decision model to obtain a final decision result; and detecting the target model based on the final decision result.

Description

Technical Field

[0001] This specification relates to the field of artificial intelligence, and in particular to a model detection method, apparatus, medium, and electronic device. Background Technology

[0002] With the rapid development of artificial intelligence, large language models are widely used in various fields. Large language models refer to deep learning models trained on large amounts of text data that can generate natural language text or understand the meaning of language text. Large language models can handle various natural language tasks, such as question answering, knowledge dialogue, and text summarization (a task where large language models extract key information from given text and generate a short summary or conclusion), bringing great convenience to people's daily lives.

[0003] However, the content generated by large language models is prone to the phantom problem, which means that when processing certain inputs, it may produce outputs that do not conform to reality. The phantom problem will cause the results generated by large language models to be inaccurate. Existing technical means cannot effectively detect whether the results generated by large language models contain inaccurate parts, thus failing to play a positive incentive role in the subsequent training of large language models, causing large language models to output incorrect results in subsequent use.

[0004] Therefore, how to effectively detect whether a model contains illusion problems is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a model testing method, apparatus, medium, and electronic device to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a model testing method, including:

[0008] Obtain the statement output by the target model in response to the input statement, and use it as the statement to be detected;

[0009] Based on a preset knowledge graph, the entity relationships corresponding to at least some of the entity words contained in the statement to be detected are determined, and a statement describing the entity relationships is generated as a supplementary statement.

[0010] For each anomaly category corresponding to the statement to be detected, the question statement used to detect whether the target model has that anomaly category is determined, and it is used as the question statement corresponding to that anomaly category.

[0011] The question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statement are input into a preset decision model to obtain a decision result and criterion text for each anomaly category, indicating whether the target model has the anomaly category. The criterion text is used to represent the text content on which the decision model gives the decision result.

[0012] Determine the comprehensive question statement corresponding to the statement to be detected;

[0013] The comprehensive question statement, the statement to be detected, the supplementary statement, the judgment result and criterion text corresponding to each anomaly category are input into the judgment model to obtain the final judgment result;

[0014] Based on the final judgment result, the target model is detected.

[0015] Optionally, based on a preset knowledge graph, the entity relations corresponding to at least some of the entity words contained in the statement to be detected are determined, specifically including:

[0016] Based on the entity words contained in the statement to be detected, determine the keywords contained in the statement to be detected;

[0017] Based on the keywords, determine the associated subgraph corresponding to each keyword from the preset knowledge graph;

[0018] Based on the association subgraph corresponding to each keyword, determine the entity relationships corresponding to at least some of the entity words contained in the statement to be detected.

[0019] Optionally, based on the entity words contained in the statement to be detected, the keywords contained in the statement to be detected are determined, specifically including:

[0020] For each entity word contained in the statement to be detected, determine the relevance weight between the entity word and each other entity word, and determine the importance value corresponding to the entity word based on the relevance weight between the entity word and each other entity word;

[0021] Based on the importance value corresponding to each entity word, the keywords contained in the statement to be detected are determined.

[0022] Optionally, before inputting the question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statement into the preset decision model, the method further includes:

[0023] For each anomaly category, obtain demonstration example data for that anomaly category. The demonstration example data includes: positive sample instances and negative sample instances. The positive sample instances are used to represent the question and answer text of questions that exist in that anomaly category, and the negative sample instances are used to represent the question and answer text of questions that do not exist in that anomaly category.

[0024] The question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statement are input into a preset decision model, specifically including:

[0025] The question statement corresponding to each anomaly category, the statement to be detected, the supplementary statement, and the demonstration example data for each anomaly category are input into the preset decision model.

[0026] Optionally, the method further includes:

[0027] When a new anomaly category is detected, the question statement used to detect whether the target model has the new anomaly category is determined as the question statement corresponding to the new anomaly category. The target model is used to obtain the statement to be detected for the new anomaly category. Based on the knowledge graph, the entity relations corresponding to at least some of the entity words contained in the statement to be detected for the new anomaly category are determined to generate supplementary statements for the new anomaly category.

[0028] The query statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, and the supplementary statement for the new anomaly category are input into the decision model to determine whether the target model has the decision result and criterion text for the new anomaly category.

[0029] Determine the comprehensive question statement corresponding to the statement to be detected for the new anomaly category, and use it as the comprehensive question statement corresponding to the new anomaly category;

[0030] The comprehensive question statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, the supplementary statement for the new anomaly category, the judgment result and the criterion text for the new anomaly category are input into the judgment model to obtain the final judgment result for the new anomaly category.

[0031] The target model is then detected based on the final judgment result for the new anomaly category.

[0032] This specification provides a model testing device, including:

[0033] The acquisition module is used to acquire the output statement of the target model in response to the input statement, which is used as the statement to be detected;

[0034] The supplementary module is used to determine the entity relationships corresponding to at least some of the entity words contained in the statement to be detected based on a preset knowledge graph, and to generate a statement describing the entity relationships as a supplementary statement.

[0035] The determination module is used to determine, for each anomaly category corresponding to the statement to be detected, the question statement used to detect whether the target model has that anomaly category, and to use it as the question statement corresponding to that anomaly category;

[0036] The input module is used to input the question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statement into the preset decision model, so as to obtain the decision result and criterion text of whether the target model has the anomaly category for each anomaly category. The criterion text is used to represent the text content on which the decision model gives the decision result.

[0037] The questioning module is used to determine the comprehensive questioning statement corresponding to the statement to be detected;

[0038] The decision module is used to input the comprehensive question statement, the statement to be detected, the supplementary statement, the decision result and criterion text corresponding to each anomaly category into the decision model to obtain the final decision result;

[0039] The detection module is used to detect the target model based on the final judgment result.

[0040] Optionally, the supplementary module is specifically used to: determine the keywords contained in the statement to be detected based on the entity words contained in the statement to be detected; determine the association subgraph corresponding to each keyword from a preset knowledge graph based on the keywords; and determine the entity relationships corresponding to at least some of the entity words contained in the statement to be detected based on the association subgraph corresponding to each keyword.

[0041] Optionally, the supplementary module is specifically used to: for each entity word contained in the statement to be detected, determine the relevance weight between the entity word and each other entity word, and determine the importance value corresponding to the entity word based on the relevance weight between the entity word and each other entity word; and determine the keywords contained in the statement to be detected based on the importance value corresponding to each entity word.

[0042] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for model detection.

[0043] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method for model detection.

[0044] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0045] In the model detection method provided in this specification, the target model's statement to be detected is obtained. Based on a preset knowledge graph, the entity relations corresponding to at least some of the entity words contained in the statement to be detected are determined, and supplementary statements describing the entity relations are generated. For each anomaly category corresponding to the statement to be detected, the corresponding question statement for that anomaly category is determined. The question statement, the statement to be detected, and the supplementary statements for each anomaly category are input into a preset decision model to obtain a decision result and criterion text for whether the target model possesses that anomaly category. The comprehensive question statement corresponding to the statement to be detected is determined. The comprehensive question statement, the statement to be detected, the supplementary statements, the decision result and criterion text for each anomaly category are input into the decision model to obtain the final decision result. Based on the final decision result, the target model is detected.

[0046] As can be seen from the above method, in the model detection method provided in this specification, for the phantom problem of the target model, firstly, based on each anomaly category of the phantom problem, the corresponding question statement for each anomaly category is obtained. The decision model then obtains the decision result for each anomaly category based on the corresponding question statement, the statement to be detected, and the supplementary statement, and provides the corresponding criterion text. Then, based on the decision result and criterion text corresponding to each anomaly category, combined with the previous input data, the statement to be detected is judged again. This further and accurately determines whether the target model contains a phantom problem and what type of anomaly it is, thereby training the target model in subsequent processes to ensure the accuracy of the target model's output results. Attached Figure Description

[0047] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0048] Figure 1 This is a flowchart illustrating a model detection method provided in this specification;

[0049] Figure 2 This specification provides a schematic diagram of a process for obtaining a final judgment result;

[0050] Figure 3This is a schematic diagram of a model testing device provided in this specification;

[0051] Figure 4 This specification provides a corresponding Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0053] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0054] Figure 1 This is a flowchart illustrating a model detection method provided in this specification, including the following steps:

[0055] S101: Obtain the statement output by the target model in response to the input statement, and use it as the statement to be detected.

[0056] With the rapid development of artificial intelligence, large language models (LLMs) have emerged. A LLM is a deep learning model trained on large amounts of text data that can generate natural language text or understand the meaning of language text. It can handle various natural language tasks, such as text classification, question answering, and dialogue. However, the results generated by large language models are not always accurate. There are several reasons for the inaccuracy of the output results, one of which is the phantom problem in the output.

[0057] The phantom problem refers to the possibility that large language models may produce outputs that contradict common sense or reality when generating or understanding text. This usually occurs because the large language model misunderstands and interprets certain information when processing the input data, resulting in generated content that does not match the actual situation.

[0058] The phantom problem in different large language models may have multiple types of anomalies. The following is a detailed explanation of the types of anomalies in the phantom problem of different large language models:

[0059] For large language models performing question-answering tasks, there are four types of anomalies in phantom questions: comprehension phantoms, reality phantoms, specificity phantoms, and reasoning phantoms. Comprehension phantoms refer to the large language model's failure to understand the question's context and intent. For example, given the question "Which of Tang Sanzang's disciples in Journey to the West can perform the 72 transformations: Sun Wukong, Zhu Bajie, or Sha Wujing?", the large language model incorrectly outputs "Tang Sanzang can perform the 72 transformations." This example demonstrates that while the large language model excels at understanding questions, it falls short when faced with questions containing grammatical errors or ambiguity. The challenging interrogative pronoun "which disciple" poses a challenge to the large language model. Therefore, instead of recognizing the question as a choice between three disciples, the large language model incorrectly interprets it as a choice between four names. Reality phantoms occur when the large language model lacks the necessary supporting facts to produce an accurate answer. This may be due to the large language model's lack of knowledge about specific entities, attributes, or events. For example, if the question is "Are Tower A and Tower B the same height?", the large language model incorrectly outputs "No, Tower A is 123 meters tall, and Tower B is 380 meters tall." This example indicates that the large language model has an incorrect understanding of the heights of the two buildings. Specificity illusion refers to the large language model's inability to answer questions with an appropriate level of specificity, manifesting as answers that are either too general or too specific. For example, if the question is "What is the common profession of person A and person B?", the large language model incorrectly answers "artist." The correct answer in this example is "singer," but if the large language model answers "artist," it lacks specificity. Reasoning illusion refers to the large language model possessing the knowledge needed to answer a question but being unable to effectively reason from facts to arrive at the correct answer. For example, given the question "Are car brand A and car brand B part of the same automotive group?", the large language model incorrectly outputs "Car brand A and car brand B are not part of the same automotive group. Car brand A is a luxury car brand and was acquired by car brand B in 1998." This example shows that the large language model may know the fact that car brand A was acquired by car brand B, but still incorrectly claims that the two are not part of the same automotive group.

[0060] For large language models performing knowledge dialogue tasks, the phantom problem encompasses three types of anomalies: external soft phantoms, external hard phantoms, and external group phantoms. External soft phantoms refer to responses that introduce a new text segment similar to the expected text segment but not conforming to valid triples in the given knowledge graph. For example, given the knowledge graph [(Movie 1, Director, Director A), (Movie 1, Leading Actor, Actor A), (Actor A, Plays, Role A), (Movie 2, Director, Director B), (Movie 2, Leading Actor, Actor A)]. The question is "Can you recommend a movie with the same leading actors as Movie 1?", and the response is "I recommend you watch Movie 3." In this example, although both "Movie 1" and "Movie 3" are mythological romance films, the response does not correspond to a valid triple in the knowledge graph. External hard phantoms refer to responses that introduce a new text segment different from the expected segment and not corresponding to any triple in the given knowledge graph. For example, given the same knowledge graph and question, the answer is "I recommend you watch movie 4". In this example, "movie 1" is a mythological romance film, and "movie 4" is a family comedy film, and neither corresponds to a triple instance in the knowledge graph. Outgroup illusion refers to an answer that introduces a new text segment, different from the expected segment, but with a specific predefined type, and does not correspond to a valid triple in the knowledge graph. For example, given the same knowledge graph and question, the answer is "I recommend you watch movie 5". In this example, "movie 1" is a mythological romance film, and "movie 5" is a modern romance film, both belonging to the same broad category of romance films.

[0061] For large language models performing summarization tasks, the phantom problem encompasses three types of anomalies: factual phantoms, non-factual phantoms, and intrinsically false phantoms. Factual phantoms refer to summarization sentences containing objectively true information, the veracity of which requires world knowledge to determine, but which cannot be corroborated by supporting sentences in the given document. For example, given the text segment "Person A is a hardworking athlete (...), he won the championship in competition a in 2000 (...), and he performed (...) in this competition," the output summary "Person A won the championship in competition a in country f." Although the information provided by external knowledge, "competition a was held in country f," aligns with real-world knowledge, it is not directly stated in the original text. Therefore, "country f" in the output summary is a factual phantom. Non-factual phantoms refer to summarization sentences containing objectively false information, for which no evidence or refutation of the phantom entity can be found in an external knowledge base. For example, given the text "To celebrate the arrival of the new year, people held a grand temple fair, where they could eat snacks from all over the country (...)," the output summary "People held a temple fair in place a." However, according to online reports, the news indicated that the temple fair was held in place b. Therefore, "place a" in the generated summary is a false illusion. An intrinsic false illusion refers to a summary that may contain an information entity from the document, but that entity is actually related to another event. For example, given the text "20-year-old athlete A comes from place a and represented place a in many large-scale events (...), during a competition in place b, he accidentally injured his leg and unfortunately retired," the output summary "Athlete A was injured and retired while participating in competition b in place a." Although the entity "place a" is mentioned in the source document, it is misused in the summary and incorrectly represents the information in the document. Therefore, "place a" in the generated summary is an intrinsic false illusion.

[0062] As can be seen from the above, different large language models correspond to different types of anomalies in the phantom problem, and these phantom problems cause the output results of the large language models to be inaccurate.

[0063] Based on this, this specification provides a model detection method. First, the output statement of the target model in response to the input statement is obtained as the statement to be detected. Based on a pre-defined knowledge graph, the entity relations corresponding to at least some of the entity words contained in the statement to be detected are determined, and statements describing these entity relations are generated as supplementary statements. For each anomaly category corresponding to the statement to be detected, the question statement used to determine whether the target model possesses that anomaly category is determined as the question statement corresponding to that anomaly category. The question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statements are input into a pre-defined decision model to obtain a decision result and criterion text for each anomaly category, indicating whether the target model possesses that anomaly category. The criterion text represents the text content on which the decision model bases its decision result. Then, a comprehensive question statement corresponding to the statement to be detected is determined. The comprehensive question statement, the statement to be detected, the supplementary statements, the decision results corresponding to each anomaly category, and the criterion text are input into the decision model to obtain the final decision result. Finally, the target model is detected based on the final decision result.

[0064] This decision-making method greatly improves the ability of the decision model to determine whether the target model's statement to be detected has a phantom problem. At the same time, the introduction of supplementary statements enables the decision model to judge the statement to be detected based on external knowledge, thereby accurately detecting whether the target model has a phantom problem, so as to further train the target model and obtain a more accurate target model.

[0065] To ensure stability and reliability during the detection process, the methods provided in this specification require some preparatory work, including: preparing servers, building backend communication ports, and opening application programming interface (API) services. Specifically, one or more servers need to be prepared. These servers need sufficient hardware resources, including adequate memory, central processing unit (CPU), and disk space, to support the operation of the decision model and handle a large number of requests. Simultaneously, an operating system and software dependencies need to be provided to the servers, installing a suitable operating system, such as Linux, and necessary software dependencies, such as Python and GPU drivers. Backend communication ports are built on the servers to access the decision model over the network. This invention uses the Fastapi framework to establish an API for each decision model service, allowing interaction between decision models through request sending. The specific steps for deploying the API service are as follows: 1. Package the backend service: Package the source code, configuration files, and necessary resources of the decision model into an executable file or container image. If using Python, a virtual environment can be used to isolate dependencies, unify control and scheduling, and achieve resource isolation and access control to prevent the failure of one dependent service from affecting other services. 2. Upload the backend service: Use tools such as File Transfer Protocol (FTP), Secure Copy Protocol (SCP), or the file upload function of a cloud service provider to transfer the packaged file to the server. 3. Start the backend service: Run the packaged backend service file on the server. Startup scripts, command lines, or process management tools can be used to manage the startup and operation of the service. 4. Test the backend service: Use Postman (a tool for API testing) to access the API interface of the decision model. Send requests and check the returned results to ensure the service responds correctly. 5. Test different types of input data to verify whether the deployed decision model can perform normally in various scenarios.

[0066] Furthermore, the server obtains the statement output by the target model in response to the input statement as the statement to be detected, wherein the statement to be detected includes the input statement and the statement output by the target model in response to the input statement.

[0067] S102: Based on a preset knowledge graph, determine the entity relationships corresponding to at least some of the entity words contained in the statement to be detected, and generate a statement to describe the entity relationships as a supplementary statement.

[0068] For each entity word in the statement to be detected, the server determines the relevance weight between that entity word and all other entity words, and determines the importance value of that entity word based on its relevance weight. Based on the importance value of each entity word, the server identifies the keywords contained in the statement to be detected. Based on the keywords, a subgraph corresponding to each keyword is determined from a pre-defined knowledge graph. Based on the subgraph corresponding to each keyword, the server determines the entity relationships corresponding to at least some of the entity words in the statement to be detected. The pre-defined knowledge graph can be a knowledge graph formed from factual knowledge obtained from Wikipedia, FreeBase, etc.

[0069] Specifically, keywords are extracted from the statements to be detected based on the PageRank algorithm. The server treats each entity word in the statement as a node, and for each node, determines the relevance weight between that node and all other nodes. This relevance weight is used to represent the strength of the relevance between any two nodes, that is, the strength of the semantic relevance between any two entity words.

[0070] For each node, its importance value is determined based on the correlation weights between that node and other nodes. The specific formula is as follows:

[0071]

[0072] Where d is the damping coefficient, with a value of 0.85, WS(v i ) represents node v i Importance value, WS(v j ) represents node v j Importance value, In(v i ) indicates pointing to node v i The set of all nodes, W ji Represents node v i With node v j The correlation weight between them Indicates pointing to node v k The sum of the relevance weights of all nodes, node v k This refers to the relationship with node v i All nodes that are connected.

[0073] Furthermore, the importance values ​​of each node are sorted in descending order, that is, from largest to smallest, to obtain the N most important entity words, which are then used as keywords in the statement to be detected. The server determines the associated subgraph corresponding to each keyword from a pre-defined knowledge graph, which can be specifically represented as follows:

[0074] Gsub =SPARQL Query(G KG ,q ′ key ),q ′ key ∈Q key

[0075] Among them, G sub G represents a related subgraph. KG This represents a predefined knowledge graph, specifically, G. KG =V KG ×R KG G KG For all entity nodes V KG The relationship between R and entity nodes KG The query set composed of keywords is Q. key ={x1;x2;…;x N},x1,x2,…,x N Each keyword represents a keyword. SPARQL is a query language specifically designed for graphs represented in Resource Description Framework (RDF) schemas. It can extract specific parts from knowledge represented in RDF format through selection, extraction, and other methods. The SPARQL protocol is used to send SPARQL queries to query the associated subgraph G. sub and associated subgraphs

[0076] The server converts the association subgraph corresponding to each keyword into a triple format. In other words, based on the correspondence between entity words in the association subgraph, the association subgraph is converted into a linear triple format (entity word-relationship-entity word). Then, this triple format information is converted into a statement to describe the entity relationship between entity words, as a supplementary statement.

[0077] S103: For each anomaly category corresponding to the statement to be detected, determine the question statement used to detect whether the target model has that anomaly category, and use it as the question statement corresponding to that anomaly category.

[0078] Whether the target model has a phantom problem can be determined by considering various anomaly types of phantom problems. For different anomaly types of phantom problems, the corresponding question statement is determined. That is, for each anomaly category, the server can determine the question statement corresponding to that anomaly category.

[0079] For example, when judging whether the target model has an illusion of understanding, the question could be, "Are you trying to determine whether the answer misunderstands the context and intent of the question?"; when judging whether the target model has an illusion of reality, the question could be, "Are you trying to determine whether there is a factual contradiction between the answer and world knowledge? Some information in the answer may be fabricated"; when judging whether the target model has an illusion of specificity, the question could be, "Are you trying to determine whether the answer is too general or too specific to answer the question with an appropriate level of specificity?"; and when judging whether the target model has an illusion of reasoning, the question could be, "Are you trying to determine whether the answer can be correctly inferred from knowledge?"

[0080] For a target model performing a knowledge dialogue task, when determining whether the target model has an external soft illusion, the question could be "You are trying to determine whether the real entity in the response has been replaced by a highly similar entity"; when determining whether the target model has an external hard illusion, the question could be "You are trying to determine whether the real entity in the response has been replaced by a different entity"; when determining whether the target model has an external group illusion, the question could be "You are trying to determine whether the real entity in the response has been replaced by a different entity from a different entity type".

[0081] For a target model performing a text summarization task, when determining whether the target model has a factual illusion, the question could be "You are trying to determine whether the summary is true, but some information cannot be directly inferred from or contained in the document"; when determining whether the target model has a non-factual illusion, the question could be "You are trying to determine whether there is some untrue and incorrect information in the summary"; when determining whether the target model has an inherent error illusion, the question could be "You are trying to determine whether there is a factual contradiction between the summary and the document".

[0082] S104: Input the question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statement into the preset decision model, so as to obtain the decision result and criterion text for each anomaly category regarding whether the target model has the anomaly category. The criterion text is used to represent the text content on which the decision model gives the decision result.

[0083] For each anomaly category, the server retrieves demonstration example data for that category. This demonstration example data includes positive sample instances and negative sample instances. Positive sample instances represent question-and-answer text for questions where that anomaly category exists. A positive sample instance may include a section of text to be detected, supplementary statements for the text to be detected, and a judgment result indicating the presence of that anomaly category in the text. Negative sample instances represent question-and-answer text for questions where that anomaly category does not exist. A negative sample instance may include a section of text to be detected, supplementary statements for the text to be detected, and a judgment result indicating the absence of that anomaly category in the text. Providing demonstration example data to the decision model allows it to better determine whether the detected statement belongs to that anomaly category.

[0084] Furthermore, the server can input the question statement, the statement to be detected, the supplementary statement, and the demonstration example data of each anomaly category into the preset decision model, so as to obtain the decision result and the criterion text for each anomaly category regarding whether the target model has that anomaly category. The criterion text is used to represent the text content on which the decision model gives the decision result.

[0085] In other words, for each anomaly category, the corresponding question statement, the statement to be detected, the supplementary statement, and the demonstration example data of that anomaly category are input into the preset decision model to obtain the decision result and criterion text for whether the target model has that anomaly category. That is, there will be as many decision models as there are anomaly categories to output the decision result and criterion text for whether the target model has that anomaly category.

[0086] The decision model can be an LLAMA-2 model, a large open-source model whose general working steps are as follows: The input text is tokenized, meaning it is segmented into tokens, typically basic units of text such as words or punctuation marks. Each token is then converted into a vector representation. This vector is usually a point in a high-dimensional space, used to capture the semantic information of the token. The LLAMA-2 model uses a Transformer encoder to encode the input token sequence. The encoder, through a self-attention mechanism and a feedforward neural network, captures the contextual information and semantic relationships between tokens, generating a context vector. After obtaining the context vector, the LLAMA-2 model uses a decoder to decode and generate the output text. The decoder, through a self-attention mechanism and a feedforward neural network, progressively generates the token sequence of the output text based on the context vector and known input information. After decoding, the LLAMA-2 model outputs the generated text result.

[0087] Of course, the above-mentioned decision model can also be other models. This specification does not limit the specific form of the decision model, as long as it can guarantee the functional requirements required by this specification.

[0088] It's important to note that the server, along with the input data, needs to pre-set an intent statement for the decision model to clarify its purpose. This intent statement prompts the decision model to determine whether the statement to be detected contains a phantom problem based on the question statement, the statement to be detected, and the supplementary statement corresponding to each type of anomaly, and to provide a reasonable decision. For example, this intent statement could be: "You are a phantom detector. You must determine whether the provided answer contains a phantom based on your knowledge of the world. Your answer must be 'yes' or 'no'."

[0089] S105: Determine the comprehensive question statement corresponding to the statement to be detected.

[0090] The server determines the comprehensive question statement corresponding to the statement to be tested. Specifically, the server obtains the number of anomaly types of the phantom questions corresponding to the statement to be tested (the number of anomaly types can be understood as the number of phantom questions mentioned in the statement to be tested that indicate the existence of the target model). Based on the number of anomaly types, the server determines a comprehensive question statement from a pre-defined statement library of comprehensive question statements that matches the number of anomaly types of the phantom questions corresponding to the statement to be tested, and uses this as the comprehensive question statement corresponding to the statement to be tested. For example, for the target model performing the question-answering task, the comprehensive question statement could be "You attempt to summarize the final judgment based on the judgment results and criterion texts corresponding to the four anomaly types of phantom questions output by the judgment model."

[0091] S106: Input the comprehensive question statement, the statement to be detected, the supplementary statement, the judgment result and criterion text corresponding to each anomaly category into the judgment model to obtain the final judgment result.

[0092] The server inputs the query statement, the statement to be detected, the supplementary statement, each anomaly category, and the corresponding judgment result and criterion text into the decision model. The decision model then uses this input data to arrive at a final judgment. This approach allows the decision model to analyze the judgment result and criterion text corresponding to each anomaly category, determine the anomaly category for the phantom problem, and further evaluate the statement to be detected based on the judgment result and criterion text corresponding to each anomaly category, resulting in a more accurate judgment.

[0093] S107: Detect the target model based on the final judgment result.

[0094] The server performs detection on the target model based on the final judgment result. Based on the detection results, it trains the target model to obtain a more accurate output model. Here, the target model can refer to a large language model for question-answering tasks, a large language model for knowledge dialogue tasks, a large language model for summarizing tasks, etc.

[0095] It should be noted that when the server detects a new anomaly category, it determines the question statement used to check whether the target model possesses the new anomaly category, which serves as the question statement corresponding to the new anomaly category. It also obtains the statement to be detected for the new anomaly category from the target model. Based on the knowledge graph, it determines the entity relations corresponding to at least some of the entity words in the statement to be detected for the new anomaly category, thereby generating supplementary statements for the new anomaly category. Further, the server inputs the question statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, and the supplementary statements for the new anomaly category into the decision model to determine the decision result and criterion text for whether the target model possesses the new anomaly category.

[0096] Then, the server determines the comprehensive question statement corresponding to the statement to be detected for the new anomaly category. As the comprehensive question statement corresponding to the new anomaly category, the comprehensive question statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, the supplementary statement for the new anomaly category, the judgment result and criterion text for the new anomaly category are input into the judgment model to obtain the final judgment result for the new anomaly category. Then, the target model is detected based on the final judgment result for the new anomaly category.

[0097] Alternatively, the server determines the comprehensive question statement corresponding to the statement to be detected for the new anomaly category, and uses it as the comprehensive question statement corresponding to the new anomaly category. The comprehensive question statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, the supplementary statement for the new anomaly category, the judgment result and criterion text for the new anomaly category, and the judgment result and criterion text of the original anomaly category are all input into the judgment model to obtain the final judgment result. Then, the target model is detected based on the final judgment result.

[0098] In other words, when a new anomaly category appears, the decision model judges the new anomaly category and obtains the judgment result and criterion text corresponding to the new anomaly category. Furthermore, the decision model judges the phantom problem of the statement to be detected again based on the judgment result and criterion text corresponding to the new anomaly category and the judgment result and criterion text corresponding to the original anomaly category, so as to obtain the final judgment result of the phantom problem for the target model.

[0099] In this specification, for large language models performing different tasks, there may be multiple categories of phantom problems with various exceptions. For each category of phantom problem exceptions, the corresponding question statement, demonstration example data, the statement to be detected, and supplementary statements are input into the decision model to obtain the decision result and criterion text corresponding to that category of phantom problem exceptions. Then, the decision model uses the decision result and criterion text corresponding to each category of phantom problem exceptions to comprehensively consider each category of phantom problem exceptions to obtain the final decision result. In other words, it is equivalent to subdividing a whole phantom problem into many exception categories and using the decision result and criterion text corresponding to each exception category to obtain the final decision result. This method makes the final decision result more accurate, and can effectively detect whether the model contains phantom problems using the method provided in this specification. It can also better detect the target model based on the final decision result and make corresponding adjustments to the target model to obtain a more accurate target model.

[0100] Furthermore, when determining whether the target model exhibits a phantom problem, the decision model incorporates supplementary statements. This approach allows the model to utilize external knowledge data to evaluate the statements being tested, significantly improving accuracy. For each anomaly category, the decision model can more accurately determine whether the target model possesses that anomaly, leading to a more correct final decision. When new phantom problem anomaly categories emerge, the decision model can also classify them, thus providing flexibility in the method described in this specification.

[0101] In the process of obtaining the final judgment result described above, the same judgment model is used. That is, different data are input into the judgment model to obtain different results. In order to better describe the method provided in this specification, a model detection method using multiple judgment models is also provided here.

[0102] Here's an example: a large language model implementing a question-answering task can have four anomaly categories for phantom questions. Therefore, each of these four anomaly categories corresponds to a different decision model. Each decision model has a higher accuracy rate for its corresponding anomaly category. Then, the decision results and criterion text of each decision model for its corresponding anomaly category are input into the decision model to obtain the final decision result. Here, the decision model that yields the final decision result can refer to a decision model that effectively summarizes the output results of multiple decision models for each anomaly category, such as... Figure 2 As shown.

[0103] Figure 2 This is a schematic diagram illustrating a process for obtaining a final judgment result, as provided in this specification.

[0104] from Figure 2 As can be seen, the process first involves obtaining a pre-defined knowledge graph, determining the association subgraphs corresponding to the keywords contained in the statement to be detected, converting the association subgraphs into triplet format to obtain the supplementary statements corresponding to the statement to be detected, and inputting the supplementary statements, the statement to be detected, the demonstration example data corresponding to each anomaly category, and the question statement into the decision model corresponding to each anomaly category to obtain the decision result and criterion text corresponding to each anomaly category. This allows the decision model to obtain the final decision result based on the decision result and criterion text corresponding to each anomaly category, the statement to be detected, and the supplementary statements.

[0105] For example, for a large language model performing a question-answering task, there are four anomaly categories for phantom questions, each corresponding to one of four decision models. The set of these four decision models can be represented as M. QA = {m1, m2, m3, m4}, where m1 is the decision model corresponding to comprehensible illusions, m2 is the decision model corresponding to realistic illusions, m3 is the decision model corresponding to specific illusions, and m4 is the decision model corresponding to reasoning illusions, in order to judge illusion problems of different abnormal categories.

[0106] For a large language model performing a knowledge dialogue task, there are three anomaly categories for the illusion problem, each corresponding to one of three decision models. The set of these three decision models can be represented as M. Dialogue = {m1, m2, m3}, where m1 is the decision model corresponding to external soft phantoms, m2 is the decision model corresponding to external hard phantoms, and m3 is the decision model corresponding to external group phantoms, in order to judge phantom problems of different abnormal categories.

[0107] For large language models performing text summarization tasks, there are three anomaly categories for the phantom problem, each corresponding to a different decision model. The set of these three decision models can be represented as M. Summary = {m1, m2, m3}, where m1 is the decision model corresponding to factual illusions, m2 is the decision model corresponding to non-factual illusions, and m3 is the decision model corresponding to inherently erroneous illusions, in order to judge illusion problems of different abnormal categories.

[0108] Furthermore, the supplementary statement, the statement to be detected, the demonstration example data corresponding to each anomaly category, and the question statement are respectively input into the decision model, and the decision result and criterion text corresponding to each anomaly category are output, which can be expressed by the formula:

[0109] {J i B i}=f i (D,G sub ,T),m i ∈M

[0110] Among them, J i Denotes the decision model m corresponding to the i-th anomaly category. i The output judgment result, B i Denotes the decision model m corresponding to the i-th anomaly category. i The output criterion text, f represents the decision model m i The process of outputting results, where D represents the statement to be detected, and G... sub This represents the association subgraph corresponding to some entity words contained in the statement to be detected. T represents the decision template corresponding to the decision model. It is used to input a complete decision template into the decision model by forming an intent statement, a question statement corresponding to each anomaly category, a supplementary statement, a statement to be detected, and a demonstration example data corresponding to each anomaly category, so as to facilitate the decision model to process a large amount of input data.

[0111] Then, the combined question statement, the statement to be detected, the supplementary statement, the judgment result and criterion text corresponding to each anomaly category are input into the judgment model to obtain the final judgment result, which can be specifically represented as follows:

[0112] J sum =g(D,G sub ,T sum ,J1,B1,…,J N B N )

[0113] Among them, J sum Indicates the final judgment result, J i Denotes the decision model m corresponding to the i-th anomaly category. i The output judgment result, B i Denotes the decision model m corresponding to the i-th anomaly category. i The output criterion text contains N anomaly categories, where g represents the process by which the decision model outputs the final decision result, and T... sum This represents the decision template corresponding to the decision model. It is used to input a complete decision template into the decision model by forming an intent statement, a comprehensive question statement, a supplementary statement, a statement to be detected, the decision result corresponding to each anomaly category, and the judgment text. This facilitates the decision model to process the input data and then use the processed data to obtain the final decision result.

[0114] The above describes one or more implementation methods for model testing in this specification. Based on the same approach, this specification also provides corresponding model testing devices, such as... Figure 3 As shown.

[0115] Figure 3 A schematic diagram of a model testing device provided in this specification includes:

[0116] The acquisition module 301 is used to acquire the statement output by the target model in response to the input statement, as the statement to be detected;

[0117] The supplementary module 302 is used to determine the entity relationships corresponding to at least some of the entity words contained in the statement to be detected based on a preset knowledge graph, and to generate a statement describing the entity relationships as a supplementary statement;

[0118] The determination module 303 is used to determine, for each anomaly category corresponding to the statement to be detected, the question statement used to detect whether the target model has that anomaly category, and to use it as the question statement corresponding to that anomaly category.

[0119] The input module 304 is used to input the question statement corresponding to each anomaly category, the statement to be detected, and the supplementary statement into a preset decision model, so as to obtain a decision result and a criterion text for each anomaly category regarding whether the target model has the anomaly category. The criterion text is used to represent the text content on which the decision model gives the decision result.

[0120] Questioning module 305 is used to determine the comprehensive questioning statement corresponding to the statement to be detected;

[0121] The decision module 306 is used to input the comprehensive question statement, the statement to be detected, the supplementary statement, the decision result and criterion text corresponding to each anomaly category into the decision model to obtain the final decision result;

[0122] The detection module 307 is used to detect the target model based on the final judgment result.

[0123] Optionally, the supplementary module 302 is specifically used to: determine the keywords contained in the statement to be detected based on the entity words contained in the statement to be detected; determine the association subgraph corresponding to each keyword from a preset knowledge graph based on the keywords; and determine the entity relations corresponding to at least some of the entity words contained in the statement to be detected based on the association subgraph corresponding to each keyword.

[0124] Optionally, the supplementary module 302 is specifically used to: determine the relevance weight between each entity word contained in the statement to be detected and other entity words, and determine the importance value corresponding to the entity word based on the relevance weight between the entity word and other entity words; and determine the keywords contained in the statement to be detected based on the importance value corresponding to each entity word.

[0125] Optionally, before inputting the question statement, the statement to be detected, and the supplementary statement corresponding to each anomaly category into the preset decision model, the input module 304 is further configured to obtain demonstration example data for each anomaly category, the demonstration example data including: positive sample instances and negative sample instances, the positive sample instances being used to represent the question and answer text of questions that exist in this anomaly category, and the negative sample instances being used to represent the question and answer text of questions that do not exist in this anomaly category;

[0126] The input module 304 is specifically used to input the question statement corresponding to each anomaly category, the statement to be detected, the supplementary statement, and the demonstration example data of each anomaly category into the preset decision model.

[0127] Optionally, the decision module 306 is further configured to, when a new anomaly category is detected, determine the question statement used to detect whether the target model has the new anomaly category, as the question statement corresponding to the new anomaly category; and, through the target model, obtain the statement to be detected for the new anomaly category, and based on the knowledge graph, determine the entity relations corresponding to at least some of the entity words contained in the statement to be detected for the new anomaly category, so as to generate supplementary statements for the new anomaly category; and combine the question statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, and the statement to be detected for the new anomaly category... Supplementary statements are input into the decision model to determine whether the target model has the decision result and criterion text for the new anomaly category; a comprehensive question statement corresponding to the statement to be detected for the new anomaly category is determined as the comprehensive question statement corresponding to the new anomaly category; the comprehensive question statement corresponding to the new anomaly category, the statement to be detected for the new anomaly category, the supplementary statement for the new anomaly category, the decision result and criterion text for the new anomaly category are input into the decision model to obtain the final decision result for the new anomaly category; the target model is detected based on the final decision result for the new anomaly category.

[0128] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 A model detection method is provided.

[0129] This instruction manual also provides Figure 4 The one shown corresponds to Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 4At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The model detection method described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0130] Improvements in a technology can be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many improvements to the methodology can now be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that an improvement in methodology cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are the most commonly used. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0131] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0132] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0133] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0134] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0136] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0137] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0138] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0139] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0140] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0141] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0144] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0145] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A model checking method, characterized by, The method comprises the following steps: acquiring a statement output by a target model for an input statement as a to-be-detected statement, the to-be-detected statement comprising the input statement and the statement output by the target model for the input statement; determining an entity relationship corresponding to at least part of entity words contained in the to-be-detected statement based on a preset knowledge graph, and generating a statement for describing the entity relationship as a supplementary statement; for each abnormal category corresponding to the to-be-detected statement, determining a question statement used for detecting whether the target model has the abnormal category as a question statement corresponding to the abnormal category; inputting the question statement corresponding to each abnormal category, the to-be-detected statement and the supplementary statement into a preset decision model to obtain, for each abnormal category, a decision result and a criterion text of a question of whether the target model has the abnormal category, the criterion text being used for representing a basis text content of the decision result given by the decision model; determining a comprehensive question statement corresponding to the to-be-detected statement, the comprehensive question statement being matched from a preset statement library of comprehensive question statements according to a number of abnormal categories of a phantom question corresponding to the to-be-detected statement; inputting the comprehensive question statement, the to-be-detected statement, the supplementary statement, the decision result and the criterion text of each abnormal category into the decision model to obtain a final decision result; detecting the target model according to the final decision result.

2. The method of claim 1, wherein, The method comprises the following steps: determining an entity relationship corresponding to at least part of entity words contained in the to-be-detected statement based on a preset knowledge graph, and generating a statement for describing the entity relationship as a supplementary statement; determining a key word contained in the to-be-detected statement according to an entity word contained in the to-be-detected statement; determining an associated subgraph corresponding to each key word from a preset knowledge graph according to the key word; 3. The method of claim 2, wherein, determining an entity relationship corresponding to at least part of entity words contained in the to-be-detected statement according to the associated subgraph corresponding to each key word. The method comprises the following steps: determining a key word contained in the to-be-detected statement according to an entity word contained in the to-be-detected statement, specifically comprising the following steps:

4. The method of claim 1, wherein, determining a correlation weight between each entity word and each other entity word contained in the to-be-detected statement, and determining an importance value corresponding to the entity word according to the correlation weight between the entity word and each other entity word; determining a key word contained in the to-be-detected statement according to the importance value corresponding to each entity word. Before inputting the question statement corresponding to each abnormal category, the to-be-detected statement and the supplementary statement into the preset decision model, the method further comprises the following steps: for each abnormal category, acquiring demonstration example data for the abnormal category, the demonstration example data comprising a positive sample instance and a negative sample instance, the positive sample instance being used for representing a question and answer text of a question of the abnormal category, and the negative sample instance being used for representing a question and answer text of a question without the abnormal category; inputting the question statement corresponding to each abnormal category, the to-be-detected statement and the supplementary statement into the preset decision model, specifically comprising the following steps: input the question statement corresponding to each abnormality category, the to-be-detected statement, the supplementary statement, and the demonstration example data of each abnormality category into a preset decision model.

5. The method of claim 1, wherein, The method further includes: When a new abnormality category is monitored, determining a question statement used to detect whether the target model has the new abnormality category as the question statement corresponding to the new abnormality category, and obtaining, through the target model, a to-be-detected statement for the new abnormality category, and determining an entity relationship corresponding to at least part of entity words contained in the to-be-detected statement for the new abnormality category based on the knowledge graph to generate a supplementary statement for the new abnormality category; inputting the question statement corresponding to the new abnormality category, the to-be-detected statement for the new abnormality category, and the supplementary statement for the new abnormality category into the decision model to obtain a decision result and criterion text of whether the target model has the new abnormality category; determining a comprehensive question statement corresponding to the to-be-detected statement for the new abnormality category as a comprehensive question statement corresponding to the new abnormality category; inputting the comprehensive question statement corresponding to the new abnormality category, the to-be-detected statement for the new abnormality category, the supplementary statement for the new abnormality category, the decision result and criterion text of the new abnormality category into the decision model to obtain a final decision result for the new abnormality category; detecting the target model according to the final decision result for the new abnormality category.

6. A model checking apparatus characterized by comprising: includes: an obtaining module, configured to obtain a statement output by a target model for an input statement as a to-be-detected statement, the to-be-detected statement including the input statement and a statement output by the target model for the input statement; a supplementing module, configured to determine an entity relationship corresponding to at least part of entity words contained in the to-be-detected statement based on a preset knowledge graph, and generate a statement used to describe the entity relationship as a supplementary statement; a determining module, configured to determine, for each abnormality category corresponding to the to-be-detected statement, a question statement used to detect whether the target model has the abnormality category as a question statement corresponding to the abnormality category; an inputting module, configured to input the question statement corresponding to each abnormality category, the to-be-detected statement, and the supplementary statement into a preset decision model to obtain, for each abnormality category, a decision result and criterion text of a question of whether the target model has the abnormality category, the criterion text being used to represent basis text content of the decision result given by the decision model; a questioning module, configured to determine a comprehensive question statement corresponding to the to-be-detected statement, the comprehensive question statement being matched from a preset statement library about comprehensive question statements according to a number of abnormality categories of a phantom question corresponding to the to-be-detected statement; a decision module, configured to input the comprehensive question statement, the to-be-detected statement, the supplementary statement, the decision result and criterion text of each abnormality category into the decision model to obtain a final decision result. The detection module is configured to detect the target model according to the final decision result.

7. The apparatus of claim 6, wherein, The supplement module is specifically configured to determine a keyword contained in the to-be-detected sentence according to an entity word contained in the to-be-detected sentence, determine a corresponding associated subgraph of each keyword from a preset knowledge graph according to the keyword, and determine an entity relationship corresponding to at least part of the entity words contained in the to-be-detected sentence according to the corresponding associated subgraph of each keyword.

8. The apparatus of claim 7, wherein, The supplement module is specifically configured to determine a correlation weight between each entity word and each other entity word contained in the to-be-detected sentence, and determine an importance value corresponding to the entity word according to the correlation weight between the entity word and each other entity word. The supplement module is specifically configured to determine a keyword contained in the to-be-detected sentence according to an importance value corresponding to each entity word.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-5.

10. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the method in any one of claims 1-5.

Images