Method and apparatus for evaluating content detection strategy based on large model, electronic device

Abstract

This disclosure provides an evaluation method for content detection strategies based on a large model, relating to artificial intelligence technologies such as large models, natural language processing, content detection, and content security. The evaluation method for content detection strategies based on a large model includes: acquiring target resources and detection data obtained by performing content detection on the target resources using the content detection strategy to be evaluated; using a large model to verify the review point data based on the target resources, and generating a first detection question based on the verification result; using a large model to verify the evidence chain data based on a pre-set evidence library, and generating a second detection question based on the verification result; using a large model to verify the inference process data and tag data, and generating a third detection question based on the verification result; and obtaining the evaluation result of the content detection strategy to be evaluated based on the first, second, and third detection questions. This disclosure can improve the evaluation accuracy and efficiency of content detection strategies.

Description

Technical Field

[0001] This disclosure relates to the field of internet technology, and more particularly to artificial intelligence technologies such as large-scale models, natural language processing, content detection, and content security. It provides an evaluation method, apparatus, electronic device, and readable storage medium for content detection strategies based on large-scale models. Background Technology

[0002] Content detection is one of the core tasks in the field of content security, covering various types such as detection of false information, detection of celebrity statements without sources, detection of clickbait headlines, and detection of old news being passed off as news.

[0003] Existing technologies typically employ corresponding content detection strategies to detect content in resources. Specifically, they use a content detection model within the content detection strategy and a "search-reasoning-judgment" detection logic to obtain tags for the corresponding resource content.

[0004] For the evaluation of content detection strategies or content detection models, existing technologies typically employ traditional classification index evaluation or manual item-by-item evaluation, which suffers from problems such as poor evaluation accuracy, high evaluation cost, and low evaluation efficiency. Summary of the Invention

[0005] According to a first aspect of this disclosure, an evaluation method for a content detection strategy based on a large model is provided, comprising: acquiring a target resource and detection data obtained by performing content detection on the target resource using the content detection strategy to be evaluated, the detection data including review point data, evidence chain data, reasoning process data, and tag data; using a large model to verify the review point data based on the target resource, and generating a first detection question based on the verification result; using the large model to verify the evidence chain data based on a preset evidence library, and generating a second detection question based on the verification result; using the large model to verify the reasoning process data and the tag data, and generating a third detection question based on the verification result; and obtaining an evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question, and the third detection question.

[0006] According to a second aspect of this disclosure, an evaluation apparatus for a content detection strategy based on a large model is provided, comprising: an acquisition unit, configured to acquire a target resource and detection data obtained by performing content detection on the target resource using the content detection strategy to be evaluated, the detection data including review point data, evidence chain data, reasoning process data, and tag data; a first verification unit, configured to use the large model to verify the review point data based on the target resource, and generate a first detection question based on the verification result; a second verification unit, configured to use the large model to verify the evidence chain data based on a preset evidence library, and generate a second detection question based on the verification result; a third verification unit, configured to use the large model to verify the reasoning process data and the tag data, and generate a third detection question based on the verification result; and an evaluation unit, configured to obtain an evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question, and the third detection question.

[0007] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0008] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described above.

[0009] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0012] Figure 1 This is a schematic diagram based on the first embodiment of the present disclosure;

[0013] Figure 2 This is a schematic diagram according to the second embodiment of the present disclosure;

[0014] Figure 3 This is a schematic diagram according to the third embodiment of the present disclosure;

[0015] Figure 4 This is a schematic diagram according to the fourth embodiment of the present disclosure;

[0016] Figure 5 This is a block diagram of an electronic device used to implement the evaluation method of the content detection strategy based on a large model according to the embodiments of this disclosure. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and mechanisms are omitted in the following description.

[0018] Figure 1 This is a schematic diagram based on the first embodiment of this disclosure. (See diagram below.) Figure 1 As shown, the evaluation method for content detection strategies based on large models in this embodiment specifically includes the following steps:

[0019] S101. Obtain the target resource and the detection data obtained by performing content detection on the target resource using the content detection strategy to be evaluated. The detection data includes review point data, evidence chain data, reasoning process data and tag data.

[0020] S102. Using a large model, verify the review point data based on the target resource, and generate a first detection question based on the verification result;

[0021] S103. Using the large model, verify the evidence chain data according to the preset evidence library, and generate a second detection question based on the verification result;

[0022] S104. Using the large model, verify the inference process data and the label data, and generate a third detection question based on the verification results;

[0023] S105. Based on the first detection question, the second detection question, and the third detection question, obtain the evaluation result of the content detection strategy to be evaluated.

[0024] The content detection strategy evaluation method based on a large model in this embodiment uses a large model to complete review point verification, evidence verification, and consistency verification between reasoning and conclusion, based on the target resource and the detection data obtained when using the content detection strategy to be evaluated to perform content detection on the target resource. Thus, the evaluation result of the content detection strategy to be evaluated is obtained based on the detection questions generated by the verification results. This achieves the purpose of performing a full-link quality evaluation of the reasoning link (review point extraction, evidence citation, reasoning logic, and conclusion determination) of the content detection strategy, thereby raising the evaluation dimension of the content detection strategy from the tag level to the reasoning link level. This can improve the evaluation accuracy of the content detection strategy, achieve the purpose of automating the evaluation of the content detection strategy, and also improve the evaluation efficiency of the content detection strategy.

[0025] In this embodiment, the content detection strategy is a strategy used to perform content detection on resources to determine whether the resources contain false or misleading content. The content detection strategy includes a content detection model and may also include corresponding prompt text.

[0026] In this embodiment, non-true or misleading content is collectively referred to as general false content. Types of general false content may include false information, celebrity statements without a source, clickbait headlines, and old news masquerading as news.

[0027] In this embodiment, when executing S101, the target resource obtained can be one or more, and the number of content detection strategies to be evaluated can be one or more; that is, this embodiment can evaluate one or more content detection strategies based on one or more target resources.

[0028] In this embodiment, different content detection strategies may include different content detection models or different prompt texts.

[0029] The detection data obtained in S101 of this embodiment is text data in natural language format, including Verification Points data, Evidence Chain data, Reasoning Process data, and Label data obtained by the content detection strategy to be evaluated when performing content detection on the target resource.

[0030] The review point data includes at least one review point, which is a factual unit that needs to be verified and extracted from the target resource by the content detection strategy to be evaluated; the evidence chain data includes at least one citation data, which is the evidence obtained by the content detection strategy to be evaluated for each review point through retrieval or matching; the reasoning process data includes the judgment result made on the review point based on the evidence and the logic of obtaining the detection conclusion based on multiple judgment results; the tag data is the tag reflecting the detection result of the target resource, which can include two types of tags: "approved" and "failed".

[0031] In this embodiment, when using the content detection strategy to be evaluated to perform content detection on the target resource, the content detection strategy to be evaluated (specifically the content detection model in the content detection strategy to be evaluated) first extracts review points from the target resource, then obtains evidence for the corresponding review points, then combines the obtained evidence to reason about the review points, and finally obtains the label of the corresponding target resource based on the reasoning result.

[0032] In this embodiment, after executing S101 to obtain target resources and detection data, S102 is executed to use a large model to verify the review point data based on the target resources, and generate the first detection question based on the verification result.

[0033] The large model in this embodiment can be a large language model (LLM) or a multimodal large model; this embodiment does not limit it.

[0034] Specifically, in this embodiment, when executing S102 using a large model to verify the review point data based on the target resource and generating a first detection question based on the verification result, the following implementation method can be adopted: input the target resource and review point data into the large model, and have the large model perform accuracy verification and integrity verification on the review point data based on the target resource; based on the obtained accuracy verification result and integrity verification result, the large model generates a first detection question.

[0035] In this embodiment, the large model verifies the accuracy of the review point data. Specifically, the large model verifies whether the content of the review points is faithful to the original text and does not introduce illusions or biased interpretations (such as treating others' opinions as factual statements or incorporating incorrect dates into the statements). In this embodiment, the large model also verifies the completeness of the review point data. Specifically, the large model verifies whether the review points in the review point data are complete and do not omit any key verifiable facts in the original text.

[0036] In this embodiment, when executing S102, the large model can generate a first detection question if it is determined that the obtained accuracy verification result and / or integrity verification result includes "failed verification". The generated first detection question is used to reflect that the content detection strategy to be evaluated has problems with the accuracy of extraction and / or the integrity of extraction when extracting review points. The generated first detection question can be "problematic review points exist".

[0037] It is understandable that in this embodiment, if the accuracy verification result and the integrity verification result obtained by the large model are both "passed verification" when executing S102, the generation of the first detection question may not be performed.

[0038] In this embodiment, after executing S102 to generate the first detection question based on the verification result, S103 is executed to use a large model to verify the evidence chain data according to the preset evidence library, and generate the second detection question based on the verification result.

[0039] Specifically, in this embodiment, when executing S103 using a large model to verify the evidence chain data according to a preset evidence library and generating a second detection question based on the verification results, the following implementation method can be adopted: input the evidence chain data into the large model, and the large model performs evidence existence verification, evidence consistency verification, and evidence source attribution verification on each piece of evidence in the evidence chain data based on the preset evidence library; based on the obtained evidence existence verification results, evidence consistency verification results, and evidence source attribution verification results, the large model generates a second detection question.

[0040] In other words, this embodiment establishes the "evidence base boundary principle," which stipulates that all factual claims (i.e., evidence in the evidence chain) of the content detection strategy to be evaluated must and can only come from content that actually exists in the preset database; otherwise, they are all considered fictitious.

[0041] In this embodiment, the large model performs evidence existence verification for each piece of evidence. Specifically, the large model searches for evidence in the evidence chain data within a preset database, and determines whether there is evidence fabrication based on the search results (i.e., the model claims that the evidence is "confirmed by search," but there is no corresponding content in the preset evidence database). The large model also performs evidence consistency verification for each piece of evidence. Specifically, the large model compares the evidence in the evidence chain data with the corresponding evidence in the preset evidence database, and determines whether there is evidence tampering (e.g., distortion of numbers, people, or causal relationships) based on the comparison results. Finally, the large model verifies the source attribution of each piece of evidence. Specifically, the large model compares the source of the evidence in the preset database with the source of the evidence in the evidence chain data, and determines whether there is source fabrication based on the comparison results (e.g., claiming evidence from source A as source B).

[0042] When executing S103 in this embodiment, the following may be included: determining the verification type that failed the evidence verification, including existence verification, consistency verification and source attribution verification; the large model generates a second detection question based on the determined verification type.

[0043] In this embodiment, if there are multiple verification types that fail the evidence verification, the large model will generate a second detection question corresponding to each verification type. For example, the generated second detection question can be "there is fabricated evidence", "there is tampered evidence", "there is a fabricated source", etc., thereby improving the accuracy of the generated second detection question.

[0044] In this embodiment, after executing S103 to generate the second detection question based on the verification result, S104 is executed to use a large model to verify the inference process data and label data, and generate the third detection question based on the verification result.

[0045] Specifically, in this embodiment, when executing S103 using a large model to verify the inference process data and label data, and generating a third detection question based on the verification result, the following implementation method can be adopted: input the inference process data and label data into the large model, and have the large model verify the logical consistency between the inference process data and label data; based on the obtained logical consistency verification result, the large model generates a third detection question.

[0046] In this embodiment, the large model verifies the logical consistency between the reasoning process data and the label data. Specifically, the large model extracts the judgment results made for each review point from the reasoning process data, and then verifies whether the reasoning process when judging the review point is consistent with the label data (if the reasoning process when judging the review point believes that "the core facts are true and accurate", but the label data is "the review is not approved", then it is inconsistent; if the reasoning process when judging the review point believes that "the facts are seriously false", but the label data is "the review is approved", then it is inconsistent.

[0047] In this embodiment, when executing S104, the large model can generate a third detection problem if the obtained logical consistency verification result is "inconsistent". The generated third detection problem is used to reflect the logical collapse problem in the logical consistency judgment of the content detection strategy to be evaluated. The generated third detection problem can be "logical collapse".

[0048] In other words, this embodiment uses a large model to sequentially perform review point verification, evidence verification, and logical consistency verification, and generates corresponding detection questions based on the verification results. Compared with the existing evaluation paradigm of "only evaluating the correctness of tags", this embodiment verifies the complete inference chain of the content detection strategy link by link, realizing an evaluation paradigm that penetrates the entire inference chain, thereby effectively improving the evaluation accuracy.

[0049] When executing S104, this embodiment may also include the following: obtaining the Judgment Code, which serves as the business knowledge base for the large model during evaluation, to perform compliance checks on the conclusions during logical consistency verification; inputting the obtained Judgment Code, reasoning process data, and label data into the large model, which then verifies the logical consistency between the reasoning process data and the label data based on the Judgment Code.

[0050] In other words, the large model in this embodiment can also be combined with the judicial code for logical consistency verification, so that the large model has a certain business rule awareness capability. When the business rules change, only the content of the judicial code needs to be updated, without retraining the large model, thereby supporting the dynamic updating of business rules.

[0051] In this embodiment, the judgment code may include hierarchical control standards (e.g., P0 level is content that must be controlled, P1 level is content that requires special attention, and P2 level is content that can be handled at the discretion of the authorities), exemption rules for non-control scope (clarifying which types of content should not be marked as false), and source level system (credibility classification rules for first-level, second-level, and third-level sources), etc.; the judgment code can be injected into the large model in the form of text.

[0052] In this embodiment, after executing S104 to generate a third detection question based on the verification result, S105 is executed to obtain the evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question, and the third detection question.

[0053] In this embodiment, when executing S105, the generated detection problem can be input into the large model. Based on the output of the large model, the evaluation result of the content detection strategy to be evaluated can be obtained, and the obtained evaluation result can be the evaluation score.

[0054] It is understood that if there are multiple target resources, this embodiment will generate a corresponding detection question for each target resource, and then obtain the evaluation score of the content detection strategy for the current target resource based on the generated detection question. Based on the evaluation scores of multiple target resources, the final evaluation result is obtained (e.g., calculating the average score, calculating the score variance, etc.).

[0055] If the content detection strategies to be evaluated in this embodiment include multiple strategies, after executing S105 to obtain the final evaluation result of each content detection strategy to be evaluated, this embodiment can also sort the multiple content detection strategies to be evaluated according to the final evaluation result, and then select the content detection strategy with the best effect according to the sorting result.

[0056] Figure 2This is a schematic diagram according to the second embodiment of this disclosure. (See diagram below.) Figure 2 As shown, when executing S105 "obtaining the evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question, and the third detection question" in this embodiment, the following may be included:

[0057] S201. Obtain the target label corresponding to the target resource output by the large model;

[0058] S202. Determine the target sub-table from the preset score table based on the target label and the label data;

[0059] S203. Based on the target molecular table, determine the first score corresponding to the first detection question, the second score corresponding to the second detection question, and the third score corresponding to the third detection question.

[0060] S204. Based on the first score, the second score, and the third score, the evaluation result of the content detection strategy to be evaluated is obtained.

[0061] In other words, after obtaining the detection questions of the detection strategy corresponding to the content to be evaluated, this embodiment determines the target sub-table from the preset score table based on the target labels output by the large model, and then obtains the scores of each detection question according to the determined target sub-table to obtain the evaluation results. Obtaining the evaluation results through the preset score table can ensure the reliability and stability of the evaluation criteria, so that a target resource can obtain a consistent score in multiple evaluations, avoiding the problem of evaluation criterion drift. Moreover, the evaluation criteria are defined in a structured rule format, which can also facilitate subsequent review and iterative adjustment.

[0062] In this embodiment, the preset scoring table includes a first scoring table and a second scoring table; wherein, the first scoring table is the scoring table for those with correct corresponding labels, and the second scoring table is the scoring table for those with incorrect corresponding labels, and each scoring table includes different detection questions and their corresponding scores.

[0063] In this embodiment, when S102 is executed, the target label can be compared with the label data, and the label can be determined as correct or incorrect based on the comparison result. For example, if the target label is the same as the label data, the label is correct; otherwise, the label is incorrect.

[0064] In this embodiment, the first result table includes 7 questions and their corresponding scores, namely: Question 1 "The reasoning is correct and 100% aligned with the evidence" and its corresponding score "100", Question 2 "There is a problem with the reasoning logic" and its corresponding score "70", Question 3 "There is fabricated evidence" and its corresponding score "80", Question 4 "There is tampered evidence" and its corresponding score "70", Question 5 "There is a fabricated source" and its corresponding score "80", Question 6 "There is a problem with the review point" and its corresponding score "80", and Question 7 "There are other minor errors" and its corresponding score "80".

[0065] In this embodiment, the second result table includes 7 questions and their corresponding scores, namely, question 8 "Logical breakdown" and its corresponding score "10", question 9 "Other reasons (hallucination judgment, etc.)" and its corresponding score "40", question 10 "There is fabricated evidence or fabricated evidence supports misjudgment" and its corresponding score "30", question 11 "There is tampered evidence or tampered evidence supports misjudgment" and its corresponding score "20", question 12 "There is fabricated information source or fabricated information source supports misjudgment" and its corresponding score "30", question 13 "There is a problem review point or the problem review point leads to misjudgment" and its corresponding score "30", and question 14 "Ignoring the judgment code leads to misjudgment" and its corresponding score "30".

[0066] In this embodiment, when executing S203, the existing detection problems can be matched in the target molecular table, and then the corresponding score can be obtained based on the matching results.

[0067] In this embodiment, when executing S204, the lowest score can be selected from the obtained scores as the evaluation result of the corresponding content detection strategy to be evaluated.

[0068] Figure 3 This is a schematic diagram according to the third embodiment of this disclosure. (See diagram below.) Figure 3 As shown, the evaluation method for content detection strategies based on large models in this embodiment may further include:

[0069] S301. In response to the fact that the target resources include multiple targets, obtain the detection question corresponding to each target resource;

[0070] S302. Statistically analyze the acquired detection issues according to the issue type, and generate a heatmap of detection issues based on the statistical results;

[0071] S303. Determine the target detection problem based on the heatmap of the detection problem, and optimize the detection strategy for the content to be evaluated based on the target detection problem.

[0072] In other words, after completing the evaluation of the content detection strategy, this embodiment can further generate a heat map based on the detection problems obtained during the evaluation process, and then determine the subsequent iteration direction of the content detection strategy based on the heat map, so as to optimize the shortcomings of the content detection strategy in a targeted manner, thereby greatly shortening the iteration cycle of "discovering problems - locating causes - guiding optimization".

[0073] In this embodiment, when executing S303, the occurrence frequency of each type of detection problem can be determined according to the detection problem heatmap. Detection problems that occur more than a preset threshold frequency are taken as target detection problems. Then, the detection strategy for the content to be evaluated is optimized or iterated based on the target detection problem.

[0074] Figure 4 This is a schematic diagram according to the fourth embodiment of this disclosure. (See diagram below.) Figure 4 As shown, the evaluation device 400 for content detection strategies based on large models in this embodiment includes:

[0075] The acquisition unit 401 is used to acquire the target resource and the detection data obtained by performing content detection on the target resource using the content detection strategy to be evaluated. The detection data includes review point data, evidence chain data, reasoning process data and tag data.

[0076] The first verification unit 402 is used to use a large model to verify the review point data according to the target resource, and generate a first detection question based on the verification result;

[0077] The second verification unit 403 is used to use the large model to verify the evidence chain data according to the preset evidence library, and generate a second detection question based on the verification result;

[0078] The third verification unit 404 is used to verify the reasoning process data and the label data using the large model, and generate a third detection question based on the verification result;

[0079] Evaluation unit 405 is used to obtain the evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question and the third detection question.

[0080] The target resource acquired by the acquisition unit 401 can be one or more, and the number of content detection strategies to be evaluated can be one or more; that is, in this embodiment, one or more content detection strategies can be evaluated based on one or more target resources.

[0081] In this embodiment, different content detection strategies may include different content detection models or different prompt texts.

[0082] The detection data acquired by the acquisition unit 401 is text data in natural language format, including Verification Points data, Evidence Chain data, Reasoning Process data, and Label data obtained by the content detection strategy to be evaluated when performing content detection on the target resource.

[0083] The review point data includes at least one review point, which is a factual unit that needs to be verified and extracted from the target resource by the content detection strategy to be evaluated; the evidence chain data includes at least one citation data, which is the evidence obtained by the content detection strategy to be evaluated for each review point through retrieval or matching; the reasoning process data includes the judgment result made on the review point based on the evidence and the logic of obtaining the detection conclusion based on multiple judgment results; the tag data is the tag reflecting the detection result of the target resource, which can include two types of tags: "approved" and "failed".

[0084] In this embodiment, after the acquisition unit 401 acquires the target resource and the detection data, the first verification unit 402 uses a large model to verify the review point data based on the target resource, and generates a first detection question based on the verification result.

[0085] The large model in this embodiment can be a large language model (LLM) or a multimodal large model; this embodiment does not limit it.

[0086] Specifically, when the first verification unit 402 uses the large model to verify the review point data based on the target resources and generates the first detection question based on the verification results, the implementation method can be as follows: input the target resources and review point data into the large model, and let the large model perform accuracy verification and integrity verification on the review point data based on the target resources; and generate the first detection question based on the obtained accuracy verification results and integrity verification results.

[0087] In this embodiment, the large model verifies the accuracy of the review point data. Specifically, the large model verifies whether the content of the review points is faithful to the original text and does not introduce illusions or biased interpretations (such as treating others' opinions as factual statements or incorporating incorrect dates into the statements). In this embodiment, the large model also verifies the completeness of the review point data. Specifically, the large model verifies whether the review points in the review point data are complete and do not omit any key verifiable facts in the original text.

[0088] The large model can generate a first detection question if the obtained accuracy verification results and / or completeness verification results include "verification failed". The generated first detection question is used to reflect the accuracy and / or completeness problems of the content detection strategy under evaluation when extracting review points. The generated first detection question can be "problematic review points exist".

[0089] Understandably, if the large model determines that both the accuracy verification result and the integrity verification result are "passed verification", the first verification unit 402 may not generate the first detection question.

[0090] In this embodiment, after the first verification unit 402 generates the first detection question based on the verification result, the second verification unit 403 uses a large model to verify the evidence chain data according to a preset evidence library, and generates the second detection question based on the verification result.

[0091] Specifically, when the second verification unit 403 uses the large model to verify the evidence chain data according to the preset evidence library and generates the second detection question based on the verification results, the following implementation method can be adopted: input the evidence chain data into the large model, and the large model performs evidence existence verification, evidence consistency verification, and evidence source attribution verification on each piece of evidence in the evidence chain data based on the preset evidence library; based on the obtained evidence existence verification results, evidence consistency verification results, and evidence source attribution verification results, the large model generates the second detection question.

[0092] In other words, this embodiment establishes the "evidence base boundary principle," which stipulates that all factual claims (i.e., evidence in the evidence chain) of the content detection strategy to be evaluated must and can only come from content that actually exists in the preset database; otherwise, they are all considered fictitious.

[0093] In this embodiment, the large model performs evidence existence verification for each piece of evidence. Specifically, the large model searches for evidence in the evidence chain data within a preset database, and determines whether there is evidence fabrication based on the search results (i.e., the model claims that the evidence is "confirmed by search," but there is no corresponding content in the preset evidence database). The large model also performs evidence consistency verification for each piece of evidence. Specifically, the large model compares the evidence in the evidence chain data with the corresponding evidence in the preset evidence database, and determines whether there is evidence tampering (e.g., distortion of numbers, people, or causal relationships) based on the comparison results. Finally, the large model verifies the source attribution of each piece of evidence. Specifically, the large model compares the source of the evidence in the preset database with the source of the evidence in the evidence chain data, and determines whether there is source fabrication based on the comparison results (e.g., claiming evidence from source A as source B).

[0094] The second verification unit 403 can perform the following: determine the verification type that failed the evidence verification, including existence verification, consistency verification and source attribution verification; the large model generates a second detection question based on the determined verification type.

[0095] In this embodiment, if there are multiple verification types that fail the evidence verification, the large model will generate a second detection question corresponding to each verification type. For example, the generated second detection question can be "there is fabricated evidence", "there is tampered evidence", "there is a fabricated source", etc.

[0096] In this embodiment, after the second verification unit 403 generates the second detection question based on the verification result, the third verification unit 404 uses a large model to verify the inference process data and the label data, and generates the third detection question based on the verification result.

[0097] Specifically, when the third verification unit 404 uses the large model to verify the inference process data and the label data, and generates the third detection question based on the verification result, the following implementation method can be adopted: input the inference process data and the label data into the large model, and have the large model verify the logical consistency between the inference process data and the label data; based on the obtained logical consistency verification result, the large model generates the third detection question.

[0098] In this embodiment, the large model verifies the logical consistency between the reasoning process data and the label data. Specifically, the large model extracts the judgment results made for each review point from the reasoning process data, and then verifies whether the reasoning process when judging the review point is consistent with the label data (if the reasoning process when judging the review point believes that "the core facts are true and accurate", but the label data is "the review is not approved", then it is inconsistent; if the reasoning process when judging the review point believes that "the facts are seriously false", but the label data is "the review is approved", then it is inconsistent.

[0099] The large model can generate a third detection question when the obtained logical consistency verification result is "inconsistent". The generated third detection question is used to reflect the logical collapse problem in the logical consistency judgment of the content detection strategy to be evaluated. The generated third detection question can be "logical collapse".

[0100] In other words, this embodiment uses a large model to sequentially perform review point verification, evidence verification, and logical consistency verification, which conforms to the detection logic of the content detection network when performing content detection. Based on the verification results, corresponding detection questions are generated. Compared with the existing technology's evaluation paradigm of "only evaluating the correctness of tags", this embodiment verifies the complete inference chain of the content detection strategy link by link, realizing an evaluation paradigm that penetrates the entire inference chain, thereby effectively improving the evaluation accuracy.

[0101] The third verification unit 404 can also perform the following: obtain the judicial code, which in this embodiment serves as a business knowledge base for the large model during evaluation, to perform compliance checks on the conclusions during logical consistency verification; input the obtained judicial code, reasoning process data, and label data into the large model, and have the large model verify the logical consistency between the reasoning process data and label data based on the judicial code.

[0102] In other words, the third verification unit 404 can also be combined with the Judgment Code to verify logical consistency, enabling the large model to have a certain business rule awareness capability. When the business rules change, only the content of the Judgment Code needs to be updated, without retraining the large model, thus supporting the dynamic updating of business rules.

[0103] In this embodiment, after the third verification unit 404 generates the third detection question based on the verification result, the evaluation unit 405 obtains the evaluation result of the detection strategy of the content to be evaluated based on the first detection question, the second detection question and the third detection question.

[0104] The evaluation unit 405 can input the generated detection question into the large model, and obtain the evaluation result of the detection strategy of the content to be evaluated based on the output of the large model. The obtained evaluation result can be the evaluation score.

[0105] It is understood that if there are multiple target resources, this embodiment will generate a corresponding detection question for each target resource, and then obtain the evaluation score of the content detection strategy for the current target resource based on the generated detection question. Based on the evaluation scores of multiple target resources, the final evaluation result is obtained (e.g., calculating the average score, calculating the score variance, etc.).

[0106] If the content detection strategies to be evaluated in this embodiment include multiple strategies, after the evaluation unit 405 obtains the final evaluation result of each content detection strategy to be evaluated, it can also sort the multiple content detection strategies to be evaluated according to the final evaluation result, and then select the content detection strategy with the best effect according to the sorting result.

[0107] When the evaluation unit 405 obtains the evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question, and the third detection question, it may also include the following: obtaining the target tags corresponding to the target resources output by the large model; determining the target sub-table from the preset score table based on the target tags and tag data; determining the first score corresponding to the first detection question, the second score corresponding to the second detection question, and the third score corresponding to the third detection question based on the target sub-table; and obtaining the evaluation result of the content detection strategy to be evaluated based on the first score, the second score, and the third score.

[0108] In other words, after obtaining the detection questions corresponding to the content detection strategy to be evaluated, the evaluation unit 405 determines the target sub-table from the preset score table based on the target labels output by the large model, and then obtains the scores of each detection question according to the determined target sub-table to obtain the evaluation results. Obtaining the evaluation results through the preset score table can ensure the reliability and stability of the evaluation criteria, so that a target resource can obtain a consistent score in multiple evaluations, avoiding the problem of evaluation criterion drift; and the evaluation criteria are defined in a structured rule, which can also facilitate subsequent review and iterative adjustment.

[0109] In this embodiment, the preset scoring table includes a first scoring table and a second scoring table; wherein, the first scoring table is the scoring table for those with correct corresponding labels, and the second scoring table is the scoring table for those with incorrect corresponding labels, and each scoring table includes different detection questions and their corresponding scores.

[0110] The evaluation unit 405 can compare the target label with the label data and determine whether the label is correct or incorrect based on the comparison result. For example, if the target label is the same as the label data, the label is correct; otherwise, the label is incorrect.

[0111] The evaluation unit 405 can match the existing detection problems in the target molecular table and then obtain the corresponding score based on the matching results.

[0112] The evaluation unit 405 can select the lowest score from the obtained scores as the evaluation result of the corresponding content detection strategy to be evaluated.

[0113] The evaluation device 400 for the content detection strategy based on the large model in this embodiment may further include an optimization unit 406, which is used to perform the following: in response to the inclusion of multiple target resources, obtain detection questions corresponding to each target resource; perform statistics on the obtained detection questions according to the question type, and generate a detection question heatmap based on the statistical results; determine the target detection question based on the detection question heatmap, and optimize the content detection strategy to be evaluated based on the target detection question.

[0114] In other words, after completing the evaluation of the content detection strategy, the optimization unit 406 can further generate a heat map based on the detection problems obtained during the evaluation process, and then determine the subsequent iteration direction of the content detection strategy based on the heat map, so as to optimize the shortcomings of the content detection strategy in a targeted manner, thereby greatly shortening the iteration cycle of "discovering problems - locating causes - guiding optimization".

[0115] The optimization unit 406 can determine the frequency of each type of detection problem based on the detection problem heatmap, and take the detection problems that occur more than a preset threshold as target detection problems, and then optimize or iterate the detection strategy of the content to be evaluated based on the target detection problem.

[0116] The acquisition, storage, and application of any type of information, such as user personal information, involved in the technical solutions disclosed herein comply with relevant laws and regulations and do not violate public order and good morals.

[0117] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0118] like Figure 5 The diagram shown is a block diagram of an electronic device for evaluating a content detection strategy based on a large model according to an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0119] like Figure 5 As shown, device 500 includes a computing unit 501, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 502 or a computer program loaded from storage unit 508 into random access memory (RAM) 503. RAM 503 may also store various programs and data required for the operation of device 500. The computing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. Input / output (I / O) interface 505 is also connected to bus 504.

[0120] Multiple components in device 500 are connected to I / O interface 505, including: input unit 506, such as keyboard, mouse, etc.; output unit 507, such as various types of displays, speakers, etc.; storage unit 508, such as disk, optical disk, etc.; and communication unit 509, such as network card, modem, wireless transceiver, etc. Communication unit 509 allows device 500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0121] The computing unit 501 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as the evaluation method of a content detection strategy based on a large model. For example, in some embodiments, the evaluation method of a content detection strategy based on a large model can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as storage unit 508.

[0122] In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by computing unit 501, one or more steps of the evaluation method for the large-model-based content detection strategy described above may be performed. Alternatively, in other embodiments, computing unit 501 may be configured to perform the evaluation method for the large-model-based content detection strategy by any other suitable means (e.g., by means of firmware).

[0123] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include: implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transferring data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The program code used to implement the methods of this disclosure can be written in any combination of one or more programming languages. This program code can be provided to the processor or controller of a general-purpose computer, special-purpose computer, or other programmable evaluation device for a large-model-based content detection strategy, such that when executed by the processor or controller, the program code enables the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0125] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for showing information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0127] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0128] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem, addressing the shortcomings of traditional physical hosts and VPS (Virtual Private Server, or simply "VPS") services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0129] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0130] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An evaluation method for content detection strategies based on large models, comprising: The method involves acquiring target resources and performing content detection on those target resources using a content detection strategy to be evaluated. The detection data includes review point data, evidence chain data, reasoning process data, and tag data. Using a large model, the review point data is validated based on the target resource, and a first detection question is generated based on the validation results; Using the large model, the evidence chain data is verified according to the preset evidence library, and a second detection question is generated based on the verification result; Using the large model, the inference process data and the label data are verified, and a third detection question is generated based on the verification results; Based on the first detection question, the second detection question, and the third detection question, the evaluation result of the content detection strategy to be evaluated is obtained.

2. The method according to claim 1, wherein, The process of using a large model to validate the review point data based on the target resource, and generating a first detection question based on the validation results, includes: The target resource and the review point data are input into the large model, and the large model performs accuracy and integrity verification on the review point data based on the target resource. Based on the obtained accuracy verification results and integrity verification results, the first detection question is generated by the large model.

3. The method according to claim 1, wherein, The step of using the large model to verify the evidence chain data according to a preset evidence library, and generating a second detection question based on the verification result includes: The evidence chain data is input into the large model, and the large model performs evidence existence verification, evidence consistency verification, and evidence source attribution verification on each piece of evidence in the evidence chain data based on the preset evidence library. Based on the obtained evidence existence verification results, evidence consistency verification results, and evidence source attribution verification results, the second detection question is generated by the large model.

4. The method according to claim 1, wherein, The step of using the large model to verify the inference process data and the label data, and generating a third detection question based on the verification results, includes: The reasoning process data and the label data are input into the large model, and the large model verifies the logical consistency between the reasoning process data and the label data. Based on the obtained logical consistency verification results, the third detection problem is generated by the large model.

5. The method according to claim 1, wherein, The evaluation result of the content detection strategy to be evaluated, obtained based on the first detection question, the second detection question, and the third detection question, includes: Obtain the target label corresponding to the target resource output by the large model; Based on the target label and the label data, determine the target sub-table from the preset score table; Based on the target molecular table, determine the first score corresponding to the first detection question, the second score corresponding to the second detection question, and the third score corresponding to the third detection question. The evaluation result of the content detection strategy to be evaluated is obtained based on the first score, the second score, and the third score.

6. The method according to claim 5, wherein, The evaluation result of the content detection strategy to be evaluated, obtained based on the first score, the second score, and the third score, includes: The lowest score among the first score, the second score, and the third score is selected as the evaluation result.

7. The method according to claim 1, further comprising: Since the target resources include multiple ones, the detection question corresponding to each target resource is obtained; The detected issues are statistically analyzed based on their types, and a heatmap of detected issues is generated based on the statistical results. The target detection problem is determined based on the heatmap of the detection problem, and the detection strategy for the content to be evaluated is optimized based on the target detection problem.

8. The method according to claim 3, wherein, The step of generating the second detection question from the large model based on the obtained evidence existence verification results, evidence consistency verification results, and evidence source attribution verification results includes: Determine the type of verification that failed the evidence verification; The large model generates the second detection question based on the determined verification type.

9. An evaluation device for a content detection strategy based on a large model, comprising: The acquisition unit is used to acquire the target resource and the detection data obtained by performing content detection on the target resource using the content detection strategy to be evaluated. The detection data includes review point data, evidence chain data, reasoning process data, and tag data. The first verification unit is used to use a large model to verify the review point data based on the target resource, and generate a first detection question based on the verification result; The second verification unit is used to verify the evidence chain data using the large model according to the preset evidence library, and generate a second detection question based on the verification result. The third verification unit is used to verify the inference process data and the label data using the large model, and generate a third detection question based on the verification result; An evaluation unit is used to obtain an evaluation result of the content detection strategy to be evaluated based on the first detection question, the second detection question, and the third detection question.

10. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

12. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images