Data evaluation method, apparatus, electronic device, and storage medium

Abstract

A data evaluation method and device, electronic equipment and storage medium are disclosed, and relate to the technical field of information retrieval and evaluation. The method comprises: performing multi-round verification on a first evaluation text through an initial data graph, obtaining a plurality of verification results, and reducing the subjectivity of manual evaluation based on logical reasoning on a large amount of objective original data; updating the verification information in each round of verification result to the graph of the next round of verification, continuously learning the latest technological progress through the mode of continuous updating, and ensuring the technological frontiers and dynamic adaptability of the evaluation capability; obtaining a weighted evaluation score of the plurality of verification results through a target evaluation label and a corresponding evaluation score, and then obtaining an evaluation report of the first evaluation text based on the plurality of verification results, the target evaluation label and the weighted evaluation score, realizing efficient evaluation of massive technical news, and improving the objectivity and accuracy of the evaluation result.

Description

Technical Field

[0001] This application relates to the field of information retrieval and evaluation technology, and more specifically, to a data evaluation method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of large language model technology, retrieval-augmented generation (RAG) techniques have been used to enhance the knowledge acquisition framework of the model. RAG typically performs one-off, static knowledge retrieval, making it difficult to handle complex evaluation tasks requiring multi-step reasoning and dynamic information gathering. However, related proxy-based retrieval enhancement frameworks, such as ReAct (Reasoning and Acting), aim to allow the large language model to operate within a "think-act" loop, actively acquiring information by invoking external tools. However, this proxy framework primarily relies on keyword searches or semantic vector retrieval from unstructured text corpora, and cannot perform complex tasks such as structured, multi-hop relational reasoning and historical context analysis.

[0003] Therefore, there are technical problems in the technical assessment, such as insufficient logical reasoning ability, difficulty in analyzing the technical context, and lack of transparency and interpretability of the assessment results. Summary of the Invention

[0004] In view of this, embodiments of this application propose a data evaluation method, apparatus, electronic device, and storage medium that can achieve efficient evaluation of massive amounts of technical news and improve the objectivity and accuracy of the evaluation results.

[0005] The following technical solution is adopted in this application.

[0006] According to a first aspect of the embodiments of this application, a data evaluation method is provided, the method comprising: Obtain an initial data graph and a first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data and the relationships between the data nodes; perform multiple rounds of verification on the first evaluation text based on the initial data graph to obtain multiple verification results; each verification result includes verification information and a verification status; the verification status is either confirmed or disproven; match the multiple verification results with multiple configured evaluation tags and the evaluation score of each evaluation tag to determine the target evaluation tag and the weighted evaluation score of the multiple verification results; based on the multiple verification results, the target evaluation tag, and the weighted evaluation score of the multiple verification results, obtain an evaluation report for the first evaluation text.

[0007] In some embodiments, obtaining the initial data map includes: Acquire multimodal data; perform preprocessing operations on the multimodal data to obtain initial data; the preprocessing operations include one or more combinations of the following: data cleaning, format conversion, entity recognition and relation extraction; perform structured processing on the initial data to construct an initial data map carrying timestamps.

[0008] In some embodiments, obtaining the first evaluation text includes: The obtained initial evaluation text is defined as multiple sets of texts to be evaluated; the verification status of each set of texts to be evaluated is initialized to be verified, and the first evaluation text is obtained.

[0009] In some embodiments, the step of performing multiple rounds of verification on the first evaluation text based on the initial data map to obtain multiple verification results includes: The first evaluation text is verified in one round based on the initial data graph to obtain the first verification information and the first verification status; the first verification information is updated in the initial data graph as the initial data graph for the next round of verification.

[0010] In some embodiments, the step of performing a round of verification on the first evaluation text based on the initial data map to obtain first verification information and a first verification status includes: Determine the verification tool corresponding to each group of texts to be evaluated; the verification tool includes at least one of the following: graph inferencer, text retrieval tool, key information extractor; perform one round of verification on the first evaluation text based on the initial data graph and the verification tool to obtain the first verification information and the first verification status.

[0011] In some embodiments, matching the plurality of verification results with a plurality of configured evaluation labels and the evaluation score of each evaluation label to determine a weighted evaluation score for a target evaluation label and the plurality of verification results includes: Determine the target evaluation label that matches each of the verification results; determine the confidence score corresponding to each target evaluation label; and obtain the weighted evaluation score of the multiple verification results based on the evaluation score and confidence score corresponding to the target evaluation label.

[0012] In some embodiments, the method for obtaining a weighted evaluation score of the plurality of verification results based on the evaluation score and confidence score corresponding to the target evaluation label further includes: The median value is obtained by multiplying the evaluation score corresponding to each target evaluation label by the confidence score; the median value of all target evaluation labels is summed and then divided by the sum of the confidence scores of all target evaluation labels to obtain the weighted evaluation score.

[0013] According to a second aspect of the embodiments of this application, a data evaluation apparatus is provided, the apparatus comprising: An acquisition module is used to acquire an initial data graph and a first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data and the relationships between the data nodes; a verification module is used to perform multiple rounds of verification on the first evaluation text based on the initial data graph to obtain multiple verification results; each verification result includes verification information and a verification status; the verification status is either verified or disproven; an evaluation module is used to match the multiple verification results with multiple configured evaluation tags and the evaluation score of each evaluation tag to determine the weighted evaluation score of the target evaluation tag and the multiple verification results; a generation module is used to obtain an evaluation report of the first evaluation text based on the multiple verification results, the target evaluation tag, and the weighted evaluation score of the multiple verification results.

[0014] According to a third aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, the data evaluation method described above is implemented.

[0015] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which computer-readable instructions are stored, which, when executed by a processor or electronic device, implement the data evaluation method described above.

[0016] In this application, an initial data graph constructed from multimodal data is used to perform multiple rounds of verification on the first evaluation text, resulting in multiple verification results. By using logical reasoning to construct a knowledge graph from massive, objective papers and patent data, the subjective bias of manual evaluation is effectively reduced, improving the objectivity and accuracy of the evaluation results. Each verification result includes verification information and verification status. The verification information from each round of verification results is updated in the data graph as evidence for the next round of verification. Through an iterative reasoning verification process, deeper innovative points are extracted from the first evaluation text. By continuously learning the latest technological advancements through a continuous update model, the technological frontier of the evaluation capability is ensured, exhibiting good dynamic adaptability.

[0017] Secondly, by determining the target evaluation label and evaluation score of the verification results, a weighted evaluation score of multiple verification results is obtained. Based on the multiple verification results, the target evaluation label, and the weighted evaluation score of multiple verification results, an evaluation report of the first evaluation text is obtained. This automated evaluation frees up the human resources of professionals, achieves high-efficiency evaluation of massive amounts of technical news, and further improves the objectivity and accuracy of the evaluation results.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1 This is a schematic diagram of a data evaluation system provided in an embodiment of this application.

[0021] Figure 2 This is a flowchart illustrating a data evaluation method provided in an embodiment of this application.

[0022] Figure 3 This is a flowchart illustrating a data acquisition method provided in an embodiment of this application.

[0023] Figure 4 This is a flowchart illustrating a multi-round verification method provided in an embodiment of this application.

[0024] Figure 5 This is a flowchart illustrating a method for obtaining a weighted evaluation score, provided as an embodiment of this application.

[0025] Figure 6 This is a flowchart illustrating another data evaluation method provided in an embodiment of this application.

[0026] Figure 7 This is a schematic diagram of the structure of a data evaluation device provided in an embodiment of this application.

[0027] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0028] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through specific embodiments. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] In typical technologies, technological innovation does not exist in isolation but is rooted in a complex network of relationships. Traditional proxy frameworks cannot effectively utilize the structured relationships (such as referencing, inheritance, and competition) between technological entities for in-depth analysis. This makes it difficult to accurately determine a technology's position within the overall technological ecosystem; for example, it cannot distinguish whether a technology is a new "leaf node" (representing incremental improvement) or a completely new "root node" (representing disruptive innovation) on an existing technology tree. Secondly, tracing the complete evolutionary path of a technology and identifying key paradigm shifts in unstructured text is difficult and unreliable, thus proxy frameworks have weak analytical capabilities in the time dimension. Finally, because the reasoning process is entirely based on understanding massive amounts of unstructured text, its internal decision-making logic is like a "black box," making it difficult for outsiders to understand the specific logical chain that determines a technology's high innovativeness, resulting in poor interpretability of the evaluation results and difficulty in gaining user trust.

[0031] The data evaluation method provided in this application aims to solve the above-mentioned technical problems of the prior art.

[0032] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0033] Figure 1 This is a schematic diagram of a data evaluation system provided in an embodiment of this application. Figure 1 As shown, the data evaluation system 100 provided in this application embodiment includes a data acquisition and preprocessing module 110, a time-series knowledge graph construction module 120, a tagging system and scoring module 130, and an interactive graph reasoning and evaluation module 140.

[0034] In one optional implementation, the data acquisition and preprocessing module 110, the temporal knowledge graph construction module 120, the tagging system and scoring module 130, and the interactive graph reasoning and evaluation module 140 refer to software units or modules. Specifically, the data acquisition and preprocessing module 110 acquires multimodal data and performs data cleaning, format conversion, entity recognition, and relation extraction; the temporal knowledge graph construction module 120 performs structured processing on the preprocessed multimodal data to construct an initial data graph with timestamps; the tagging system and scoring module 130 defines the evaluation tagging system; and the interactive graph reasoning and evaluation module 140 performs multiple rounds of verification on the text to be evaluated based on the evaluation tagging system and the initial data graph, determines the verification results, evaluation tags, and evaluation scores, and generates an evaluation report.

[0035] In another alternative implementation, the data acquisition and preprocessing module 110, the time-series knowledge graph construction module 120, the tagging system and scoring module 130, and the interactive graph reasoning and evaluation module 140 refer to hardware devices, which may include, but are not limited to, electronic devices with data processing capabilities such as computers, host machines, servers, or data centers.

[0036] The following is combined Figure 1 The data evaluation system 100 shown illustrates the data evaluation method provided in this application embodiment: The data acquisition and preprocessing module 110 automatically acquires initial data related to automotive technology from multiple preset data sources of different types, such as academic databases, patent databases, and professional industry news websites, and performs preprocessing operations on the initial data, including at least data cleaning, format conversion, entity recognition, and relation extraction, to generate structured data and send it to the time-series knowledge graph construction module 120.

[0037] In addition, the temporal knowledge graph construction module 120 receives structured data, organizes the structured data into a knowledge graph containing rich semantic information, and sends it as the initial data graph to the interactive graph reasoning and evaluation module 140. The knowledge graph includes nodes and edges, with technical entities, papers, patents, etc. as nodes and relationships such as citation, improvement, and implementation as edges, and timestamps are added to key nodes and edges.

[0038] Furthermore, the labeling system and scoring module 130 predefines an innovative labeling system, defines evaluation labels according to the type of innovation, and quantifies a basic score for each evaluation label, providing a standardized reference for the interactive graph reasoning and evaluation module 140. For example, evaluation labels include: "disruptive technology first", "major breakthrough in key performance indicators", "significant improvement of existing technology", etc.

[0039] Finally, the interactive graph reasoning and evaluation module 140 uses a large language model as its core controller, namely the intelligent agent. This agent performs multiple verifications through a dynamic iterative cycle of planning, reasoning, retrieval, and reflection. In each verification cycle, the agent uses the initial data graph as its primary reasoning sandbox, utilizing the structured information of the graph for deep logical reasoning. Simultaneously, it can also utilize external retrieval tools to supplement external unstructured text data when graph information is insufficient. Furthermore, through iterative verification, using an innovative tagging system as a standardized reference, it performs multiple rounds of verification on the first evaluation text, matching evaluation tags, quantifying verification results, and outputting an evaluation report for the first evaluation text. This evaluation report is based on a fully visualized path within the data graph.

[0040] Below Figure 1 Based on the data evaluation system 100 shown, the data evaluation method provided in the embodiments of this application will be further described, such as... Figure 2 The diagram illustrates a data evaluation method. In a specific embodiment, this data evaluation can be applied to, for example... Figure 7 The data evaluation device 700 and the electronic device 800 equipped with the data evaluation device 700 are shown. Figure 8 The specific process of the embodiments of this application will be described below. Of course, it is understood that this method can be executed by a cloud server with computing power. The following will focus on... Figure 2 The process shown is described in detail, and the data evaluation method may specifically include the following steps 201 to 204.

[0041] Step 201: Obtain the initial data graph and the first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data and the relationships between the data nodes.

[0042] In this embodiment of the application, the initial data graph is a time-series knowledge graph that traces the technological evolution, with document types such as technical entities, papers, patents, and performance indicators as nodes, citations, improvements, and implementations as relational edges, and both nodes and relations carrying timestamps; the first evaluation text is obtained by processing the automotive technology news text to be evaluated through a planning function.

[0043] For example, the constructed initial data map This map can be formally defined as a tuple. ,in: It is the set of all nodes in the graph, representing various types of entities, i.e. ,in, Representing a technology entity, Representative document entities (papers, patents). Represents the entity that controls the performance metrics; It is the set of all edges in the graph, representing the relationships between entities, i.e. ,in, It is a collection of relation types, such as {improves_on, cites, achieves}.

[0044] in addition, Indicates that the node ( ) mapped to a timestamp ( A function that takes the timestamp as the publication date of the document; Indicates that the relation edge ( ) mapped to a timestamp ( The function of ).

[0045] For example, an intelligent agent A (Agent) with a large language model as its core controller is used to evaluate the acquired automotive technology news text (D). news Automatic evaluation is performed.

[0046] Among them, based on the obtained automotive technology news text to be evaluated (D news The intelligent agent A initializes its internal state, including creating a hypothesis set, a state set, and an evidence set; resetting the loop iteration steps; and clearing historical reasoning information, thus establishing an initial operating environment for the subsequent iterative evaluation process of planning, reasoning, retrieval, and reflection.

[0047] As also exemplarily, intelligent agent A processes the input automotive technology news text (D... news Semantic analysis and technical feature extraction are performed using a planning function. Constructing the first evaluation text, i.e., the initial set of structured analytical hypotheses used for determining innovativeness. , For the elements (hypotheses) in the hypothesis set, this process can be represented by formula (1).

[0048] Formula (1) Among them, the hypothesis set Each hypothesis in All of them need to be verified.

[0049] Furthermore, construct the state set of the hypothesis set. Initialize the state set, that is, each The initial values ​​are all set to "unverified" to provide the initial state basis for subsequent iterative verification.

[0050] Furthermore, intelligent agent A will generate the initial data map. As a set of evidence.

[0051] Step 202: Perform multiple rounds of verification on the first evaluation text based on the initial data map to obtain multiple verification results; each verification result includes verification information and verification status; the verification status is either confirmed or disproven.

[0052] In this embodiment, the verification results are structured graph data, text fragments, or other forms of information. The verification results obtained in each round are updated to the initial data graph as new evidence, serving as the evidence set for the next round of verification.

[0053] In one implementation, after the internal state initialization of intelligent agent A is completed, the first evaluation text is subjected to multiple rounds of innovativeness-related verification based on the initial data graph; that is, the intelligent agent... At time step The process involves iterative verification, with each round generating a verification result, gradually completing the evaluation of the innovation of the first evaluation text.

[0054] For example, at each time step In this context, the operational state of an intelligent agent can be defined as a state tuple. Among them, the hypothesis set Includes currently unverified and / or verified hypotheses, and the state set. This includes a hypothesis state corresponding to each hypothesis, used to record the current verification status of each hypothesis (pending verification, confirmed, or falsified), and an evidence set. This includes all evidence collected to date, as well as the total set of evidence accumulated through initial data graphs, graph reasoning, tool calls, and other verification information.

[0055] Furthermore, at each time step In China, intelligent agents Based on the current internal state Select the hypothesis to be tested And select a suitable verification tool from toolset T for the hypothesis. To perform the action The toolset can be represented as: The toolset includes at least graph inference tools. Text search engine Key Information Extractor Tools such as...

[0056] Optionally, smart agent The current action is generated using an action decision function. The action selection process can be expressed as the following formula (2).

[0057] Formula (2) in, It calls the verification tool. Required parameters. For example, based on the hypothesis to be verified, when selecting a graph inferencer... When verifying hypotheses about technological evolution, the graph inferencer is invoked. Required motion parameters It includes a formalized graph query.

[0058] Furthermore, intelligent agents After completing the action Then, the hypothesis was obtained. A corresponding verification result includes verification status and verification information. The verification information is typically structured graph data, text fragments, or other forms, indicating whether the verification tool has been approved. Perform actions based on the assumption The subsequent observations.

[0059] Optionally, smart agent Verification information As new evidence, updated to the evidence set. In this process, the update process can be expressed as the following formula (3).

[0060] Formula (3) in, This action was recorded. and corresponding observation results .

[0061] Furthermore, intelligent agents Based on the new set of evidence For the hypothesis The state is updated, and the process can be expressed as the following formula (4).

[0062] Formula (4) For each hypothesis Its verification status It may be updated to be confirmed or refuted.

[0063] Furthermore, intelligent agents The validation of the first evaluation text is terminated based on the termination condition of the iterative process. The termination condition is either that all hypotheses have been validated or the maximum number of iterations has been reached. If the termination condition is not met, the process proceeds to the next time step. Repeat the process of validating the first evaluation text until the termination condition is met, and then end the validation.

[0064] Step 203: Match the multiple verification results with the configured multiple evaluation labels and the evaluation score of each evaluation label to determine the weighted evaluation score of the target evaluation label and the multiple verification results.

[0065] In this embodiment of the application, the evaluation label is a label defined according to the type of innovation for evaluating innovation, such as a disruptive first-time innovation, a major breakthrough in key performance indicators, or a significant improvement in existing technology; the evaluation score is a quantitative basic score assigned to each label.

[0066] In this embodiment of the application, the weighted evaluation score is used to quantify the innovativeness of the first evaluation text based on a predefined innovativeness labeling system.

[0067] For example, an innovative labeling system is predefined, wherein the evaluation labels in the labeling system can be defined as follows: and defining each evaluation label Corresponding to a base score .

[0068] In this embodiment of the application, the target evaluation label is the evaluation label that successfully matches the verification result, and is used to indicate the innovation type corresponding to the verification information in the verification result.

[0069] In one implementation, the smart agent performs the operation after all hypotheses in the hypothesis set have been verified. Based on the verification results obtained in the final round of verification, the final evidence set can be determined. .

[0070] Furthermore, intelligent agents According to the final set of evidence The evaluation labels are matched with the evaluation labels in the labeling system and the evaluation scores of each evaluation label to determine the innovation type corresponding to all evidence in the evidence set, and the target evaluation label of the first evaluation text is obtained.

[0071] Optionally, based on the target evaluation label and the corresponding evaluation score, a decision function is used. Calculate a confidence score for each target evaluation label. The confidence set is obtained. It can be expressed as the following formula (5).

[0072] Formula (5) Furthermore, based on the confidence set The base score is determined by evaluating all target labels. A confidence-weighted average is performed to obtain the weighted evaluation score of the first evaluation text, which is the total score for innovation. It can be expressed as the following formula (6).

[0073] Formula (6) Step 204: Based on the multiple verification results, the target evaluation label, and the weighted evaluation score of the multiple verification results, obtain the evaluation report of the first evaluation text.

[0074] In this embodiment of the application, the evaluation report is a comprehensive report that includes evaluation scores, matching labels, and a complete chain of evidence.

[0075] For example, based on the final set of evidence determined in step 203 The system uses target assessment labels and weighted assessment scores to generate an assessment report for the first assessment text.

[0076] In this embodiment, an initial data graph constructed from multimodal data is used to perform multiple rounds of verification on the first evaluation text, resulting in multiple verification results and improving the objectivity and accuracy of the evaluation results. Specifically, the verification information from each round of verification results is updated in the data graph as evidence for the next round of verification, enabling deeper exploration of the innovative points in the first evaluation text. Furthermore, by continuously learning from the latest technological advancements through a continuous update model, the technological advancement of the evaluation capability is ensured. Simultaneously, from a configured innovative tagging system, target evaluation tags and evaluation scores for the verification results are determined, resulting in a weighted evaluation score for multiple verification results. Based on these multiple verification results, target evaluation tags, and the weighted evaluation score, an evaluation report for the first evaluation text is generated. This achieves highly efficient evaluation of massive amounts of technical news and further improves the objectivity and accuracy of the evaluation results.

[0077] Based on the above, this application provides an optional implementation method for obtaining the initial data map and the first evaluation text, such as... Figure 3 The flowchart shown is a data acquisition method that may include the following steps 301 to 308.

[0078] Step 301: Obtain multimodal data.

[0079] In this embodiment of the application, multimodal data refers to raw data in various forms obtained from multiple data sources; wherein, the data sources include, but are not limited to, the IEEE Xplore API (digital library application programming interface) and automotive news.

[0080] For example, the data evaluation system deploys a distributed crawler framework and API client to call the open application programming interfaces of academic databases, patent databases and automotive industry news platforms, send data requests to the corresponding servers, periodically obtain the latest technical documents, patent information and industry news data, and parse the returned raw data to provide data source support for subsequent preprocessing and knowledge graph construction.

[0081] Step 302: Perform preprocessing operations on the multimodal data to obtain initial data; the preprocessing operations include one or more combinations of the following: data cleaning, format conversion, entity recognition, and relation extraction.

[0082] In the embodiments of this application, the initial data is semantic data that can be used for data graph construction after preprocessing the unstructured raw data.

[0083] For example, based on a language detection library, regular expressions and a standardized dictionary are used to clean and convert the format of multimodal data. For instance, "500km", "500-km", and "five hundred kilometers" are uniformly standardized into structured data {"value": 500, "unit": "km"}.

[0084] For example, the BERT-CRF model based on Transformer is used to perform Named Entity Recognition (NER) on multimodal data. The model is fine-tuned on a private dataset containing tens of thousands of automotive-related technical terms and entity annotations to ensure the accuracy of identifying technical terms, company entities, performance metrics, etc.

[0085] As another example, a method based on prompt-based large language model initial extraction and distant supervision is used to extract relations from multimodal data. Existing knowledge bases (such as DBpedia) are used to automatically label and verify the extracted relations in order to continuously optimize the performance of the relation extraction model.

[0086] Step 303: Perform structured processing on the initial data to construct an initial data map carrying timestamps.

[0087] For example, the Neo4j graph database is used to build a graph of the initial data. The schema of this graph is designed to be more detailed. For example, the "Technical Keyword" node includes two attributes: description (text description) and embedding (vector generated by the SBERT model). Another example is that the "Performance Indicator" node is designed as a composite node, which includes attributes such as metric_name (metric name, such as energy density), value (metric value), unit (unit of measurement), and test_condition (test conditions, such as laboratory environment, room temperature) to support more refined evaluation and comparison in the future.

[0088] Furthermore, the construction of the initial data map is an incremental update process. Whenever a new piece of initial data (such as news literature) is processed, the data evaluation system will search in the map to see if a node representing the initial data already exists. If it does not exist, a new node will be created in the map, and its relationship with other nodes such as authors, technical keywords, and cited literature will be established based on the results of entity recognition and relationship extraction in step 302.

[0089] Meanwhile, when creating "cites" relationships, the system checks the publication date of the cited literature and adds it to the graph as a timestamp to generate an initial data graph (i.e., a time-series technology knowledge graph) to ensure the graph's correctness in terms of time logic.

[0090] Optionally, to support more advanced semantic queries, the system will periodically run GraphSAGE (graph sampling aggregation algorithm) or GAT (graph attention network algorithm) to iteratively train all entity nodes in the time-series technology knowledge graph, and update the vector representation (embedding) of all nodes in real time. The generated node vectors have stronger semantic representation capabilities and scene adaptability.

[0091] In the process of vector training and updating, the algorithm not only fully encodes the core information of the nodes themselves, including textual semantic content such as technical entity names, performance indicators, parameter attributes, technical descriptions, and innovative features, but also deeply mines and integrates the global neighborhood structure information of the nodes through the structural learning capabilities of graph neural networks, which greatly improves the accuracy of technical similarity calculation and emerging technology cluster identification.

[0092] Step 304: Define the obtained initial evaluation text as multiple sets of texts to be evaluated.

[0093] For example, since the raw data (such as automotive technology news text) is unstructured natural language content, it has the characteristics of broad expression, many promotional phrases, blurred technical boundaries, and implicit innovation points. It cannot be directly adapted to the tools called by the intelligent agent A. Therefore, before the evaluation, it is necessary to transform the unstructured raw data into multiple sets of texts to be evaluated, that is, a standardized set of hypotheses that can be verified one by one. H This supports the system's multi-round iterative reasoning logic, enabling the verification, evidence tracing, and objective judgment of each innovative point in the original data.

[0094] Step 305: Initialize the verification status of each group of texts to be evaluated to be verified, and obtain the first text to be evaluated.

[0095] For example, the states of multiple sets of texts (hypotheses) to be evaluated are all initialized to be verified, so as to ensure that the system performs a verification action for each hypothesis.

[0096] Step 306: Perform multiple rounds of verification on the first evaluation text based on the initial data map to obtain multiple verification results; each verification result includes verification information and verification status; the verification status is either confirmed or disproven.

[0097] Step 307: Match the multiple verification results with the configured multiple evaluation labels and the evaluation score of each evaluation label to determine the weighted evaluation score of the target evaluation label and the multiple verification results.

[0098] Step 308: Based on the multiple verification results, the target evaluation label, and the weighted evaluation score of the multiple verification results, obtain the evaluation report of the first evaluation text.

[0099] The specific steps of steps 306 to 308 can be found in steps 202 to 204, and will not be repeated here.

[0100] In this embodiment of the application, an initial data graph constructed from multimodal data is obtained, and the first evaluation text is verified in multiple rounds to obtain multiple verification results. By performing logical reasoning on the knowledge graph constructed from massive and objective paper and patent data, the subjective bias of manual evaluation is effectively reduced, and the objectivity and accuracy of the evaluation results are improved.

[0101] Based on the above, this application provides an optional implementation method for performing multiple rounds of verification on the first evaluation text based on the initial data map to obtain multiple verification results, such as... Figure 4 The flowchart shown is a multi-round verification method, which may specifically include the following steps 401 to 405.

[0102] Step 401: Obtain the initial data graph and the first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data and the relationships between the data nodes.

[0103] Step 402: Perform a round of verification on the first evaluation text based on the initial data map to obtain the first verification information and the first verification status.

[0104] For example, the intelligent agent adopts the GeminiPro model that supports function calling. Based on this model, the system prompt is designed in a refined and customized manner, and its main structure includes the following aspects.

[0105] Persona: "You are a top automotive industry technology analyst with 20 years of experience. Your task is to objectively, rigorously, and interpretably evaluate the innovativeness of technology news."

[0106] Tool Schema: Using JSON format in the OpenAPI Schema (Open Application Programming Interface Schema), the calling rules and parameter specifications for all callable tools are clearly defined to the intelligent agent. For example, for Graph_Reasoner, its callable function is predefined with the name find_evolution_path (technology evolution path query function), and its input parameters, return data format, and exception type (such as EntityNotFound) are also specified. The input parameters include entity_name:string (entity name) and max_hops:integer (maximum number of hops), and the return data format is standardized JSON structured data containing graph nodes and associated edges.

[0107] Output Instruction: Instructs the intelligent agent to evaluate innovation in a chain-of-thought manner and output a JSON object containing an innovation score and a structured chain of evidence.

[0108] The Graph_Reasoner tool's internal implementation includes a natural language-to-graph query conversion module. This module employs semantic parsing; when it receives a natural language instruction from the agent (such as "find the technology source of 'Company A's LiDAR'"), it first identifies the core entity ('Company A's LiDAR') and the intent ('find the technology source'), then maps this intent to one or more predefined Cypher query templates. For example, "find the source" might map to a traversal query in the graph that performs inverse improves_on or cites relationships.

[0109] Furthermore, based on the above model, the first evaluation text is validated multiple times according to the initial data map. The process of the intelligent agent's chain of thought and action logic (thinking-action-observation) is shown in the following example.

[0110] Reflection: The performance breakthrough has been confirmed. However, innovation lies not only in a single metric, but also in the technological path taken to achieve that metric. Minor improvements to existing, mature technologies will limit the innovation score. Adopting a completely new technological paradigm will result in a higher innovation score. I need to explore its technological roadmap.

[0111] Action: Call Graph_Reasoner.find_evolution_path(entity_name='Company A's LiDAR', relation_type='improves_on').

[0112] Observation: The tool returns {"path": null, "error": "No direct 'improves_on' parentfound."}.

[0113] Furthermore, based on the verification of the above-mentioned thinking chain (thinking-action-observation), the graph inference engine returns an empty path and prompts an exception after retrieval, such as not finding a direct iterative improvement technology or performance exceeding the record.

[0114] In one implementation, step 402, which involves performing a round of verification on the first evaluation text based on the initial data map to obtain first verification information and first verification status, includes steps 412 to 422.

[0115] Step 412: Determine the verification tool corresponding to each group of texts to be evaluated; the verification tool includes at least one of the following: graph inferencer, text retrieval tool, key information extractor.

[0116] Step 422: Based on the initial data map and the verification tool, perform a round of verification on the first evaluation text to obtain the first verification information and the first verification status.

[0117] Step 403: Update the first verification information into the initial data map as the initial data map for the next round of verification.

[0118] For example, combining the "think-action-observation" verification process of the intelligent agent in step 402, the observation results of the first round are updated to the initial data graph as evidence for the next round of verification. The process of the intelligent agent's chain of thoughts and action logic continues as follows.

[0119] Reflection: There are no direct inheritance relationships in the graph. This could be due to two possibilities: either the initial data graph was incomplete, or this is a relatively new technique. I should first try broadening the search scope to find broader correlations, rather than focusing on direct inheritance relationships.

[0120] Action: Call Graph_Reasoner.find_related_clusters(entity_name='Company A's LiDAR'). This function finds clusters of technology nodes that are close to the entity node in the vector space, or connected through indirect paths (such as shared authors or similar citations).

[0121] Observation: The tool returns {"related_clusters": ["SPAD detectors","Quantum dots"], "confidence":[0.8,0.3]}.

[0122] Reflection: The observations strongly suggest a high correlation between this technology and 'SPAD detectors'. This is significantly different from mainstream FMCW or MEMS lidar technologies. This is strong evidence of an innovative technological approach. Now, I need to confirm this by combining it with external information.

[0123] Action: Call Text_Retriever.search(query='Company A's SPAD LiDAR technology').

[0124] Observation: The tool returned an interview with the company's chief technology officer, who mentioned that "we achieved a breakthrough in both detection range and accuracy by utilizing the latest single-photon avalanche diode (SPAD) array technology."

[0125] The final judgment was: "The chain of evidence is complete. This news not only reported record-breaking performance metrics, but its SPAD (Special Application Development) approach also differs from mainstream technologies, constituting an innovation in technological paradigm. Therefore, it should be labeled with both 'major breakthrough in key performance metrics' and 'pioneering new technology application areas'."

[0126] Step 404: Match the multiple verification results with the configured multiple evaluation labels and the evaluation score of each evaluation label to determine the weighted evaluation score of the target evaluation label and the multiple verification results.

[0127] Step 405: Based on the multiple verification results, the target evaluation label, and the weighted evaluation score of the multiple verification results, obtain the evaluation report of the first evaluation text.

[0128] The specific steps of steps 401, 404 to 405 can be found in steps 201, 203 to 204, and will not be repeated here.

[0129] In this embodiment of the application, by updating the verification information in each round of verification results to the data graph as evidence for the next round of verification, the verification process of iterative reasoning enables the discovery of deeper innovative points in the first evaluation text. Furthermore, by continuously learning the latest technological advancements through a continuous update model, the technological frontier of the evaluation capability is ensured, and it has good dynamic adaptability.

[0130] Building upon the above, this application provides an optional implementation method for matching the multiple verification results with the configured multiple evaluation labels and the evaluation score of each evaluation label, and determining the weighted evaluation score of the target evaluation label and the multiple verification results. For example... Figure 5 The flowchart shown is a method for obtaining weighted evaluation scores, which may include the following steps 501 to 506.

[0131] Step 501: Obtain the initial data graph and the first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data and the relationships between the data nodes.

[0132] Step 502: Perform multiple rounds of verification on the first evaluation text based on the initial data map to obtain multiple verification results; each verification result includes verification information and verification status; the verification status is either confirmed or disproven.

[0133] Step 503: Determine the target evaluation label that matches each of the verification results.

[0134] For example, as in the final judgment returned in step 403, the target evaluation labels include major breakthroughs in key performance indicators and pioneering new technology application areas.

[0135] Step 504: Determine the confidence score corresponding to each of the target evaluation labels.

[0136] For example, in the innovation scoring system, the confidence score corresponding to each evaluation label. The judgment is not based on a single dimension, but rather on a weighted calculation that combines the quantity and strength of various pieces of evidence. For example, evidence of structured data directly returned by Graph_Reasoner (such as performance surpassing records), which is generated based on entity and relation data in a knowledge graph, is assigned a higher weight, with an example weight of 0.9. Unstructured text evidence returned by Text_Retriever (a text retrieval tool), which is characterized by fragmented expression, a lot of implicit information, and the need for human-assisted semantic interpretation, is assigned a relatively lower weight, with an example weight of 0.6.

[0137] Step 505: Obtain the weighted evaluation score of the multiple verification results based on the evaluation score and confidence score corresponding to the target evaluation label.

[0138] For example, the intelligent agent iterates through all the verifications and summarizes them to form a complete final chain of evidence. Then, a weighted summation method was used to calculate the confidence level of each evaluation label. That is, based on the entire chain of evidence... Each target evaluation label supported in The weighted sum of the evidence yields the weighted evaluation score.

[0139] In step 505, the process of obtaining the weighted evaluation score of the multiple verification results based on the evaluation score and confidence score corresponding to the target evaluation label includes the following steps 515 to 525.

[0140] Step 515: Multiply the evaluation score corresponding to each target evaluation label by the confidence score to obtain the median value.

[0141] Step 525: After summing the median values ​​of all the target evaluation labels, divide by the sum of the confidence scores of all the target evaluation labels to obtain the weighted evaluation score.

[0142] For a detailed description of the steps, please refer to step 203.

[0143] Step 506: Based on the multiple verification results, the target evaluation label, and the weighted evaluation score of the multiple verification results, obtain the evaluation report of the first evaluation text.

[0144] The specific steps of steps 501 to 502 and 506 can be found in steps 201 to 202 and 204, and will not be repeated here.

[0145] In this embodiment of the application, the target evaluation label and evaluation score of the verification result are determined from the configured innovative label system, and the weighted evaluation score of multiple verification results is obtained. Based on the multiple verification results, the target evaluation label and the weighted evaluation score of multiple verification results, the evaluation report of the first evaluation text is obtained. The automated evaluation frees up the human resources of professionals, realizes the high-efficiency evaluation of massive amounts of technical news, and further improves the objectivity and accuracy of the evaluation results.

[0146] Based on the foregoing, this application provides an optional implementation method, such as... Figure 6 The flowchart shown is for another data evaluation method, which may include the following:

[0147] First, for the collection and preprocessing of multi-source heterogeneous data, the system obtains raw data from three types of data sources. After preprocessing such as data cleaning, format conversion, entity recognition and relation extraction, the data is transformed into structured data, providing data support for the construction of knowledge graphs.

[0148] Secondly, for the construction of a time-series technology knowledge graph, based on the preprocessed data, nodes and edges are constructed. Nodes include technical entities, patents, documents, performance indicators, etc., and edges include relationships such as improvements, citations, and achievements, thereby generating a time-series technology knowledge graph.

[0149] The toolbox provides standardized tool interfaces for intelligent agents, encapsulating tools such as Graph_Reasoner and Text_Retriever.

[0150] In addition, the intelligent agent, as the core reasoning engine, inputs news text (D news The algorithm transforms news texts into a structured set of hypotheses and, based on a temporal knowledge graph, uses tools from the toolbox to verify the hypotheses in multiple iterations, gradually building a chain of evidence.

[0151] Finally, the label matching and scoring are output. Based on the final evidence chain, the confidence level of each innovation assessment label is calculated according to the evidence weight. Combined with the label base score, the final innovation score and an interpretable assessment report are output. The innovation score is used to quantify the level of technological innovation. The assessment report includes the reasoning process, evidence chain and innovation judgment basis, realizing full traceability of the assessment process.

[0152] To achieve the functions of the above embodiments, the data evaluation method includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and method steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0153] exist Figures 2 to 6 Based on the data evaluation method shown, the embodiments of this application also provide a data evaluation apparatus for further explanation, such as... Figure 7 The diagram shows a data evaluation device 700, which includes: an acquisition module 710, a verification module 720, an evaluation module 730, and a generation module 740.

[0154] Acquisition module 710 is used to acquire an initial data graph and a first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data, and the relationships between the data nodes; wherein, acquisition module 710 may include, for example, Figure 1 The data evaluation system 100 shown includes a data acquisition and preprocessing module 110 and a time-series knowledge graph construction module 120.

[0155] Verification module 720 is configured to perform multiple rounds of verification on the first evaluation text based on the initial data map, obtaining multiple verification results; each verification result includes verification information and a verification status; the verification status is either verified or disproven; wherein, verification module 720 may include, for example, Figure 1 The interactive graph reasoning and evaluation module 140 in the data evaluation system 100 shown.

[0156] Evaluation module 730 is used to match the plurality of verification results with a plurality of configured evaluation labels and an evaluation score for each evaluation label to determine a weighted evaluation score for the target evaluation label and the plurality of verification results; wherein, evaluation module 730 may include, for example, Figure 1 The data evaluation system 100 shown includes a labeling system and scoring module 130 and an interactive graph reasoning and evaluation module 140.

[0157] The generation module 740 is configured to generate an evaluation report of the first evaluation text based on the plurality of verification results, the target evaluation label, and the weighted evaluation score of the plurality of verification results; wherein, the generation module 740 may include, for example, Figure 1 The interactive graph reasoning and evaluation module 140 in the data evaluation system 100 shown.

[0158] In some embodiments, the acquisition module 710 includes: a data acquisition module for acquiring multimodal data; a preprocessing module for performing preprocessing operations on the multimodal data to obtain initial data; the preprocessing operations include one or more combinations of the following: data cleaning, format conversion, entity recognition and relation extraction; and a structured processing module for performing structured processing on the initial data to construct an initial data map carrying timestamps.

[0159] In some embodiments, the acquisition module 710 further includes: a data grouping module, configured to define the acquired initial evaluation text as multiple groups of texts to be evaluated; and an initialization module, configured to initialize the verification status of each group of texts to be evaluated to be verified, thereby obtaining the first evaluation text.

[0160] In other embodiments, the verification module 720 includes: a text verification module, used to perform one round of verification on the first evaluation text based on the initial data graph to obtain first verification information and a first verification status; and a data graph update module, used to update the first verification information to the initial data graph as the initial data graph for the next round of verification.

[0161] In other embodiments, the verification module 720 further includes: a tool determination module, used to determine the verification tool corresponding to each group of texts to be evaluated; the verification tool includes at least one of the following: a graph inferencer, a text retrieval device, and a key information extractor; and a verification information acquisition module, used to perform one round of verification on the first evaluation text based on the initial data graph and the verification tool, to obtain the first verification information and the first verification status.

[0162] In some embodiments, the evaluation module 730 includes: a label matching module for determining a target evaluation label matched to each of the verification results; a label quantization module for determining a confidence score corresponding to each of the target evaluation labels; and a weighted evaluation module for obtaining a weighted evaluation score of the plurality of verification results based on the evaluation score and confidence score corresponding to the target evaluation label.

[0163] In some embodiments, the evaluation module 730 further includes: a first evaluation module, configured to multiply the evaluation score corresponding to each target evaluation label by the confidence score to obtain an intermediate value; and a second evaluation module, configured to sum the intermediate values ​​of all the target evaluation labels and then divide them by the sum of the confidence scores of all the target evaluation labels to obtain the weighted evaluation score.

[0164] According to one aspect of the embodiments of this application, Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8As shown, the electronic device 800 includes a processor 810 and one or more memories 820. The one or more memories 820 are used to store program instructions executed by the processor 810. When the processor 810 executes the program instructions, it implements the data evaluation method described above.

[0165] Furthermore, the processor 810 may include one or more processing cores. The processor 810 runs or executes instructions, programs, code sets, or instruction sets stored in the memory 820, and retrieves data stored in the memory 820. Optionally, the processor 810 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 810 may integrate one or a combination of several of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the processor and may be implemented using a separate communication chip.

[0166] According to one aspect of this application, a computer-readable storage medium is also provided, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable storage medium carries computer-readable instructions that, when executed by a processor, implement the methods in any of the above embodiments.

[0167] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. Computer-readable storage media can be, for example, but not limited to: electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such transmitted data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0168] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0170] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0171] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A data evaluation method, characterized in that, include: Obtain the initial data map and the first evaluation text; The initial knowledge graph is used to indicate multiple data nodes extracted from the initial data, and the relationships between the data nodes; Based on the initial data map, multiple rounds of verification are performed on the first evaluation text to obtain multiple verification results; each verification result includes verification information and verification status; the verification status is either confirmed or disproven. The multiple verification results are matched with multiple configured evaluation labels and the evaluation score of each evaluation label to determine the target evaluation label and the weighted evaluation score of the multiple verification results; An evaluation report for the first evaluation text is obtained based on the multiple verification results, the target evaluation label, and the weighted evaluation score of the multiple verification results.

2. The method according to claim 1, characterized in that, The process of obtaining the initial data map includes: Acquire multimodal data; Preprocessing operations are performed on the multimodal data to obtain initial data; the preprocessing operations include one or more combinations of the following: data cleaning, format conversion, entity recognition, and relation extraction; The initial data is structured to construct an initial data map carrying timestamps.

3. The method according to claim 1, characterized in that, The step of obtaining the first evaluation text includes: The acquired initial evaluation text is defined as multiple sets of texts to be evaluated; The verification status of each group of texts to be evaluated is initialized to be verified, thus obtaining the first text to be evaluated.

4. The method according to claim 1, characterized in that, The first evaluation text is subjected to multiple rounds of verification based on the initial data map, resulting in multiple verification results, including: The first evaluation text is verified based on the initial data map to obtain the first verification information and the first verification status. The first verification information is updated into the initial data map and used as the initial data map for the next round of verification.

5. The method according to claim 4, characterized in that, The step of performing a round of verification on the first evaluation text based on the initial data map to obtain first verification information and a first verification status includes: Determine the verification tool corresponding to each group of texts to be evaluated; the verification tool includes at least one of the following: graph inferencer, text retrieval unit, key information extractor; Based on the initial data map and the verification tool, the first evaluation text is verified in one round to obtain the first verification information and the first verification status.

6. The method according to claim 1, characterized in that, The step of matching the multiple verification results with multiple configured evaluation labels and the evaluation score of each evaluation label to determine the weighted evaluation score of the target evaluation label and the multiple verification results includes: Determine the target evaluation label that matches each of the verification results; Determine the confidence score corresponding to each of the target evaluation labels; The weighted evaluation score of the multiple verification results is obtained based on the evaluation score and confidence score corresponding to the target evaluation label.

7. The method according to claim 6, characterized in that, The step of obtaining a weighted evaluation score for the multiple verification results based on the evaluation score and confidence score corresponding to the target evaluation label includes: Multiply the evaluation score corresponding to each target evaluation label by the confidence score to obtain the median value; The weighted evaluation score is obtained by summing the median values ​​of all the target evaluation labels and dividing by the sum of the confidence scores of all the target evaluation labels.

8. A data evaluation device, characterized in that, The device includes: The acquisition module is used to acquire an initial data graph and a first evaluation text; the initial knowledge graph is used to indicate multiple data nodes extracted from the initial data and the relationships between the data nodes. The verification module is used to perform multiple rounds of verification on the first evaluation text based on the initial data map to obtain multiple verification results; each verification result includes verification information and verification status; the verification status is either confirmed or disproven; An evaluation module is used to match the multiple verification results with multiple configured evaluation labels and the evaluation score of each evaluation label to determine the weighted evaluation score of the target evaluation label and the multiple verification results; The generation module is used to obtain an evaluation report of the first evaluation text based on the multiple verification results, the target evaluation label, and the weighted evaluation score of the multiple verification results.

9. An electronic device, characterized in that, The electronic device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor or electronic device to execute the method as described in any one of claims 1 to 7.

Images