Academic declaration assessment method for knowledge graph enhanced large model reasoning

Abstract

The invention provides an academic declaration assessment method for knowledge graph enhanced large model reasoning, and belongs to the field of combination of a large model and a knowledge graph. Through deep fusion of the structured ability of the knowledge graph and the reasoning agent of the large language model, the spanning of the academic ability assessment from shallow text matching to deep logical reasoning is realized. The method comprises the following steps: constructing a knowledge-driven intelligent reasoning framework; performing deep semantic extraction from an unstructured academic text by utilizing the powerful semantic analysis capability of a large language model; scattered text information is converted into a structured logic network by constructing a dynamic knowledge graph. According to the method, a thinking-action reasoning circulation mechanism is innovatively introduced, multi-hop exploration is carried out on a knowledge graph through an intelligent agent, and the inherent factual illusion and reasoning fault problems of a traditional large model during complex academic logic processing can be solved; and an explanation generation strategy based on a structured evidence chain is supported, and the reliability, the professional degree and the transparency of an academic evaluation system are remarkably improved.

Description

Technical Field

[0001] This invention belongs to the field of combining large models and knowledge graphs, and relates to an academic application evaluation method that enhances large model reasoning with knowledge graphs, and more particularly to a method for evaluating academic applications that combines large models with knowledge graph reasoning. Background Technology

[0002] With the expansion of scientific research, academic evaluation plays an increasingly important role in key areas such as talent selection, research funding allocation, and institutional policy formulation. Its fairness and accuracy are directly related to the rational allocation of resources and the career development of researchers. Traditional evaluation systems mainly rely on the number of papers, citation indices (such as impact factor and H-index), and peer review. However, purely quantitative indicators often only focus on surface data, neglecting the deeper dimensions of academic ability, such as originality, conceptual integration ability, and interdisciplinary reasoning. Furthermore, traditional peer review suffers from strong subjectivity, lack of transparency, and susceptibility to institutional bias. This situation has driven technological innovation in academic evaluation, aiming to improve the efficiency and objectivity of evaluation through automation, which is valuable for maintaining fairness in academia and promoting knowledge innovation.

[0003] Meanwhile, automated academic assessment faces a series of challenges, the core of which lies in capturing the complex logical structure and deep cognitive reasoning within academic texts, rather than merely relying on surface-level textual features. In recent years, research on automated scoring has made some progress. Early methods primarily utilized bibliometric analysis, citation network modeling, and natural language processing techniques based on machine learning (such as SVM) for assessment. Subsequent research attempted to introduce heterogeneous academic networks or topic models to characterize researchers' expertise. However, these traditional data-driven models often lack interpretability and are difficult to adapt to specific assessment criteria, limiting their further application in complex academic scenarios.

[0004] In recent years, Large Language Models (LLMs) have emerged as a powerful alternative to traditional methods, achieving significant breakthroughs in tasks such as document summarization, proposal scoring, and automated review generation thanks to their superior language understanding and generation capabilities. Furthermore, Retrieval-Enhanced Generation (RAG) techniques have been introduced, attempting to improve the factual accuracy of evaluations by integrating external knowledge sources.

[0005] In terms of theory, *Large Language Models for Automated Scholarly Paper Review* (arXiv, 2025) systematically explores the feasibility of using large language models to achieve automated paper review. It attempts to generate review opinions from dimensions such as paper structure, originality, and experimental adequacy, thereby assisting or partially replacing the traditional peer review process. This work verifies the potential of large language models in language understanding and text generation; however, its evaluation process still essentially relies on end-to-end understanding and generation of the paper text. It lacks explicit modeling of the academic knowledge system, making it difficult to depict the deep logical connections between research background, theoretical framework, and methodology, and it cannot guarantee the interpretability and traceability of the evaluation reasoning path.

[0006] At the engineering practice level, patent CN118313462A—"Article Scoring Method, Device, and Electronic Equipment Based on Large Language Model"—proposes an automatic scoring framework based on a large language model. By constructing scoring templates and combining them with language model output, it achieves automated scoring of article quality. However, this solution mainly focuses on general article or text quality evaluation, emphasizing surface-level feature modeling such as language expression and structural integrity. It lacks a systematic characterization of academic argumentation structure, knowledge dependencies, and research logic chains, making it difficult to adapt to highly complex and specialized academic evaluation scenarios such as research proposals.

[0007] In summary, existing academic assessment research and application schemes based on large language models generally have the following shortcomings: First, the assessment process is mainly based on "black box" text reasoning, lacking explicit knowledge graph structure support, making it difficult to conduct interpretable logical inferences; second, it is difficult to depict the complex knowledge connections across chapters, concepts, and fields in academic texts.

[0008] To address the aforementioned issues, this invention proposes a graph agent evaluation framework for academic application documents that integrates a large language model and a dynamic knowledge graph. By deeply fusing the structured representation capabilities of the knowledge graph with the reasoning agent of the large language model, a knowledge-driven intelligent reasoning framework, Graph Agent, is constructed, thereby enabling a leap in academic competence assessment from "shallow text matching" to "deep logical reasoning." This framework can systematically model multi-dimensional characteristics in academic evaluation, such as logical rigor, rationality of argumentation structure, and completeness of knowledge associations. It effectively compensates for the shortcomings of existing technologies in terms of cognitive reasoning depth, knowledge structure characterization, and evaluation interpretability, providing intelligent support for highly reliable and accurate scientific research review. Summary of the Invention

[0009] To address the problems existing in the prior art, this invention proposes an evaluation framework for academic application graph agents that integrates large language models and dynamic knowledge graphs. By deeply integrating the structured expression capabilities of knowledge graphs with the reasoning agents of large language models, a knowledge-driven intelligent reasoning framework, Graph Agent, is constructed, thereby enabling the assessment of academic ability to leap from "shallow text matching" to "deep logical reasoning".

[0010] This intelligent reasoning framework first leverages the powerful semantic parsing capabilities of large language models to extract deep semantics from unstructured academic texts (such as application documents and papers). Then, by constructing a dynamic knowledge graph, it transforms fragmented textual information into a structured logical network. In particular, this framework innovatively introduces a "Think-Act" reasoning loop mechanism. Through multi-hop exploration of the knowledge graph by an agent, it collaboratively verifies the implicit reasoning of LLM with the explicit paths of the graph, effectively overcoming the inherent "fact illusion" and reasoning gaps problems of traditional large language models when dealing with complex academic logic. Compared to traditional evaluation methods that rely on shallow text matching or single-model probability generation, this invention supports an explanation generation strategy based on structured evidence chains. This mechanism not only enables real-time knowledge alignment through incremental retrieval of graph nodes but also ensures that every evaluation conclusion can be traced back to specific logical entities and relational paths, significantly improving the reliability, professionalism, and transparency of the academic evaluation system.

[0011] The following section introduces the relevant methods of the intelligent reasoning specification framework.

[0012] (1) Large language model: Large Language Models (LLMs) represent a key leap forward in language modeling within the field of intelligent computing. These models rely on large-scale corpora for training, gradually learning the statistical patterns and semantic structures of language, thus acquiring relatively mature natural language understanding and generation capabilities. Compared to traditional Natural Language Processing (NLP) techniques that primarily rely on rules or shallow features, LLMs typically employ self-supervised pre-training strategies, uncovering latent patterns in text without explicit manual annotation. This allows the model to simultaneously characterize semantic information, syntactic structure, and dependencies between contexts, resulting in significant performance improvements in tasks such as text generation, automatic question answering, and cross-language translation.

[0013] From a model structure perspective, LLMs are mostly built upon the Transformer specification framework. Their core lies in introducing attention-based modeling to directly characterize the relationships between arbitrary positions in a sequence. This self-attention mechanism effectively handles long-distance dependencies, avoiding the information decay and limited receptive field problems faced by Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs) during sequence modeling. Furthermore, multi-head self-attention, by focusing on inter-word relationships in different subspaces in parallel, enables the model to capture semantic associations, such as those between pronouns and their potential referents, in greater detail. The formula is expressed as: (1) Where Q, K, and V represent the query, key, and value matrices, respectively. d k This is the dimension scaling factor.

[0014] (2) Knowledge graph integration: In addressing the challenges of deep academic evaluation of large language models, a natural and promising solution is to integrate external structured knowledge, such as knowledge graphs (KG), to enhance the reasoning performance of LLMs. KG provides a structured, explicit, and editable knowledge representation, offering a complementary strategy to compensate for the limitations of LLMs in logical coherence and factual accuracy in long texts.

[0015] In the context of application evaluation, the role of knowledge graphs extends beyond simply querying general knowledge. It transforms unstructured, lengthy academic texts (such as research background, technical routes, and innovative points) into semantic triples of "entity-relationship-entity" (e.g., specific algorithm, solution, key scientific question or experimental design, verification, theoretical hypothesis). This transformation makes the previously obscure textual logic clearly visible, supporting the system's graph-based logical reasoning. For example, unlike general knowledge bases (such as Freebase) that store massive amounts of encyclopedic facts, graph construction in academic evaluation focuses on capturing the conceptual hierarchy and argumentation chains within the application. By explicitly modeling the intrinsic connections between concepts, methodological depth, and academic innovation, the model can perform multi-hop reasoning and evidence gathering along logical paths, much like an expert, thus achieving a more reliable and interpretable assessment of academic capabilities.

[0016] To address the problems existing in the prior art, the present invention adopts the following technical solution: An evaluation method for academic applications using knowledge graph-enhanced large language model reasoning, the evaluation method comprising the following steps: Step 1: Constructing a Pattern-Driven Knowledge Graph; This step aims to transform unstructured academic texts into a knowledge graph with topological structure and semantic features by explicitly modeling academic logic. It consists of four stages: initial reasoning (Step 1.1), knowledge extraction (Step 1.2), confidence assessment and threshold selection (Step 1.3), and formal representation of effective triples (Step 1.4), as detailed below: Step 1.1, in the initial reasoning stage of constructing the academic knowledge graph, the primary task is to establish an ontology schema S (Schema) with rigorous logic, providing a standardized framework for the structured transformation of unstructured text, and laying a unified semantic constraint for subsequent structured extraction.

[0017] The ontology schema defines the entity type space. (such as research objectives, methodology, innovative points, etc.) and relational type space (such as "utilize", "verify", "solve", etc.), among which, Represents the first type of graph node, Represents nodes of the second type of graph. Represents the nth type of graph node, This represents the first type of connecting edge. This represents a second type of connecting edge. This represents the m-th type of connection edge. Through this predefined pattern, the large language model can map fragmented academic text to a unified semantic dimension. For any input text paragraph, the goal of modeling its logical structure is to construct a knowledge graph. This lays a structured foundation for subsequent in-depth reasoning.

[0018] Step 1.2: Under the constraints of the ontology pattern S determined in Step 1.1, the input academic text is subjected to structured semantic parsing. Based on the predefined entity types and relation types in the ontology pattern, the large language model is used to identify entity pairs that conform to the ontology pattern from the text and determine the semantic relationships between entities, thereby generating a set of triples of academic knowledge consisting of subject entities, relation types and object entities, realizing the mapping from unstructured text to structured graph data.

[0019] In the knowledge extraction stage, this invention utilizes a large language model as the information extraction function. , the original text Mapped to a set of triples The specific process is as follows: (2) Where S represents the ontology pattern defined in step 1.1, which serves as the extraction constraint; T represents the original text, specifically the input original academic text paragraph. Representative subject; Represents an object entity; This represents the predicate relationship between the two; V represents the set of nodes; E represents the set of directed edges.

[0020] Step 1.3: To ensure the reliability of knowledge construction, this invention introduces a confidence assessment and threshold screening mechanism to evaluate the set of triples obtained in Step 1.2. Consistency checks and reliability enhancements are performed. Then, the relation types in the triplet set are mapped to directed edges connecting the corresponding entity nodes, thereby constructing an academic knowledge graph composed of a node set and an edge set.

[0021] This invention applies to each extracted triplet Introducing confidence rating factors , ,in, This represents the posterior probability of the logical truth of the triple given the original text T. A rigor threshold is preset. Only when At that time, the triple is admitted into the knowledge graph as a valid triple. The rigor threshold... The preset probability threshold used to measure the logical credibility of triplet sets ranges from 0.5 to 0.9.

[0022] This probabilistic filtering mechanism effectively mitigates the "fact illusion" problem that LLM may encounter when processing complex academic texts, achieves alignment and conflict detection between entities, and ensures the authenticity of the subsequent underlying evidence chain.

[0023] After screening, valid triples that meet the rigor threshold are treated as valid knowledge units and subjected to graph structuring. Specifically, different entities appearing in the valid triples are treated as nodes in the academic knowledge graph; and the relation types in the valid triples are mapped to directed edges connecting the corresponding subject entity nodes and object entity nodes, thus forming an academic knowledge graph composed of a set of nodes and a set of edges. This is used for subsequent graph reasoning and evidence chain mining.

[0024] Step 1.4: After completing entity alignment and conflict detection, the extracted valid triples are formally represented.

[0025] To enhance the graph's ability to capture academic content, the proposed framework integrates semantic embedding, using a pre-trained encoder to map the textual descriptions of nodes into high-dimensional vectors. : (3) in, Represents a node The original text attributes.

[0026] At this point, the logical structure of the academic text is jointly represented by the adjacency matrix A and the feature matrix H; the adjacency matrix A originates from the academic knowledge graph extracted in step 1.3. Direct generation: If there is an academic logic r between entity J and entity L that conforms to the ontology schema S, then the elements in the adjacency matrix A... Otherwise, it is 0. This method characterizes the explicit topological skeleton of academic texts; the feature matrix H is derived from semantic embedding, and the system uses the pre-trained encoder in step 1.4 to embed each node. The original text attributes are mapped to high-dimensional vectors. Then, the high-dimensional vectors of all nodes are arranged in order to form the feature matrix. .in, This represents the high-dimensional vector of the first node. This represents the high-dimensional vector of the second node. Let H represent the high-dimensional vector of the b-th node, and T represent the transpose of the matrix. Matrix H characterizes the implicit semantic information of the academic content.

[0027] Finally, a knowledge graph is obtained. The topological structure is represented by the adjacency matrix A, and the semantic features of the nodes are represented by the feature matrix H. This representation method enables the graph to not only have explicit topological associations but also implicit semantic continuity. This two-layer representation mechanism provides the core impetus for the agent to perform semantic jumps and multi-step reasoning by calculating cosine similarity, realizing the paradigm evolution from text-driven to knowledge-driven.

[0028] Step Two: Based on the knowledge graph constructed in Step One, the agent performs pathfinding through a "think-action" loop to achieve in-depth mining and automated generation of academic argumentation evidence chains. Finally, the agent completes its exploration on the knowledge graph.

[0029] In the intelligent reasoning framework, the second step, based on the agent's "think-act" cycle, is the core of achieving dynamic deep reasoning. This framework consists of three core components: a semantic deconstructor, a heuristic pathfinding agent, and a state convergence determiner. It mimics the cognitive process of human experts reviewing applications—"identifying problems—finding evidence—verifying hypotheses"—by linking discrete knowledge points into a rigorous chain of evidence through multi-hop navigation on the knowledge graph. Specifically, it involves three steps: First, the semantic deconstructor performs semantic parsing and task decomposition of the academic application's evaluation requirements, generating an initial reasoning state to guide graph reasoning. Then, guided by this initial reasoning state, the heuristic pathfinding agent drives the agent to perform multi-hop pathfinding operations in the knowledge graph, obtaining a set of evidence chains. Finally, the state convergence determiner updates the state of the evidence chain set and determines its convergence, forming a structured set of evidence chains that meets the evaluation requirements. The above process is executed sequentially: first, goal generation; then, path exploration; and finally, evidence convergence. Specifically: Step 2.1, Inference State Initialization and Goal Decomposition, is the starting point of the entire inference process. It receives the evaluation requirements from external input and generates the initial inference state through semantic deconstruction. This provides clear "navigational instructions" for subsequent exploration. Specifically: In this step, the specification framework receives the original text of the academic application to be evaluated. The evaluation criteria are taken as input, along with a predefined set of evaluation dimensions C. A semantic deconstructor based on a large language model is used to analyze the evaluation requirements, breaking down macro-evaluation goals such as "innovation" and "technical feasibility" into several specific knowledge verification tasks. Finally, the semantic deconstruction process maps the fuzzy evaluation criteria into a logical query vector q in a high-dimensional semantic space, and simultaneously generates a corresponding initial task stack to characterize the knowledge directions that the current inference needs to focus on.

[0030] Through the above processing, the transformation from the original evaluation requirements to the structured reasoning goal is completed. The generated logical query vector q and the initial task stack together constitute the initial reasoning state. This provides clear semantic guidance for subsequent path exploration based on knowledge graphs.

[0031] Step 2.2, Graph-based "Action" Execution and Path Exploration: This step is executed under the guidance of the logical query vector q output in Step 2.1. The agent maps "assessing needs" to "pathfinding actions" on the graph, while simultaneously extracting real logical connections using the knowledge graph generated in Step 1. Specifically: In this step, the canonical framework takes the knowledge graph G (containing the topology A and node feature matrix H) constructed in step 1.4 and the logical query vector q output in step 2.1 as input, and executes action decisions on the graph structure through a heuristic pathfinding agent. At each time step t, the agent matches and scores its neighboring nodes based on the current anchor node and the dynamically generated inference intent vector, and selects the next-hop node under topological constraints. The transition probability is determined by the following formula: (4) in, This represents the anchor node where the intelligent agent is currently located, i.e., the academic entity it is currently focusing on; This represents the target node that the agent points to in the candidate set; This indicates that the current node is Under these conditions, the agent chooses to jump to the node. The transition probability; It is a normalized exponential function used to transform the matching scores of all candidate neighbor nodes into a probability distribution, ensuring that the sum of the probabilities of all possible actions is 1; It is a cosine similarity metric function, used to calculate the cosine of the angle between two vectors in a multidimensional semantic space. The closer the value is to 1, the higher the semantic relevance. The inference intent vector at time step t is dynamically generated by the large language model based on the global objective q and the currently obtained evidence, representing what the agent "wants to find" at this moment. Represents the nodes in the characteristic matrix H The semantic embedding vector, generated by the pre-trained encoder in step 1.4, represents the academic connotation of the node; It is a topological constraint condition, which means that the value of the i-th row and j-th column in the adjacency matrix A must be 1, that is, only logical jumps between physically connected nodes in the graph are allowed for the agent.

[0032] Through the aforementioned agent-driven multi-step pathfinding process, the system gradually obtains a set of logical path segments composed of multiple entity nodes and their relational edges. This set of path segments constitutes an evidence chain set. , used to characterize the potential argumentation structure in an academic application.

[0033] The output of this step marks the completion of the normative framework's leap from "intent understanding" to "empirical pathfinding". Instead of haphazard keyword extraction, this output presents a logical chain of evidence with topological constraints. Through the physical constraints of the adjacency matrix A, each hop of the agent must adhere to the inherent argumentative structure of academic papers, thus eliminating the "semantic jumps" or "causal inversions" that often occur in traditional large language models during evaluation. In summary, this output provides a physically traceable "factual skeleton" for subsequent evaluation conclusions, ensuring that every sentence in the review comments is verifiable.

[0034] Step 2.3, Dynamic determination of state update and termination condition: This step receives the path segments discovered in each iteration of Step 2.2. The information is then compared with the logical query vector q defined in step 2.1. If the evidence is sufficient, the reasoning terminates; otherwise, a new intent vector is generated and fed back to step 2.2 to continue the search.

[0035] In this step, the canonical framework will use the path segments discovered in each iteration of step 2.2. Gradually integrate into the global reasoning context, and incrementally update the currently constructed knowledge state through a gating update mechanism. The state transition process is shown in formula (5). Whenever the agent completes a graph action at time step t, And capture new logical subgraphs Then, the system incrementally updates the global inference context through a state transition function. Mathematical expression: (5) in, Represents the currently constructed knowledge context; This indicates capturing a new logical subgraph. The subsequent knowledge context, These are newly extracted entity relation triples from the graph; It is a gated update function (similar to the GRU mechanism) used to filter out redundant information that is irrelevant to the evaluation dimension C, and only retain the core argument chain.

[0036] The gated update function is used to filter redundant information irrelevant to the current evaluation dimension, retaining only the core argument relationships that contribute to the evaluation objective. Then, based on the logical convergence criterion, the evidence chain set obtained in step 2.2 is calculated. The degree of uncertainty of the evaluation results determines whether to continue graph exploration. The determination process is shown in formula (6). To prevent reasoning from getting stuck in an infinite loop or over-searching, the normative framework introduces a dynamic termination decision function based on evidence entropy. By calculating the knowledge context after capturing the new logical subgraph. The determination formula is defined for the uncertainty contribution of the logical query vector q: (6) in, Indicates the judgment result; This indicates the termination of this round of exploration. This indicates that the current exploration will continue; This indicates capturing a new logical subgraph. The knowledge context following; D is a divergence function (such as KL divergence) used to measure the volatility of the model's judgment results after adding new evidence; q is the logical query vector defined in step 2.1; This is the convergence threshold. When volatility is below this value, it indicates that the existing evidence is sufficient to form a stable assessment. The range is defined as 0 to 0.3. t is the maximum number of iterations, and t is the current number of iterations.

[0037] When the judgment result When the preset convergence condition is met or the maximum number of iterations is reached, the agent's graph exploration process is terminated, and... This final set of structured, complete evidence is designated as evidence chain set B. Evidence chain set B provides a credible factual basis for subsequent multi-dimensional scoring and review report generation.

[0038] The output of step 2.3 (here, the output refers to the evidence chain set B) marks the achievement of the "evidence chain mining" stage goal of this invention. Through the state update mechanism, the agent integrates the scattered path fragments discovered in step 2.2 into a complete set of structured evidence chains.

[0039] Ultimately, step two successfully achieved the "autonomy" and "closed-loop" of the evaluation logic. The resulting review report was not based on probabilistic random guesses, but rather on the most comprehensive fact-checking of the academic application under the logical convergence criterion. Thus, step two, through a complete cycle of "discovering the target (step 2.1) – finding the path (step 2.2) – determining convergence (step 2.3)," provides immutable factual evidence for the subsequent scoring prediction and report generation in step three.

[0040] Step 3: Structured Evidence Integration and Multi-Dimensional Scoring; This step receives the evidence chain set B finally locked in Step 2 through the "think-action" cycle, and initiates the "evaluation" process. By performing dimensional modeling on the evidence chains in evidence chain set B, the discrete logical paths are transformed into authoritative review scores and structured reports, realizing the final closed loop from fact discovery to value judgment.

[0041] Step 3.1: Dimensional Logical Deduction and Quantitative Scoring: After obtaining the complete evidence chain set B, proceed to this step, and use the thought chain model to conduct in-depth logical deduction on specific dimensions to generate preliminary quantitative scores for each dimension.

[0042] In this step, using the evidence chain set B output from step 2.3 and the predefined evaluation index system C as input, a multi-dimensional thinking chain reasoning model based on a large language model is used to perform logical deduction for each evaluation dimension. Specifically, for a certain evaluation dimension m in the index system C, some evidence paths highly related to that evaluation dimension are selected from the evidence chain set B. (i.e., a part of the evidence chain in evidence chain set B), and combined with the special evaluation prompt template preset for this evaluation dimension, guide the model to reason step by step around the corresponding academic evaluation logic.

[0043] In the above reasoning process, the special assessment prompts the model based on the evidence path. The performance of the evaluated object on the corresponding evaluation dimension is logically summarized and quantitatively mapped to obtain the preliminary score of the evaluation dimension. The calculation process is shown in formula (7): (7) in, This represents the quantitative score of the m-th evaluation dimension (such as the score for the "innovation" dimension). This represents the reasoning mapping function of the thought chain, which represents the logical deduction process performed by the model given facts; It is a subset of path segments that are highly correlated with dimension m, filtered from the total evidence set; This indicates a specific evaluation prompt template designed for dimension m, used to guide the model to focus on specific academic review logic.

[0044] Through the above-described multi-dimensional logical deduction process, the agent transforms the structured evidence chain into a multi-dimensional preliminary scoring vector, laying the foundation for the subsequent stabilization of the scoring results.

[0045] Step 3.2: Multi-path voting consensus and review report generation (achieving robustness of the final conclusion): This step receives the preliminary score vectors for each dimension generated in Step 3.1. To eliminate random errors in the model generation of the special assessment prompts, parallel inference and consensus voting are performed here. Finally, the aggregated scores are associated with the evidence chain set B from Step 2.3 to generate the final report.

[0046] In this step, for each evaluation dimension, the reasoning process in step 3.1 is sampled multiple times in parallel to obtain multiple sets of scoring results under the same evaluation dimension. These multiple sets of scoring results are then used as input, combined with a predefined evaluation index weight vector W, and statistically integrated through a multi-path consensus aggregation function. The multi-path consensus aggregation function weakens the random fluctuations that may be caused by a single inference and improves the robustness of the scoring results. Its calculation process is shown in formula (8): (8) Where Score represents the final weighted overall score of the academic application; n represents the total number of predefined evaluation dimensions, and i represents the i-th evaluation dimension. Let represent the weight coefficient of the i-th evaluation dimension, and satisfy . ; This represents a multi-path consensus aggregation function (using majority voting) to improve the stability of the results; This represents the random path score obtained when performing the k-th parallel sampling inference on the i-th dimension, where k represents the total number of samples performed for parallel inference on the same dimension (used to eliminate bias through multi-path verification). Simultaneously with obtaining the final weighted overall score, the final weighted overall score is associated with the evidence chain set B obtained in step 2.3, automatically generating a review report R containing the scoring basis and logical path description, thus completing the transformation from structured evidence to authoritative evaluation conclusions.

[0047] Step three integrates the evidence chain into the multi-dimensional reasoning and supplements it with a multi-path voting consensus mechanism, thus completing the transformation from structured evidence to authoritative evaluation conclusions. Through the local logical deduction of formula (7) and the global consensus aggregation of formula (8), the normative framework not only solves the pain point of unstable scoring of large language models in complex tasks, but also ensures the openness, transparency and logical interpretability of the evaluation process through the evidence chain set in the final report.

[0048] The beneficial effects of this invention are: (1) This invention transforms unstructured application texts into explicit knowledge graph structures by constructing a knowledge graph based on academic models, forcing the model to anchor to real-world knowledge nodes during reasoning. This structured constraint mechanism ensures that the evaluation process strictly follows the logical flow of the original text, fundamentally reducing the uncertainty and factual errors caused by the model's probability generation. It significantly improves the logical depth and factual accuracy of complex academic text evaluations and effectively suppresses the "illusion" phenomenon of large language models.

[0049] (2) This invention innovatively introduces a "think-action" cyclical pathfinding mechanism for the intelligent agent. Unlike the passive, one-time retrieval methods in the prior art, the intelligent agent of this invention can autonomously plan the search path based on the current reasoning state and dynamically judge the sufficiency of evidence. By performing multi-hop reasoning on the graph, the intelligent agent can discover implicit causal relationships across paragraphs and chapters (such as the supporting role of technical solutions for innovative points), realizing a technological leap from "keyword matching" to "cognitive reasoning," and significantly improving the evaluation accuracy in complex academic scenarios.

[0050] In summary, this invention establishes an intelligent evaluation system with high reliability, high interpretability, and deep reasoning capabilities through the synergistic effect of graph construction and agent dynamic reasoning. It has broad application value in fields such as scientific research management, project review, and academic quality monitoring. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the overall specification framework of the present invention.

[0052] Figure 2 This is a diagram illustrating a reasoning example combining a large language model and a knowledge graph. Detailed Implementation

[0053] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings and technical solutions.

[0054] A general normative framework for a reliable and interpretable academic competence assessment method based on graph agents, as follows: Figure 1 As shown, it includes the following steps: First, this embodiment uses a specially constructed domain-specific academic proposal dataset. This dataset covers three major scientific disciplines: physics, chemistry, and biology, aiming to simulate real-world expert review scenarios. All proposal texts have been preprocessed to remove irrelevant formatting information, and the average length of each sample is approximately 8,000 tokens. The dataset is divided into three levels based on the overall expert review scores: Good (70-80 points), Excellent (80-90 points), and Outstanding (90-100 points). Detailed statistical information for the dataset is shown in Table 1. Table 1: Statistics of Academic Proposal Dataset

[0055] Secondly, this embodiment uses Qwen2.5-7B as the core Large Language Model (LLM) backbone to drive the agent's reasoning and decision-making. During the graph construction phase, long texts are segmented into chunks of up to 200 tokens in length, and then triples are extracted. During the graph exploration phase, the agent maintains an evidence chain set with a maximum capacity of 10 records to balance information content and reasoning efficiency within a limited context window. Decoding parameters are set to Temperature 0.7 and Top_p 0.9 to balance the diversity and coherence of the generated data. The method of this invention runs on an Ubuntu 20.04 system with an Intel Xeon CPU E5-2650 v4 @2.20GHz and an NVIDIA GeForce RTX 4090 GPU (24GB VRAM).

[0056] by Figure 2To further illustrate, the text to be evaluated is an academic application on "Bioorthogonal Labeling Technology for Proteins at the Live Cell Level." This application belongs to the interdisciplinary field of chemical biology, and its research focuses on the bioorthogonal labeling technology for proteins in a live cell environment, involving key issues such as the introduction of non-natural amino acids, molecular evolution screening, and the design of biocompatible reaction systems. Based on steps one to three of this invention, the application is subjected to structured analysis, graph reasoning, and multi-dimensional academic evaluation to verify the feasibility and effectiveness of the method in a real academic evaluation scenario.

[0057] In step one, this invention first constructs an ontology model oriented towards academic evaluation tasks, specifically tailored to the academic expression characteristics of chemical biology application documents, to constrain the subsequent knowledge extraction process. This ontology model clearly distinguishes semantic roles and entity relationships such as research objects, research methods, research objectives, technical problems, and innovative points, enabling unstructured text to be parsed within a unified semantic framework. Examples of research object categories include: proteins, mammalian living cells, and non-natural amino acids; methodological techniques include: orthogonal biological reactions, molecular evolution, and click chemistry; and problems and innovations include: copper ion toxicity, limited expression efficiency, and copper-free labeling strategies.

[0058] Subsequently, under this ontology model, a segment-by-segment semantic analysis was performed on the entire application, extracting triples that conform to academic logic. For example, in the statement of research objectives in the application, the large language model identified that the applicant aimed to achieve efficient and specific labeling of proteins in mammalian living cells, thus extracting the structured expression "project objective—achievement—specific labeling of living cell proteins". In the methodological description section, the technical route of screening PylRS variants through molecular evolution methods to improve the efficiency of introducing non-natural amino acids was further identified, thus forming knowledge triples such as "molecular evolution methods—used for—screening PylRS variants" and "PylRS variants—improvement—efficiency of introducing non-natural amino acids".

[0059] To ensure the reliability of the knowledge construction results, the large language model performs a confidence assessment on each triple after extraction to measure its logical truthfulness within the original text context. When the confidence of a triple falls below a preset rigor threshold (set to 0.6 here), the triple is automatically filtered out, thus preventing speculative or imprecise statements from being introduced into the knowledge graph. For example, triples containing hypothetical or forward-looking descriptions in applications are often removed due to insufficient confidence.

[0060] After confidence screening, the agent further maps the retained triples into a graph structure representation. Specifically, different academic entities are represented as nodes in the graph, and the academic relationships between entities are represented as directed edges connecting the nodes. The corresponding adjacency matrix and feature matrix are then calculated. Through the above processing, an academic knowledge graph for the application is finally constructed. This graph clearly depicts the inherent logical connections between research methods, technical problems, and innovative points, and provides a foundation for subsequent graph-based reasoning processes.

[0061] Step Two: After completing the construction of the academic knowledge graph, the normative framework proceeds to Step Two, conducting evidence-based reasoning analysis on the academic value of the application. First, the model receives an evaluation task request, such as assessing the academic level of the application in terms of innovation and technical feasibility. The intelligent system uses a semantic deconstructor to parse this evaluation request, breaking it down into several sub-tasks with clearly defined judgment objectives, such as whether it has overcome the toxicity bottleneck of traditional copper-catalyzed click chemistry in intracellular applications, or whether it has established a feasible non-natural amino acid expression system for mammalian cells, thus forming the initial reasoning state.

[0062] Guided by the initial reasoning state, the agent performs multi-hop exploration within the academic knowledge graph constructed in step one. Starting with nodes highly relevant to the evaluation target, such as "traditional click chemistry," "non-natural amino acids," and "PylRS variants," the agent gradually expands its exploration path within the graph based on semantic similarity and graph topological constraints. During this process, the agent can generate multiple candidate argumentation paths. For example, in the innovation analysis, the agent starts with traditional noble metal catalysis methods, passes through the "copper ion toxicity problem," and further points to the copper-free bioorthogonal reaction and novel ligand design scheme proposed in the application, thus forming a complete chain of evidence for technological substitution and improvement. In the technical feasibility analysis, the agent constructs a technical path from molecular evolution screening to stable expression in mammalian cells, focusing on the introduction mechanism of non-natural amino acids.

[0063] As the exploration process progresses, the agent continuously incorporates newly acquired evidence paths into the current reasoning context and evaluates the consistency and coverage among the evidence. When multiple evidence paths converge on key conclusions, and the contribution of newly added paths to the evaluation results gradually weakens, the agent determines that the reasoning process has reached convergence, terminates the graph exploration operation, and outputs the final set of structured evidence chains. This set of evidence chains provides a clear and traceable logical foundation for subsequent scoring and conclusion generation.

[0064] Step Three: In Step Three, the agent performs a multi-dimensional academic evaluation of the application based on the structured evidence chain set obtained in Step Two. First, according to the predefined evaluation index system, a logical deduction process is executed for each evaluation dimension. For example, under the dimension of innovativeness, evidence paths related to the proposal of new methods, technological breakthroughs, and improvements to existing methods are selected. A multi-dimensional reasoning model is used to analyze the evidence step by step, and based on this, it is determined whether the application proposes a new solution with substantial progress based on existing research. In this example, based on the evidence chain combining a copper-free reaction system and a molecular evolution strategy, the agent determines that the research has a high level of innovation in the field of live-cell protein labeling, and initially gives an innovativeness score of 85. Similarly, under the dimension of technical feasibility, the agent also uses the evidence chain to reason about the completeness of the experimental route, the clarity of the technical implementation path, and the reproducibility of the method, and forms a corresponding score of 85. Through the above multi-dimensional reasoning process, preliminary scoring results covering multiple evaluation dimensions are generated.

[0065] Subsequently, to reduce the random fluctuations that might arise from a single inference, the agent further performed multiple parallel inferences on each evaluation dimension and adopted a consensus aggregation mechanism to integrate multiple sets of scoring results according to weights, ultimately determining a stable comprehensive score of 85 points. Based on this, the scoring results were correlated with the corresponding evidence chain path, automatically generating a review report containing the scoring conclusion and its supporting explanation: This research, focusing on the cutting-edge issue of live-cell protein labeling, addresses the toxicity limitations of traditional noble metal catalytic systems in intracellular applications by proposing a technical solution combining copper-free bioorthogonal reactions and molecular evolutionary screening strategies. The overall research objectives are clear, and the technical route is well-defined. In the innovation evaluation section, the review report, combining the reasoning path in the evidence chain regarding the mechanism of introducing non-natural amino acids and the screening process of PylRS variants, concludes that the application breaks through the existing technical paradigm that mainly relies on copper-catalyzed click chemistry, demonstrating significant innovation in achieving protein-specific labeling at the live-cell level.

[0066] The review report not only provides quantitative evaluation results, but also clearly shows the knowledge nodes and reasoning paths on which each evaluation conclusion depends, thereby making the evaluation process transparent and explainable.

[0067] The above embodiments are merely illustrative of the implementation methods of the present invention, but should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the protection scope of the present invention.

Claims

1. A method for evaluating academic applications using knowledge graph-enhanced large-scale model reasoning, characterized in that, The academic application evaluation method includes the following steps: Step 1: Construct a pattern-driven knowledge graph; transform unstructured academic texts into knowledge graphs with topological structure and semantic features. It includes four stages: initial reasoning stage, knowledge extraction stage, confidence assessment and threshold screening, and formal representation of effective triples. Step 2: Based on the knowledge graph constructed in Step 1, the agent performs pathfinding through a thinking-action loop to achieve in-depth mining and automated generation of academic argumentation evidence chains. Finally, the agent completes its exploration on the knowledge graph. The intelligent reasoning framework consists of a semantic deconstructor, a heuristic pathfinding agent, and a state convergence determiner. First, the semantic deconstructor performs semantic parsing and task decomposition of the academic application evaluation requirements, generating an initial reasoning state to guide graph reasoning. Then, guided by the initial reasoning state, the heuristic pathfinding agent drives the agent to perform multi-hop pathfinding operations in the knowledge graph, obtaining a set of evidence chains. Finally, the state convergence determiner updates the state of the evidence chain set and determines convergence, forming a structured set of evidence chains B that meets the evaluation requirements. The above process is executed sequentially in the order of goal generation, path exploration, and evidence convergence. Step 3: Structured Evidence Integration and Multi-Dimensional Scoring; Receive the evidence chain set B finally locked in Step 2 through the thinking-action cycle, start the evaluation process, and transform the discrete logical paths into authoritative review scores and structured reports by dimensional modeling of the evidence chains in evidence chain set B, thus realizing the final closed loop from fact discovery to value judgment.

2. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 1, characterized in that, The first step is as follows: Step 1.1: In the initial reasoning stage of constructing the academic knowledge graph, an ontology model S is established to provide a normative framework for the structured transformation of unstructured text, and at the same time to lay a unified semantic constraint for subsequent structured extraction. Step 1.2: Under the constraints of the ontology schema S, perform structured semantic parsing on the input academic text. Based on the predefined entity types and relation types in the ontology schema, use a large language model to identify entity pairs that conform to the ontology schema from the text, determine the semantic relationships between entities, and generate a set of triples of academic knowledge consisting of subject entities, relation types, and object entities. This enables the mapping from unstructured text to structured graph data; Step 1.3 introduces a confidence assessment and threshold screening mechanism to evaluate the set of triples obtained in Step 1.

2. Perform consistency verification and reliability enhancement; map the relation types in the triple set to directed edges connecting the corresponding entity nodes, and construct an academic knowledge graph composed of a node set and an edge set; Step 1.4: After completing entity alignment and conflict detection, the extracted valid triples are formally represented. Finally, a knowledge graph is obtained. The topological structure is represented by the adjacency matrix A, and the semantic features of the nodes are represented by the feature matrix H.

3. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 1, characterized in that, In step one: Step 1.1 specifically includes: The ontology schema defines the entity type space. and relation type space ,in, Represents the first type of graph node, Represents nodes of the second type of graph. Represents the nth type of graph node, This represents the first type of connecting edge. This represents a second type of connecting edge. This represents the m-th type of connection edge; through predefined patterns, the large language model maps fragmented academic text to a unified semantic dimension; for any input text paragraph, the goal of modeling its logical structure is to construct a knowledge graph. ; Step 1.2 specifically includes: In the knowledge extraction stage, a large language model is used as the information extraction function. , the original text Mapped to a set of triples The specific process is as follows: （2）； Where S represents the ontology pattern defined in step 1.1, which serves as the extraction constraint; T represents the original text, specifically the input original academic text paragraph. Represents the main entity; Represents the entity or object; Then, it represents the predicate relationship between the two; V represents the set of nodes; E represents the set of directed edges; Step 1.3 specifically includes: For each extracted triplet Introducing confidence rating factors , ,in, This represents the posterior probability of the logical truth of the triple given the original text T; and a presupposed rigor threshold is used. Only when At that time, the triple is admitted into the knowledge graph as a valid triple; After screening, valid triples that meet the rigor threshold are treated as valid knowledge units and subjected to graph structuring. Specifically, different entities appearing in the valid triples are treated as nodes in the academic knowledge graph; and the relation types in the valid triples are mapped to directed edges connecting the corresponding subject entity nodes and object entity nodes, thus forming an academic knowledge graph composed of a set of nodes and a set of edges. ; Step 1.4 specifically involves: The canonical framework integrates semantic embedding, mapping the textual descriptions of nodes into high-dimensional vectors through a pre-trained encoder. : （3）； in, Represents a node The original text attributes; At this point, the logical structure of the academic text is jointly represented by the adjacency matrix A and the feature matrix H; the adjacency matrix A originates from the academic knowledge graph extracted in step 1.

3. Direct generation: If there is an academic logic r between entity J and entity L that conforms to the ontology schema S, then the elements in the adjacency matrix A... Otherwise, it is 0; the feature matrix H is derived from semantic embedding, and each node is encoded by the pre-trained encoder in step 1.

4. The original text attributes are mapped to high-dimensional vectors. Then, the high-dimensional vectors of all nodes are arranged in order to form the feature matrix. ;in, This represents the high-dimensional vector of the first node. This represents the high-dimensional vector of the second node. Let H represent the high-dimensional vector of the b-th node, and T represent the transpose of the matrix; matrix H describes the implicit semantic information of the academic content.

4. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 3, characterized in that, In step 1.3, the rigor threshold The preset probability threshold used to measure the logical credibility of triplet sets ranges from 0.5 to 0.

9.

5. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 4, characterized in that, The second step is specifically as follows: Step 2.1, Inference State Initialization and Target Decomposition: Receive external input evaluation requirements and generate the initial inference state through semantic deconstruction. This provides clear navigation instructions for subsequent exploration; Step 2.2, Graph-based action execution and path exploration, is performed under the guidance of the logical query vector q output in Step 2.1; The agent maps the evaluation requirements to pathfinding actions on the graph, while using the knowledge graph generated in step one to extract the real logical connections. Step 2.3, Dynamic determination of state update and termination condition: This step receives the path segments discovered in each iteration of Step 2.

2. The information is compared with the logical query vector q defined in step 2.1; if the evidence is sufficient, the reasoning is terminated; otherwise, a new intent vector is generated and fed back to step 2.2 to continue the search.

6. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 5, characterized in that, In step two: Specifically, step 2.1 is as follows: The normative framework accepts the original text of academic applications to be evaluated. The evaluation requirements are analyzed by a semantic deconstructor based on a large language model, and a predefined set of evaluation dimensions C is used as input. The semantic deconstruction process maps the fuzzy evaluation criteria into a logical query vector q in a high-dimensional semantic space, and generates the corresponding initial task stack at the same time. Through the above processing, the transformation from the original evaluation requirements to the structured reasoning objective is completed. The generated logical query vector q and the initial task stack together constitute the initial reasoning state. This provides semantic guidance for subsequent path exploration based on knowledge graphs; Specifically, step 2.2 involves: In this step, the canonical framework takes the knowledge graph G constructed in step 1.4 and the logical query vector q output in step 2.1 as input, and executes action decisions on the graph structure through a heuristic pathfinding agent. At each time step t, the agent matches and scores its neighboring nodes based on the current anchor node and the dynamically generated inference intent vector, and selects the next hop node under topological constraints. The transition probability is determined by the following formula: （4）； in, This represents the anchor node where the intelligent agent is currently located, i.e., the academic entity it is currently focusing on; This represents the target node that the agent points to in the candidate set; This indicates that the current node is Under these conditions, the agent chooses to jump to the node. The transition probability; It is a normalized exponential function; It is a cosine similarity metric function; The inference intent vector at time step t is dynamically generated by the large language model based on the global objective q and the currently obtained evidence. Represents the nodes in the characteristic matrix H The semantic embedding vector is generated by the pre-trained encoder in step 1.4; It is a topological constraint condition, which means that the value of the i-th row and j-th column of the adjacency matrix A is 1, that is, only logical jumps between physically connected nodes in the graph are allowed; Through a multi-step pathfinding process driven by an intelligent agent, the system gradually obtains a set of logical path segments consisting of multiple entity nodes and their relational edges. This set of path segments constitutes an evidence chain set. , used to characterize the potential argumentation structure in academic applications; Specifically, step 2.3 is as follows: The specification framework will identify the path segments discovered in each iteration of step 2.

2. Gradually integrate into the global reasoning context, and incrementally update the currently constructed knowledge state through a gating update mechanism. The state transition process is shown in formula (5). Whenever the agent completes a graph action at time step t, And capture new logical subgraphs Subsequently, the system incrementally updates the global inference context through a state transition function; mathematical expression: （5）； in, Represents the currently constructed knowledge context; This indicates capturing a new logical subgraph. The subsequent knowledge context, These are newly extracted entity relation triples from the graph; It is a gated update function used to filter out redundant information that is irrelevant to the current evaluation dimension, and retain only the core argument relationships that contribute to the evaluation objective; Next, based on the logical convergence criterion, the evidence chain set is calculated. The degree of impact of the uncertainty of the assessment results determines whether to continue graph exploration. The determination process is shown in formula (6). The normative framework introduces a dynamic termination determination function based on evidence entropy. ; knowledge context captured by computation after obtaining a new logical subgraph The determination formula is defined for the uncertainty contribution of the logical query vector q: （6）； in, Indicates the judgment result; This indicates the termination of this round of exploration. This indicates that the current exploration will continue; This indicates capturing a new logical subgraph. The subsequent knowledge context; D is a divergence function; q is the logical query vector defined in step 2.1; It is the convergence threshold; t is the maximum number of iterations, and t is the current number of iterations. When the judgment result When the preset convergence condition is met or the maximum number of iterations is reached, the agent's graph exploration process is terminated, and... This is determined to be the final structured full set of evidence, serving as evidence chain set B; Through the state update mechanism, the agent integrates the scattered path fragments discovered in step 2.2 into a complete set of structured evidence chains.

7. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 6, characterized in that, In step 2.3, when the volatility is below this value, it indicates that the existing evidence is sufficient to form a stable assessment, and a convergence threshold is defined. Between 0 and 0.

3.

8. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 7, characterized in that, Step three specifically refers to: Step 3.1: Dimensional Logical Deduction and Quantitative Scoring: After obtaining the complete evidence chain set B, proceed to this step, use the thought chain model to conduct in-depth logical deduction on specific dimensions, and generate preliminary quantitative scores for each dimension; Step 3.2: Multi-path voting consensus and review report generation to achieve robustness of the final conclusion: Receive the preliminary score vectors of each dimension generated in Step 3.1; perform parallel reasoning and consensus voting, associate the aggregated scores with the evidence chain set B in Step 2.3, and generate the final report.

9. The academic application evaluation method for knowledge graph-enhanced large-scale model reasoning according to claim 8, characterized in that, In step three: Step 3.1 specifically involves: Using the evidence chain set B output in step 2.3 and the predefined evaluation index system C as input, a multi-dimensional thinking chain reasoning model based on a large language model is used to perform logical deduction for each evaluation dimension. Specifically, for a certain evaluation dimension m in the index system C, some evidence paths that are highly related to that evaluation dimension are selected from the evidence chain set B. Furthermore, by combining the pre-set special assessment prompt templates for this evaluation dimension, the model is guided to reason step by step around the corresponding academic evaluation logic. During the reasoning process, the special assessment prompts the model to follow the evidence path. The performance of the evaluated object on the corresponding evaluation dimension is logically summarized and quantitatively mapped to obtain the preliminary score of the evaluation dimension. The calculation process is shown in formula (7): （7）； in, This represents the quantitative score of the m-th evaluation dimension; This represents the reasoning mapping function of the thought chain; It is a subset of path segments that are highly correlated with dimension m, filtered from the total evidence set; This is a template for a specific assessment tool designed for dimension m. The agent transforms the structured chain of evidence into a multi-dimensional preliminary scoring vector; Step 3.2 specifically refers to: For each evaluation dimension, multiple parallel samplings are performed on the reasoning process in step 3.1 to obtain multiple sets of scoring results under the same evaluation dimension. These multiple sets of scoring results are used as inputs and combined with the predefined evaluation index weight vector W. The scoring results are statistically integrated through a multi-path consensus aggregation function. The multi-path consensus aggregation function weakens the random fluctuations that may be caused by a single reasoning and improves the robustness of the scoring results. Its calculation process is shown in formula (8): （8）； Where Score represents the final weighted overall score of the academic application; n represents the total number of predefined evaluation dimensions, and i represents the i-th evaluation dimension. Let represent the weight coefficient of the i-th evaluation dimension, and satisfy . ; This represents a multi-path consensus aggregation function; This represents the random path score obtained when performing the k-th parallel sampling inference on the i-th dimension, where k represents the total number of samplings performed for parallel inference on the same dimension; While obtaining the final weighted overall score, the final weighted overall score is associated with the evidence chain set B obtained in step 2.3, and an evaluation report R containing the scoring basis and logical path description is automatically generated, thereby completing the transformation from structured evidence to authoritative evaluation conclusion.

Images