A method and system for identifying innovative points in academic papers

Abstract

This invention discloses a method and system for identifying innovative points in academic papers, relating to the field of artificial intelligence technology. The method first processes the paper abstract through a concept path extraction module, including: structured semantic segmentation, concept pair extraction and verification, constraint relation triple generation, hierarchical verification, and path optimization, to construct a complete set of concept paths for the paper. Then, in a rare path discovery module, the popularity index of each concept path in the global path frequency dictionary is calculated and compared with a threshold to identify rare paths as the scientific innovative points of the paper. This invention effectively suppresses the illusion of language models through a knowledge graph constraint mechanism, significantly improving the accuracy of concept path extraction and its coverage of long-tail concepts, achieving accurate and efficient identification of innovative points in academic papers at the path level.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for identifying innovative points in academic papers. Background Technology

[0002] With the surge in the number of academic publications worldwide, researchers face the severe challenge of information overload. Existing academic search and analysis systems primarily rely on keyword matching, semantic similarity calculation, or document retrieval methods based on pre-trained language models. For example, the SciBERT model improves the semantic understanding of scientific vocabulary through pre-trained scientific text corpora (I. Beltagy, K. Lo, and A. Cohan, "Scibert: A pretrained language model for scientific text," arXiv preprint arXiv:1903.10676, 2019.), and TechNet utilizes a technology semantic network to support concept association analysis (S. Sarica, J. Luo, and KL Wood, "Technet: Technology semantic network based on patent data," Expert Systems with Applications, vol.142, p.112995, 2020.).

[0003] However, existing methods have the following technical drawbacks:

[0004] 1. Insufficient concept association modeling: Most systems only focus on concept similarity matching or static graph embedding, lacking the ability to model the dynamic and structured paths between concepts within a paper (Y.Wang, Y.Peng, and J.Guo, “Enhancing knowledge graph embedding with structure and semantic features: Y.wang et al.” Applied Intelligence, vol.54, no.3, pp.2900–2914, 2024.), (D.Wang, Y.Li, S.Wang, X.Chen, J.Liao, D.Li, and X.Li, “Ckemi: Concept knowledge enhanced metaphor identification framework,” Information Processing & Management, vol.62, no.1, p.103946, 2025.).

[0005] 2. Poor coverage of long-tail concepts: Existing academic search and analysis systems based on pre-trained language models fail to fully utilize knowledge graph constraint mechanisms to guide the generation of small language models, resulting in unrobust analysis results, low accuracy in identifying innovative concepts, and a tendency to "illusion" or miss detection when dealing with low-frequency or emerging concepts. This makes them difficult to apply to large-scale academic literature tracking scenarios (M. Iori, M. Fontana et al., "Novelty as recombination of knowledge," in 17th International Conference on Scientometrics and Informetrics, ISSI 2019-Proceedings, vol. 1. International Society for Scientometrics and Informetrics, 2019, pp. 1210–1213.).

[0006] 3. Lack of path-level innovation discovery: Existing methods often focus on the novelty of the concepts themselves and the updating of macro-level knowledge graphs, neglecting the fine-grained integration of concept paths in individual papers. This leads to insufficient accuracy in rare path identification and innovation point discovery, and easily misses innovative combinations of mainstream concepts (R.K.Amplayo, S. Hong, and M. Song, “Network-based approach to detect novelty of scholarly literature,” Informationsciences, vol.422, pp.542–557, 2018.), (C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P.J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol.21, no.140, pp.1–67, 2020.), (J. Priem, H. Piwowar, and R. Orr, “Openalex: A fully-open index of scholarly works, authors, venues, institutions, and concepts,” arXiv preprint arXiv:2205.01833, 2022.).

[0007] Therefore, there is an urgent need for an intelligent academic literature analysis system that can deeply integrate paper content with knowledge graphs, possess path-level analysis capabilities, and effectively identify structured innovation points. Summary of the Invention

[0008] The purpose of this invention is to address the technical problems of existing academic analysis systems, such as incomplete conceptual path modeling, inaccurate long-tail concept identification, and insufficient ability to discover path-level innovation points. Based on knowledge graph constraints and intelligent agent technology, this invention provides a method and system for identifying innovation points in academic papers.

[0009] To achieve the above objectives, this invention provides a method for identifying innovative points in academic papers, comprising the following steps:

[0010] (1) Obtain the abstract text of the academic paper to be analyzed;

[0011] (2) Perform structured parsing on the abstract text to generate semantic fragments; extract candidate concept pairs from the semantic fragments and perform concept verification through the domain knowledge graph; generate relation triples based on the verified concept pairs under the constraints of the domain knowledge graph; assemble the relation triples into an initial concept hierarchy graph, optimize the structure of the initial concept hierarchy graph by iteratively proposing and verifying intermediate concepts, and obtain an optimized concept hierarchy graph; extract a complete set of concept paths from the optimized concept hierarchy graph.

[0012] (3) Match each concept path in the concept path set with the pre-built global path frequency dictionary and calculate the popularity index of each concept path;

[0013] (4) Compare the popularity index with the preset popularity threshold, mark the concept path with the popularity index below the popularity threshold as a rare path, and take all the rare paths of the paper as the result of its innovation point identification.

[0014] Furthermore, the semantic fragment includes relevant research, research methods, and conclusions.

[0015] Furthermore, in step (2), for each concept in the candidate concept pair, it is determined whether it is in the domain knowledge graph; if so, it is confirmed as a valid concept and stored in the database; otherwise, an external knowledge base is queried for fuzzy matching. If the fuzzy matching fails, the concept is submitted to experts for manual verification and annotation, and the verified concept is updated to the knowledge base.

[0016] Further, in step (2), the semantic fragment is used as the context for model reasoning. The verified concept pair (concept A, concept B) and the context are input into the language model together. The model analyzes the context and selects one of the two given relation directions (concept A, is-a, concept B) or (concept B, is-a, concept A). In the relation direction, is-a represents the child concept and is-a represents the parent concept. The model determines whether there is an "is-a" relationship between concept A and concept B and generates a relation triplet.

[0017] Furthermore, the step of optimizing the structure of the initial concept hierarchy graph by iteratively proposing and verifying intermediate concepts to obtain the optimized concept hierarchy graph specifically includes the following iterative steps:

[0018] Scan the initial concept hierarchy diagram to identify problematic direct connections A→B;

[0019] A language model is used to propose one or more intermediate concepts C for the direct connection relationship A→B;

[0020] Based on the system's or expert's decision, perform add, retain, or delete operations on the proposed intermediate concept C and the new relations (A, is-a, C) and (C, is-a, B) that constitute it, and update the concept hierarchy diagram;

[0021] When the loop termination condition is met, stop the iteration and output the optimized concept hierarchy graph.

[0022] Furthermore, the loop termination condition includes at least one of the following: the concept hierarchy diagram no longer changes during a complete iteration; or the number of iterations reaches a preset maximum number of iterations.

[0023] Furthermore, the extraction of the complete set of concept paths from the optimized concept hierarchy graph is achieved through a breadth-first search algorithm.

[0024] Furthermore, the global path frequency dictionary is constructed in the following manner:

[0025] On the academic paper knowledge graph, the analysis process described in step (2) is applied to multiple papers to extract the conceptual path of each paper;

[0026] The frequency of each unique concept path in the knowledge graph of the academic paper is counted to form a path-frequency mapping dictionary.

[0027] Furthermore, the popularity index D = log(1 + F), where F is the frequency of the concept path in the global path frequency dictionary.

[0028] To achieve the above objectives, the present invention also provides an academic paper innovation point identification system that implements the above method, comprising:

[0029] The acquisition module is used to acquire the abstract text of the academic paper to be analyzed;

[0030] The concept path extraction module is used to perform structured parsing of the summary text to generate semantic fragments; extract candidate concept pairs from the semantic fragments and perform concept verification through a domain knowledge graph; generate relation triples based on the verified concept pairs under the constraints of the domain knowledge graph; assemble the relation triples into an initial concept hierarchy graph; optimize the structure of the initial concept hierarchy graph by iteratively proposing and verifying intermediate concepts to obtain an optimized concept hierarchy graph; and extract a complete set of concept paths from the optimized concept hierarchy graph.

[0031] The rare path discovery module is used to match each concept path in the concept path set with a pre-built global path frequency dictionary, calculate the popularity index of each concept path, compare the popularity index with a preset popularity threshold, and mark concept paths with popularity indices lower than the popularity threshold as rare paths.

[0032] The identification results module is used to identify all rare paths in the paper as its innovative points.

[0033] The present invention has the following significant beneficial effects:

[0034] 1. Significantly improved concept path extraction accuracy: Through the knowledge graph constraint mechanism, the F1 score in the constraint relation triple generation stage increased from 41.29% without constraints to 82.26%, achieving an accuracy of 95.19%. The overall end-to-end system F1 score reached 97.24%, significantly higher than the 27.98% concept path extraction accuracy of the direct generation method and the less than 9% concept path extraction accuracy of the zero-sample large model.

[0035] 2. Significantly improved accuracy in identifying innovative points: Based on the analysis of the rarity of conceptual paths, the innovation rate of low-prevalence paths reached 57.7%, which is more than 2.48 times that of high-prevalence paths (23.2%), providing a new and effective indicator for academic innovation discovery.

[0036] 3. Enhanced coverage of long-tail concepts: Through the concept verification mechanism guided by knowledge graph, emerging concepts in the long-tail distribution are effectively identified and processed, solving the problem of insufficient coverage of rare concepts by traditional methods.

[0037] 4. Model illusion is effectively suppressed: Knowledge graph constraints improve the accuracy of constraint relation triple generation from 56.90% to 95.19%, proving the strong constraint effect of structured external knowledge on the output of the language model.

[0038] 5. Optimized computational efficiency: The use of a small language model significantly reduces the computational resource requirements compared to a large model with hundreds of billions of parameters, making it more suitable for practical deployment. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort:

[0040] Figure 1 This is a schematic diagram of the overall processing flow of the present invention;

[0041] Figure 2 This is a flowchart illustrating the implementation of the structured semantic segmentation stage in the concept path extraction module of this invention.

[0042] Figure 3 This is a flowchart illustrating the implementation of the concept pair extraction and verification stage in the concept path extraction module of this invention.

[0043] Figure 4 This is a flowchart illustrating the implementation of the constraint relationship triple generation stage in the conceptual path extraction module of this invention.

[0044] Figure 5 This is a flowchart illustrating the implementation of the hierarchical verification and path optimization stages in the conceptual path extraction module of this invention.

[0045] Figure 6 This is a flowchart illustrating the implementation of the rare path discovery module of the present invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0048] This invention provides a method and system for identifying innovative points in academic papers, which integrates a small language model and an open academic knowledge graph. It is used to automatically extract conceptual paths in papers and identify scientific innovation points, and is applicable to specific application scenarios such as intelligent analysis of academic literature, discovery of scientific research trends, and mining of technological opportunities.

[0049] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] 1. Overall architecture.

[0051] The overall process of the system of this invention is as follows: Figure 1 As shown. The system consists of two modules connected in series, which take paper abstracts as input and output complete concept paths with innovation annotations.

[0052] For a new paper to be analyzed, firstly, the abstract is processed by the concept path extraction module to obtain the complete set of concept paths for the paper. Then, in the rare path discovery module, the popularity index of each path in the complete set of concept paths is calculated, and rare paths are selected. Finally, the system outputs all rare paths in the paper as the innovation point identification result.

[0053] 2. Concept path extraction module.

[0054] The concept path extraction module employs a four-stage framework for its analysis process, which includes the following steps:

[0055] Phase 1: Structured Semantic Segmentation. Use a language model to perform structured parsing on the paper abstract, generating three semantic segments (related research, research methods, and conclusions), which serve as context anchors for subsequent concept extraction.

[0056] See Figure 2 , use a language model (such as the T5-base model) to split the paper abstract into sentences and semantic chunks. The model input is the original abstract, and the output is structured segmented text (including related research, research methods, and conclusions). The output is stored in the database as a context anchor, providing a reliable context basis for the subsequent phase.

[0057] One prompt template for inputting into the language model is as follows:

[0058] “Parse A into three segments: <related_research>...< / related_research><research_methods>...< / research_methods> <conclusions> ...< / conclusions> ”

[0059] The Chinese meaning corresponding to the above prompt is: “Parse A into the following three segments: <related research>...< / related research><research methods>...< / research methods><conclusions>...< / conclusions>”.

[0060] Phase 2: Concept Pair Extraction and Verification. Use a language model to extract candidate concept pairs from the semantic segments, in the format [domain, specific concept], and verify the concepts through a domain knowledge graph. Unmatched concepts are manually verified by an expert system.

[0061] See Figure 3 , the input for this phase is the structured segmented text output from Phase 1, and the output is standardized concept pairs, where the format of each concept is [domain, specific concept]. For each segmented text, use a language model (such as the Qwen3-4B-Instruct-2507 model) to extract standardized concept pairs. For each extracted specific concept, first determine whether it is in the concept system of a scientific research paper database (such as the OpenAlex database). If so, confirm it as a valid concept and store it in the database; if not, query an external knowledge base (such as Wikidata or DBpedia) for fuzzy matching. If the fuzzy matching is successful, confirm and standardize the concept, and then store it in the database; if the fuzzy matching fails, submit the specific concept to human experts for verification. After the experts confirm and annotate it, update the concept to the knowledge base (i.e., knowledge base = OpenAlex database + external knowledge base + expert-annotated concepts), and then store it in the database.

[0062] Phase 3: Constraint Relation Triple Generation. Under the constraints of the knowledge graph, "is-a" relation triples are generated based on verified concept pairs. Only concepts verified in the previous phase are allowed to participate in relation construction, effectively suppressing model illusion.

[0063] See Figure 4 The input for this stage is the concept pair (concept A, concept B) from the second stage, and the structured segmented text obtained from the first stage (as the context for reasoning). The concept pair and context are then fed into a language model (such as the Qwen3-4B-Instruct-2507 model), which analyzes the context to determine if an "is-a" relationship exists between concept A and concept B, and identifies the correct direction. The model must choose between two options:

[0064] (A,is-a,B): means "A is a kind of B", that is, A is a child concept and B is a parent concept.

[0065] (B,is-a,A): means "B is a kind of A", that is, B is a sub-concept and A is a parent concept.

[0066] Key constraint: The model is not allowed to use any new concepts outside of concept pairs. This hard rule is the core mechanism to prevent illusions.

[0067] The generated candidate triple (Child, is-a, Parent) is immediately compared with the OpenAlex knowledge graph to check whether Parent is a direct or indirect parent node of Child in the OpenAlex hierarchy. If it exists, the triple is considered valid and stored in the database. If it does not exist, the triple is marked as invalid or incorrect.

[0068] The output of this stage is a series of "is-a" relation triples validated by the knowledge graph. These triples will be passed to the fourth stage.

[0069] Phase 4: Hierarchical Validation and Path Optimization. Through iterative intermediate concept proposals and expert validation, the conceptual hierarchy is improved, and a complete generalized path is finally output.

[0070] See Figure 5The input for this stage is the set of relation triples from the third stage. First, these triples are assembled into an initial, directed hierarchical graph G. At this point, graph G may be disconnected, have gaps, or contain redundancy. Then, graph G is scanned (scanning methods include connectivity analysis using depth-first search, cycle detection using depth-first search, redundant edge identification based on graph transitivity reduction, and gap detection based on transitive closure analysis) to identify problematic structures, such as A→B. Then, a language model (such as the Qwen3-4B-Instruct-2507 model) is used to propose one or more reasonable intermediate concepts C to complete the problematic structures. For each proposed intermediate concept C and the new relations (A, is-a, C) and (C, is-a, B) it constitutes, the system (or a human expert) will determine:

[0071] "Add": The system considers the intermediate concept and relationship to be correct and necessary. It then formally adds it to graph G. This changes the graph structure and triggers a new round of iterations.

[0072] "Retain": This means that the existing relationship A→B is considered sufficient and does not need to be modified. Graph G remains unchanged.

[0073] "Delete": If the existing relationship A→B is found to be incorrect, the system will delete the incorrect relationship from graph G. This will also change the graph structure and trigger iteration.

[0074] The above cycle of "detection → proposal → decision → update" will continue. The cycle will terminate under at least one of the following conditions:

[0075] Condition 1: In a complete iteration, graph G no longer changes.

[0076] Condition 2: Reach the preset maximum number of iterations to prevent infinite loops.

[0077] When the loop terminates, the final optimized concept hierarchy graph G is output.

[0078] 3. The rare path discovery module uses the "concept path popularity index" proposed in this invention to screen for highly innovative concept paths. The calculation of this index relies on a "global path frequency dictionary." Before analyzing any specific paper, it is necessary to know which paths are common and which are rare in the entire academic field—that is, the "global path frequency dictionary." When calculating the concept path popularity index of a paper, each concept path in the paper is matched with the global path frequency dictionary to obtain its global occurrence frequency, i.e., the concept path popularity. Subsequently, all popularity scores are compared with a predefined popularity threshold to classify concept paths as "rare" and "common," thereby identifying the innovative points in the paper.

[0079] See Figure 6 The input to this module is the concept hierarchy graph G from the concept path extraction module. First, a breadth-first search algorithm is used in graph G to find all paths from the root node to the leaf node, resulting in a complete and high-quality set of concept paths for this paper. Each path in the set Match it with the global path frequency dictionary to find its global occurrence frequency. Then, according to calculate Popularity index .like Less than the popularity threshold Then Mark as a rare path, otherwise... The paths are marked as common paths. Finally, all rare paths in the analyzed paper are output as the results of the paper's innovation identification.

[0080] A global path frequency dictionary can be constructed based on publicly available scientific research paper knowledge graphs, such as OpenAlex. The construction method is as follows: for each paper in the database, generate all its concept paths using the scheme described in the concept path extraction module. Summarize and statistically analyze all extracted paths, calculating the frequency of each unique concept path in the entire knowledge graph. For example, the path "Computer Science → Artificial Intelligence → Natural Language Processing" appears 1500 times, while the path "Physics → Materials Science → Topological Insulators → Quantum Computing" appears only 25 times. This yields a global path frequency dictionary.

[0081] Conceptual path Popularity index The calculation method is as follows:

[0082]

[0083] in, yes The frequency of occurrence in the global path frequency dictionary.

[0084] Popularity threshold Set to the median of popularity in the global path frequency dictionary.

[0085] 4. Optional implementation methods.

[0086] In other implementations, the OpenAlex knowledge graph can be replaced by Microsoft Academic Graph or Aminer, and the language model can be replaced by models with fine-tuning capabilities such as LLaMA or ChatGLM. The path construction algorithm can be replaced by a depth-first search algorithm or a random walk algorithm, as long as it can guarantee the integrity and interpretability of the path.

[0087] The above description is merely an embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of the claims of the present invention.

Claims

1. A method for identifying innovative points in academic papers, characterized in that, Includes the following steps: (1) Obtain the abstract text of the academic paper to be analyzed; (2) Perform structured parsing on the abstract text to generate semantic fragments; extract candidate concept pairs from the semantic fragments and perform concept verification through the domain knowledge graph; generate relation triples based on the verified concept pairs under the constraints of the domain knowledge graph; assemble the relation triples into an initial concept hierarchy graph, optimize the structure of the initial concept hierarchy graph by iteratively proposing and verifying intermediate concepts, and obtain an optimized concept hierarchy graph; extract a complete set of concept paths from the optimized concept hierarchy graph. The process of optimizing the structure of the initial concept hierarchy graph by iteratively proposing and verifying intermediate concepts to obtain the optimized concept hierarchy graph specifically includes the following loop: Scan the initial concept hierarchy diagram to identify problematic direct connections A→B, where A and B both represent concepts; A language model is used to propose one or more intermediate concepts C for the direct connection relationship A→B; Based on the system's or expert's decision, perform add, retain, or delete operations on the proposed intermediate concept C and the new relations (A, is-a, C) and (C, is-a, B) that constitute it, and update the concept hierarchy diagram; where is-a represents the child concept and is-a represents the parent concept. When the loop termination condition is met, the iteration stops and the optimized concept hierarchy diagram is output. (3) Match each concept path in the concept path set with the pre-built global path frequency dictionary and calculate the popularity index of each concept path; The global path frequency dictionary is constructed in the following way: On the academic paper knowledge graph, the analysis process of step (2) is applied to multiple papers to extract the concept path of each paper; the frequency of each unique concept path in the academic paper knowledge graph is counted to form a path-frequency mapping dictionary. The popularity index D = log(1 + F), where F is the frequency of the concept path in the global path frequency dictionary; (4) Compare the popularity index with the preset popularity threshold, mark the concept path with the popularity index below the popularity threshold as a rare path, and take all the rare paths of the paper as the result of its innovation point identification.

2. The method for identifying innovative points in academic papers according to claim 1, characterized in that, In step (2), for each concept in the candidate concept pair, it is determined whether it is in the domain knowledge graph; if so, it is confirmed as a valid concept and stored in the database; otherwise, an external knowledge base is queried for fuzzy matching. If the fuzzy matching fails, the concept is submitted to experts for manual verification and annotation, and the verified concept is updated to the knowledge base.

3. The method for identifying innovative points in academic papers according to claim 1, characterized in that, In step (2), the semantic fragment is used as the context for model reasoning. The verified concept pair (concept A, concept B) and the context are input into the language model together. The model analyzes the context and selects one of the two given relation directions (concept A, is-a, concept B) or (concept B, is-a, concept A). In the relation direction, is-a represents the child concept and is-a represents the parent concept. The model determines whether there is an "is-a" relation between concept A and concept B and generates a relation triplet.

4. The method for identifying innovative points in academic papers according to claim 1, characterized in that, The loop termination condition includes at least one of the following: the concept hierarchy diagram no longer changes during a complete iteration; or the number of iterations reaches the preset maximum number of iterations.

5. The method for identifying innovative points in academic papers according to claim 1, characterized in that, The extraction of the complete set of concept paths from the optimized concept hierarchy graph is achieved through a breadth-first search algorithm.

6. An academic paper innovation point identification system implementing the method described in claim 1, characterized in that, include: The acquisition module is used to acquire the abstract text of the academic paper to be analyzed; The concept path extraction module is used to perform structured parsing of the summary text to generate semantic fragments; extract candidate concept pairs from the semantic fragments and perform concept verification through a domain knowledge graph; generate relation triples based on the verified concept pairs under the constraints of the domain knowledge graph; assemble the relation triples into an initial concept hierarchy graph; optimize the structure of the initial concept hierarchy graph by iteratively proposing and verifying intermediate concepts to obtain an optimized concept hierarchy graph; and extract a complete set of concept paths from the optimized concept hierarchy graph. The rare path discovery module is used to match each concept path in the concept path set with a pre-built global path frequency dictionary, calculate the popularity index of each concept path, compare the popularity index with a preset popularity threshold, and mark concept paths with popularity indices lower than the popularity threshold as rare paths. The identification results module is used to identify all rare paths in the paper as its innovative points.

Images