A method and apparatus for generating literature keywords based on large model feedback reinforcement learning

By combining retrieval enhancement generation and reinforcement learning, and utilizing a large language model evaluation and subject classification system to generate high-quality keywords, the problems of inconsistent generation and insufficient coverage in existing technologies are solved, and efficient organization and retrieval of scientific literature are achieved.

CN123068657APending Publication Date: 2026-11-10COMP NETWORK INFORMATION CENT CHINESE ACADEMY OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611353160.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-02
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing technologies lack a deep understanding of the semantics of documents in the generation of keywords for scientific literature. The generated results lack quality control mechanisms and are difficult to utilize weak supervision signals in large-scale unlabeled scientific corpora, resulting in inconsistent generated keywords and insufficient coverage.

Method used

By combining Retrieval Enhancement Generation (RAG) and Reinforcement Learning (RL), candidate keywords are generated using a lightweight large language model, the quality is evaluated using a large-scale large language model, and the model is optimized through reinforcement learning. In addition, preference signals are constructed by combining subject classification system and user feedback to achieve high-quality keyword generation.

Benefits of technology

The generated keywords are semantically accurate and have high coverage. They can automatically generate high-quality keyword sets, improving the efficiency of organizing, retrieving, and analyzing scientific literature, and adapting to the ever-evolving scientific terminology and user preferences.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123068657A_ABST
    Figure CN123068657A_ABST
Patent Text Reader

Abstract

This invention discloses a method and apparatus for generating keywords in scientific literature based on large-scale model feedback reinforcement learning, belonging to the field of natural language processing technology. The method includes: retrieving semantically relevant documents from an unlabeled scientific corpus; feeding the input document and semantically relevant documents into a lightweight large-scale language model to generate candidate keywords, obtaining a candidate keyword set; using a large-scale language model to evaluate the quality of the candidate keyword set, obtaining a preference reward signal for each candidate keyword; fine-tuning the lightweight large-scale language model based on the preference reward signal using a reinforcement learning algorithm to obtain a keyword generation model; deploying the keyword generation model and combining it with the unlabeled scientific corpus to obtain the keyword generation results for the target document. This invention can automatically generate high-quality, semantically accurate keyword sets from scientific literature, improving the efficiency of organizing, retrieving, and analyzing scientific literature.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention belongs to the field of natural language processing and scientific literature information management technology, specifically relating to a method and device for generating literature keywords based on large model feedback reinforcement learning (hereinafter referred to as SciKGen). Background Technology

[0002] Scientific literature is the primary carrier of human knowledge; however, with millions of new papers published annually, researchers struggle to comprehensively track the latest advancements across all fields. Against this backdrop, transforming massive collections of digital literature into structured, machine-understandable knowledge resources has become crucial, with efficient indexing and summarizing mechanisms being key. Keywords, as concise expressions of the core themes of a document, play a central role in literature retrieval and organization. However, a large amount of scientific literature (especially in computer science) lacks author-provided keywords, or existing author-annotated keywords are incomplete or inconsistent, severely limiting the effectiveness of current methods.

[0003] Existing keyword extraction methods can be mainly divided into the following categories: (1) Traditional statistical and graph methods, such as TF-IDF, TextRank, and TopicRank, mainly rank and select phrases that appear explicitly in a document based on word frequency, co-occurrence, or graph centrality signals. These methods can only operate on the surface form of the document and have limited coverage of the underlying semantic space.

[0004] (2) Methods based on pre-trained language models: such as KeyBERT, EmbedRank, SIFRank, etc., which use pre-trained language models to encode contextual semantics. Although the capabilities are improved, they are essentially still fine-grained extractors based on surface expressions and cannot generate semantic keywords that do not appear directly in the document.

[0005] (3) Direct generation method based on large language model: Although LLM can understand complex scientific texts and generate keywords, it often produces suboptimal results when used directly - the generated terms may be too broad or inconsistent, and general generation models are prone to deviating from the topic when there is a lack of document content anchoring.

[0006] The core challenges faced by existing technologies include: (a) lack of understanding of the deep semantics of documents, making it impossible to generate conceptual keywords that do not appear on the surface of the documents; (b) lack of quality control mechanisms for the generated results, making it impossible to systematically optimize the results in terms of semantic relevance, coverage, diversity, and other dimensions; and (c) difficulty in utilizing weak supervision signals in large-scale unlabeled scientific corpora to enhance the quality of the generated results.

[0007] Therefore, there is an urgent need to develop a scientific literature keyword generation method that can combine Retrieval Enhancement Generation (RAG) and Reinforcement Learning (RL) to overcome the above limitations. Summary of the Invention

[0008] To address the limitations of existing technologies in generating keywords for scientific literature, this invention proposes a method and apparatus for generating keywords based on large-model feedback reinforcement learning. This invention integrates retrieval enhancement with weak supervision, lightweight language model training, preference-based evaluation, and reinforcement learning optimization into a unified pipeline, enabling the automatic generation of high-quality, semantically accurate keyword sets from scientific literature, thereby improving the efficiency of organizing, retrieving, and analyzing scientific literature.

[0009] To achieve the above objectives, the technical solution of the present invention includes the following:

[0010] A method for generating literature keywords based on large-model feedback reinforcement learning, the method comprising: Retrieve semantically relevant documents of the input document from an unlabeled scientific corpus; wherein each document in the unlabeled scientific corpus Associated with one or more subject category tags, and the document No keywords were marked; The input document and semantically related documents are fed into a lightweight large language model to generate candidate keywords, resulting in a set of candidate keywords. A large-scale language model is used to evaluate the quality of the candidate keyword set and obtain the preference reward signal for each candidate keyword. Based on the preference reward signal, the lightweight large language model is fine-tuned using a reinforcement learning algorithm to obtain a keyword generation model; Deploy a keyword generation model and combine it with an unlabeled scientific corpus to obtain keyword generation results for the target document.

[0011] Furthermore, semantically relevant documents of the input document are retrieved from an unlabeled scientific corpus, including: Calculate the input document and the document separately. The dense vector representation is used to obtain the input document and the document based on the dense vector representation. The similarity between them; Select the K documents with the highest similarity Semantic-related documents that serve as input documents.

[0012] Furthermore, the preference reward signal is based on keyword-level quality scores. Category alignment score Topic specificity score and excessively dispersed penalty scores Generate; where, Represents the first in the candidate keyword set 10 candidate keywords.

[0013] Furthermore, the keyword-level quality score The classification alignment score The topic specificity score The excessively dispersed penalty score ;in, Indicates candidate keywords Semantic relevance to the input document Indicates candidate keywords Thematic coverage of the input document, Indicates candidate keywords semantic and lexical diversity Indicates candidate keywords Simplicity Indicates candidate keywords Semantic consistency with subject classification labels; Indicates candidate keywords The ability to reflect the core contributions of input documents and indicate a single topic category; Indicates candidate keywords The false dispersion extends to the level of unrelated subject classification labels.

[0014] Furthermore, based on the preference reward signal, the lightweight large language model is fine-tuned using a reinforcement learning algorithm to obtain a keyword generation model, including: Calculate the relative advantage of each candidate keyword in the candidate keyword set based on preference reward signals. Calculate the policy gradient based on relative advantage; The model parameters of the lightweight large language model are updated using the policy gradient.

[0015] Furthermore, after deploying the keyword generation model, the method also includes: We constructed a manually labeled dataset based on user feedback and calculated the online preference reward for each user feedback. For each data point in the manually labeled dataset, calculate the preference reward information for that data. The preference reward signal of this data is weighted and calculated with the corresponding online preference reward to generate the final reward; Based on the final reward, the keyword generation model is fine-tuned using a reinforcement learning algorithm.

[0016] A document keyword generation device based on large model feedback reinforcement learning, the device comprising: A retrieval enhancement module is used to retrieve semantically relevant documents from an unlabeled scientific corpus containing an input document; wherein each document in the unlabeled scientific corpus... Associated with one or more subject category tags, and the document No keywords were marked; The keyword generation module is used to feed the input document and semantically related documents into a lightweight large language model to generate candidate keywords and obtain a set of candidate keywords. The preference evaluation module is used to evaluate the quality of the candidate keyword set using a large-scale language model and obtain the preference reward signal for each candidate keyword. The reinforcement learning optimization module is used to fine-tune the lightweight large language model based on the preference reward signal using a reinforcement learning algorithm to obtain a keyword generation model. The human-computer interaction module is used to deploy the keyword generation model and, in conjunction with an unlabeled scientific corpus, obtain the keyword generation results for the target document.

[0017] A computer device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the document keyword generation method based on large model feedback reinforcement learning as described above.

[0018] A computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the document keyword generation method based on large model feedback reinforcement learning as described in any one of the claims.

[0019] A computer program product, when run on a computer device, causes the computer device to execute the literature keyword generation method based on large model feedback reinforcement learning as described above.

[0020] Compared with the prior art, the present invention has at least the following beneficial effects.

[0021] (1) Existing keyword generation technologies either rely on manually annotated preference data or on reward models that require separate training. Moreover, their domain constraints are usually derived from general corpora, lacking a specific design for scientific literature and failing to have a closed-loop pathway for feeding back real user feedback for training. This invention constructs preference signals through a collaborative approach of "automatic scoring by a large language model - closed-loop calibration by manual feedback - weak supervision by a scientific literature subject classification system".

[0022] (2) Using a large language model as an automatic scorer to replace manually labeled preference data and independently trained reward models. This invention directly uses a powerful large language model as a multi-dimensional automatic evaluator, scoring multiple candidate keyword sets of the same document one by one and outputting normalized sub-scores, which are then aggregated into a scalar reward. Unlike existing reinforcement learning schemes based on human feedback, which require collecting a large number of human preference pairs and then training an independent reward model as a scoring agent, this invention provides the reward signal in real time by a large language model evaluator. This eliminates the cost of manually labeling preference data item by item, as well as the training and maintenance of the reward model, and avoids reward distortion caused by training data distribution shifts. Furthermore, this scoring covers keyword-level quality (relevance). Coverage diversity Simplicity ), category alignment Theme specificity With excessively dispersed punishment By employing multiple orthogonal dimensions and uniform weight aggregation, keyword quality can be constrained simultaneously across multiple dimensions, overcoming the shortcomings of traditional keyword extraction methods (such as TF-IDF, TextRank, KeyBERT, etc.) which completely lack quality control mechanisms and suffer from overly broad or inconsistent terminology when directly calling large language models for generation. (3) Introducing a human feedback loop to achieve online dynamic calibration of the reward function and continuous model evolution. This invention deploys the optimized model as a Web service, collecting explicit corrections (such as adding, deleting, and rewriting keywords) and implicit behavioral cues (such as adoption, copying, and ignoring) from real users through a human-computer interaction module. Based on this, an online preference dataset is constructed, and then... The reward function is dynamically calibrated and then periodically retrained to reinject the generator. Unlike existing solutions that treat manual annotations as a one-time offline training input that remains fixed after model deployment, this invention continuously collects manual feedback within the deployment environment and directly applies it to the reward function, thus forming a closed loop of "generation-use-feedback-re-optimization": First, it corrects the systematic bias between automatic scorers and human expert judgments, ensuring the optimization objective is always anchored to real user preferences; second, it allows the model to continuously adapt to evolving scientific terminology (such as emerging method names and new task names), mitigating the model's timeliness degradation caused by rapid terminology updates in scientific literature scenarios; third, weights... Adjustable, allowing for flexible configuration of the influence ratio between automatic ratings and human preferences; (4) Specialized design for scientific literature: Using the subject classification system as a weak supervision signal, domain-level semantic constraints can be obtained without manual annotation. This invention takes advantage of the fact that scientific literature has its own structured metadata. The retrieval module retrieves the K documents with the most similar semantics from a large-scale scientific corpus and extracts their subject classification tags (such as arXiv topic categories cs.CL, cs.AI, cs.LG, etc.) as weak tags, and then constructs classification alignment based on these tags. Theme specificity With excessively dispersed punishment Three reward components. Unlike existing general text keyword generation technologies that only constrain the content of the document itself or use general corpora unrelated to the subject system as search sources, this invention utilizes an established research classification system accumulated over a long period in the field of scientific literature. This allows the generated keywords to simultaneously satisfy "local alignment" (fitting the specific content of the document) and "global alignment" (fitting the recognized terminology system and knowledge organization structure of the discipline). As a result, the generator can produce conceptual keywords that are not explicitly stated on the surface of the document but are inferred from semantics, significantly improving the semantic coverage and domain alignment of the keywords, and the entire process does not require any additional manual annotation. (5) This invention uses the GRPO algorithm for grouping relative strategy optimization. It calculates the advantage estimate based only on the relative ranking of $G$ candidates within the same document, without the need for an additional value network (critic) or manual adjustment of the weight scheme for each evaluation dimension. This design is naturally compatible with the automatic scoring mechanism of the above-mentioned large language model - since the absolute score scales of automatic scoring on multiple heterogeneous dimensions are not strictly comparable, while the relative superiority and inferiority relationship between candidates in the same group is stable and reliable, the use of intra-group relative comparison can effectively suppress the variance caused by the drift of the scoring scale, and achieve stable convergence while reducing training complexity; (6) This invention outperforms traditional keyword extraction methods and thirteen similarly sized LLM baseline models on multiple large-scale scientific literature datasets (AI-DM, DBLP V10, PubMed) and six standard benchmark datasets, and achieves leading performance in automatic evaluation metrics and human expert evaluation. Among them, the ablation experiment further verifies the necessity of the above key design: the coverage decreased significantly after removing the retrieval module, proving that subject classification evidence is a comprehensive source; the relevance and cohesion were damaged after replacing the large language model evaluator with a weaker open source model, proving that the preference supervision provided by the strong scorer is indispensable. Attached Figure Description

[0023] Figure 1 This is a framework diagram of a literature keyword generation method based on large model feedback reinforcement learning.

[0024] Figure 2This is a flowchart of a document keyword generation method based on large model feedback reinforcement learning.

[0025] Figure 3 Radar chart comparison.

[0026] Figure 4 Human assessment quadrant diagram.

[0027] Figure 5 Inference throughput comparison chart. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0029] This invention, for the first time, combines retrieval enhancement generation with reinforcement learning based on large language model feedback in the task of scientific literature keyword generation, developing a scientific literature keyword generation device based on reinforcement learning and large language model feedback, hereinafter referred to as SciKGen. SciKGen mainly comprises five modules: a retrieval enhancement module, a keyword generation module, a preference evaluation module, a reinforcement learning optimization module, and a human-computer interaction module. Specifically, the retrieval enhancement module retrieves semantically relevant documents and their subject classification information from the large-scale arXiv corpus, providing coarse-grained semantic guidance; the keyword generation module uses lightweight instructions to fine-tune the LLM (Large Language Model), generating a set of candidate keywords for each document in conjunction with the retrieval context; the preference evaluation module uses a powerful LLM evaluator to assess candidate quality from multiple dimensions, generating preference signals; the reinforcement learning optimization module uses the GRPO algorithm to iteratively optimize the generator using preference signals; and the human-computer interaction module deploys the model as a web service, collecting user feedback for continuous improvement. Through the combined effects of RAG guidance, multi-dimensional preference evaluation, and GRPO optimization, SciKGen generates high-quality, semantically accurate scientific keywords.

[0030] like Figure 1 and Figure 2 As shown, the scientific literature keyword generation device based on reinforcement learning and large language model feedback of the present invention specifically includes the following contents.

[0031] (a) Search enhancement module.

[0032] To leverage the rich semantic structure of arXiv without requiring manual annotation, the retrieval enhancement module of this invention retrieves semantically relevant documents from a large-scale unannotated scientific corpus for a given input document.

[0033] Specifically, dense representations are used to calculate semantic similarity before retrieval. Most similar document: in, This represents the set of relevant documents retrieved for the input document x, i.e., the enhanced search context. This indicates that input document x and the j-th document in the unlabeled corpus... The semantic similarity between them is calculated from their dense vector representations (e.g., taking the cosine similarity of their encoded vectors). This indicates the selection operation of sorting the similarity from high to low and taking the top K items; K is the number of documents returned by the retrieval, and in this embodiment of the invention, K=3; This refers to a large-scale unlabeled scientific corpus.

[0034] The retrieval process employs the all-MiniLM-L6-v2 model as the retrieval encoder. Each retrieved document is associated with one or more arXiv topic categories (e.g., cs.CL, cs.AI, cs.LG). These categories serve as weak labels, providing domain-level semantic constraints for the downstream keyword generation task. The generation process is guided by the retrieval context and categories, ensuring that keywords are simultaneously aligned with the specific content of the document and the broader established research classification system.

[0035] (ii) Keyword generation module.

[0036] Keyword generation model The implementation is a lightweight, instruction-based fine-tuning language model to balance expressive power and computational efficiency. The model is initialized from Meta-Llama-3.1-8B-Instruct and fine-tuned using 4-bit LoRA quantization (rank r=32, maximum sequence length 2560 tokens). Given an input document x and its retrieval context... The model generates a set of candidate keywords: in, This represents the set of candidate keywords output by the model. This indicates that the document content x is used in conjunction with the search enhanced context. A keyword generation model for combined inputs. Its trainable parameters Included in the suggestions as evidence to enhance the search.

[0037] To ensure stable and well-formatted output, a fixed system-level instruction template is used to constrain content and format. Specifically, the instruction model prompts: (a) generating concise, domain-relevant scientific keywords, focusing on core themes, methods, tasks, datasets, and research findings; (b) prioritizing the use of multi-word technical phrases over generic words; (c) excluding stop words, vague terms, and overly broad words (such as "research," "method," "data," and "system"); and (d) outputting 8 to 12 keywords by default. Output is limited to keywords containing `...` <answer>The structured pattern of labels facilitates reliable parsing and downstream evaluation. This standardized hint design improves generation consistency across models and supports efficient reward calculation in subsequent preference-based optimization.

[0038] (iii) Preference assessment module.

[0039] To provide fine-grained supervision beyond classification labels, this invention introduces a preference-based evaluation mechanism, employing a powerful LLM (GPT-4.1-nano) as the automatic evaluator. For each generated keyword set... The evaluator assesses its quality from multiple dimensions.

[0040] The preference-based evaluation includes: assessing the semantic correctness of the generated keywords and the input document, the alignment with the subject classification signal obtained by the RAG module, and the consistency with manually annotated keywords (if any); the above evaluation criteria are aggregated into a unified reward function: in, Indicates keywords The scalar reward value, i.e., the preference reward signal; k is the index of the evaluation dimension; This indicates that, given an input document x, the evaluator evaluates the keywords... The given normalized score takes values ​​in the range [0,1]. Let be the weight coefficient of the k-th evaluation dimension, and In practice, uniform weighting is adopted. To avoid manual parameter tuning and ensure balanced contributions across all dimensions. Indicates the number of keywords.

[0041] In a preferred embodiment, the overall reward Defined as the arithmetic mean of the four advanced evaluation components: The components are defined as follows: Keyword-level quality score It consists of the averages of four fine-grained standards: in: Measure the semantic relevance between generated keywords and input documents; Assess topic coverage, i.e., whether the keywords cover the important aspects of the gold standard set; Encourage semantic and lexical diversity, avoid redundancy, and provide different concepts; Assess conciseness and format conformity, i.e., whether the keywords are short and clear phrases.

[0042] Category alignment score , To align scores for classification, capture the semantic consistency between the generated keyword set and the subject classification signal obtained from the retrieval (considering both description and examples).

[0043] Topic specificity score , The topic-specific score assesses whether the generated keywords reflect the document’s main contribution and can confidently indicate a single primary category (the sharper the distribution, the better).

[0044] Excessive Dispersion of Punishment , The overdispersion score measures the degree to which generated keywords are falsely dispersed into irrelevant categories. A higher value indicates a more dispersed distribution of keywords. The reward is converted into a penalty for overly scattered or unconcentrated keyword distribution; the values ​​of the above three scores are all in the range of [0,1].

[0045] Scores for each dimension range from 0 to 1, using uniform weights to avoid manual parameter tuning. The evaluator simultaneously outputs suggested primary categories and up to two secondary categories, providing rich feedback to the generator.

[0046] (iv) Enhanced learning optimization module.

[0047] To optimize the keyword generation model under structured preference signals, this invention employs the Grouped Relative Policy Optimization (GRPO) algorithm. GRPO relies on relative comparisons between multiple candidate outputs generated from the same input document, making it particularly suitable for the setting of this invention, as the reward consists of several heterogeneous evaluation dimensions, and the relative ranking among candidates is more meaningful than the absolute score magnitude.

[0048] In one embodiment, for a given document, the keyword generation model samples K times to produce a set of candidate keywords: Each candidate set is scored by an automated LLM evaluator, which outputs normalized sub-scores that reflect different aspects of keyword quality. Calculate the relative strength of all candidates within the same group: Policy gradient update based on GRPO optimization objective: in, Represents the training dataset. Represents the parameters of a lightweight, large language model. Indicates the use of parameters A lightweight large language model. The maximization step shifts the model's probabilistic quality toward the preferred keyword set, achieving stable optimization through intra-group relative preference learning, without requiring an additional value network or manual adjustment of the weight scheme.

[0049] GRPO training consists of 3 epochs, using the AdamW optimizer, cosine learning rate scheduling, and peak learning rate. The weight decay, warm-up ratio, and gradient clipping were all set to 0.1, and the experiment was performed on dual NVIDIA A100-80GB GPUs with BF16 precision.

[0050] (v) Human-computer interaction module.

[0051] This framework is designed to support online human-computer interaction extensions. The system has been deployed as a web service, and user feedback—including explicit corrections and implicit behavioral cues—is collected to build an online preference dataset, dynamically calibrating the reward function. in, Controlling the influence of human feedback on weights, This represents online preference rewards constructed from online user feedback (explicit corrections and implicit behavioral cues). Through periodic retraining, the model can continuously adapt to evolving scientific terminology and user preferences.

[0052] Furthermore, this invention can also utilize small-scale manually labeled datasets. Supervised training is performed on the deployed keyword model.

[0053] The invention was then validated on multiple real-world datasets.

[0054] 1. Dataset Overview This invention has been tested on a large scale on the following datasets: (1) Standard benchmark datasets (exact match evaluation): Inspec (500 abstracts), SemEval-2017 (100 documents), SemEval-2010 (100 papers), DUC-2001 (308 articles), NUS (211 papers) and Krapivin (2,304 papers).

[0055] (2) Large-scale scientific corpora (no reference assessment): AI-DM (78,020 papers from 13 conferences), DBLP V10 (827,533 papers) and PubMed (142,432 papers).

[0056] (3) Training data: RAG retrieval was performed using the arXiv corpus, and the arXiv-Abstracts-Keywords dataset (290,000+ pairs) provided supervision signals.

[0057] 2. Competition Methods This invention compares SciKGen with the following methods: (A) Traditional keyword extraction methods: statistical method YAKE!; graph methods TextRank, SingleRank, TopicRank, MultipartiteRank; embedding methods KeyBERT, EmbedRank, SIFRank, MDERank; prompt-based method PromptRank; reference-aligned unsupervised generator ERU-KG; zero-shot LLM baseline Vanilla (Meta-Llama-3.1-8B-Instruct); and hybrid methods.

[0058] (B) Thirteen 8B-scale LLM baselines: Apertus-8B, aya-expanse-8b, DeepAnalyze-8B, DeepSeek-R1-Distill-Llama-8B, granite-3.3-8b-instruct, Hermes-3-Llama-3.1-8B, Hypnos-i1-8B, marin-8b-instruct, Meta-Llama-3.1-8B-Instruct, MiniCPM4.1-8B, MiroThinker-v1.0-8B, Qwen3-8B, and UserLM-8b. All baselines used the same hints, and hyperparameters were optimized on the validation set.

[0059] 3. Analysis of Experimental Results (1) Evaluation of standard benchmark datasets.

[0060] Table 1 compares the existing keyword F1 scores of SciKGen with traditional keyword extraction baselines on six standard benchmarks. As shown in Table 1, SciKGen achieves the highest average F1 score at all cutoff points (F1@5=29.30, F1@10=35.57, F1@15=36.41), exceeding the Hybrid baseline by up to 23.1%. Its performance is particularly outstanding on short document corpora that match the training domain (e.g., a 12.74 improvement in F1@5 on Inspec). SciKGen sweeps all six benchmark datasets at F1@15, demonstrating its comprehensive generative capabilities even with a high budget. Table 1 (2) Evaluation of large-scale scientific corpus.

[0061] Table 2 compares the keyword generation performance of SciKGen with traditional keyword extraction baselines on the AI-DM, DBLP, and PubMed datasets. Table 3 compares the keyword generation performance of SciKGen with an 8B-level large language model (LLM) baseline on the AI-DM, DBLP, and PubMed datasets. As shown in Tables 2 and 3, without reference metrics, SciKGen outperforms traditional extractors. It sweeps all five metrics on AI-DM and DBLP, and leads in four metrics on PubMed, maintaining a significant advantage in relevance and cohesion, while remaining competitive in coverage and diversity. Table 2 Table 3 (3) Fine-tuning model compared with LLM of the same scale.

[0062] Compared to thirteen 8B-scale LLMs, SciKGen outperforms its closest competitors by 14.8% in average relevance, 50.3% in cohesion, 2.9% in coverage, and 13.3% in representativeness. The Wilcoxon signed-rank test confirms the significance of relevance, cohesion, and representativeness across all corpora. ), and the significance of coverage on AI-DM / DBLP ( ).

[0063] (4) Human assessment.

[0064] like Figure 4 As shown, 20 computer science PhD students blind-reviewed 1,000 AI-DM abstracts, rating them on a 0-5 scale based on semantic correctness and information content. SciKGen achieved the highest overall score of 4.018, significantly outperforming the strongest baseline Qwen3-8B's 3.994 (Wilcoxon signed-rank test). This study confirms that reinforcement learning-based alignment produces a substantial, expert-perceived improvement over the standard 8B baseline on dense scientific texts.

[0065] (5) Cost-benefit assessment.

[0066] like Figure 3 As shown, SciKGen captures document semantics with fidelity approaching that of GPT-4.1-nano, and even surpasses it in representativeness (0.7531 vs. 0.7363). Through GRPO tuning, SciKGen significantly improves the severe redundancy problem of the base Meta-Llama-3.1-8B, increasing diversity from 0.0671 to 0.4704. Although the human score is slightly lower than GPT-4.1-nano (4.018 vs. 4.537), SciKGen offers a quality-to-cost ratio far superior to the standard 8B baseline, demonstrating practical value in large-scale tasks where commercial models are expensive.

[0067] (6) Ablation experiment.

[0068] Table 4 presents the ablation experiment results. As shown in Table 4, the ablation analysis on the AI-DM dataset indicates that removing the retrieval module (SciKGen w / o RAG) significantly reduces coverage, demonstrating that categorical evidence drives comprehensiveness; replacing the evaluator with the open-source model (SciKGen r / p GPT-OSS) impairs relevance and cohesion, highlighting the necessity of strong preference supervision. The peak score of the complete model confirms the synergistic effect of RAG guidance and preference tuning in producing accurate scientific metadata. Table 4 (7) Runtime performance.

[0069] like Figure 5 As shown, SciKGen achieves an inference throughput of 15.29 terms / second, second only to aya-expanse-8b (16.01 terms / second), and significantly outperforming models such as DeepSeek-R1-8B (3.05 terms / second), Qwen3-8B (3.02 terms / second), and DeepAnalyze-8B (1.05 terms / second). Combined with its superior keyword quality and human scoring, SciKGen offers a highly favorable quality-efficiency tradeoff for practical deployments.

[0070] In summary, this invention provides SciKGen, a scientific literature keyword generation framework based on reinforcement learning and large language model feedback. This framework combines retrieval-enhanced weak supervision with preference-based reinforcement learning. Through RAG classification evidence and GRPO optimization, the output of the lightweight 8B generator is aligned with reward signals reflecting relevance, diversity, and conciseness. On AI-DM, DBLP V10, and PubMed, SciKGen outperforms traditional extraction methods and thirteen 8B-scale LLM baselines, achieving the highest expert scores while maintaining high inference throughput. These results demonstrate the effectiveness of combining weak supervision with preference tuning for scalable, high-quality scientific indexing. This framework has broad application potential and can be applied to the automated analysis of large-scale scientific literature across multiple research fields, providing an efficient and reliable solution for managing the ever-growing volume of scientific literature.

[0071] Although specific embodiments of the invention have been disclosed for illustrative purposes to aid in understanding and implementing the invention, those skilled in the art will understand that various substitutions, variations, and modifications are possible without departing from the spirit and scope of the invention and the appended claims. Therefore, the invention should not be limited to the content disclosed in the preferred embodiments, and the scope of protection claimed by the invention is defined by the claims.< / answer>

Claims

1. A method for generating literature keywords based on large-model feedback reinforcement learning, characterized in that, The method includes: Retrieve semantically relevant documents of the input document from an unlabeled scientific corpus; wherein each document in the unlabeled scientific corpus Associated with one or more subject category tags, and the document No keywords were marked; The input document and semantically related documents are fed into a lightweight large language model to generate candidate keywords, resulting in a set of candidate keywords. A large-scale language model is used to evaluate the quality of the candidate keyword set and obtain the preference reward signal for each candidate keyword. Based on the preference reward signal, the lightweight large language model is fine-tuned using a reinforcement learning algorithm to obtain a keyword generation model; Deploy a keyword generation model and combine it with an unlabeled scientific corpus to obtain keyword generation results for the target document.

2. The method according to claim 1, characterized in that, Retrieve semantically relevant documents of the input document from an unlabeled scientific corpus, including: Calculate the input document and the document separately. The dense vector representation is used to obtain the input document and the document based on the dense vector representation. The similarity between them; Select the K documents with the highest similarity Semantic-related documents that serve as input documents.

3. The method according to claim 1, characterized in that, The preference reward signal is based on keyword-level quality scores. Category alignment score Topic specificity score and excessively dispersed penalty scores Generate; where, Represents the first in the candidate keyword set 10 candidate keywords.

4. The method according to claim 3, characterized in that, Keyword-level quality score The classification alignment score ; The topic specificity score The excessively dispersed penalty score ;in, Indicates candidate keywords Semantic relevance to the input document Indicates candidate keywords Thematic coverage of the input document, Indicates candidate keywords semantic and lexical diversity Indicates candidate keywords Simplicity Indicates candidate keywords Semantic consistency with subject classification labels; Indicates candidate keywords The ability to reflect the core contributions of input documents and indicate a single topic category; Indicates candidate keywords The false dispersion extends to the level of unrelated subject classification labels.

5. The method according to claim 1, characterized in that, Based on the preference reward signal, the lightweight large language model is fine-tuned using a reinforcement learning algorithm to obtain a keyword generation model, including: Calculate the relative advantage of each candidate keyword in the candidate keyword set based on preference reward signals. Calculate the policy gradient based on relative advantage; The model parameters of the lightweight large language model are updated using the policy gradient.

6. The method according to any one of claims 1 to 5, characterized in that, After deploying the keyword generation model, the method further includes: We constructed a manually labeled dataset based on user feedback and calculated the online preference reward for each user feedback. For each data point in the manually labeled dataset, calculate the preference reward information for that data. The preference reward signal of this data is weighted and calculated with the corresponding online preference reward to generate the final reward; Based on the final reward, the keyword generation model is fine-tuned using a reinforcement learning algorithm.

7. A document keyword generation device based on large model feedback reinforcement learning, characterized in that, The device includes: A retrieval enhancement module is used to retrieve semantically relevant documents from an unlabeled scientific corpus containing an input document; wherein each document in the unlabeled scientific corpus... Associated with one or more subject category tags, and the document No keywords were marked; The keyword generation module is used to feed the input document and semantically related documents into a lightweight large language model to generate candidate keywords and obtain a set of candidate keywords. The preference evaluation module is used to evaluate the quality of the candidate keyword set using a large-scale language model and obtain the preference reward signal for each candidate keyword. The reinforcement learning optimization module is used to fine-tune the lightweight large language model based on the preference reward signal using a reinforcement learning algorithm to obtain a keyword generation model. The human-computer interaction module is used to deploy the keyword generation model and, in conjunction with an unlabeled scientific corpus, obtain the keyword generation results for the target document.

8. A computer device, characterized in that, The computer device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the document keyword generation method based on large model feedback reinforcement learning as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the literature keyword generation method based on large model feedback reinforcement learning as described in any one of claims 1-6.

10. A computer program product, characterized in that, When the computer program product is run on a computer device, the computer device performs the document keyword generation method based on large model feedback reinforcement learning as described in any one of claims 1-6.