A method for constructing a library of original text citations based on improved reinforcement learning

Abstract

The application provides a method for constructing a reference library based on improved reinforcement learning, comprising: constructing a data set, combining an online learning gradient update strategy to optimize an online learning large model, and obtaining an optimized online learning large model; constructing a reference library based on a multi-layer index architecture according to an output result of the optimized LLM large model; inputting a title, a content and a corresponding candidate key expression sentence into an audit system and performing manual audit for the output result of the optimized LLM large model, that is, a plurality of candidate key expression sentences; inputting an article to be checked into the optimized online learning large model, outputting all candidate key expression sentences, and constructing a candidate expression sentence list; obtaining a structured phrase set and a sentence structure regular expression through a text analysis module for the candidate expression sentence list, and extracting phrase features by using the structured phrase set; and searching in the reference library based on the multi-layer index architecture, and outputting a reference position and a similarity score.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method for constructing a source citation library based on improved reinforcement learning. Background Technology

[0002] News media often cite official standard expressions when writing articles, and the cited content must be completely consistent with the original text, including punctuation. Before publication, articles need to be reviewed by the system to ensure there are no errors in the citations. At present, the citation library is built based on rules, fixed algorithms for extraction, and manual review. This has the following problems: weak ability to adapt to new sentence structures and low extraction efficiency; relying on manual annotation or simple rule extraction makes it difficult to cover complex semantic scenarios, resulting in redundancy or omissions in the candidate set. Summary of the Invention

[0003] Technical objective: To address the shortcomings of existing technologies, this application proposes a method for constructing a source citation library based on improved reinforcement learning. This method not only overcomes the deficiencies of traditional models in extracting new sentence structures, but also continuously improves the accuracy of key sentence recognition and corpus coverage through an online learning mechanism, effectively reducing the cost of manual review.

[0004] Technical solution: To achieve the above technical objectives, the present invention adopts the following technical solution.

[0005] A method for constructing a source citation library based on improved reinforcement learning, the method includes:

[0006] Step 1: Construct a dataset and optimize the large online learning model using an online learning gradient update strategy to obtain the optimized large online learning model.

[0007] Step 2: Construct a reference library based on a multi-level index architecture based on the optimized LLM large model output results; For the optimized LLM large model output results, i.e., several candidate key statements, input the title, body content and their corresponding candidate key statements into the review system; Manually review the body and candidate key statements, and store the review conclusions in the human review information table, where the review conclusions include whether the candidate key statements are correct, incorrect or omitted.

[0008] Step 3: Input the article to be verified into the optimized online learning model, output all candidate key statements, and construct a list of candidate statements;

[0009] Step 4: Obtain a structured phrase set and sentence structure regular expression from the candidate expression list through the text analysis module, and extract phrase features using the structured phrase set;

[0010] Step 5: Search the reference library based on the multi-level index architecture and output the reference location and similarity score.

[0011] Beneficial effects:

[0012] 1. The present invention provides a method for constructing a source citation library based on improved reinforcement learning, which not only overcomes the shortcomings of traditional models in extracting new sentence patterns, but also continuously improves the recognition accuracy and corpus coverage of key expressions through an online learning mechanism, effectively reducing the cost of manual review.

[0013] 2. This invention constructs a three-level collaborative indexing system consisting of semantic indexing (based on deep vector similarity calculation), syntactic indexing (relying on syntactic tree structure pattern matching), and phrase indexing (using phrase vector embedding matching). It achieves the following through a multi-granularity feature fusion mechanism: 1) semantic space mapping; 2) syntactic structure alignment; and 3) phrase-level semantic approximation. This architecture, leveraging the dual advantages of heterogeneous feature collaborative computation and hierarchical retrieval optimization, constructs a progressive retrieval channel from semantic understanding to structural matching, thereby synergistically improving the recall and precision of the search system.

[0014] 3. This invention proposes an improved reinforcement learning strategy. By introducing a sliding window exponential decay mean model to construct a dynamic baseline mechanism, it reduces the dependence of the value function while avoiding the problem of repeated sampling of the same input. It innovatively uses a dynamic memory bank to replace the traditional reference model. By designing a penalty term based on the distribution of correctly labeled sentences, it constructs an optimization objective function with boundary constraints: it both guides the model to quickly converge to the target distribution region using labeled data and prevents the model from deviating excessively from the rules of natural language expression through distribution similarity constraints. This dual constraint mechanism improves model training efficiency while ensuring the correct direction of the optimization trajectory in the semantic space. Attached Figure Description

[0015] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0016] Figure 2 This is a flowchart illustrating the construction of a reference library based on improved reinforcement learning, as described in an embodiment of the present invention.

[0017] Figure 3 This is a flowchart illustrating the construction of the reference library index structure according to an embodiment of the present invention;

[0018] Figure 4 This is a flowchart illustrating the online learning and training process of a large model according to an embodiment of the present invention. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0020] Example

[0021] As attached Figure 1 As shown in this embodiment, a method for constructing a source citation library based on improved reinforcement learning includes the following steps:

[0022] Step 1, as shown in the attached document Figure 4 As shown, a dataset is constructed, and an online learning gradient update strategy is used to update the large online learning model.

[0023] The model is optimized to obtain an optimized large-scale online learning model; specifically, the following steps are included:

[0024] Step 11: Preprocess several original article contents (referred to as the original text) to output the title and body text, which will serve as the source material.

[0025] Dataset;

[0026] The preprocessing process mainly includes removing redundant blank lines, images, and image titles from the original text, extracting the original text title, and separating the title from the main text content.

[0027] Step 12: Obtain new sentence structures for key expressions from the quotation pattern table. The quotation pattern table is a pre-constructed table.

[0028] This is a MySQL table, and the table data is continuously updated during the training process of this invention;

[0029] Step 13: Concatenate the title, body content, key statement sentences, and prompt words to form the text input for the online learning model. The output of the online learning model is the candidate key statement sentences. Here, the online learning model is abbreviated as LLM model, and the prompt words are expert-initialized prompt words validated by thousands of data points.

[0030] The output format and content requirements for large LLM models are specified.

[0031] Step 14: Optimize the online learning large model using an online learning gradient update strategy. The online learning large model extracts all candidate key statements in the main text.

[0032] Step 14 includes the following steps:

[0033] Step 141: Enter the title, prompt word template, key expression sentence format, and the i-th original text. i The LLM large model outputs candidate key statements from the original text in O_Text format. i1 O_Text i2 ,...,O_Text ij ,...,O_Text in Where 1 ≤ j ≤ n, and n is the original text. i The number of key statements output by the large online learning model.

[0034] Step 142: Calculate the key statement O_Text i1 O_Text i2 ,...,O_Text ij ,...,O_Text in The reward value is obtained through a reward function, the formula of which is defined as follows:

[0035]

[0036] I(s)=α*Position(s)+β*TF-IDF(s)+γ*cos(embedding(s), embedding(text))

[0037] Where reward is the reward value, C(s) is the sentence completeness, I(s) is the importance of the extracted key statement in the original text, s represents the s-th key statement, the number of core components is the number of core syntactic components actually present in the s-th key statement (e.g., the number of subject, verb, and object), the number of required components is the total number of required core components in the s-th key statement (obtained using dependency parsing), λ is a hyperparameter, embedding(s) is the Bert embedding vector of the s-th key statement, embedding(context) is the Bert embedding vector of the context text, cos() is the cosine similarity, α is a hyperparameter, Position(s) is the position weight of sentence s, β is a hyperparameter, and TF-IDF(s) is the TF-IDF value of sentence s. TF-IDF refers to Term Frequency–Inverse. DocumentFrequency is a commonly used weighted statistical algorithm for information retrieval and text mining. γ is a hyperparameter, and embedding(text) is the BERT embedding vector of the context text. α+β+γ=1. In this embodiment, α=0.3, β=0.4, γ=0.3, and λ=0.4. This reward function considers both the completeness of the sentence and its importance in the original text, avoiding situations where C(s) or I(s) are high alone, thus optimizing the output of the large model towards the goal of extracting key expressions.

[0038] Step 143: Calculate the baseline value of the LLM large model at the current time. The present invention designs a baseline that is adaptively adjusted as the LLM large model is updated and optimized, so that it more accurately reflects the current level of the model, thereby improving the efficiency and stability of the Advantage calculation.

[0039] The dynamic baseline of the large LLM model is calculated using the exponentially decaying mean of a sliding window, retaining only the O_Text corresponding to the N most recent prompt words. ij The reward value is calculated, and a dynamic mean is derived through exponential decay weighting to balance short-term fluctuations and long-term trends. The formula for calculating the dynamic baseline, which is adaptively adjusted as the LLM model is updated and optimized, is as follows:

[0040]

[0041] Among them, B t B is the dynamic mean reward for extracting key statements in the t-th iteration. t-1 Let N be the dynamic mean reward for extracting key statements in the (t-1)th iteration, N be the sliding window size, and ξ be the decay coefficient.

[0042] r tLet O_Text be the total reward value for all key statements output by the LLM large model in the t-th iteration, and let O_Text be the total reward value for all output key statements. ti The corresponding reward value r ti The mean, i.e. M represents the number of key statements extracted in the t-th iteration, and O_Text ti For the key statement output in the t-th iteration, O ti For the t-th iteration (prompt, Text) t The text outputs the i-th key statement;

[0043] Step 144: Calculate the prompt word template for the input text and the original text. i The corresponding advantage of its output is calculated using the following formula:

[0044] A t =r t -B t +α*n_sytax(O ti )

[0045] Among them, A t To provide prompt word templates based on the input text, and the original text... i The advantage is calculated from the extracted key statements, and n_sytax is the reward value for new sentence structures appearing in the output of the key statements. If O ti A new sentence structure was discovered and r t If the expression is positive, then n_sytax is 1; otherwise, it is 0. The rule for discovering new sentence patterns is to check if a corresponding dependency parsing sentence pattern exists in the correct expression sentence pattern library.

[0046] Step 145: Calculate the prompt word template and original text of the text pair. i and its output O_Text t The penalty is calculated using the following formula:

[0047] penalty = σ * (1 - cos_sim)

[0048]

[0049] Where penalty is the penalty term, σ is the penalty coefficient hyperparameter, and cos_sim is the similarity used to measure the current semantic features. K historical features in the dynamic memory Similarity, For semantic model f θ Extracting model output O_Textt semantic features, Semantic features of manually annotated sentences recorded in the dynamic memory are extracted for the semantic model; the semantic model uses the BERT model as the backbone and is trained using human-reviewed text pairs for similarity training.

[0050] This penalty term guides the model to optimize towards the distribution of correctly labeled statements, but at the same time, it cannot deviate too much from the distribution of correctly labeled statements. This improves the optimization speed of the model while ensuring the correct optimization direction.

[0051] The dynamic memory stores correctly labeled sentences and key sentences that were missed by manual analysis, processed by a semantic model f. θ Extracted semantic features. The dynamic memory update principle is based on the O_Text of the statement to be added to the memory. i1 The similarity between the feature and the semantic features in the dynamic memory is calculated. If the similarity is less than 0.7, the feature is added to the dynamic memory to ensure the diversity of the dynamic memory.

[0052] Step 146: Calculate the gradient update loss value of the large LLM model. The calculation formula is as follows:

[0053]

[0054] Where B is the number of batches. For the expected distribution, logπ θ (O i ,t) represents the output of logits during large model inference, and logπ old (O i ,t) represents the logits output corresponding to the old large model strategy, A i,t The advantage is the benefit in step 144, and the penalty is the penalty in step 145.

[0055] Step 147: Use the loss value obtained in step 146 to perform backpropagation, update the parameters of the online learning large model online until the loss value is within the preset threshold, and obtain the optimized LLM large model. The optimized LLM large model is used to extract all candidate key expression statements in the text content based on the text input content.

[0056] Step 2, as attached Figure 2 and attached Figure 3As shown, a reference library based on a multi-layered index architecture is constructed based on the optimized LLM large model output. For the optimized LLM large model output, i.e., several candidate key statements, the title, body text, and their corresponding candidate key statements are input into the review system. The body text and candidate key statements are manually reviewed, and the review conclusions are stored in the human review information table. The review conclusions include whether the candidate key statements are correct, incorrect, or omitted.

[0057] The process of building a reference library based on a multi-level indexing architecture includes:

[0058] Step 21: Perform NLP (Natural Language Processing) semantic analysis on the input correct key statements and missing statements to remove pronouns and non-statement parts, and obtain the standard statements.

[0059] Step 22: Input the built-in phrase library. This library collects domain-specific phrases, weight values ​​(dict_conf) calculated using TF-IDF, and the result indicating whether automatic decay is enabled. Automatic decay is an attribute of the words collected in the built-in phrase library, indicating whether the word weights automatically decay. TF-IDF, or Term Frequency-Inverse Document Frequency, is a statistical method used to evaluate the importance of a word in a document collection. The weight value of each word in a document is obtained by calculating the product of the term frequency (TF) and the inverse document frequency (IDF). These weight values ​​can be stored in a dictionary, such as dict_conf.

[0060] Step 23: Input the standard expression sentence and the built-in phrase library into the text analysis module. The text analysis module extracts structured phrases from the standard expression sentence based on dependency parsing (DP, which reveals the syntactic structure by analyzing the dependency relationships between components within a language unit) and part-of-speech tagging. The extracted structured phrases and the built-in phrase library are fused using a confidence-based adaptive weighting method. The confidence-based adaptive weighting method, i.e., the dynamic weight calculation formula, is as follows:

[0061]

[0062] DictScore = dict_conf * (1 + λ * log(len(built-in phrases)))

[0063] ModelScore = 0.6 * (Subject-Verb-Object Completeness) + 0.4 * (1 - Percentage of Abnormal Modifiers)

[0064] Wherein, FinalScore is the dynamic weight, representing the final weight value of the fused phrase. For dynamic weighting coefficients, `dict_conf` is the weight value calculated by TF-DIF for this structured phrase, and `k` is a hyperparameter that can be customized according to the actual situation, with a suggested value range of 2-5; `DictScore` is the confidence score of the dictionary phrase, `λ` is the word length gain coefficient, with a suggested range of 0.05-0.1, and `len` (built-in phrase) is the character length of the built-in phrase; `ModelScore` is the semantic extraction confidence score, `η` is a hyperparameter, with a default value of 0.01, and `e`... (-ηt) This indicates the daily automatic decay of inactive phrase weights, corresponding to whether automatic decay is enabled in step S22; Subject-verb-object completeness is the score of the completeness of the main components of the subject-verb-object pair (1.0 for complete subject-verb-object, 0.5 for missing one component, and 0.3 for other cases); Abnormal modifier percentage is the number of unreasonable modifiers obtained from part-of-speech analysis / the total number of modifiers.

[0065] The phrase fusion rules are as follows:

[0066] (1) When DictScore±e (-ηt) If the similarity is ≥0.9 and com_sim (built-in phrase, phrase extracted by dependency parsing) >0.6, the built-in phrase is used; where com_sim is the cosine similarity.

[0067] (2) When 0.6 ≤ DictScore ± e (-ηt) When the value is less than 0.9 and com_sim (built-in phrases, dependency parsing extracted phrases) is less than or equal to 0.6, both built-in phrases and dependency parsing extracted phrases are used.

[0068] (3) When DictScore±e (-ηt) When the value is less than 0.6 or in other cases, use dependency parsing to extract phrases;

[0069] This method not only solves the problem of domain-specific vocabulary segmentation in traditional structured phrase extraction, but also addresses the timeliness issue of domain-specific vocabulary by incorporating automatically decaying weights for inactive phrases. The dynamic weight FinalScore of the fused phrase is calculated as the final weight value for the fused phrase, which is used for phrase extraction and ranking.

[0070] Step 24: Construct a multi-level index structure based on semantic index, syntactic index, and phrase index to support efficient multi-dimensional retrieval. The semantic index is calculated based on deep vector similarity, the syntactic index is obtained through pattern matching based on syntactic tree structure, and the phrase index is obtained through phrase vector embedding matching. The specific steps are as follows:

[0071] Step 241: Store the preprocessed correct key statements as standard statements in the reference statement information table, and obtain the unique index value of the reference statement, such as the id value. Preprocessing includes removing pronouns and non-statement parts.

[0072] Step 242: Perform dependency semantic analysis on the standard expression sentences to obtain dependency relations and stems. Based on the dependency relations and stems, construct regular expression expressions for sentence structures, store them in the reference sentence structure table, and build a trie-tree search tree. Specifically, the regular expression expressions for sentence structures based on dependency relations and stems are generated using dependency parsing, and regular expressions are constructed using dependency relations and stems. The reference sentence structure table includes the regular expression for sentence structure, the ID of the standard expression sentence, and a flag indicating whether it is a new sentence structure. The flag indicating whether it is a new sentence structure is updated periodically based on the correct key expression sentences extracted by the online learning model to ensure the timeliness of new sentence structures. If an expression sentence is a sentence structure missed by the reviewer, it is considered a new sentence structure. If the subsequent model correctly identifies the correct expression sentence corresponding to this sentence structure, the new sentence structure flag is removed.

[0073] Step 243: Extract phrase embeddings, i.e., phrase features, from the structured phrases obtained in Step 23 using the bge-m3 model. Store the structured phrases, phrase embeddings, and a list of unique index values ​​for all quoted expressions containing the phrases in a vector database. This vector database is used to implement phrase indexing in a multi-level index.

[0074] The multi-level index structure (semantic, syntactic, and phrase indexes) constructed using the above methods significantly improves retrieval efficiency, recall, and precision through multi-dimensional collaboration. In terms of comprehensiveness, BGE-M3 is used for semantic similarity comparison to achieve semantic indexing. The semantic index leverages the vector similarity matching of the BGE-M3 model to overcome literal limitations, capturing synonyms and long-tail content. The syntactic index generalizes sentence variations based on Trie tree regular expressions generated by dependency analysis, and dynamically updated new sentence markers ensure language timeliness. The phrase index anchors core keywords for precise supplementation. In terms of accuracy, a three-level progressive mechanism—initial screening using phrase indexes, filtering using syntactic tree structures, and semantic vector ranking based on semantic index similarity comparisons—combined with dependency constraints to eliminate grammatical ambiguity, while BGE-M3's context-aware capabilities effectively distinguish polysemous words.

[0075] Step 3: Input the article to be verified into the optimized online learning model, output all candidate key statements, and construct a list of candidate statements;

[0076] Step 4: Obtain a structured phrase set and sentence structure regular expression from the candidate expression list through the text analysis module, and extract phrase features using the structured phrase set;

[0077] Step 5: Search the reference database based on the multi-level index architecture and output the reference location and similarity score; including the following steps:

[0078] In the reference library based on the multi-layer index architecture, the highly relevant candidate set is first quickly recalled by the phrase feature index based on the structured phrase vector embedding matching algorithm, that is, the candidate key expression sentences that are highly relevant to the candidate sentences collected in the original library.

[0079] Subsequently, based on the sentence structure regular expression, the dependency parsing Trie tree regular expression engine of the syntactic index is used to perform sentence structure generalization matching to cover variant expressions, and the expression sentences are obtained, which are the candidate expression sentences in the article to be reviewed.

[0080] Next, the BGE-M3 model was used to calculate the multi-dimensional semantic similarity of the dual-indexed recall results, and the results were filtered and verified by a cosine similarity threshold > 0.85.

[0081] Finally, the unique index value of the original text is associated with the matching sentence, and the citation position and similarity score are output.

[0082] The present invention also discloses an electronic device, the device comprising: a memory for storing a computer program; and a processor for executing the computer program to cause the device to perform the aforementioned original text citation method based on an improved citation library.

[0083] The present invention also provides a computer storage medium on which a computer program is stored, wherein when the computer program is run, a device running the computer program implements the aforementioned original text citation method based on an improved citation library.

[0084] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that all or part of the steps in the methods of the above embodiments can be implemented by means of software plus a general-purpose hardware platform. Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This computer software product can be stored in a storage medium. The memory can be various types of memory, such as random access memory, read-only memory, flash memory, etc., such as read-only memory (ROM) / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which can be a personal computer, server, or network communication device such as a router) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0085] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for constructing a source text citation library based on improved reinforcement learning, characterized in that, The methods include: Step 1: Construct a dataset and optimize the online learning large model by combining it with the online learning gradient update strategy to obtain the optimized online learning large model, which is abbreviated as LLM large model; Step 2: Construct a reference library based on a multi-level index architecture based on the optimized LLM large model output results; For the optimized LLM large model output results, i.e., several candidate key statements, input the title, body content and their corresponding candidate key statements into the review system; Manually review the body and candidate key statements, and store the review conclusions in the human review information table, where the review conclusions include whether the candidate key statements are correct, incorrect or omitted. Step 3: Input the article to be verified into the optimized online learning model, output all candidate key statements, and construct a list of candidate statements; Step 4: Obtain a structured phrase set and sentence structure regular expression from the candidate expression list through the text analysis module, and extract phrase features using the structured phrase set; Step 5: Search the reference library based on the multi-level indexing architecture and output the reference location and similarity score; Step 1 includes the following steps: Step 11: Preprocess several original articles (hereinafter referred to as "original texts") and output the titles and body text as a dataset. Step 12: Obtain new sentence structures for key expressions from the table of quoted sentence structures; Step 13: Concatenate the title, body content, key expression sentence structure, and prompt words to form the text input content of the online learning big model. The output of the online learning big model is the candidate key expression sentences. Step 14: Optimize the online learning large model using an online learning gradient update strategy. The online learning large model extracts all candidate key expressions in the main text. The process of building a reference library based on a multi-level indexing architecture includes: Step 21: Perform NLP semantic analysis on the input correct key statements and missing statements to remove pronouns and non-statement parts, and obtain the standard statements; Step 22: Input the built-in phrase library, which collects domain phrase vocabulary, weight values ​​dict_conf calculated by tf-idf, and results indicating whether automatic decay is enabled; Step 23: Input the standard expression sentence and the built-in phrase library into the text analysis module. The text analysis module extracts the structured phrases from the standard expression sentence based on dependency parsing and part-of-speech tagging. The extracted structured phrases and the built-in phrase library are fused using an adaptive weighting method based on confidence. Step 24: Construct a multi-level index structure based on semantic index, syntactic index, and phrase index to support efficient multi-dimensional retrieval; among which, the semantic index is calculated based on deep vector similarity, the syntactic index is obtained by pattern matching based on syntactic tree structure, and the phrase index is obtained by phrase vector embedding matching; Step 24 includes the following steps: Step 241: Store the preprocessed correct key statements as standard statements in the reference statement information table and obtain the unique index value of the reference statement. Step 242: Perform dependency semantic analysis on the standard expression sentences to obtain dependency relations and stems. Based on the dependency relations and stems, form regular expression expressions of sentence structure, store them in the reference sentence expression table, and construct a trie-tree search tree. Step 243: Extract phrase embeddings, i.e. phrase features, from the structured phrases obtained in Step 23 using the bge-m3 model; store the structured phrases, phrase embeddings, and a list of unique index values ​​for all quoted expressions in which the phrases appear in a vector database; this vector database is used to implement phrase indexing in multi-level indexing.

2. The method for constructing a source citation library based on improved reinforcement learning according to claim 1, characterized in that... Step 14 includes the following steps: Step 141: Enter title, prompt word template, key expression sentence structure, and the i-th original text. The LLM large model outputs candidate key statements from the original text. ;in, n is the original text The number of key statements output by the large online learning model; Step 142: Calculate key statements The reward value is obtained through a reward function; Step 143: Calculate the baseline value of the current LLM large model; Step 144: Calculate the text pair between the input prompt word template and the original text. and the corresponding advantage of its output; Step 145: Calculate the text pair between the input prompt word template and the original text. and its output Penalties; Step 146: Calculate the gradient update loss value of the large LLM model; Step 147: Use the loss value obtained in step 146 to perform backpropagation, update the parameters of the online learning large model online until the loss value is within the preset threshold, and obtain the optimized LLM large model. The optimized LLM large model is used to extract all candidate key expression statements in the text content based on the text input content.

3. The method for constructing a source citation library based on improved reinforcement learning according to claim 2, characterized in that: The formula for the reward function includes: ， ， ， in, As a reward value, For sentence completeness, To extract the importance of key statements in the original text, s represents the s-th key statement extracted. Let be the number of core syntactic components actually present in the s-th key statement. The total number of core components required in the s-th key statement is given by the number of core components and the number of required components, which are obtained using dependency parsing. For hyperparameters, Let be the BERT embedding vector of the s-th key statement. For the BERT embedding vector of the context text, For cosine similarity, For hyperparameters, The position weight of sentence s, For hyperparameters, Let s be the TF-IDF value of sentence s. For hyperparameters, The Bert embedding vector for the context text.

4. The method for constructing a source citation library based on improved reinforcement learning according to claim 3, characterized in that: The dynamic baseline calculation formula includes: ， in, The reward is the dynamic mean value when extracting key statements in the t-th iteration. The reward is the dynamic mean when extracting key statements in the (t-1)th iteration, where N is the sliding window size. The attenuation coefficient is, and , Let be the total reward value for all key statements output by the LLM large model in the t-th iteration, and let be the reward value for each key statement output. The mean, i.e. M is the number of key statements extracted in the t-th iteration. For the t-th iteration (prompt, The text represents the i-th key statement in the output.

5. The method for constructing a source citation library based on improved reinforcement learning according to claim 1, characterized in that: The confidence-based adaptive weighting method, i.e., the dynamic weight calculation formula, includes: ， in, The dynamic weight represents the final weight value of the fused phrase. For dynamic weighting coefficients, The weight value is calculated for this structured phrase using tf-dif, and k is a hyperparameter. Confidence of dictionary phrases This is the word length gain coefficient. The length of the built-in phrase character set; To extract confidence scores for semantics, For hyperparameters, This indicates that the weight of inactive phrases will automatically decay daily. Subject-verb-object completeness score This is the number of unreasonable modifiers obtained from part-of-speech analysis / the total number of modifiers.

6. The method for constructing a source citation library based on improved reinforcement learning according to claim 1, characterized in that: Step 5 includes: In the reference library based on the multi-layer index architecture, firstly, the highly relevant candidate set is quickly recalled by the phrase feature index based on the structured phrase vector embedding matching algorithm, that is, the candidate key expression sentences that are highly relevant to the candidate sentences collected in the original library; Subsequently, based on the sentence structure regular expression, the dependency parsing Trie tree regular expression engine of the syntactic index is used to perform sentence structure generalization matching to cover variant expressions, and the expression sentences are obtained, which are the candidate expression sentences in the article to be reviewed. Next, the BGE-M3 model was used to calculate the multi-dimensional semantic similarity of the dual-indexed recall results, and the results were filtered and verified by a cosine similarity threshold > 0.

85. Finally, the unique index value of the original text is associated with the matching sentence, and the citation position and similarity score are output.

Images