A library document abstract generation method based on a deep learning model

Abstract

This invention belongs to the field of information retrieval technology, specifically a library document abstract generation method based on a deep learning model. The method includes data preprocessing, model building, semantic understanding and core idea extraction, abstract generation and optimization, and quality assessment. This deep learning-based library document abstract generation method combines rule-based and deep learning technologies through hybrid annotation techniques, fully leveraging the advantages of both. Rules provide the basic framework, while deep learning compensates for the shortcomings of rules, enabling a more accurate understanding of the meaning of domain terminology. A domain entity knowledge base, through rigorous extraction and disambiguation methods, ensures accurate identification and understanding of relationships among entities in the document, laying a semantic foundation for high-quality abstract generation. Furthermore, by accurately understanding domain terminology and effectively capturing semantic relationships in long texts, the accuracy, completeness, and readability of the generated abstracts are improved, thus better meeting users' needs for quickly obtaining key information from documents.

Description

Technical Field

[0001] This invention belongs to the field of information retrieval technology, specifically a method for generating library document summaries based on a deep learning model. Background Technology

[0002] Among the vast amount of library literature, users often spend a lot of time and effort manually searching for documents of interest. In addition, according to surveys, academic search engines have a 3 to 5 times higher click-through rate for abstracts than for full texts, meaning that most users tend to read the abstract first before deciding whether to read the full text in detail. With more than 3 million scientific papers published globally each year, traditional manual abstracting is time-consuming and labor-intensive, thus necessitating the use of library document abstract generation methods.

[0003] Traditional abstract generation methods rely on statistical features, making it difficult to understand domain terminology and semantic relationships within long texts. Long texts contain complex semantic connections, such as causal and progressive relationships. As a result, traditional methods struggle to capture these long-distance semantic connections, leading to fragmented and disorganized abstracts that fail to accurately convey the main idea of ​​the document. Consequently, the accuracy and completeness of the abstracts are reduced, and readability suffers.

[0004] Therefore, this invention provides a method for generating library document summaries based on a deep learning model. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, at least one technical problem raised in the background art is solved.

[0006] The technical solution adopted by this invention to solve its technical problem is: a library document abstract generation method based on a deep learning model, comprising:

[0007] Data preprocessing: noise removal and structured sentence segmentation of the document text;

[0008] A hybrid annotation technique based on rules and deep learning is used to add semantic tags to the text, including topic categories, sentiment tendencies, and academic value classifications.

[0009] Model building: Select a model as the basic architecture based on the characteristics of the data and task, and conduct domain pre-training on academic literature datasets in the field of library science, including journal articles, dissertations and conference literature in the fields of library and information science, bibliometrics and knowledge organization, with a training round of ≥10 rounds;

[0010] We introduce dynamic context-aware technology, calculate long-distance dependencies in text using a Transformer encoder, and dynamically adjust the attention weight distribution by combining the document chapter titles.

[0011] Semantic understanding and core idea extraction: Identify agent, patient and action relationships in sentences through semantic role labeling, and perform entity disambiguation by combining domain entity knowledge base;

[0012] use and The hybrid model is used to extract and filter core keywords;

[0013] Abstract generation and optimization: An initial abstract is generated using a multi-scale attention mechanism, where the local attention layer has a window size of 5 sentences and the global attention layer covers the entire text;

[0014] The summary length is adjusted through parameterized length constraints and reinforcement learning;

[0015] Quality assessment: Design a hybrid reward function and iteratively optimize the model through reinforcement learning.

[0016] Furthermore, the noise includes special symbols, HTML tags, and non-text content. In step S1, when the document length exceeds the threshold of 10,000 words, it is divided into segments of 2,000 words each, and a hierarchical abstracting strategy is adopted to generate sub-abstracts before integration.

[0017] Furthermore, in the hybrid annotation technology based on rules and deep learning, the rule part includes a pre-set list of topic category keywords, a sentiment lexicon, and an academic value grading standard. The deep learning part uses a pre-trained language model to perform semantic understanding and auxiliary annotation of the text. When the rule annotation and the deep learning annotation results are inconsistent, the deep learning annotation result shall prevail. At the same time, it supports manual intervention and correction for special cases.

[0018] Furthermore, during domain pre-training, the model employs a dynamic learning rate adjustment strategy, with the initial learning rate set to 10. −5 Based on the changes in the loss function during training, when the loss function decreases by less than 0.01 for five consecutive rounds, the learning rate is reduced to 0.8 times the original value to accelerate model convergence.

[0019] The model constructs an academic literature dataset in the field of library science, including ≥10,000 journal articles, dissertations, and conference papers in the areas of library and information science, bibliometrics, and knowledge organization.

[0020] Furthermore, the dynamic context-aware technology includes:

[0021] The long-distance dependencies between words are calculated using a self-attention mechanism to analyze the importance of contextual information to the current word. The calculation formula for the self-attention mechanism is as follows:

[0022] ;

[0023] in, This is the query vector, representing the information that needs to be focused on at the moment, used to search for relevant information among all key-value pairs; This is the key vector, used in conjunction with the query vector. Perform matching calculations to determine And the correlation between each key-value pair; It is a value vector that contains the actual information content; Key vector The dimension;

[0024] The method for generating the title vector in the dynamic context-aware technology includes:

[0025] Extract the title semantic vector using an encoder that shares parameters with the main model;

[0026] The similarity between the title vector and the text segment vector is calculated using the following formula:

[0027] ;

[0028] in, For the title vector, This is a vector of text fragments.

[0029] Furthermore, the construction of the domain entity knowledge base includes:

[0030] Entities and their relationships are extracted from authoritative literature in the field, and the entity matching similarity threshold is set to 0.8.

[0031] Cosine similarity is used for entity disambiguation. The formula for cosine similarity is:

[0032] ;

[0033] in, These represent two entity vectors. After extracting entities and their relationships from authoritative literature in the field, the entities are represented in vector form. and These are the vectors corresponding to the two entities that need to be compared for similarity. Representing vectors sum vector The dot product operation measures the similarity of two vectors in a direction. Representing vectors respectively sum vector The modulus is the length. The cosine similarity of two vectors is obtained by dividing the dot product by the product of the moduli of the two vectors. Its value ranges between [−1, 1]. The closer the value is to 1, the more similar the two vectors are, and the closer it is to -1, the less similar the two vectors are.

[0034] Regularly collect new authoritative literature in the field and update entities and their relationships. The condition for updating entities is that the proportion of unmatched entities in the new literature is greater than 20%, which triggers the knowledge base update.

[0035] Furthermore, the hybrid model includes:

[0036] Weights are calculated, and the formula for weight calculation is as follows:

[0037] ;

[0038] in, Words of The weight value, which combines word frequency and inverse document frequency information, is used to measure the importance of a word in a document. The higher the value, the more important the word is.

[0039] Vocabulary The frequency of a word in the current document reflects its relative importance within the document; the word's importance in the current document is positively correlated with its frequency.

[0040] The word count represents the total number of words in the current document. The word count can be obtained by dividing the number of occurrences of word w by the total number of words in the document. Word frequency in this document ,Right now Word frequency reflects the importance of words in a single document;

[0041] It represents the total number of documents in a document collection; it is used to calculate the inverse document frequency. A fundamental quantity used to measure the distribution of words throughout the entire document collection;

[0042] It refers to the number of documents in a document set that contain the word w, inverse document frequency. Inverse document frequency reflects the general importance of words;

[0043] The model uses 3 layers of convolutional kernels and bidirectional... structure;

[0044] Keyword selection strategy is based on Weights and The model outputs keyword probabilities. A comprehensive score threshold is set, and keywords with comprehensive scores higher than the threshold are selected as core words. The comprehensive score is calculated as follows: Weights and The model outputs a weighted sum of probabilities, with weights of 0.6 and 0.4 respectively.

[0045] In the semantic understanding and core idea extraction steps, semantic role labeling adopts a neural network-based semantic role labeling model.

[0046] Furthermore, in the multi-scale attention mechanism, the weight allocation of the local attention layer and the global attention layer can be dynamically adjusted according to the document type;

[0047] The parameterized length constraint adjustment includes adjusting the max_length and min_length parameters of the model to control the length of the summary, where max_length is 300 words and min_length is 100 words.

[0048] Furthermore, the hybrid reward function includes automatic indicators. The formula for the hybrid reward function is: (Based on the combined score and human evaluation.)

[0049] ;

[0050] in, It is an automatic evaluation metric used to assess the similarity between the generated summary and the reference summary;

[0051] The human scoring is done by evaluators who score the generated abstracts based on their accuracy, completeness, and readability, with a score range of 0-10.

[0052] 0.6 and 0.4 are respectively The weighting coefficients of the scores and human ratings indicate the automatic metrics in the hybrid reward function. The contribution of human scoring accounts for 60%, while the contribution of manual scoring accounts for 40%.

[0053] Furthermore, it also includes model deployment and updates: deploying the trained model into the library document processing system, regularly collecting user feedback on the generated abstracts, and using incremental learning to optimize the model based on user feedback. The update trigger condition is: user negative review rate >10% or automatic evaluation index decreases by more than 15%.

[0054] The beneficial effects of this invention are as follows:

[0055] 1. In the data preprocessing stage, a hybrid annotation technique combining rules and deep learning is used to fully leverage the advantages of both. Rules provide the basic framework, while deep learning compensates for the shortcomings of rules and can more accurately understand the meaning of domain terms. The domain entity knowledge base ensures accurate identification and understanding of relationships among entities in the document through rigorous extraction and disambiguation methods, laying the foundation for high-quality abstract generation from a semantic level. In addition, by accurately understanding domain terms and effectively capturing semantic associations in long texts, the accuracy, completeness, and readability of the generated abstracts are improved, thereby better meeting users' needs for quickly obtaining key information from documents.

[0056] 2. In the model building stage, dynamic context awareness technology, with the help of Transformer and self-attention mechanism, effectively captures long-distance semantic relationships in long texts, enabling the model to grasp complex logical relationships; multi-scale attention mechanism focuses on the text from both local and global dimensions, further strengthening the understanding of semantic hierarchy and logical relationships, so that the abstract content can coherently, orderly and accurately convey the main idea of ​​the document. Attached Figure Description

[0057] Figure 1 This is the overall flowchart of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] Please see Figure 1 ,

[0060] This embodiment provides a method for generating library document summaries based on a deep learning model, including:

[0061] Data preprocessing: noise removal and structured sentence segmentation of the document text;

[0062] A hybrid annotation technique based on rules and deep learning is used to add semantic tags to the text, including topic categories, sentiment tendencies, and academic value classifications.

[0063] Noise includes special symbols, HTML tags, and non-text content. In step S1, when the document length exceeds the threshold of 10,000 words, it is divided into segments of 2,000 words each, and a hierarchical abstracting strategy is used to generate sub-abstracts before integration.

[0064] In data preprocessing, the hybrid annotation technology based on rules and deep learning includes a pre-defined list of keywords for topic categories, a sentiment lexicon, and academic value grading standards. The deep learning part uses a pre-trained language model to perform semantic understanding and auxiliary annotation of the text. When the rule annotation and the deep learning annotation results are inconsistent, the deep learning annotation result shall prevail. At the same time, it supports manual intervention and correction for special cases.

[0065] It is worth noting that special symbols, HTML tags, and non-textual content were removed from the document text to make it cleaner and more standardized, facilitating subsequent processing. Structured sentence segmentation rationally divides the text, laying the foundation for accurate semantic labeling. A hybrid labeling technique based on rules and deep learning is employed to add semantic tags such as topic categories, sentiment tendencies, and academic value ratings to the text, enriching its semantic information and helping the model better understand the content. The rule-based part utilizes a pre-set list of topic category keywords, a sentiment lexicon, and academic value rating standards to provide clear labeling criteria. The deep learning part uses a pre-trained language model for semantic understanding-assisted labeling; when the rule-based labeling and deep learning labeling results are inconsistent, the deep learning labeling results take precedence, balancing accuracy and flexibility. Manual intervention and correction for special cases are supported to ensure the accuracy and reliability of the labeling results and improve data quality. When the document length exceeds the threshold (10,000 words), it is segmented into 2,000-word segments, and a hierarchical summarization strategy is used to generate sub-summaries before integration, effectively solving the problem of difficult long document processing and improving the feasibility and quality of summary generation.

[0066] Data preprocessing is the foundation of the entire solution. High-quality preprocessing provides accurate and standardized data for subsequent model building, semantic understanding, and other processes, directly affecting the model training effect and the quality of summary generation.

[0067] Model building: Select a model as the basic architecture based on the characteristics of the data and task. In practical applications, a more suitable model can be selected according to the characteristics of the data and task. Conduct domain pre-training on academic literature datasets in the field of library science, including journal articles, dissertations and conference literature in the fields of library and information science, bibliometrics, and knowledge organization, with a training round of ≥10 rounds.

[0068] We introduce dynamic context-aware technology, calculate long-distance dependencies in text using a Transformer encoder, and dynamically adjust the attention weight distribution by combining the document chapter titles.

[0069] During domain pre-training of the model, a dynamic learning rate adjustment strategy is adopted, with the initial learning rate set to 10. −5Based on the changes in the loss function during training, when the loss function decreases by less than 0.01 for five consecutive rounds, the learning rate is reduced to 0.8 times the original value to accelerate model convergence.

[0070] The model constructs an academic literature dataset in the field of library science, including journal articles, dissertations, and conference papers in the areas of library and information science, bibliometrics, and knowledge organization, with a total of ≥10,000 articles.

[0071] Dynamic context-aware technologies include:

[0072] The long-distance dependencies between words are calculated using a self-attention mechanism to analyze the importance of contextual information to the current word. The calculation formula for the self-attention mechanism is as follows:

[0073] ;

[0074] in, This is the query vector, representing the information that needs to be focused on at the moment, used to search for relevant information among all key-value pairs; This is the key vector, used in conjunction with the query vector. Perform matching calculations to determine And the correlation between each key-value pair; It is a value vector that contains the actual information content; Key vector The dimension;

[0075] Methods for generating title vectors in dynamic context-aware technology include:

[0076] Extract the title semantic vector using an encoder that shares parameters with the main model;

[0077] The similarity between the title vector and the text segment vector is calculated using the following formula:

[0078] ;

[0079] in, For the title vector, This is a vector of text fragments.

[0080] It is important to note that the model was selected as the basic architecture based on the characteristics of the data and the task, demonstrating flexibility and adaptability to meet the needs of different scenarios. Domain-specific pre-training was performed on academic literature datasets in the field of library science, including journal articles, dissertations, and conference proceedings (≥10,000 articles) in the areas of library and information science, bibliometrics, and knowledge organization. The training run consisted of ≥10 epochs, allowing the model to better adapt to the characteristics and language style of the specific domain, thereby improving its performance in the document summarization task. A dynamic learning rate adjustment strategy was adopted, with an initial learning rate set to 10. −5 Based on the changes in the loss function during training, when the loss function decreases by less than 0.01 for five consecutive rounds, the learning rate is reduced to 0.8 times its original value to accelerate model convergence and improve training efficiency. The Transformer encoder calculates long-distance dependencies in the text, and the self-attention mechanism is used to calculate long-distance dependencies between words, analyzing the importance of contextual information to the current word, enabling the model to better understand the semantics and structure of the text. The attention weight distribution is dynamically adjusted based on the document chapter titles to more accurately capture key information in the text. Title semantic vectors are extracted using an encoder that shares parameters with the main model, and similarity is used to calculate the similarity between the title vector and the text segment vector, further optimizing the attention weight allocation.

[0081] Model building is the core of the solution. A reasonable model architecture, effective training strategies, and advanced technology applications determine the model's ability to understand text and generate summaries, which directly affects the quality of the final summary.

[0082] Semantic understanding and core idea extraction: Identify agent, patient and action relationships in sentences through semantic role labeling, and perform entity disambiguation by combining domain entity knowledge base;

[0083] use and The hybrid model is used to extract and filter core keywords;

[0084] The construction of a domain entity knowledge base includes:

[0085] Entities and their relationships are extracted from authoritative literature in the field, and the entity matching similarity threshold is set to 0.8.

[0086] Cosine similarity is used for entity disambiguation. The formula for cosine similarity is:

[0087] ;

[0088] in, These represent two entity vectors. After extracting entities and their relationships from authoritative literature in the field, the entities are represented in vector form. and These are the vectors corresponding to the two entities that need to be compared for similarity. Representing vectors sum vector The dot product operation measures the similarity of two vectors in a direction. Representing vectors respectively sum vector The modulus is the length. The cosine similarity of two vectors is obtained by dividing the dot product by the product of the moduli of the two vectors. Its value ranges between [−1, 1]. The closer the value is to 1, the more similar the two vectors are, and the closer it is to -1, the less similar the two vectors are.

[0089] Regularly collect new authoritative literature in the field and update entities and their relationships. The condition for updating entities is that the proportion of unmatched entities in the new literature is greater than 20%, which triggers the knowledge base update.

[0090] Hybrid models include:

[0091] Weights are calculated, and the formula for weight calculation is as follows:

[0092] ;

[0093] in, Words of The weight value, which combines word frequency and inverse document frequency information, is used to measure the importance of a word in a document. The higher the value, the more important the word is.

[0094] Vocabulary The frequency of a word in the current document reflects its relative importance within the document; the word's importance in the current document is positively correlated with its frequency.

[0095] The word count represents the total number of words in the current document. The word count can be obtained by dividing the number of occurrences of word w by the total number of words in the document. Word frequency in this document ,Right now Word frequency reflects the importance of words in a single document;

[0096] It represents the total number of documents in a document collection; it is used to calculate the inverse document frequency. A fundamental quantity used to measure the distribution of words throughout the entire document collection;

[0097] It refers to the number of documents in a document set that contain the word w, inverse document frequency. Inverse document frequency reflects the general importance of words;

[0098] The model uses 3 layers of convolutional kernels and bidirectional... structure;

[0099] Keyword selection strategy is based on Weights and The model outputs keyword probabilities. A comprehensive score threshold is set, and keywords with comprehensive scores higher than the threshold are selected as core words. The comprehensive score is calculated as follows: Weights and The model outputs a weighted sum of probabilities, with weights of 0.6 and 0.4 respectively.

[0100] In semantic understanding and core idea extraction, semantic role labeling adopts a neural network-based semantic role labeling model.

[0101] It is worth noting that semantic role labeling identifies the agent, patient, and action relationships in sentences. Combined with a neural network-based semantic role labeling model, this allows for a more accurate understanding of the sentence's semantic structure, providing a foundation for subsequent core idea extraction. Entity disambiguation is performed using a domain entity knowledge base, extracting entities and their relationships from authoritative domain literature. The entity matching similarity threshold is set to 0.8, and cosine similarity is used for entity disambiguation. New authoritative domain literature is collected regularly to update entities and their relationships (an update is triggered when the proportion of unmatched entities in new literature exceeds 20%), improving the accuracy of entity recognition and avoiding semantic misunderstandings caused by entity ambiguity. A hybrid model consisting of weight calculation and a model is used to extract and filter core vocabulary. The weight calculation integrates word frequency and inverse document frequency information, and the model uses a 3-layer convolutional kernel and a bidirectional structure. The keyword selection strategy sets a comprehensive scoring threshold based on the weights and the keyword probabilities output by the model, selecting keywords with a comprehensive score higher than the threshold as core vocabulary. This allows for more accurate extraction of words reflecting the core content of the text.

[0102] Semantic understanding and core idea extraction are key steps in generating accurate summaries. Accurately understanding the semantics of the text and extracting core ideas can ensure that the generated summary is accurate and highlights the key points.

[0103] Abstract generation and optimization: An initial abstract is generated using a multi-scale attention mechanism, where the local attention layer has a window size of 5 sentences and the global attention layer covers the entire text;

[0104] The summary length is adjusted through parameterized length constraints and reinforcement learning;

[0105] In multi-scale attention mechanisms, the weight allocation between local attention layers and global attention layers can be dynamically adjusted according to the document type;

[0106] The parameterized length constraint adjustment includes adjusting the max_length and min_length parameters of the model to control the length of the summary, where max_length is 300 words and min_length is 100 words.

[0107] It is worth noting that a multi-scale attention mechanism is used to generate the initial summary. The local attention layer has a window size of 5 sentences, focusing on local information in the text; the global attention layer covers the entire text, grasping the overall structure. The weight allocation of the local and global attention layers can be dynamically adjusted according to the document type, making the generated summary more suitable for different types of documents. The model's max_length and min_length parameters (max_length = 300 words, min_length = 100 words) are adjusted through parameterized length constraints to control the length of the summary and meet the needs of different users. Reinforcement learning is used to adjust the summary length, achieving a better balance between length and content quality in the generated summary.

[0108] The abstract generation and optimization process directly determines the format and quality of the final abstract presented to the user. Reasonable generation strategies and optimization methods can improve the readability and usability of the abstract.

[0109] Quality assessment: Design a hybrid reward function and iteratively optimize the model through reinforcement learning.

[0110] Hybrid reward functions include automatic metrics The formula for the combined reward function, which combines the score and the human rating, is as follows:

[0111] ;

[0112] in, It is an automatic evaluation metric used to assess the similarity between the generated summary and the reference summary;

[0113] The human scoring is done by evaluators who score the generated abstracts based on their accuracy, completeness, and readability, with a score range of 0-10.

[0114] 0.6 and 0.4 are respectively The weighting coefficients of the scores and human ratings indicate the automatic metrics in the hybrid reward function. The contribution of human scoring accounts for 60%, while the contribution of manual scoring accounts for 40%.

[0115] It is worth noting that the hybrid reward function is designed, which combines the scores of automatic indicators and human evaluations, taking into account both the objectivity of automatic evaluation and the subjectivity of human evaluation, and thus more comprehensively and accurately assessing the quality of the generated summaries. The model is iteratively optimized through reinforcement learning, and the model parameters are continuously adjusted based on the quality evaluation results to improve the quality of the model-generated summaries.

[0116] The quality assessment stage involves monitoring and providing feedback on the entire abstract generation process. Through accurate evaluation and continuous optimization, the performance of the model and the quality of the abstracts are constantly improved.

[0117] It also includes model deployment and updates: deploying the trained model to the library document processing system, regularly collecting user feedback on the generated abstracts, and using incremental learning to optimize the model based on user feedback. The update trigger condition is: user negative review rate >10% or automatic evaluation index decreases by more than 15%.

[0118] It is worth noting that the trained model is deployed into the library's document processing system to automate and intelligently generate document abstracts, thereby improving the efficiency of library document processing. User feedback on the generated abstracts is collected regularly, and the model is optimized through incremental learning based on user feedback. The update trigger condition is when the user negative review rate is >10% or the automatic evaluation index decreases by more than 15%, so that the model can continuously adapt to new data and user needs and continuously improve the quality of abstract generation.

[0119] The model deployment and update process ensures that the solution can be practically applied and continuously improved, enabling the system to operate stably for a long time and provide high-quality summary services.

[0120] Example 1

[0121] Generate medical literature summaries in the field of cancer research

[0122] Data preprocessing optimization

[0123] Biomedical-Specific Cleaning Rules: In medical literature, PubMed's MeSH terms are crucial for accurately understanding the subject matter. Preserving these unique metadata helps the model better grasp the core content of the literature. For example, MeSH terms can clearly identify key information such as the type of disease and research methods involved in the literature.

[0124] Special symbol handling: Greek letters and other special symbols are standardized and converted, such as "α→alpha", to avoid model recognition errors caused by special symbols. In medical literature, many technical terms contain Greek letters. After standardization, the model can process the text more accurately.

[0125] Segmentation strategy adjustment: For clinical trial literature, mandatory segmentation is carried out according to the "background-methods-results" structure, which conforms to the logical structure of medical research. In this way, the model can better understand the content of different parts in subsequent processing. For example, when extracting core viewpoints, it can more effectively extract key information from each part.

[0126] Domain pre-training enhancement

[0127] ClinicalBERT is used as the base model. For example, when processing cancer research literature, ClinicalBERT is better able to identify and understand technical terms such as cancer-related gene names and treatment methods, thereby generating more accurate summaries.

[0128] New training data:

[0129] Data source Document volume Professional coverage PubMed 50000 General Practice TCGA 2500 Cancer Genome

[0130] Dynamic context awareness improvement

[0131] Title Weight Enhancement: In cancer research literature, key sections such as "survival rate" and "adverse reactions" are crucial for understanding the core content. Increasing the attention weight of these sections by 30% allows the model to focus more on this key information when processing the text, thus better reflecting the main points of the literature in the generated abstract.

[0132] Improved similarity calculation: using a similarity formula The use of this method comprehensively considers the similarity between the title vector and the text fragment vector, as well as the title words. This allows the model to focus more on key information in the title when calculating similarity, further improving the accuracy of the summary.

[0133] Validation Result Analysis

[0134] index Before improvement Improved Increase Terminology accuracy 88% 95% +7% Key conclusion retention rate 76% 89% +13% Doctor satisfaction 3.8 / 5 4.5 / 5 +18%

[0135] The validation results show that the accuracy rate of terminology improved from 88% to 95%, the retention rate of key conclusions improved from 76% to 89%, and the physician satisfaction rate improved from 3.8 / 5 to 4.5 / 5.

[0136] This indicates that the preprocessing and model optimization measures for the medical field are effective and significantly improve the clinical applicability of the abstracts.

[0137] Example 2

[0138] Updates to intellectual property legal documents

[0139] Entity knowledge base update mechanism

[0140] The update trigger logic is refined: incremental updates are initiated when the proportion of new regulations exceeds 20% or the proportion of important entities among unmatched entities exceeds 15%. Such triggering conditions ensure timely responses to changes in the legal field and guarantee the timeliness of the entity knowledge base. For example, when new intellectual property regulations are introduced, the knowledge base is updated promptly, enabling the model to learn the latest legal information.

[0141] Entity Relationship Mining: A new three-dimensional relationship network of "legal provisions → case law → scholarly opinions" has been added, enriching the information on the relationships between entities. When processing legal documents, the model can better understand the application of legal provisions in actual case law and scholars' opinions on relevant legal issues, thereby generating more comprehensive and accurate summaries.

[0142] Hybrid model optimization

[0143] TF-IDF weight adjustment: A minimum threshold of 0.2 is set for legally specific expressions (such as the "fair use principle") to ensure that these key expressions receive sufficient attention in the model's processing. In legal literature, these specific expressions often carry important legal concepts and principles, and adjusting the weights helps the model better capture key information.

[0144] CNN-LSTM Structure Adjustment:

[0145] use The structure allows for better processing of sequential information in legal texts; one-dimensional convolution can extract local features, bidirectional LSTM can capture long-distance dependencies in the text, and the attention mechanism can further focus on key information, thus improving the model's ability to process legal documents.

[0146] Evaluation Standards Upgrade

[0147] A new legal validity assessment has been added, where practicing lawyers score the accuracy of clause citations in the summary. This assessment standard measures the quality of the summary from a professional perspective, ensuring that the generated summary is accurate and reliable from a legal standpoint.

[0148] Validation Result Analysis

[0149] Scene F1 before update F1 after update Typical Improvement Cases Patent infringement 0.71 0.83 Accuracy in interpreting the "principle of equivalence" increased by 22%. Copyright 0.68 0.79 The complete mention rate of the four elements of "reasonable use" increased by 15%.

[0150] The verification results show that in patent infringement scenarios, the F1 score improved from 0.71 to 0.83, and the accuracy of interpreting the "equivalence doctrine" increased by 22%; in copyright scenarios, the F1 score improved from 0.68 to 0.79, and the complete mention rate of the four elements of "fair use" increased by 15%.

[0151] This indicates that the dynamic update mechanism and model optimization measures have enabled the system to maintain legal timeliness and significantly improved the ability to identify key concepts.

[0152] The terms "front," "back," "left," "right," "top," and "bottom" all refer to the figures in the accompanying drawings. Figure 1 Based on the perspective of the observer, the side of the device facing the observer is defined as the front, the left side of the observer is defined as the left, and so on.

[0153] In the description of this invention, it should be understood that the terms "center", "longitudinal", "lateral", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting the scope of protection of this invention.

[0154] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for generating library document summaries based on a deep learning model, characterized in that: include: Data preprocessing: noise removal and structured sentence segmentation of the document text; A hybrid annotation technique based on rules and deep learning is used to add semantic tags to the text, including topic categories, sentiment tendencies, and academic value classifications. Model building: Select a model as the basic architecture based on the characteristics of the data and the task, and conduct domain pre-training on academic literature datasets in the field of library science, including journal articles, dissertations and conference literature in the fields of library and information science, bibliometrics and knowledge organization, with a training round of ≥10 rounds; We introduce dynamic context-aware technology, calculate long-distance dependencies in text using a Transformer encoder, and dynamically adjust the attention weight distribution by combining the document chapter titles. Semantic understanding and core idea extraction: Identify agent, patient and action relationships in sentences through semantic role labeling, and perform entity disambiguation by combining domain entity knowledge base; use and The hybrid model is used to extract and filter core keywords; Abstract generation and optimization: An initial abstract is generated using a multi-scale attention mechanism, where the local attention layer has a window size of 5 sentences and the global attention layer covers the entire text; in the multi-scale attention mechanism, the weight allocation of the local attention layer and the global attention layer can be dynamically adjusted according to the document type; The summary length is adjusted through parameterized length constraints and reinforcement learning; The parameterized length constraint adjustment includes: adjusting the model's max_length and min_length parameters to control the length of the abstract, where max_length is 300 words and min_length is 100 words; dynamically adjusting the weight allocation of the local attention layer and the global attention layer according to the document type to ensure that the generated abstract conforms to the characteristics of different document types; adjusting the model's max_length and min_length parameters through parameterized length constraints, with max_length set to 300 words and min_length to 100 words, to control the length of the abstract and meet the needs of different users for abstract length; and adjusting the abstract length through reinforcement learning to achieve a balance between length and content quality in the generated abstract. Quality assessment: Design a hybrid reward function and iteratively optimize the model through reinforcement learning.

2. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: The noise includes special symbols, HTML tags and non-text content. In step S1, when the document length exceeds the threshold of 10,000 words, it is divided into segments of 2,000 words each, and a hierarchical abstracting strategy is used to generate sub-abstracts before integration.

3. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: In the hybrid annotation technology based on rules and deep learning, the rule part includes a pre-set list of topic category keywords, a sentiment lexicon, and an academic value grading standard. The deep learning part uses a pre-trained language model to perform semantic understanding and auxiliary annotation of the text. When the rule annotation and the deep learning annotation results are inconsistent, the deep learning annotation result shall prevail. At the same time, it supports manual intervention and correction for special cases.

4. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: When performing domain pre-training, the model construction employs a dynamic learning rate adjustment strategy, with the initial learning rate set to 10. −5 Based on the changes in the loss function during training, when the loss function decreases by less than 0.01 for five consecutive rounds, the learning rate is reduced to 0.8 times the original value to accelerate model convergence. The model constructs an academic literature dataset in the field of library science, including ≥10,000 journal articles, dissertations, and conference papers in the areas of library and information science, bibliometrics, and knowledge organization.

5. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: The dynamic context-aware technology includes: The long-distance dependencies between words are calculated using a self-attention mechanism to analyze the importance of contextual information to the current word. The calculation formula for the self-attention mechanism is as follows: ， in, This is the query vector, representing the information that needs to be focused on at the moment, used to search for relevant information among all key-value pairs; This is the key vector, used in conjunction with the query vector. Perform matching calculations to determine And the correlation between each key-value pair; It is a value vector that contains the actual information content; Key vector The dimension; The method for generating the title vector in the dynamic context-aware technology includes: Extract the title semantic vector using an encoder that shares parameters with the main model; The similarity between the title vector and the text segment vector is calculated using the following formula: ， in, For the title vector, This is a vector of text fragments.

6. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: The construction of the domain entity knowledge base includes: Entities and their relationships are extracted from authoritative literature in the field, and the entity matching similarity threshold is set to 0.

8. Cosine similarity is used for entity disambiguation. The formula for cosine similarity is: ， in, These represent two entity vectors. After extracting entities and their relationships from authoritative literature in the field, the entities are represented in vector form. and These are the vectors corresponding to the two entities that need to be compared for similarity. Representing vectors sum vector The dot product operation measures the similarity of two vectors in a direction. Representing vectors respectively sum vector The modulus is the length. The cosine similarity of two vectors is obtained by dividing the dot product by the product of the moduli of the two vectors. Its value ranges between [−1, 1]. The closer the value is to 1, the more similar the two vectors are, and the closer it is to -1, the less similar the two vectors are. Regularly collect new authoritative literature in the field and update entities and their relationships. The conditions for updating entities are as follows: When the proportion of unmatched entities in a new document exceeds 20%, a knowledge base update is triggered.

7. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: The hybrid model includes: Weights are calculated, and the formula for weight calculation is as follows: ， in, Words of The weight value, which combines word frequency and inverse document frequency information, is used to measure the importance of a word in a document. The higher the value, the more important the word is. Vocabulary The frequency of a word in the current document reflects its relative importance within the document; the word's importance in the current document is positively correlated with its frequency. Represents the total number of words in the current document, by using vocabulary Dividing the number of occurrences of a word by the total number of words in the document yields the vocabulary. Word frequency in this document , Word frequency reflects the importance of words in a single document; It represents the total number of documents in a document collection; it is used to calculate the inverse document frequency. A fundamental quantity used to measure the distribution of words throughout the entire document collection; This refers to a collection of documents containing vocabulary. Number of documents, inverse document frequency Inverse document frequency reflects the general importance of words; The model uses 3 layers of convolutional kernels and bidirectional... structure; Keyword selection strategy is based on Weights and The model outputs keyword probabilities. A comprehensive score threshold is set, and keywords with comprehensive scores higher than the threshold are selected as core words. The comprehensive score is calculated as follows: Weights and The model outputs a weighted sum of probabilities, with weights of 0.6 and 0.4 respectively. In the semantic understanding and core idea extraction steps, semantic role labeling adopts a neural network-based semantic role labeling model.

8. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: The hybrid reward function includes automatic indicators. The formula for the hybrid reward function is: (Based on the combined score and human evaluation.) ， in, It is an automatic evaluation metric used to assess the similarity between the generated summary and the reference summary; The human scoring is done by evaluators who score the generated abstracts based on their accuracy, completeness, and readability, with a score range of 0-10. 0.6 and 0.4 are respectively The weighting coefficients of the scores and human ratings indicate the automatic metrics in the hybrid reward function. The contribution of human scoring accounts for 60%, while the contribution of manual scoring accounts for 40%.

9. The library document abstract generation method based on a deep learning model according to claim 1, characterized in that: It also includes model deployment and updates: deploying the trained model to the library document processing system, regularly collecting user feedback on the generated abstracts, and using incremental learning to optimize the model based on user feedback. The update trigger condition is: user negative review rate >10% or automatic evaluation index decreases by more than 15%.

Images