Long text intelligent output reference system

Abstract

The invention discloses a long text intelligent output reference system, which relates to the technical field of intelligent text processing, and comprises a long text processing module, a reference demand analysis module, a reference extraction module, an optimization module and a reference output module, the long text processing module is used for receiving multiple formats of long texts and converting the long texts into internal formats, determining a core theme field and a sentence structure relationship through vocabulary statistics, syntactic analysis and semantic recognition and constructing a semantic map, and providing a text analysis basis for subsequent reference; according to the method, the long text reference efficiency and accuracy are remarkably improved, the time consumption of manual retrieval and screening is reduced through automatic processing of long text data, meanwhile, the professionality and correlation of reference content are ensured through deep analysis of the text content, the system can understand the reference requirement of a user, and the user experience is improved. And the demands are converted into accurate screening conditions, so that reference contents highly matched with the user demands are provided.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent text processing, and in particular to a long text intelligent output citation system. Background Art

[0002] In the fields of academic research, technical writing, and literary creation, citing long texts is a complex and time-consuming process. Traditional long text citations rely on manual retrieval and screening, which is not only inefficient but also easily influenced by personal experience and preferences, resulting in inaccurate and incomplete citations. With the explosive growth of information, long text materials are becoming more and more numerous, and it is becoming increasingly difficult to manually process these materials. In addition, the diversity and complexity of long texts also increase the difficulty of citations, making it difficult for the cited content to meet the user's requirements for accuracy and relevance.

[0003] In order to solve the above problems, an intelligent long text output citation system is needed, which can automatically process long text data, understand the user's citation needs, and provide accurate, relevant and coherent citation content. Such a system needs to have the ability of text analysis, including vocabulary statistics, syntactic analysis and semantic recognition to determine the core subject areas and sentence structure relationships of the text. At the same time, the system also needs to be able to understand the user's citation needs and convert these needs into system instructions and parameters in order to accurately screen and extract the required citation content.

[0004] Although existing technologies have made some progress in text processing and semantic analysis, most systems still cannot meet the complex requirements of long text citations. These systems often lack an in-depth understanding of user needs and cannot provide personalized citation services. In addition, existing technologies often ignore the relevance and coherence between text fragments when processing long texts, resulting in the extracted citation content being semantically incoherent and unable to form an organic whole. Therefore, developing a long text intelligent output citation system that can comprehensively consider text analysis, user demand understanding, and citation content optimization is of great significance to improving citation efficiency and quality.

[0005] In order to solve the above defects, a technical solution is now provided. Summary of the invention

[0006] The purpose of the present invention is to solve the problems of low efficiency, low accuracy and lack of personalized service in the traditional long text citation process, and to propose a long text intelligent output citation system.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] Long text intelligent output citation system, including:

[0009] The long text processing module receives long texts in various formats and converts them into internal formats. It determines the core subject areas, sentence structure relationships, and constructs semantic maps through vocabulary statistics, syntactic analysis, and semantic recognition, providing a text analysis basis for subsequent citations.

[0010] The reference demand analysis module receives reference demand information through the interactive interface, analyzes and classifies it, and converts it into system instructions and parameters to ensure that the system understands user needs;

[0011] The citation extraction module, based on the screening conditions determined by the demand analysis module, preliminarily screens text fragments and evaluates their relevance based on the semantic graph and text structure, and then uses machine learning algorithms to accurately evaluate and select matching and relevant and coherent citation content;

[0012] The optimization module processes the extracted citation content according to the citation format specifications determined by the demand analysis module, adds information and adjusts the typesetting format, and uses text summarization technology to simplify the content, ensuring that the simplified content is complete, logically coherent, and meets the simplification standards through various means;

[0013] The citation output module converts the optimized citation content into the user-specified format, inserts it into document editing software or generates an independent document, and adds the citation source, location and credibility score metadata. It comprehensively evaluates the credibility based on the authority of the original text, the relevance of the citation content to the topic, and the timeliness of the citation content, to assist users in judging the reliability and value of the citation.

[0014] Furthermore, the execution steps of the long text processing module are as follows:

[0015] Receive long text data in multiple formats and convert it into the internal text format processed by the system;

[0016] Conduct in-depth analysis of imported long texts, including vocabulary statistics, counting the frequency of occurrence of different words, and identifying high-frequency professional words to determine the core subject areas of the text;

[0017] In the process of determining high-frequency professional vocabulary, the importance weight of the vocabulary in the text is determined by the formula: Get vocabulary weight TF-IDF j , where f ij represents the frequency of word i in document j, n is the total number of words in document j, N is the total number of documents in the document collection, df i is the number of documents containing word i;

[0018] The obtained vocabulary weights are TF-IDE ij Compare with the preset weight threshold, when the vocabulary weight TF-IDF ijWhen it is greater than the preset weight threshold, the word is judged to be a high-frequency professional word;

[0019] Syntactic analysis: analyzing the sentence structure of the text and determining the relationship between sentence components, providing a structural basis for subsequent semantic understanding and reference extraction;

[0020] The semantic recognition submodule uses natural language processing technology and pre-trained semantic models to identify semantic units in the text and construct a semantic map of the text.

[0021] Furthermore, the specific operation steps of the reference demand analysis module are as follows:

[0022] Receive the citation requirement information input by the user through the interactive interface, including the purpose of the citation, the type of the citation text, and the application scenario of the citation;

[0023] Parse and classify the received reference requirements and convert them into instructions and parameters recognized by the system. The specific process is as follows:

[0024] Analysis for the purpose of citation: When the purpose of the user's citation is to prove a certain point, extract the keywords in the point of view, convert these keywords into semantic search conditions, and map these keywords to predefined semantic tags or concepts; construct text screening conditions based on these semantic tags and concepts, including finding sentences or paragraphs containing these keywords or semantically related words in long texts, and these contents have the nature of argumentation;

[0025] When the purpose of citing is to serve as a comparative case, analyze the comparative objects involved by the user, establish corresponding text features for each comparative object, and convert these features into screening conditions to extract the comparative content from the long text;

[0026] When citations are used to supplement background knowledge, identify the topics of the background knowledge needed and translate these concepts into search terms for chronological, historical event-related text features in long texts;

[0027] Parsing of quoted text types: When the user specifies a quoted paragraph or chapter, the paragraph number or chapter title information entered by the user is parsed, and the corresponding index or positioning parameters are established to extract the content of the chapter or paragraph from the long text;

[0028] For different types of sentences, identify the characteristics of data description sentences, build screening conditions based on these characteristics, and for opinion elaboration sentences, analyze the language patterns of opinion elaboration sentences;

[0029] Parsing of citation application scenarios: When the citation application scenario is an academic paper, call the predefined academic paper citation format template, including the typesetting format of the author's name, publication year, paper title, journal name, volume number, and page number information; convert these templates into codes or parameters to format the citation content when the citation is output;

[0030] For the citation scenario of technical reports, identify the characteristics of the citation format of technical reports, including the report author, report number, publishing organization, and publishing year information, and convert these format characteristics into corresponding format codes;

[0031] In the citation scenario of literary creation, the format of literary citation is analyzed, including chapters, paragraphs, and character dialogues in the quoted works; the citation format requirements in these literary works are converted into codes or parameters executed by the system to correctly format the quoted content.

[0032] Furthermore, the specific operation steps of the reference extraction module are as follows:

[0033] According to the screening conditions determined by the citation demand analysis module, based on the semantic graph and text structure constructed by the long text processing module, the text fragments that meet the requirements are preliminarily screened out;

[0034] The relevance score is obtained by evaluating the relevance of the text fragment, using the formula: To get the relevance score Score(s) of the text segment s, w j is the weight of the jth feature, f j (s) is the score function of the text segment s for the jth feature, and k is the number of features; the text segments whose relevance score Score(s) is greater than the preset score threshold are used as the preliminary screening criteria for meeting the requirements;

[0035] Use machine learning algorithms to evaluate the accuracy of the initially screened text fragments, and based on the evaluation results, screen out text content that matches the citation requirements again; at the same time, analyze the relevance and coherence between text fragments to avoid extracting isolated, semantically incoherent citation content.

[0036] Furthermore, the specific process of using machine learning algorithms to conduct accuracy assessment and re-screening of the initially screened text fragments is as follows:

[0037] A. Data preparation: Extract multiple features from the initially selected text fragments for subsequent machine learning model training and evaluation; count the word frequency and inverse document frequency of vocabulary to reflect the importance of vocabulary in the text fragment; analyze the syntactic structure of sentences, including sentence length, sentence type, and grammatical structure complexity; use the pre-trained word vector model to convert the vocabulary in the text fragment into vectors, and then calculate the semantic vector representation of the text fragment, and measure the semantic relevance by calculating the cosine similarity between the text fragment vector and the citation demand vector; label each initially selected text fragment to indicate its matching degree with the citation demand;

[0038] B. Model training: Select the support vector machine model and find an optimal hyperplane to divide samples of different categories to maximize the interval between samples of different categories; train the model based on the extracted features and annotated labels to learn the relationship between the features of the text fragment and the matching degree of the citation requirements;

[0039] Divide the text snippet dataset with features and labels into a training set and a validation set; use the training set data to train the model and determine the model parameters by optimizing the objective function; during the training process, use cross-validation technology to evaluate the performance of the model on the validation set; adjust the model parameters based on the evaluation results to improve the performance of the model;

[0040] C. Accuracy evaluation and screening: For each initially screened text segment, its features are input into the trained model, and the probability or score of the text segment belonging to different categories is output; for the multi-classification model, based on the output scores of different categories, the category with the highest score is selected as the predicted category of the text segment, and then based on the definition of the degree of matching between the category and the citation requirement, it is determined whether the text segment highly matches the citation requirement;

[0041] Second screening: Based on the evaluation results, select text fragments that are predicted to match the citation requirements and meet the standards as candidates for further screening; at the same time, analyze the relevance and coherence between text fragments, and measure them by calculating the semantic similarity between text fragments or the relevance based on text structure; for isolated text fragments, even if they themselves match the citation requirements and meet the standards, they are excluded to ensure that the final extracted citation content is semantically coherent and complete, and can form an organic whole.

[0042] Furthermore, the specific operation steps of the optimization module are as follows:

[0043] According to the reference format specification corresponding to the application scenario determined by the reference requirement analysis module, the extracted reference content is formatted, specifically:

[0044] For academic paper citations, add the reference number, author name, publication year, page number information according to the requirements of academic journals, and adjust the text layout format, including font, font size, and indentation;

[0045] Use text summarization technology to simplify the quoted content, remove redundant information, and retain the core semantics; in the process of simplifying, ensure the integrity and logic of the quoted content;

[0046] After simplification, the simplification ratio of the cited content is calculated to measure the simplification degree of the cited content, using the formula: Get the content reduction ratio Ratio, where L c is the length of the condensed quoted content, L o is the length of the original quoted content; the obtained content reduction ratio Ratio is compared with the preset content reduction ratio standard. When the content reduction ratio Ratio is greater than the content reduction ratio standard, it is judged that the content reduction has reached the standard. Otherwise, it is judged that the content reduction has not reached the standard. Under the premise of ensuring the integrity and logic of the quoted content, redundant information is removed again until the content reduction reaches the standard.

[0047] Furthermore, the optimization module removes redundant information again until the content is streamlined to meet the standard. The specific operation steps are as follows:

[0048] Simplification based on vocabulary and grammar rules: identify and delete redundant words, re-scan the quoted content, and find words that appear continuously or repeatedly in similar positions; analyze the function words and auxiliary words in the sentence and delete them without affecting the core semantics of the sentence;

[0049] Find words with the same semantics and replace words that exceed the threshold with concise synonyms; simplify sentence structure and merge semantically related short sentences; break down complex long sentences into several simple sentences and remove repeated or redundant parts;

[0050] Simplification based on semantic analysis: extract the core semantics, use the keyword extraction algorithm in natural language processing technology to determine the keyword in the quoted content; extract key information, identify the key information units in the sentence; divide the quoted content into several semantic blocks, and calculate the semantic similarity between these semantic blocks; merge or select one of the semantic blocks that reach the threshold of semantic similarity to be retained;

[0051] Simplification based on text summarization technology: When using an attention-based summary generation algorithm, rerun the algorithm and readjust the attention weights;

[0052] For statistical summary algorithms, the word weights in the quoted content are re-analyzed, and the words and sentences containing words with weights greater than the preset weights are selected and retained;

[0053] For graph-based summary algorithms, the relationship graph between sentences is reconstructed, and the sentences to be retained are determined through the centrality analysis method of the graph, so as to further streamline the content.

[0054] Furthermore, the specific operation steps of the reference output module are as follows:

[0055] Convert optimized citation content into a user-defined output format, including formats for direct insertion into document editing software, or to generate a stand-alone citation document;

[0056] Add metadata information to the output citation content, including detailed information about the citation source, the location of the citation content in the original text, and the credibility score of the citation content;

[0057] The credibility score of the cited content is obtained by evaluating the credibility of the output content. Specifically, the credibility evaluation value is obtained by analyzing different evaluation factors comprehensively. Based on the credibility evaluation value, a comprehensive credibility evaluation is provided for the output cited content to help users judge the reliability and value of the cited content.

[0058] Furthermore, the analysis steps of the credibility evaluation value in the reference output module are as follows:

[0059] Evaluation factors include:

[0060] Authoritative factors of original text: determined by the journal impact factor score, author reputation score and institution authority score respectively. After normalization, a cone model is established with the journal impact factor score as the base circle radius and the author reputation score as the height. The vertex of the cone model is selected as the center of the sphere, and the institution authority score is used as the sphere diameter to establish a spherical model. The volume of the heteromorphic model formed by the cone model and the spherical model is calculated and recorded as the weighted judgment value, which is used as the standard for measuring the authoritative factors of the original text;

[0061] Factors related to the relevance of the cited content and the topic: A comprehensive analysis is conducted through the vocabulary matching score and the semantic similarity score. The normalization process is performed through weighted summation to obtain the relevant judgment value, which is then used as the standard for measuring the relevance of the cited content and the topic.

[0062] Timeliness factor of cited content: analyzed through time difference and timeliness score. Time difference is calculated by calculating the time difference between the publication time of the cited content and the current time. Timeliness score is scored by scoring the time difference. Several time difference intervals are preset. Different time difference intervals correspond to different timeliness scores. Determine the time difference interval to which the current time difference belongs, and then obtain the timeliness score corresponding to the time difference.

[0063] The obtained time difference and timeliness score are then marked as st and sp respectively, and after normalization, they are entered into the following formula: The time judgment value SYZ is obtained, where τ is the correction factor; and the time judgment value is used as the standard for measuring the timeliness factor of the cited content;

[0064] The obtained weighted judgment value and related judgment value are calibrated as QPZ and XGP respectively, and after normalization with the time judgment value SYZ, they are entered into the following formula: KXP=QPZ×λ1+XGP×λ2+SYZ×λ3 to obtain the credibility evaluation value KXP, where λ1, λ2, and λ3 are the preset weight coefficients of the weighted judgment value, related judgment value, and time judgment value respectively.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] (1) The present invention significantly improves the efficiency and accuracy of long text citations. By automatically processing long text data, the time consumption of manual retrieval and screening is reduced. At the same time, by deeply analyzing the text content, the professionalism and relevance of the cited content are ensured. The system can understand the user's citation needs and convert these needs into accurate screening conditions, thereby providing citation content that highly matches the user's needs;

[0067] (2) The present invention, when extracting the citation content, takes into account the relevance and coherence between text fragments, avoids isolated and semantically incoherent citation content, and ensures that the citation content finally extracted is semantically coherent and complete, and can form an organic whole, which is particularly important for academic papers, technical reports, and literary creation, because the citations in these fields need to provide smooth and logical citation content while maintaining the original meaning;

[0068] (3) The present invention also provides a comprehensive credibility assessment mechanism, which provides users with a credibility score by evaluating the authority of the original text, the relevance of the cited content to the topic, and the timeliness of the cited content. This score helps users judge the reliability and value of the cited content, making the citation not only accurate but also authoritative, thereby enhancing the credibility and practicality of the cited content. In this way, not only the efficiency and accuracy of citations are improved, but also the quality and reliability of the cited content are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;

[0070] Figure 1 This is the overall system block diagram of the present invention. DETAILED DESCRIPTION

[0071] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0073] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0074] like Figure 1 As shown, the long text intelligent output citation system includes a long text processing module, a citation demand analysis module, a citation extraction module, an optimization module and a citation output module;

[0075] The long text processing module receives long texts in various formats and converts them into internal formats. It determines the core subject areas, sentence structure relationships, and constructs semantic maps through vocabulary statistics, syntactic analysis, and semantic recognition, providing a text analysis basis for subsequent citations.

[0076] Receive long text data in multiple formats (such as .txt, .pdf, .docx, etc.) and convert it into an internal text format that the system can process; perform in-depth analysis on the imported long text, including but not limited to vocabulary statistics, counting the frequency of occurrence of different words, identifying high-frequency professional words to determine the core subject area of ​​the text; in the process of determining high-frequency professional words, determine the importance weight of the words in the text, and use the formula: Get vocabulary weight TF-IDF ij , where fij represents the frequency of word i in document j, n is the total number of words in document j, N is the total number of documents in the document collection, df i is the number of documents containing word i; the resulting word weight TF-IDF ij Compare with the preset weight threshold, when the vocabulary weight TF-IDE ij When it is greater than the preset weight threshold, the word is judged to be a high-frequency professional word;

[0077] Syntactic analysis: analyzing the sentence structure of the text and determining the relationship between sentence components, providing a structural basis for subsequent semantic understanding and reference extraction;

[0078] The semantic recognition submodule uses natural language processing technology and pre-trained semantic models (such as semantic models based on the BERT architecture) to identify semantic units in the text, such as topics, events, and character relationships, and construct a semantic map of the text in order to fully understand the semantic context of the text.

[0079] The citation requirement analysis module receives citation requirement information through the interactive interface, parses and classifies it, and converts it into system instructions and parameters. For example, the citation purpose, text type, and application scenario are converted into screening conditions, positioning parameters, and format codes, respectively, to ensure that the system understands user requirements.

[0080] Through the interactive interface, the user inputs citation requirement information, including the purpose of the citation (to prove a specific point of view, as a comparative case, to supplement background knowledge), the type of text to be cited (specific paragraphs, chapters, specific types of sentences such as data description sentences, point of view exposition sentences), and the application scenario of the citation (academic papers, technical reports, literary creations, so as to determine the corresponding citation format specifications);

[0081] Parse and classify the received citation requirements and convert them into instructions and parameters that the system can recognize. For example, convert the citation purpose into screening conditions for specific semantic tags or text features, and convert the application scenario into the corresponding citation format template code.

[0082] Analysis for the purpose of citation: When the user's citation purpose is to prove a specific point of view, extract the keywords in the point of view, convert these keywords into semantic search conditions, and map these keywords to predefined semantic tags or concepts; construct text screening conditions based on these semantic tags and concepts, including finding sentences or paragraphs containing these keywords or semantically related words in long texts, and these contents should have the nature of argumentation;

[0083] When the purpose of citing is to serve as a comparative case, analyze the comparative objects involved by the user, establish corresponding text features for each comparative object, and convert these features into screening conditions to extract the comparative content from the long text;

[0084] When citations are used to supplement background knowledge, identify the topics of the background knowledge needed and translate these concepts into search terms for chronological, historical event-related text features in long texts;

[0085] Parsing of quoted text types: If the user specifies to quote a specific paragraph or chapter, parse the paragraph number or chapter title and other information entered by the user, and establish corresponding indexes or positioning parameters to accurately extract the content of the chapter or paragraph from the long text, which specifically involves the structural analysis of the text;

[0086] For specific types of sentences, identify the characteristics of the data description sentences and build filtering conditions based on these characteristics. For opinion-explanation sentences, analyze the language patterns of opinion-explanation sentences.

[0087] Parsing of citation application scenarios: When the citation application scenario is an academic paper, call the predefined academic paper citation format templates, which usually include the typesetting format of information such as author name, publication year, paper title, journal name, volume number, page number, etc.; convert these templates into codes or parameters to format the citation content when the citation is output;

[0088] For the citation scenario of technical reports, identify the characteristics of the citation format of technical reports, including information such as the report author, report number, publishing organization, and publishing year, and convert these format characteristics into corresponding format codes;

[0089] In the citation scenario of literary creation, the special format of literary citation is analyzed, including chapters, paragraphs, and character dialogues in the quoted works; the citation format requirements in these literary works are converted into system-executable codes or parameters for correct formatting of the quoted content.

[0090] The citation extraction module preliminarily screens text fragments and evaluates relevance based on the screening conditions determined by the demand analysis module, based on the semantic graph and text structure, and then uses machine learning algorithms to accurately evaluate and select highly matched, relevant and coherent citation content;

[0091] According to the screening conditions determined by the citation demand analysis module, based on the semantic map and text structure constructed by the long text processing module, the text fragments that meet the requirements are preliminarily screened out; the relevance score is obtained by evaluating the relevance of the text fragments, and the formula is: To get the relevance score Score(s) of the text segment s, w jis the weight of the jth feature (such as vocabulary matching, syntactic structure rationality, semantic similarity, etc.), f j (s) is the score function of the text segment s for the jth feature (for example, the vocabulary matching degree can be scored based on the number of matches with the referenced keywords, the syntactic structure rationality can be scored based on whether it complies with specific sentence structure rules, and the semantic similarity can be calculated based on the above semantic similarity calculation formula), k is the number of features; the text segments with a relevance score Score(s) greater than the preset score threshold are used as the preliminary screening criteria for meeting the requirements;

[0092] Use machine learning algorithms (such as text classification algorithms based on support vector machines) to evaluate the accuracy of the initially selected text fragments, and further select text content that highly matches the citation requirements based on the evaluation results. At the same time, consider the relevance and coherence between text fragments to avoid extracting isolated and semantically incoherent citation content. The specific steps are as follows:

[0093] A. Data Preparation

[0094] Feature extraction: extract multiple features from the initially screened text fragments for subsequent machine learning model training and evaluation;

[0095] Lexical features: Statistical word frequency, inverse document frequency (TF-IDF), etc., to reflect the importance of the word in the text segment; for example, a text segment containing more words related to the keywords required for citation and with a higher TF-IDF value may be more in line with the requirements;

[0096] Syntactic features: Analyze the syntactic structure of sentences, such as sentence length, sentence type (declarative, interrogative, exclamatory, etc.), grammatical structure complexity (such as the number of clauses included), etc. For example, some citation requirements may prefer complete and complex declarative sentences, while the demand for simple interrogative sentences is lower;

[0097] Semantic features: Use pre-trained word vector models (such as Word2Vec, GloVe, etc.) to convert the words in the text fragment into vectors, and then calculate the semantic vector representation of the text fragment; the semantic relevance can be measured by calculating the similarity (such as cosine similarity) between the text fragment vector and the citation demand vector (the vector generated based on the citation demand keywords);

[0098] Label annotation: Label each initially screened text segment to indicate its degree of match with the citation requirement. Binary classification labels can be used, such as "match" (indicating that it highly meets the citation requirement) and "mismatch" (indicating that it does not meet or partially meets the citation requirement). Label annotation can be done by manually annotating a small number of samples, and then using semi-supervised learning or active learning methods to expand the annotated data.

[0099] Alternatively, multiple classification labels, such as "high match", "medium match", "low match", etc., can be used to more finely divide the degree of match between text fragments and citation requirements;

[0100] B. Model Training

[0101] Select the support vector machine (SVM) model: SVM is a commonly used binary or multi-classification machine learning model, which is suitable for text classification problems. It divides samples of different categories by finding an optimal hyperplane to maximize the interval between samples of different categories. In this system, SVM can train the model based on the extracted features and annotated labels to learn the relationship between the features of the text segment and the matching degree of the citation requirements.

[0102] Model training process: Divide the text snippet dataset with features and labels into a training set and a validation set; usually, 70%-80% of the data is used for training, and 20%-30% of the data is used for validation; Use the training set data to train the SVM model, and determine the model parameters (such as support vectors, hyperplane parameters, etc.) by optimizing the objective function (such as minimizing structural risk); During the training process, use cross-validation and other techniques to evaluate the performance of the model on the validation set, such as accuracy, recall rate, F1 value, etc.; Adjust the model parameters (such as kernel function type, penalty parameters, etc.) according to the evaluation results to improve the performance of the model;

[0103] C. Accuracy evaluation and screening

[0104] Text fragment evaluation: For each text fragment initially screened, its features are input into the trained SVM model, and the model will output the probability or score of the text fragment belonging to different categories; for example, for a binary classification model, a probability value indicating that it belongs to the "match" category is output; if the probability value is greater than a preset threshold (such as 0.5), the text fragment is considered to be highly matched with the citation requirement; otherwise, the matching degree is considered to be low; for a multi-classification model, based on the output scores of different categories, the category with the highest score is selected as the predicted category of the text fragment, and then based on the definition of the matching degree between the category and the citation requirement, it is determined whether the text fragment is highly matched with the citation requirement;

[0105] Further screening: Based on the evaluation results, select text segments predicted to be highly matched to the citation requirements as candidates for further screening; at the same time, consider the relevance and coherence between text segments; this can be measured by calculating the semantic similarity between text segments or the relevance based on text structure; for example, calculate the cosine similarity between two text segment vectors, and if the similarity is higher than a certain threshold (such as 0.6), they are considered to have a high relevance; or analyze the positional relationship of text segments in the original text, if they are in adjacent paragraphs or chapters and the topics are related, they are also considered to have relevance;

[0106] For isolated text fragments (i.e., fragments with low relevance to other candidate fragments), even if they highly match the citation requirements themselves, they may be excluded to ensure that the finally extracted citation content is semantically coherent and complete, can form an organic whole, and better meet the citation requirements;

[0107] The optimization module processes the extracted citation content according to the citation format specifications determined by the requirements analysis module, adds necessary information and adjusts the typesetting format, and also uses text summarization technology to streamline the content, ensuring that the streamlined content is complete, logically coherent, and meets the streamlining standards through various means;

[0108] According to the citation format specifications corresponding to the application scenarios determined by the citation requirements analysis module, format processing is performed on the extracted citation content. Specifically: for academic paper citations, according to the requirements of specific academic journals, citation numbers, author names, publication years, and page numbers are added, and the typesetting format of the text, font, font size, and indentation are adjusted;

[0109] The text summarization technology (such as the abstract generation algorithm based on the attention mechanism) is used to streamline the citation content, remove redundant information, and retain the core semantics; during the streamlining process, the integrity and logic of the citation content are ensured, and semantic loss or distortion caused by excessive streamlining is avoided; after streamlining, the streamlining degree of the citation content is measured by calculating the streamlining ratio of the citation content, through the formula: The content streamlining ratio Ratio is obtained, where L c is the length of the streamlined citation content (calculated by the number of words or characters), and L o is the length of the original citation content; the obtained content streamlining ratio Ratio is compared with the preset content streamlining ratio standard. When the content streamlining ratio Ratio is greater than the content streamlining ratio standard, it is judged that the content streamlining meets the standard. Otherwise, it is judged that the content streamlining does not meet the standard, and then, on the premise of ensuring the integrity and logic of the citation content, redundant information is further removed until the content streamlining meets the standard; specifically:

[0110] Streamlining based on lexical and grammatical rules: Identify and delete redundant words, scan the citation content again to find words that appear continuously or repeatedly in similar positions; analyze function words (such as "de", "le", "ne", etc.) and auxiliary words (such as "to a certain extent", "basically", etc.) in the sentence, and delete them without affecting the core semantics of the sentence; find words with the same or similar semantics, and replace longer or more complex words with more concise synonyms; simplify the sentence structure, merge short sentences with closely related semantics; decompose complex long sentences into several simple sentences, and then remove the repeated or redundant parts;

[0111] Simplification based on semantic analysis: extract the core semantics, use the keyword extraction algorithm in natural language processing technology to determine the keyword in the quoted content; extract key information, identify the key information units in the sentence, such as events, actions, objects, etc.; divide the quoted content into smaller semantic blocks, and calculate the semantic similarity between these semantic blocks; merge the semantic blocks whose semantic similarity reaches the threshold or select the most representative one to retain;

[0112] Simplification based on text summarization technology: When using a summary generation algorithm based on the attention mechanism, re-run the algorithm and adjust the parameters to achieve deeper simplification. By re-adjusting the attention weight, the core content can be extracted more accurately. For statistical-based summary algorithms (such as the word frequency-inverse document frequency algorithm), the word weights in the quoted content can be re-analyzed, and the words with high weights and the sentences they are in can be further screened out for retention. For graph-based summary algorithms, the relationship graph between sentences can be reconstructed, and key sentences can be determined for retention through methods such as graph centrality analysis to achieve further simplification of the content.

[0113] The citation output module converts the optimized citation content into a user-specified format that can be inserted into document editing software or generated as an independent document. It also adds metadata such as the citation source, location, and credibility score. It comprehensively evaluates credibility based on factors such as the authority of the original text, the relevance of the citation content to the topic, and the timeliness of the citation content, helping users judge the reliability and value of the citation.

[0114] Convert the optimized citation content to the output format specified by the user, such as a format that can be directly inserted into a document editing software (Word, Latex), or generate an independent citation document (such as .txt, .html format); add metadata information to the output citation content, including detailed information on the citation source (such as the file name of the long text, the source URL, the chapter title in the original text, etc.), the location information of the citation content in the original text (such as the starting paragraph number, the starting line number, the ending paragraph number, the ending line number, etc.) and the credibility score of the citation content (based on a comprehensive evaluation of factors such as the authority of the original text and the relevance of the citation content to the topic); after determining the output citation content, evaluate the credibility of the output content, specifically by analyzing it through a comprehensive analysis of different evaluation factors, including:

[0115] Authoritative factors of original text: determined by journal impact factor score, author reputation score and institutional authority score respectively. After normalization, a cone model is established with the journal impact factor score as the base circle radius and the author reputation score as the height. The vertex of the cone model is selected as the center of the sphere, and the institutional authority score is used as the sphere diameter to establish a spherical model. The volume of the heteromorphic model formed by the cone model and the spherical model is calculated and recorded as the weighted judgment value. This weighted judgment value is used as the standard for measuring the authoritative factors of the original text; Factors related to the relevance of the cited content and the topic: comprehensive analysis is performed through the vocabulary matching score and the semantic similarity score, and normalization is performed. The relevant judgment value is obtained by weighted summation, and this relevant judgment value is used as a standard to measure the relevance factor between the cited content and the topic; the timeliness factor of the cited content: analyzed by time difference and timeliness score, the time difference is calculated by calculating the time difference between the publication time of the cited content and the current time, the timeliness score is scored by scoring the time difference, and several time difference intervals are preset. Different time difference intervals correspond to different timeliness scores. The time difference interval to which the current time difference belongs is determined, and the timeliness score corresponding to the time difference is obtained; the obtained time difference and timeliness score are marked as st and sp respectively, and after normalization, they are entered into the following formula: The time judgment value SYZ is obtained, where τ is the correction factor, and the value is set between 0.95-1.23; and the time judgment value is used as a standard to measure the timeliness factor of the cited content;

[0116] The obtained weighted judgment value and related judgment value are calibrated as QPZ and XGP respectively, and after normalization with the time judgment value SYZ, they are entered into the following formula: KXP=QPZ×λ1+XGP×λ2+SYZ×λ3 to obtain the credibility evaluation value KXP, where λ1, λ2, and λ3 are the preset weight coefficients of the weighted judgment value, the related judgment value, and the time judgment value, respectively. This credibility evaluation value KXP is used as the credibility evaluation standard for measuring the output content; based on this, a comprehensive credibility evaluation is provided for the output cited content to help users better judge the reliability and value of the cited content.

[0117] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. Long text intelligent output citation system, characterized by: include: The long text processing module receives long texts in various formats and converts them into internal formats. It determines the core subject areas, sentence structure relationships, and constructs semantic maps through vocabulary statistics, syntactic analysis, and semantic recognition, providing a text analysis basis for subsequent citations. The reference demand analysis module receives reference demand information through the interactive interface, analyzes and classifies it, and converts it into system instructions and parameters to ensure that the system understands user needs; The citation extraction module, based on the screening conditions determined by the demand analysis module, preliminarily screens text fragments and evaluates relevance based on the semantic map and text structure, and then uses machine learning algorithms to accurately evaluate and select matching and relevant and coherent citation content; The optimization module processes the extracted citation content according to the citation format specifications determined by the demand analysis module, adds information and adjusts the typesetting format, and uses text summarization technology to simplify the content, ensuring that the simplified content is complete, logically coherent, and meets the simplification standards through various means; The citation output module converts the optimized citation content into the user-specified format, inserts it into document editing software or generates an independent document, and adds the citation source, location and credibility score metadata. It comprehensively evaluates the credibility based on the authority of the original text, the relevance of the citation content to the topic, and the timeliness of the citation content, to assist users in judging the reliability and value of the citation.

2. The long text intelligent output citation system according to claim 1, characterized in that: The execution steps of the long text processing module are as follows: Receive long text data in multiple formats and convert it into the internal text format processed by the system; Conduct in-depth analysis of imported long texts, including vocabulary statistics, counting the frequency of occurrence of different words, and identifying high-frequency professional words to determine the core subject areas of the text; In the process of determining high-frequency professional vocabulary, the importance weight of the vocabulary in the text is determined by the formula: Get vocabulary weight TF-IDF j , where f ij represents the frequency of occurrence of word i in document j, n is the total number of words in document j, N is the total number of documents in the document collection, df i is the number of documents containing word i; The obtained vocabulary weight TF-IDF ij Compare with the preset weight threshold, when the vocabulary weight TF-IDF j When it is greater than the preset weight threshold, the word is judged to be a high-frequency professional word; Syntactic analysis: analyzing the sentence structure of the text and determining the relationship between sentence components, providing a structural basis for subsequent semantic understanding and reference extraction; The semantic recognition submodule uses natural language processing technology and pre-trained semantic models to identify semantic units in the text and construct a semantic map of the text.

3. The long text intelligent output citation system according to claim 1 is characterized in that: The specific operation steps of the reference demand analysis module are as follows: Receive the citation requirement information input by the user through the interactive interface, including the purpose of the citation, the type of the citation text, and the application scenario of the citation; Parse and classify the received reference requirements and convert them into instructions and parameters recognized by the system. The specific process is as follows: Analysis for the purpose of citation: When the purpose of the user's citation is to prove a certain point, extract the keywords in the point of view, convert these keywords into semantic search conditions, and map these keywords to predefined semantic tags or concepts; construct text screening conditions based on these semantic tags and concepts, including finding sentences or paragraphs containing these keywords or semantically related words in long texts, and these contents have the nature of argumentation; When the purpose of citing is to serve as a comparative case, analyze the comparative objects involved by the user, establish corresponding text features for each comparative object, and convert these features into screening conditions to extract the comparative content from the long text; When citations are used to supplement background knowledge, identify the topics of the background knowledge needed and translate these concepts into search terms for chronological, historical event-related text features in long texts; Parsing of quoted text types: When the user specifies a quoted paragraph or chapter, the paragraph number or chapter title information entered by the user is parsed, and the corresponding index or positioning parameters are established to extract the content of the chapter or paragraph from the long text; For different types of sentences, identify the characteristics of data description sentences, build screening conditions based on these characteristics, and for opinion elaboration sentences, analyze the language patterns of opinion elaboration sentences; Parsing of citation application scenarios: When the citation application scenario is an academic paper, call the predefined academic paper citation format template, including the typesetting format of the author's name, publication year, paper title, journal name, volume number, and page number information; convert these templates into codes or parameters to format the citation content when the citation is output; For the citation scenario of technical reports, identify the characteristics of the citation format of technical reports, including the report author, report number, publishing organization, and publishing year information, and convert these format characteristics into corresponding format codes; In the citation scenario of literary creation, the format of literary citation is analyzed, including chapters, paragraphs, and character dialogues in the quoted works; the citation format requirements in these literary works are converted into codes or parameters executed by the system to correctly format the quoted content.

4. The long text intelligent output citation system according to claim 1 is characterized in that: The specific operation steps of the reference extraction module are as follows: According to the screening conditions determined by the citation demand analysis module, based on the semantic graph and text structure constructed by the long text processing module, the text fragments that meet the requirements are preliminarily screened out; The relevance score is obtained by evaluating the relevance of the text fragment, using the formula: To get the relevance score Score(s) of the text segment s, w j is the weight of the jth feature, f j (s) is the score function of the text segment s for the jth feature, and k is the number of features; the text segments whose relevance score Score(s) is greater than the preset score threshold are used as the preliminary screening criteria for meeting the requirements; Use machine learning algorithms to evaluate the accuracy of the initially screened text fragments, and based on the evaluation results, screen out text content that matches the citation requirements again; at the same time, analyze the relevance and coherence between text fragments to avoid extracting isolated, semantically incoherent citation content.

5. The long text intelligent output citation system according to claim 4, characterized in that: The specific process of using machine learning algorithms to evaluate the accuracy of the initially screened text fragments and re-screen them is as follows: A. Data preparation: Extract multiple features from the initially selected text fragments for subsequent machine learning model training and evaluation; count the word frequency and inverse document frequency of vocabulary to reflect the importance of vocabulary in the text fragment; analyze the syntactic structure of sentences, including sentence length, sentence type, and grammatical structure complexity; use the pre-trained word vector model to convert the vocabulary in the text fragment into vectors, and then calculate the semantic vector representation of the text fragment, and measure the semantic relevance by calculating the cosine similarity between the text fragment vector and the citation demand vector; label each initially selected text fragment to indicate its matching degree with the citation demand; B. Model training: Select the support vector machine model and find an optimal hyperplane to divide samples of different categories to maximize the interval between samples of different categories; train the model based on the extracted features and annotated labels to learn the relationship between the features of the text fragment and the matching degree of the citation requirements; Divide the text snippet dataset with features and labels into a training set and a validation set; use the training set data to train the model and determine the model parameters by optimizing the objective function; during the training process, use cross-validation technology to evaluate the performance of the model on the validation set; adjust the model parameters based on the evaluation results to improve the performance of the model; C. Accuracy evaluation and screening: For each text segment initially screened, its features are input into In the trained model, the probability or score of the text fragment belonging to different categories is output; for the multi-classification model, according to the output scores of different categories, the category with the highest score is selected as the predicted category of the text fragment, and then according to the definition of the degree of matching between the category and the citation requirement, it is determined whether the text fragment highly matches the citation requirement; Second screening: Based on the evaluation results, select text fragments that are predicted to match the citation requirements and meet the standards as candidates for further screening; at the same time, analyze the relevance and coherence between text fragments, and measure them by calculating the semantic similarity between text fragments or the relevance based on text structure; for isolated text fragments, even if they themselves match the citation requirements and meet the standards, they are excluded to ensure that the final extracted citation content is semantically coherent and complete, and can form an organic whole.

6. The long text intelligent output citation system according to claim 1, characterized in that: The specific operation steps of the optimization module are as follows: According to the reference format specification corresponding to the application scenario determined by the reference requirement analysis module, the extracted reference content is formatted, specifically: For academic paper citations, add the reference number, author name, publication year, page number information according to the requirements of academic journals, and adjust the text layout format, including font, font size, and indentation; Use text summarization technology to simplify the quoted content, remove redundant information, and retain the core semantics; in the process of simplifying, ensure the integrity and logic of the quoted content; After simplification, the simplification ratio of the cited content is calculated to measure the simplification degree of the cited content, using the formula: Get the content reduction ratio Ratio, where L c is the length of the condensed quoted content, L o is the length of the original quoted content; the obtained content reduction ratio Ratio is compared with the preset content reduction ratio standard. When the content reduction ratio Ratio is greater than the content reduction ratio standard, it is judged that the content reduction has reached the standard. Otherwise, it is judged that the content reduction has not reached the standard. Under the premise of ensuring the integrity and logic of the quoted content, redundant information is removed again until the content reduction reaches the standard.

7. The long text intelligent output citation system according to claim 6, characterized in that: The optimization module removes redundant information again until the content is streamlined to meet the standard. The specific operation steps are as follows: Simplification based on vocabulary and grammar rules: identify and delete redundant words, re-scan the quoted content, and find words that appear continuously or repeatedly in similar positions; analyze the function words and auxiliary words in the sentence and delete them without affecting the core semantics of the sentence; Find words with the same semantics and replace words that exceed the threshold with concise synonyms; simplify sentence structure and merge semantically related short sentences; break down complex long sentences into several simple sentences and then remove repeated or redundant parts; Simplification based on semantic analysis: extract the core semantics and use the keyword extraction algorithm in natural language processing technology to determine the keyword in the quoted content; Extract key information, identify key information units in the sentence; divide the quoted content into several semantic blocks, calculate the semantic similarity between these semantic blocks; merge or select one of the semantic blocks that reach the threshold of semantic similarity to be retained; Simplification based on text summarization technology: When using an attention-based summary generation algorithm, rerun the algorithm and readjust the attention weights; For statistical summary algorithms, the word weights in the quoted content are re-analyzed, and the words and sentences containing words with weights greater than the preset weights are selected and retained; For graph-based summary algorithms, the relationship graph between sentences is reconstructed, and the sentences to be retained are determined through the centrality analysis method of the graph, so as to further streamline the content.

8. The long text intelligent output citation system according to claim 1 is characterized in that: The specific operation steps of the reference output module are as follows: Convert optimized citation content into a user-defined output format, including formats for direct insertion into document editing software, or to generate a stand-alone citation document; Add metadata information to the output citation content, including detailed information about the citation source, the location of the citation content in the original text, and the credibility score of the citation content; The credibility score of the cited content is obtained by evaluating the credibility of the output content. Specifically, the credibility evaluation value is obtained by analyzing different evaluation factors comprehensively. Based on the credibility evaluation value, a comprehensive credibility evaluation is provided for the output cited content to help users judge the reliability and value of the cited content.

9. The long text intelligent output citation system according to claim 8, characterized in that: The analysis steps of the credibility evaluation value in the reference output module are as follows: Evaluation factors include: Authoritative factors of original text: determined by the journal impact factor score, author reputation score and institution authority score respectively. After normalization, a cone model is established with the journal impact factor score as the base circle radius and the author reputation score as the height. The vertex of the cone model is selected as the center of the sphere, and the institution authority score is used as the sphere diameter to establish a spherical model. The volume of the heteromorphic model formed by the cone model and the spherical model is calculated and recorded as the weighted judgment value, which is used as the standard for measuring the authoritative factors of the original text; Factors related to the relevance of the cited content and the topic: A comprehensive analysis is conducted through the vocabulary matching score and the semantic similarity score. The normalization process is performed through weighted summation to obtain the relevant judgment value, which is then used as the standard for measuring the relevance of the cited content and the topic. Timeliness factor of cited content: analyzed through time difference and timeliness score. Time difference is calculated by calculating the time difference between the publication time of the cited content and the current time. Timeliness score is scored by scoring the time difference. Several time difference intervals are preset. Different time difference intervals correspond to different timeliness scores. Determine the time difference interval to which the current time difference belongs, and then obtain the timeliness score corresponding to the time difference. The obtained time difference and timeliness score are then marked as st and sp respectively, and after normalization, they are entered into the following formula: The time judgment value SYZ is obtained, where τ is the correction factor; and the time judgment value is used as the standard for measuring the timeliness factor of the cited content; The obtained weighted judgment value and related judgment value are calibrated as QPZ and XGP respectively, and after normalization with the time judgment value SYZ, they are entered into the following formula: KXP=QPZ×λ1+XGP×λ2+SYZ×λ3 to obtain the credibility evaluation value KXP, where λ1, λ2, and λ3 are the preset weight coefficients of the weighted judgment value, related judgment value, and time judgment value respectively.

Images