Literature quality review method based on natural language large model

Abstract

The embodiment of the invention discloses a literature quality review method based on a natural language large model. The method comprises the following steps: establishing a comment forbidden word bank; collecting literatures over the years to form a literature database; using the natural language large model to generate a second literature item evaluation opinion of literatures over the years; filtering the first literature subitem evaluation opinions by using a comment forbidden word bank to obtain literature subitem evaluation opinions based on expert review; filtering the second literature subitem evaluation opinions by using a comment forbidden lexicon to obtain literature subitem evaluation opinions based on a natural language large model; generating a training sample set by utilizing the literature item evaluation opinions reviewed by experts and the literature item evaluation opinions based on the natural language large model; training the text semantic similarity model by using the training sample set to obtain a text semantic similarity calculation tool; and reviewing the to-be-reviewed literature by using the text semantic similarity calculation tool, and obtaining the literature item evaluation opinions of the to-be-reviewed literature by using the natural language large model.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing. More specifically, it relates to a method for reviewing the quality of documents based on a large natural language model. Background Art

[0002] Currently, as Chinese universities, research institutions, enterprises and other units continue to promote the work of document quality monitoring, national ministries and commissions also advocate the selection of document review experts, manual review and spot checks on some systematic platforms. Along with the sharp increase in the number of college graduates and scientific and technological workers every year, experts in various fields across the country have to invest a considerable amount of time and energy in the document review work of graduates and researchers. Especially with the significant expansion of the number of undergraduate students in national universities currently, quite a number of these students and some in-service scientific researchers are not good at writing documents. The problems are prominently manifested as follows: on the one hand, the majority of students and in-service personnel do not accurately grasp the writing requirements of documents and cannot truly meet the requirements of document review in terms of document topic selection, innovation, basic knowledge and scientific research capabilities, etc.; on the other hand, many students and in-service personnel make basic mistakes in document standardization, including spelling mistakes, document formats, ungrammatical sentences, unclear language expressions, etc., and these obvious common problems are particularly prominent. Experts in various fields across the country spend a lot of time and effort, are extremely troubled and inefficient in the process of reviewing this part of the documents, resulting in a great waste of the originally precious expert resources.

[0003] In recent years, many text processing methods have emerged in the field of natural language processing. Typical ones are ChatGPT and Bert, etc. This kind of fine-tunable pre-trained model based on Transformer, hybrid model combined with attention mechanism, convolutional neural network, self-attention mechanism and efficient parallel processing ability have shown excellent effects in aspects such as question-and-answer and document abstract extraction.

[0004] Therefore, how to apply natural language processing technology to document review work to reduce the workload is an urgent problem to be solved. Summary of the Invention

[0005] The purpose of the present invention is to provide a method for reviewing the quality of documents based on a large natural language model to solve at least one of the problems existing in the prior art.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] The first aspect of the present invention provides a method for reviewing the quality of documents based on a large natural language model, including:

[0008] Establish a library of prohibited words for comments;

[0009] Collect the basic information, expert comments, and scoring records of previous years' literature to form a literature database, where the literature database includes the first sub - evaluation opinions of the literature;

[0010] Use a natural language large model to generate the second sub - evaluation opinions of the previous years' literature;

[0011] Filter the first sub - evaluation opinions of the literature using the comment prohibited word library to obtain the sub - evaluation opinions of the literature based on expert review;

[0012] Filter the second sub - evaluation opinions of the literature using the comment prohibited word library to obtain the sub - evaluation opinions of the literature based on the natural language large model;

[0013] Generate a training sample set using the sub - evaluation opinions of the literature based on expert review and the sub - evaluation opinions of the literature based on the natural language large model;

[0014] Use the training sample set to train a text semantic similarity model to obtain a text semantic similarity calculation tool;

[0015] Use the text semantic similarity calculation tool to review the literature to be reviewed, and use the natural language large model to obtain the sub - evaluation opinions of the literature to be reviewed.

[0016] Optionally,

[0017] The method further includes:

[0018] Optimize the natural language large model using the sub - evaluation opinions of the literature based on expert review and the low - rank adaptation algorithm to obtain an optimized natural language large model;

[0019] Use the optimized natural language large model to generate the second sub - evaluation opinions of the previous years' literature.

[0020] Optionally, the filtering the first sub - evaluation opinions of the literature using the comment prohibited word library to obtain the sub - evaluation opinions of the literature based on expert review includes:

[0021] Obtain the first sub - evaluation opinions of each literature in the literature database;

[0022] Segment the first sub - evaluation opinions of the literature based on punctuation marks and conjunctions to obtain a short - sentence sequence;

[0023] Judge whether there is a prohibited word in the comment prohibited word library in the short - sentence sequence. If so, the short - sentence sequence is invalid text content;

[0024] Delete the invalid text content in the first literature sub - evaluation opinion to obtain the literature sub - evaluation opinion based on expert review.

[0025] Optionally, before filtering the first literature sub - evaluation opinion using the comment disabling word library, the method further includes: obtaining the first literature sub - evaluation opinions of each literature in the literature database;

[0026] Segment the first literature sub - evaluation opinion based on punctuation marks to obtain a short - sentence sequence;

[0027] Use a text similarity detection tool to judge the similarity between short - sentence sequences. When the similarity between multiple short - sentence sequences is greater than or equal to the first threshold, retain the first - appearing short - sentence sequence among the multiple short - sentence sequences in the first literature sub - evaluation opinion and delete the remaining short - sentence sequences among the multiple short - sentence sequences.

[0028] Optionally, before filtering the first literature sub - evaluation opinion using the comment disabling word library, the method further includes:

[0029] When the first literature sub - evaluation opinion is missing, judge whether there are keywords in the academic comment; if so, copy the sentence where the keyword is located as the first literature sub - evaluation opinion;

[0030] If not, use the natural language large - model to obtain the first literature sub - evaluation opinion according to the academic comment.

[0031] Optionally, the review of the literature to be reviewed using the text semantic similarity calculation tool includes:

[0032] Load the literature database and the literature to be reviewed in a system platform including the text semantic similarity calculation tool;

[0033] Use the literature database to filter the literature to be reviewed to obtain a first candidate literature range;

[0034] Match and screen the literature to be reviewed and the first candidate literature range according to the second threshold to obtain a second candidate literature range;

[0035] Obtain the abstract of the literature to be reviewed to get a second abstract data set, obtain the chapter summaries of the literature to be reviewed and the summary and outlook of the literature to be reviewed to get a second summary data set, perform text error correction on the literature to be reviewed to get a second text error correction data set, and use the natural language large - model to obtain a third literature sub - evaluation opinion according to the second abstract data set, the second summary data set and the second text error correction data set;

[0036] Filter the third literature item evaluation opinions using the above-mentioned evaluation term ban library to obtain the literature item evaluation opinions of the literature to be reviewed based on the large natural language model;

[0037] Screen and rank the literature item evaluation opinions of the literature to be reviewed based on the large natural language model and the literature item evaluation opinions of each literature in the second candidate literature range according to the third threshold to obtain the third candidate literature range;

[0038] Obtain the literature item evaluation score of the literature to be reviewed according to the third candidate literature range;

[0039] Summarize the literature item evaluation opinions of the literature to be reviewed to obtain the academic evaluation of the literature to be reviewed, and filter the academic evaluation using the above-mentioned evaluation term ban library to obtain the target academic evaluation;

[0040] Perform arithmetic averaging and rounding on the literature item evaluation scores of the literature to be reviewed to obtain the overall score of the literature to be reviewed;

[0041] Judge whether the overall score of the literature to be reviewed is less than the fourth preset value. If it is less than, the literature to be reviewed is evaluated as unqualified, and the literature item evaluation opinions based on the large natural language model corresponding to the literature item evaluation scores of the literature to be reviewed that are less than the fourth preset value are integrated to obtain the deficiencies and suggestions of the literature to be reviewed.

[0042] Optionally, the filtering the literature to be reviewed using the literature database to obtain the first candidate literature range includes:

[0043] Match the first-level discipline, second-level discipline, degree type, and study type of the literature to be reviewed with the first-level discipline, second-level discipline, degree type, and study type of each literature in the literature database respectively, and obtain the first candidate literature range from the literature database according to the matching results.

[0044] Optionally, the matching and screening of the literature to be reviewed and the first candidate literature range according to the second threshold to obtain the second candidate literature range includes:

[0045] Calculate the similarity between the research directions of each literature in the first candidate literature range and the research direction of the literature to be reviewed, and screen the literatures with similarity greater than or equal to the second threshold to obtain the second candidate literature range.

[0046] Optionally, the screening and ranking of the literature item evaluation opinions of the literature to be reviewed based on the large natural language model and the literature item evaluation opinions of each literature in the second candidate literature range according to the third threshold to obtain the third candidate literature range includes:

[0047] Calculate the similarity between the literature item evaluation opinions of the literature to be reviewed based on the natural language large model and the literature item evaluation opinions of each literature in the second candidate literature range based on expert review, screen the literatures with similarity greater than or equal to the third threshold and sort them in descending order to obtain the third candidate literature range.

[0048] Optionally, the obtaining the literature item evaluation score of the literature to be reviewed according to the third candidate literature range includes:

[0049] Obtain the literature item evaluation scores of the first M literatures in the third candidate literature range based on expert review, perform arithmetic mean and rounding, and use it as the literature item evaluation score of the literature to be reviewed based on the natural language large model, where M is an integer greater than zero;

[0050] Or when the number of literatures in the third candidate literature range is less than M, perform arithmetic mean and rounding on the literature item evaluation scores of each literature in the third candidate literature range, and use it as the literature item evaluation score of the literature to be reviewed based on the natural language large model;

[0051] Or when the number of literatures in the third candidate literature range is zero, set the literature item evaluation score of the literature to be reviewed based on the natural language large model to the first preset value.

[0052] The beneficial effects of the present invention are as follows:

[0053] The technical solution of the present invention, by introducing a natural language large model and a text semantic similarity model in the literature review process, can quickly complete the initial screening of literature reviews, return unqualified literatures to the authors for modification in a timely manner, greatly reduce the original inefficient and simple manual workload, enable the majority of experts to invest time and energy in more socially valuable work such as the review of literature innovation, and can effectively improve efficiency, enhance review quality, reduce the inefficient repetitive labor of experts, and reduce human bias. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The following further describes in detail the specific embodiments of the present invention with reference to the drawings.

[0055] Figure 1 Show a flowchart of the literature quality review method based on the natural language large model provided by the embodiment of the present invention.

[0056] Figure 2 Show a model training flowchart of the literature quality review method based on the natural language large model provided by the embodiment of the present invention.

[0057] Figure 3The flowchart of the literature review for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown.

[0058] Figure 4 The flowchart of the text filtering based on the comment disabling word library for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown.

[0059] Figure 5 The flowchart of the conditional filtering for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown.

[0060] Figure 6 The flowchart of generating sub-item opinion comments for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown.

[0061] Figure 7 The flowchart of calculating similarity for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown.

[0062] Figure 8 The flowchart of scoring the sub-item evaluation opinions for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown.

[0063] Figure 9 The flowchart of the overall score and review result of the literature to be reviewed for the literature quality review method based on a natural language large model provided by an embodiment of the present invention is shown. Detailed implementation manners

[0064] To illustrate the present invention more clearly, the present invention will be further described below in conjunction with embodiments and the accompanying drawings. Similar components in the drawings are denoted by the same reference numerals. Those skilled in the art should understand that the content described specifically below is illustrative rather than restrictive, and should not be used to limit the protection scope of the present invention.

[0065] Currently, there are some problems in literature review. On the one hand, the majority of students and working people have an inaccurate grasp of the writing requirements of the literature and cannot truly meet the literature review requirements in terms of literature topic selection, innovation, basic knowledge, and scientific research ability. On the other hand, many students and working people make basic mistakes in the literature standardization, including spelling mistakes, document formats, ungrammatical sentences, and unclear language expressions, etc. These obvious common problems are particularly prominent. Experts in various fields across the country spend a lot of time, rack their brains, and have low efficiency in reviewing this part of the literature, resulting in a great waste of the originally precious expert resources.

[0066] In recent years, many text processing methods have emerged in the field of natural language processing. Typical examples include ChatGPT and Bert. Such pre-trained models based on Transformer that can be fine-tuned, hybrid models combined with attention mechanisms, convolutional neural networks, self-attention mechanisms, and efficient parallel processing capabilities have shown excellent effects in question-and-answer sessions, document summary extraction, etc.

[0067] Therefore, how to apply natural language processing technology to the literature review work to reduce the workload is an urgent problem to be solved.

[0068] In view of this, an embodiment of the present invention provides a method for evaluating the quality of literature based on a large natural language model, as Figure 1 shown, including: establishing a banned comment word library; collecting the basic information, expert comments, and scoring records of previous years' literature to form a literature database, where the literature database includes the first sub-item evaluation opinions of the literature; using the large natural language model to generate the second sub-item evaluation opinions of the previous years' literature; using the banned comment word library to filter the first sub-item evaluation opinions of the literature to obtain the sub-item evaluation opinions of the literature based on expert review; using the banned comment word library to filter the second sub-item evaluation opinions of the literature to obtain the sub-item evaluation opinions of the literature based on the large natural language model; using the sub-item evaluation opinions of the literature based on expert review and the sub-item evaluation opinions of the literature based on the large natural language model to generate a training sample set; using the training sample set to train a text semantic similarity model to obtain a text semantic similarity calculation tool; using the text semantic similarity calculation tool to review the literature to be reviewed, and using the large natural language model to obtain the sub-item evaluation opinions of the literature to be reviewed.

[0069] In a specific example, the large natural language model refers to a model that can extract the abstract of a text paragraph, and the output is also text content, such as Qianwen, Zhipu, etc.

[0070] In a specific example, the text semantic similarity model is the SBert model.

[0071] This embodiment can quickly complete the initial screening of literature reviews by introducing a large natural language model and a text semantic similarity model in the literature review process, return unqualified literature to the author for modification in a timely manner, greatly reduce the original inefficient and simple manual workload, enable the majority of experts to devote their time and energy to more socially valuable work such as the review of the innovation of literature, and can effectively improve efficiency, enhance the review quality, reduce the inefficient and repetitive labor of experts, and reduce human bias.

[0072] The following will further explain the present invention in conjunction with Figures 2 to 9 this.

[0073] In a possible implementation, the generation of the second sub - evaluation opinion of the annual literature using the natural language large - model includes: obtaining the first abstract data set by getting the abstracts of the annual literature, obtaining the first summary data set by getting the summaries of each chapter of the annual literature and the summary and outlook of the annual literature, performing text error correction on the annual literature to obtain the first text error - corrected data set, and obtaining the second sub - evaluation opinion of the literature by using the natural language large - model based on the first abstract data set, the first summary data set, and the first text error - corrected data set;

[0074] The generation of the training sample set using the sub - evaluation opinion of the expert review and the sub - evaluation opinion of the literature based on the natural language large - model includes: obtaining the positive sample data set by using the sub - evaluation opinion of the expert review and the sub - evaluation opinion of the literature based on the natural language large - model of the same historical literature, and obtaining the negative sample data set by using the sub - evaluation opinion of the expert review and the sub - evaluation opinion of the literature based on the natural language large - model of different historical literature;

[0075] The training of the semantic similarity model using the training sample set to obtain the text semantic similarity calculation tool includes: inputting the positive sample data set and the negative sample data set into the text semantic similarity model for training to obtain the text semantic similarity calculation tool.

[0076] This method further includes: optimizing the natural language large - model using the sub - evaluation opinion of the expert review and the low - rank adaptation algorithm to obtain an optimized natural language large - model; using the optimized natural language large - model to generate the second sub - evaluation opinion of the annual literature.

[0077] In a specific example, this method includes collecting the basic information of the annual literature, and combining the comments and scores, academic comments, deficiencies and suggestions of the previous experts in writing the sub - evaluation opinions of the literature to form a literature database. At the same time, using the natural language large - model technology to analyze the abstracts of the literature, the summaries of the table of contents chapters, the summary and outlook of the literature, and performing text error correction on the full text of the literature, thereby generating automatic comments on the sub - evaluation opinions of the literature.

[0078] Furthermore, pairing the comments written by experts with the comments generated by the natural language large - model technology to form a positive and negative sample set, and providing it to the SBert model for training to obtain a similarity calculation method based on the text semantic content. In addition, using the expert comment data of the annual literature, and combining the literature abstracts, the summaries of each chapter, and the summary and outlook of the literature for fine - tuning, so that the natural language large - model technology is more adapted to the literature review application scenario, and making the language style and expression mode of the generated content closer to those of industry experts.

[0079] In a specific example, basic information, expert comments, and scoring records of millions of documents over the years are collected on a relevant system platform to form a document database. As shown in Table 1, each record includes the document title, first-level discipline, second-level discipline, degree type (academic type or professional type), study category (undergraduate, master's, or doctoral), research direction of the document, expert comments and scores for each item of the document (document topic selection, innovation and document value, basic knowledge and scientific research ability, and document standardization), academic comments, deficiencies, and suggestions. This database is denoted as DataBase.

[0080] Table 1 Document Database Fields

[0081]

[0082] It should be noted that the following points included in the evaluation opinions for document items all have clear evaluation indicators.

[0083] (1) Document topic selection, the forefront and openness of the topic selection; the theoretical and practical significance of the research; the summary of the development status of this topic and related fields at home and abroad.

[0084] (2) Innovation and document value, whether new laws are discovered; whether new propositions and new methods are proposed; whether it plays a role in solving important problems in humanities, social sciences, natural sciences, or engineering technologies. The impact and contribution of the document and its achievements to scientific and technological development and social progress.

[0085] (3) Basic knowledge and scientific research ability, the solid and broad degree of the scientific theoretical foundation; the systematic and in-depth degree of specialized knowledge; the scientific nature of research methods; the authenticity and adequacy of cited materials; the ability of the author to independently engage in scientific research.

[0086] (4) Document standardization, the standardization of citations; the rigor of academic style; the logic of the structure; the accuracy and fluency of text expression.

[0087] In a specific example, for the document titles in the database, retrieve their original document locations, and extract the following abstract and table of contents information from PDF or WORD documents:

[0088] (1) For the abstract, if the Chinese abstract exists, only use the Chinese abstract; otherwise, translate the English abstract into a Chinese abstract using natural language large model technology and use it to obtain the abstract dataset.

[0089] (2) For the table of contents information, check whether there is a summary section or a chapter summary section at the end of each chapter, and check whether there is a summary and outlook section at the end of the whole document. If this information exists in the i-th chapter, extract it in sequence and record it as the summary information Summarize(i) of the i-th chapter. If the corresponding information of the i-th chapter does not exist, the summary information Summarize(i) of the i-th chapter is default set to be empty. Further, the summary dataset is obtained according to the summary information as follows:

[0090] Summarize = {Summarize(1), Summarize(2),..., Summarize(n)}

[0091] In the formula, Summarize(1) is the summary of the first chapter of the original literature corresponding to index 1; Summarize(2) is the summary of the second chapter of the original literature corresponding to index 2; Summarize(n) is the summary and outlook at the end of the whole original literature corresponding to index n.

[0092] (3) For the full text of Chinese literature, use a text detection tool (such as pycorrector) to perform full text detection to obtain error results in aspects such as common spelling and grammar usage, which is recorded as the text error correction dataset; for the full text of literature written in other languages, the text error correction dataset is set to be empty.

[0093] Further, based on the abstract dataset, the summary dataset, and the text error correction dataset, use natural language large model technology to extract automatic evaluation opinions on each item of the literature (literature topic selection, innovation and literature value, basic knowledge and scientific research ability, and literature standardization). The prompt words of the natural language large model can be designed as "According to the literature abstract, chapter summary, summary and outlook, and text error correction information, respectively summarize the general opinions on the following 4 aspects: literature topic selection, innovation and literature value, basic knowledge and scientific research ability, and literature standardization".

[0094] Further, the evaluation opinions generated by the natural language large model technology are also filtered based on the evaluation disabled word library. According to the prohibited words in the evaluation disabled word library, the evaluation opinions on each item of the literature generated by the natural language large model technology are filtered to ensure the objectivity and fairness of the evaluation opinions and a good academic atmosphere, and effectively prevent the "hallucination" phenomenon of the large model technology. Thus, the evaluation opinions on each item of the natural language large model technology are formed for each literature in the database.

[0095] Further, the expert evaluation opinions and the natural language large model technology evaluation opinions on each item of the same literature can form positive sample data pairs of the training data; while the expert evaluation opinions and the natural language large model technology evaluation opinions on each item of different literatures can form negative sample data pairs of the training data.

[0096] In a specific example, based on the above positive and negative sample training data sets, the text semantic similarity model sbert_text_similarity is used for training and fine-tuning to obtain a text semantic similarity detection tool based on the SBert model, which is denoted as SBert_YuYi_Sim. This tool can generate a vector from a text content, and the vector length is, for example, 768 dimensions. Thus, comparing the semantic similarity of two text contents is converted into calculating the similarity between two vectors. The calculation method of the similarity between vectors can be cosine similarity or the similarity probability of softmax normalization, etc. For example, the model test statistical results obtained after a certain training are shown in Table 2 as follows.

[0097] Table 2 Technical indicators of model training

[0098] Accuracy Recall Comprehensive index Number of samples Negative samples 0.7 0.18 0.29 50820 Positive samples 0.91 0.99 0.95 428482 Precision -- -- 0.91 479302 Macro-average 0.81 0.59 0.62 479302 Weighted average 0.89 0.91 0.88 479302

[0099] In a specific example, in order to further adapt to the application scenario of literature review and make the comment content generated by the natural language large model technology as close as possible to the habitual expressions and ways of speaking of industry experts, it is necessary to fine-tune the natural language large model. Here, models such as ChatGLM3-6b and ollama3.2-11b can be selected.

[0100] Furthermore, through training on the dedicated data sets of literature item evaluation opinions (literature topic selection, innovation and literature value, basic knowledge and scientific research ability, and literature standardization), academic comments, deficiencies and suggestions, the natural language large model can perform better in the specific task of literature review and improve the quality of the output content of the natural language large model. In addition, adopting the method of fine-tuning the natural language large model is significantly more efficient than training a new model from scratch. Because the pre-trained models (such as ChatGLM3-6b and ollama3.2-11b) have already learned a large amount of general knowledge, and by fine-tuning the natural language large model, these knowledge can be transferred to the literature review task, which not only reduces the training time and computational resource consumption, but also enables the natural language large model technology to have a deeper understanding and performance in the field of literature review.

[0101] In a specific example, the data format for fine-tuning natural language large models typically includes dataset formats that are open-sourced in the industry and widely used, such as the Alpaca format developed by a research team at Stanford University, which is suitable for scenarios that require detailed instructions and corresponding answers. Specifically, a user inputs an instruction, and the natural language large model outputs a corresponding answer according to the instruction. Therefore, the Alpaca format is also a kind of instruction-following format, which usually includes the following items: human instruction, optional human input, model answer, and optional system prompt.

[0102] Furthermore, for the itemized evaluation opinions of the literature (literature topic selection, innovation and literature value, basic knowledge and research ability, and literature standardization), academic comments, deficiencies and suggestions, the dataset is designed in a corresponding format.

[0103] In a specific example, the Low-Rank Adaptation (LoRA) method is used to reduce the amount of parameter updates by introducing low-rank matrix adaptation to the intermediate layer of the pre-trained model, while retaining most of the advantages of the pre-trained model. The LoRA method performs well in many tasks, especially in cases where resources are limited. Its prominent advantages include fewer parameter updates and the ability to perform effective fine-tuning with fewer computational resources.

[0104] In a possible implementation, the filtering of the first itemized evaluation opinion of the literature using the comment stop word library to obtain the itemized evaluation opinion of the literature based on expert review includes: obtaining the first itemized evaluation opinion of each literature in the literature database; splitting the first itemized evaluation opinion based on punctuation marks and conjunctions to obtain a short sentence sequence; determining whether there is a prohibited word in the comment stop word library in the short sentence sequence, and if so, the short sentence sequence is invalid text content; deleting the invalid text content in the first itemized evaluation opinion to obtain the itemized evaluation opinion of the literature based on expert review.

[0105] In a specific example, whether it is the literature comment written by an expert or the text content output by the natural language large model technology, the present invention splits the comment text content according to Chinese and English punctuation marks and some Chinese conjunctions to form a string sequence. Further, using the short sentence splitting method of Python language regular expressions, the literature comment written by an expert or the literature comment output by the natural language large model is imported into the regular library, and the comment content is split according to Chinese and English punctuation marks and Chinese conjunctions; for the short sentence sequence formed by splitting the original comment, if a certain short sentence contains a comment stop word, then delete that short sentence.

[0106] In a specific example, during the literature review process, it is sometimes difficult for some experts to avoid being interfered by factors such as personal stance, emotional control, and gains and losses, resulting in strong personal feelings in the written literature comments. To solve such problems, this embodiment sets up a prohibited word library for literature comments. Its significance is mainly reflected in the following aspects:

[0107] (1) Maintaining academic norms and objectivity, the setting of prohibited words helps to ensure the objectivity and fairness of literature reviews. By avoiding using words with emotional colors or biases, review experts can focus more on the literature content itself and reduce the interference of personal emotions or non-academic factors. For example, avoiding using derogatory words or insulting terms to ensure a more rational and objective evaluation of the literature.

[0108] (2) Preventing prejudice and discrimination, prohibited words help to eliminate possible biases such as gender, race, and religion. During the review process, if review experts use inappropriate words, they may unconsciously carry biases, affecting the fair review of the literature. Prohibited words can help avoid these problems and promote equal academic evaluation.

[0109] (3) Protecting the privacy and dignity of literature authors. If experts use inappropriate words in their comments, it may hurt the feelings of literature authors or violate their privacy. Setting prohibited words can ensure that reviewers' language is more professional and respectful of literature authors, avoiding verbal violence or personal attacks.

[0110] (4) Improving the review quality, the setting of prohibited words helps to enhance the quality and professionalism of reviews. In academic reviews, reviewers are required to focus on the scientificity and innovation of the literature and avoid using words unrelated to the literature content or politically sensitive topics, thus ensuring the constructiveness and effectiveness of review feedback.

[0111] (5) Promoting a healthy atmosphere for academic exchanges. The setting of prohibited words can help maintain the rationality and constructiveness of academic discussions and prevent the academic atmosphere from being affected by overly radical or extreme views. A healthy and rational academic exchange can stimulate more innovative thinking and promote the progress of academic research.

[0112] (6) Meeting policy or legal requirements. The setting of prohibited words is also to meet the requirements of relevant policies or laws. With the increasing number of foreign students coming to China, especially in cross-border or cross-cultural academic exchanges, certain words may be considered politically sensitive or cause cultural conflicts, so special attention needs to be paid.

[0113] (7) Avoiding the unpredictability of the output results of large model technologies. Due to certain unpredictable and uncontrollable factors in the output content of natural language large model technologies, the establishment of a prohibited word library for comments can be used to make up for the deficiencies of existing large model technologies.

[0114] In this embodiment, by setting up a prohibited word library for literature reviews, on the one hand, it prevents the influence of human factors of reviewers, maintains academic integrity, promotes a fair and just review process, ensures a healthy and rational academic environment, and avoids unnecessary disputes or conflicts; on the other hand, it can effectively avoid the "hallucination effect" of natural language large model technology and prevent it from outputting bad content that goes against the academic fairness and healthy atmosphere and the requirements of policies and regulations.

[0115] In a specific example, the filtering based on the prohibited word library includes: filtering the itemized evaluation opinions, academic reviews, deficiencies and suggestions written by experts for the literature, as well as the itemized evaluation alternative opinions summarized by large model technology, according to the prohibited words in the prohibited word library, to ensure the objectivity, fairness and good academic atmosphere of the review opinions.

[0116] In this embodiment, by establishing a prohibited word library for reviews, whether it is the expert reviews used in the model training stage or the automatic reviews generated by natural language large model technology, they are filtered according to the prohibited word library, and the short sentences containing prohibited words are deleted, which not only ensures the objectivity, fairness and good academic atmosphere of the review opinions, but also effectively prevents the occurrence of the "hallucination" phenomenon of natural language large model technology.

[0117] In a possible implementation manner, before filtering the first itemized evaluation opinion of the literature by using the prohibited word library, the method further includes: obtaining the first itemized evaluation opinions of each literature in the literature database; splitting the first itemized evaluation opinions based on punctuation marks to obtain a short sentence sequence; using a text similarity detection tool to judge the similarity between each short sentence sequence, and when the similarity between multiple short sentence sequences is greater than or equal to a first threshold, retaining the first-occurring short sentence sequence among the multiple short sentence sequences in the first itemized evaluation opinion and deleting the remaining short sentence sequences among the multiple short sentence sequences.

[0118] In a specific example, check the content of the itemized evaluation opinions of the literature in the database (literature topic, innovation and literature value, basic knowledge and scientific research ability, and literature standardization), and filter out invalid text content.

[0119] Further, removing redundant and repetitive text content includes: It is observed that there are a small number of documents in the database, and when experts write itemized evaluation opinions, they tend to pad the words and reuse comments. Use regular expressions in a Python program to split the corresponding expert comments into a sequence of short sentences. For example, import the expert-written document comments into the regular expression library and split the comment content according to Chinese and English punctuation marks. Through a Python sample program, based on the obtained sequence of short sentences, use a text similarity detection tool (such as the difflib library) to judge the similarity between short sentences. For short sentences with a similarity reaching the first threshold Threshold_1 (such as 0.85), they are considered redundant, repetitive, and word-padding, and only the first-occurring short sentence is retained in the original expert itemized comments of the document.

[0120] In a possible implementation, before filtering the first document itemized evaluation opinion using the comment stop word library, the method further includes: when the first document itemized evaluation opinion is missing, determine whether there are keywords in the academic comment; if so, copy the sentence where the keyword is located as the first document itemized evaluation opinion; if not, use the natural language large model to obtain the first document itemized evaluation opinion based on the academic comment.

[0121] In a specific example, filling in the missing field content includes: for the situation where the itemized evaluation opinion of an individual document is blank and missing, extract it from the academic comment of the document. The specific method is to first use a Python program to determine whether keyword phrases (such as "topic selection", "innovation", "basic knowledge", "research methods", "writing standardization") exist in the academic comment of the document. If so, copy the complete sentence where it is located as the replacement string for the missing itemized evaluation opinion of the document; otherwise, use the natural language large model technology to read the academic comment of the document and let the natural language large model technology output a summary opinion as the replacement string for the missing itemized evaluation opinion of the document. The prompt words for the natural language large model here can be designed as: Based on the academic comment of this document, summarize the evaluation opinions in aspects such as "document topic selection", "innovation and literature value", "basic knowledge and research ability", and "document standardization".

[0122] In a possible implementation, the review of the literature to be reviewed using the text semantic similarity calculation tool includes: loading the literature database and the literature to be reviewed in a system platform including the text semantic similarity calculation tool; filtering the literature to be reviewed using the literature database to obtain a first candidate literature range; matching and screening the literature to be reviewed and the first candidate literature range according to a second threshold to obtain a second candidate literature range; obtaining an abstract of the literature to be reviewed to obtain a second abstract data set, obtaining the summaries of each chapter of the literature to be reviewed and the summary and outlook of the literature to be reviewed to obtain a second summary data set, performing text error correction on the literature to be reviewed to obtain a second text error correction data set, and obtaining a third literature sub - evaluation opinion based on the second abstract data set, the second summary data set, and the second text error correction data set using a natural language large - model; filtering the third literature sub - evaluation opinion using the comment disabling word library to obtain the literature sub - evaluation opinion of the literature to be reviewed based on the natural language large - model; screening and ranking the literature sub - evaluation opinion of the literature to be reviewed based on the natural language large - model and the literature sub - evaluation opinions of each literature in the second candidate literature range based on expert review according to a third threshold to obtain a third candidate literature range; obtaining the literature sub - evaluation score of the literature to be reviewed according to the third candidate literature range; summarizing the literature sub - evaluation opinions of the literature to be reviewed to obtain the academic comment of the literature to be reviewed, and filtering the academic comment using the comment disabling word library to obtain the target academic comment; performing arithmetic average and rounding on the literature sub - evaluation scores of the literature to be reviewed to obtain the overall score of the literature to be reviewed; determining whether the overall score of the literature to be reviewed is less than a fourth preset value. If it is less than, the literature to be reviewed is judged as unqualified, and the literature sub - evaluation opinions based on the natural language large - model corresponding to the literature sub - evaluation scores less than the fourth preset value in the literature to be reviewed are integrated to obtain the deficiencies and suggestions of the literature to be reviewed.

[0123] In a specific example, a literature database and the literature to be reviewed are loaded. First, a complete match is made in the database based on the basic information of the literature to be reviewed (including the first-level discipline, second-level discipline, degree type, and study category), and the first candidate literature range, denoted as F1, is obtained. Second, according to the research direction of the literature to be reviewed, based on the text semantic content similarity calculation method, the research directions of the literatures within the F1 candidate range are matched and restricted to obtain a new candidate range, denoted as F2. Third, for the abstract of the literature to be reviewed, the summary of the table of contents chapters, the summary and outlook of the literature, and the text error correction detection results of the full text of the literature, automatic comments on the itemized evaluation opinions of the literature are generated using natural language large model technology. Then, in combination with the text semantic content similarity calculation method, the comments on the itemized evaluation opinions of the literature generated by the large model technology are respectively matched with the comments on the itemized evaluation opinions of the literatures written by experts within the F2 candidate range, and the matching results are sorted in descending order of semantic similarity to obtain a new candidate range, denoted as F3. At this point, the expert scores corresponding to the first M (for example, the value range is 1 ≤ M ≤ 5, and M is an integer) itemized evaluation opinions within the F3 range are intercepted, and their arithmetic mean is taken and rounded to be used as the scoring result of the itemized evaluation opinions of the natural language large model technology. At this time, the arithmetic average of these scoring results of the itemized evaluation opinions of the natural language large model technology is taken and rounded to be used as the overall score of the literature to be reviewed. Here, based on the overall score of the literature, it can be judged whether the literature can pass the preliminary review. If the overall score is greater than or equal to the threshold (such as 60 points), the itemized evaluation opinion comments are integrated into academic comments using natural language large model technology, and the deficiencies and suggestions are set to be empty, and it is considered that the preliminary review status of the literature to be reviewed is passed, and the literature can be transferred to other links of the system platform for processing; otherwise, if the overall score of the literature is less than the threshold (such as 60 points), the itemized evaluation opinion comments are integrated into academic comments using natural language large model technology, and the itemized evaluation opinions with scores less than the threshold (such as 60 points) are summarized as deficiencies and suggestions, and it is considered that the preliminary review status of the literature to be reviewed is not passed, and it needs to be returned to the literature author for modification.

[0124] In a specific example, at the initial moment, a literature database is loaded, covering millions of pieces of literature information accumulated over the years. Each record includes the literature title, first-level discipline, second-level discipline, degree type (academic type or professional type), study category (undergraduate, master's, or doctoral), literature research direction, expert comments and scores on each item of the literature (literature topic selection, innovation and literature value, basic knowledge and scientific research ability, and literature standardization), academic comments, deficiencies, and suggestions.

[0125] In a specific example, based on the natural language large model technology, the opinions of each sub - comment are summarized into a complete content as the academic comment of the literature to be reviewed. The prompt of the natural language large model can be designed as "Summarize the academic comment of the literature according to the descriptions of the 4 aspects of the literature topic, innovation and literature value, basic knowledge and scientific research ability, and literature standardization."

[0126] Furthermore, the academic comments generated by the natural language large model technology are also filtered based on the comment prohibited word library. According to the prohibited words in the comment prohibited word library, the academic comments of the literature generated by the natural language large model technology are filtered to ensure the objectivity, fairness of the comment opinions and a good academic atmosphere, and effectively prevent the occurrence of the "hallucination" phenomenon of the large model technology.

[0127] In a specific example, the arithmetic mean of the scores of each sub - comment given by the natural language large model technology is taken and rounded as the overall score of this literature. The calculation formula of the overall score is expressed as:

[0128]

[0129] In the formula, round is the rounding function; sum is the summation function; Score(x) is the score of each sub - comment given by the natural language large model technology.

[0130] In a specific example, the sub - evaluation opinions generated by the natural language large model technology are summarized by the large model again as the academic comment of the literature.

[0131] Furthermore, if the overall score of the literature is less than the fourth threshold Threshold_4 (such as 60 points), it is considered that the literature to be reviewed is unqualified and fails the preliminary review of the natural language large model technology. At this time, the natural language large model integrates the sub - comments with scores less than the threshold score Threshold_4, generates the deficiencies and suggestions of this literature, and returns them to the literature author for modification.

[0132] Furthermore, if the overall score of the literature to be reviewed is greater than or equal to the fourth threshold Threshold_4, it is considered that the review is qualified and passes the preliminary review of the natural language large model technology, which also means that it is a qualified literature detected by the natural language large model technology. At this time, the item of deficiencies and suggestions is set to none. The literature to be reviewed can be transferred to other links of the system platform and recommended to suitable experts for review.

[0133] In a possible implementation manner, the filtering of the literature to be reviewed by using the literature database to obtain the first candidate literature range includes: matching the first-level subject, second-level subject, degree type, and study category of the literature to be reviewed with the first-level subject, second-level subject, degree type, and study category of each literature in the literature database respectively, and obtaining the first candidate literature range from the literature database according to the matching results.

[0134] In a specific example, for the literature to be reviewed, a complete match is made according to the first-level subject, second-level subject, degree type (academic type or professional type), and study category (undergraduate, master, or doctor) of the literature, and the first candidate literature range is obtained from the database, denoted as F1.

[0135] In a possible implementation manner, the matching and screening of the literature to be reviewed and the first candidate literature range according to the second threshold to obtain the second candidate literature range includes: calculating the similarity between the research directions of each literature in the first candidate literature range and the research direction of the literature to be reviewed, and screening the literatures with a similarity greater than or equal to the second threshold to obtain the second candidate literature range.

[0136] In a specific example, the SBert_YuYi_Sim tool is used to calculate the similarity between the research direction of the literature to be reviewed and the research directions of the literatures within the range of F1. Those with a similarity greater than or equal to the second threshold Threshold_2 (such as 0.5) are matched to obtain a new candidate literature range, denoted as F2.

[0137] In a possible implementation manner, the screening and sorting of the literature sub-item evaluation opinions of the literature to be reviewed based on the natural language large model and the literature sub-item evaluation opinions of each literature in the second candidate literature range based on expert review according to the third threshold to obtain the third candidate literature range includes:

[0138] Calculating the similarity between the literature sub-item evaluation opinions of the literature to be reviewed based on the natural language large model and the literature sub-item evaluation opinions of each literature in the second candidate literature range based on expert review, screening the literatures with a similarity greater than or equal to the third threshold and sorting them in descending order to obtain the third candidate literature range.

[0139] In a specific example, the SBert_YuYi_Sim tool is used to calculate the semantic similarity between the sub-item evaluation opinions output by the natural language large model of the literature to be reviewed and the expert sub-item opinion comments of the literatures within the range of F2 respectively, intercepting the results with a semantic relevance greater than or equal to the third threshold Threshold_3 (such as 0.5), and sorting them in descending order of semantic similarity to obtain the candidate literature range, and this sorting result is denoted as F3.

[0140] In a possible implementation, obtaining the literature sub - evaluation score of the literature to be reviewed according to the third candidate literature range includes: obtaining the arithmetic mean and rounding of the literature sub - evaluation scores based on expert review of the first M literatures in the third candidate literature range as the literature sub - evaluation score of the literature to be reviewed based on the natural language large model, where M is an integer greater than zero; or when the number of literatures in the third candidate literature range is less than M, performing arithmetic mean and rounding on the literature sub - evaluation scores based on expert review of each literature in the third candidate literature range as the literature sub - evaluation score of the literature to be reviewed based on the natural language large model; or when the number of literatures in the third candidate literature range is zero, setting the literature sub - evaluation score of the literature to be reviewed based on the natural language large model to a first preset value.

[0141] In a specific example, for each sub - comment generated by the natural language large model technology of the literature to be reviewed, to obtain the scores of the corresponding sub - evaluation opinions, the arithmetic mean of the sub - opinion scores corresponding to the top M expert comments in the F3 range can be taken and rounded as the sub - score result given by the natural language large model technology, and M can take the value of 5.

[0142] Furthermore, if there are less than M expert comment results in the F3 range, then according to the actual number of expert comments in the F3 range, the arithmetic mean of the scores of its actual number is taken and rounded as the sub - opinion score result given by the natural language large model technology.

[0143] Furthermore, if there are no expert comment results in the F3 range, considering that the literature to be reviewed may belong to a completely innovative research direction or research method, the score corresponding to the sub - evaluation opinion given by the natural language large model technology is set to the first preset value of 60 points, that is, it is defaulted to be qualified.

[0144] Furthermore, the mathematical formula description of the natural language large model technology score of the sub - comment is as follows:

[0145]

[0146] In the formula, round is the rounding function; N = min(M, Len(F3)), Len(F3) is the actual number of expert comments in the F3 range; x ∈ ['literature topic selection', 'innovation and literature value', 'basic knowledge and scientific research ability', 'literature standardization']; F3_exp_score(x) represents the expert score result of the sub - evaluation opinion sorted in descending order of semantic similarity within the third candidate literature range.

[0147] This embodiment has low resource consumption and requires very low server hardware. For natural language large models such as ChatGLM3-6B and ollama3.2-11B, the requirements of this invention for extracting abstracts and the like can be met. At this time, the server is configured with two Intel Xeon Silver 4110 CPUs, 64GB of memory, and a single NVIDIA GeForce 1080Ti independent graphics card. The video memory of the graphics card is 11GB, which can meet the usage requirements; based on the similarity calculation of the SBert model, in the model training stage, a single NVIDIA GeForce 1080Ti independent graphics card is used to train the sbert_text_similarity model, and the text semantic similarity calculation tool SBert_YuYi_Sim is obtained. This tool supports generating a vector with a length of 768 dimensions from the text content. In the tool usage stage, the CPU versions of torch and Sentence-Transformers are deployed, and only the server CPU is used to calculate the text semantic vector, and the length of a single vector is 768 dimensions.

[0148] The comment control of this embodiment is strict. By using the comment disabling word library, the solemnity of the literature review work is effectively adhered to, the fairness of the review results is ensured, the interference of human emotional factors and the "hallucination" phenomenon of natural language large model technology are strictly eliminated, and the academic research atmosphere and good social atmosphere are promoted.

[0149] The review results of the large model in this embodiment are close to those of experts. For the comments generated by natural language large model technology, whether it is the sub-item evaluation opinions (literature topic selection, innovation and literature value, basic knowledge and scientific research ability, and literature standardization), academic comments, deficiencies and suggestions, due to the fine-tuning of the natural language large model, the content generated by the natural language large model is highly professional, the text expression is natural and fluent, the thinking logic is clear, and the review attitude is objective and fair, meeting the performance of high-level experts in the field. For the scoring of the sub-item evaluation opinions of the literature and the overall literature score, due to the adoption of the matching method based on the semantic similarity of the text content, referring to the scoring results of the top several experts sorted in descending order of similarity within the third candidate range, the subjective scoring results of humans are fully utilized, which is very reasonable.

[0150] This embodiment meets the needs of literature pre-review. By being applied to the literature review system platform, as a literature quality automatic pre-review technology based on natural language large models, it can effectively detect obvious problems in the literature, promptly feedback them to the literature authors for modification, and at the same time greatly reduce the manual labor of review experts. It can be seen that this technical solution significantly improves the work efficiency of literature authors and review experts and is very prominent in terms of social benefits.

[0151] It is worth mentioning that especially for the vast number of undergraduate graduates and scientific research workers in colleges and universities, on the one hand, since they are not good at writing documents, they often make elementary mistakes in document writing norms and are not outstanding in literature research methods and innovation; on the other hand, due to the large number of undergraduates and on-the-job scientific and technological personnel, they lack targeted guiding opinions from tutoring teachers. Therefore, through the application and implementation of the present invention, more effective guiding opinions can be given to help them quickly solve the problems existing in the literature, ultimately improving the passing rate of literature submission for review and helping them graduate smoothly or publish scientific research results.

[0152] In short, the automatic pre-review technology of literature based on large natural language models meets the social needs, has broad market prospects, effectively promotes the development of literature quality monitoring work, is well received by the vast number of teachers and students, and has prominent social benefits.

[0153] Obviously, the above-mentioned embodiments of the present invention are only examples for clearly explaining the present invention, rather than limitations on the implementation manners of the present invention. For those of ordinary skill in the art, other different forms of changes or modifications can be made on the basis of the above description. It is impossible to enumerate all the implementation manners here. Any obvious changes or modifications derived from the technical solutions of the present invention still fall within the protection scope of the present invention.

Claims

1. A method for reviewing the quality of literature based on a large natural language model, characterized in that including: establishing a comment disabling word library; collecting the basic information, expert comments and scoring records of previous years' documents to form a document database, where the document database includes the first document sub - evaluation opinions; using a natural language large model to generate the second document sub - evaluation opinions of the previous years' documents; filtering the first document sub - evaluation opinions using the comment disabling word library to obtain the document sub - evaluation opinions based on expert review; filtering the second document sub - evaluation opinions using the comment disabling word library to obtain the document sub - evaluation opinions based on the natural language large model; generating a training sample set using the document sub - evaluation opinions based on expert review and the document sub - evaluation opinions based on the natural language large model; training a text semantic similarity model using the training sample set to obtain a text semantic similarity calculation tool; reviewing the document to be reviewed using the text semantic similarity calculation tool, and obtaining the document sub - evaluation opinions of the document to be reviewed using the natural language large model.

2. The method for reviewing the quality of documents based on a natural language large model according to claim 1, characterized in that, the method further includes: optimizing the natural language large model using the document sub - evaluation opinions based on expert review and the low - rank adaptation algorithm to obtain an optimized natural language large model; using the optimized natural language large model to generate the second document sub - evaluation opinions of the previous years' documents.

3. The method for reviewing the quality of documents based on a natural language large model according to claim 2, characterized in that, the filtering the first document sub - evaluation opinions using the comment disabling word library to obtain the document sub - evaluation opinions based on expert review includes: obtaining the first document sub - evaluation opinions of each document in the document database; segmenting the first document sub - evaluation opinions based on punctuation marks and conjunctions to obtain a short - sentence sequence; judging whether there is a prohibited word in the comment disabling word library in the short - sentence sequence, if so, the short - sentence sequence is invalid text content; deleting the invalid text content in the first document sub - evaluation opinions to obtain the document sub - evaluation opinions based on expert review.

4. The method for reviewing the quality of documents based on a natural language large model according to claim 3, characterized in that, before filtering the first document sub - evaluation opinions using the comment disabling word library, the method further includes: obtaining the first document sub - evaluation opinions of each document in the document database; segmenting the first document sub - evaluation opinions based on punctuation marks to obtain a short - sentence sequence; using a text similarity detection tool to judge the similarity between short - sentence sequences, when the similarity between multiple short - sentence sequences is greater than or equal to a first threshold, retaining the first - appearing short - sentence sequence in the multiple short - sentence sequences and deleting the remaining short - sentence sequences in the multiple short - sentence sequences in the first document sub - evaluation opinions.

5. The method for reviewing the quality of documents based on a natural language large model according to claim 4, characterized in that, Before filtering the first literature item evaluation opinion using the comment disabling word library, the method further includes: When the first literature item evaluation opinion is missing, determine whether there are keywords in the academic comment; if so, copy the sentence where the keyword is located as the first literature item evaluation opinion; If not, use the natural language large model to obtain the first literature item evaluation opinion based on the academic comment.

6. The method for reviewing the quality of a literature based on a natural language large model according to claim 5, wherein: The reviewing of the literature to be reviewed using the text semantic similarity calculation tool includes: Loading the literature database and the literature to be reviewed in a system platform including the text semantic similarity calculation tool; Filtering the literature to be reviewed using the literature database to obtain a first candidate literature range; Matching and screening the literature to be reviewed and the first candidate literature range according to a second threshold to obtain a second candidate literature range; Obtaining an abstract of the literature to be reviewed to obtain a second abstract data set, obtaining summaries of each chapter of the literature to be reviewed and the summary and outlook of the literature to be reviewed to obtain a second summary data set, performing text error correction on the literature to be reviewed to obtain a second text error correction data set, and using a natural language large model based on the second abstract data set, the second summary data set, and the second text error correction data set to obtain a third literature item evaluation opinion; Filtering the third literature item evaluation opinion using the comment disabling word library to obtain the literature item evaluation opinion of the literature to be reviewed based on the natural language large model; Screening and ranking the literature item evaluation opinion of the literature to be reviewed based on the natural language large model and the literature item evaluation opinions of each literature in the second candidate literature range based on expert review according to a third threshold to obtain a third candidate literature range; Obtaining the literature item evaluation score of the literature to be reviewed according to the third candidate literature range; Summarizing the literature item evaluation opinions of the literature to be reviewed to obtain the academic comment of the literature to be reviewed, and filtering the academic comment using the comment disabling word library to obtain the target academic comment; Performing arithmetic averaging and rounding on the literature item evaluation scores of the literature to be reviewed to obtain the overall score of the literature to be reviewed; Determine whether the overall score of the literature to be reviewed is less than a fourth preset value. If so, the literature to be reviewed is reviewed as unqualified, and the literature item evaluation opinions based on the natural language large model corresponding to the literature item evaluation scores of the literature to be reviewed that are less than the fourth preset value are integrated to obtain the deficiencies and suggestions of the literature to be reviewed.

7. The method for reviewing the quality of a literature based on a natural language large model according to claim 6, wherein: The filtering of the literature to be reviewed using the literature database to obtain a first candidate literature range includes: Match the first-level discipline, second-level discipline, degree type, and study category of the literature to be reviewed with those of each literature in the literature database respectively, and obtain the first candidate literature range from the literature database according to the matching results.

8. The literature quality review method based on a large natural language model according to claim 7, wherein The matching and screening of the literature to be reviewed and the first candidate literature range according to the second threshold to obtain the second candidate literature range includes: Calculate the similarity between the research directions of each literature in the first candidate literature range and the research direction of the literature to be reviewed, and screen the literatures with a similarity greater than or equal to the second threshold to obtain the second candidate literature range.

9. The literature quality review method based on a large natural language model according to claim 8, wherein The screening and sorting of the literature sub-item evaluation opinions based on the large natural language model of the literature to be reviewed and the literature sub-item evaluation opinions based on expert review of each literature in the second candidate literature range according to the third threshold to obtain the third candidate literature range includes: Calculate the similarity between the literature sub-item evaluation opinions based on the large natural language model of the literature to be reviewed and the literature sub-item evaluation opinions based on expert review of each literature in the second candidate literature range, screen the literatures with a similarity greater than or equal to the third threshold and sort them in descending order to obtain the third candidate literature range.

10. The literature quality review method based on a large natural language model according to claim 9, wherein The obtaining of the literature sub-item evaluation score of the literature to be reviewed according to the third candidate literature range includes: Obtain the arithmetic mean and round of the literature sub-item evaluation scores based on expert review of the first M literatures in the third candidate literature range as the literature sub-item evaluation score based on the large natural language model of the literature to be reviewed, where M is an integer greater than zero; Or when the number of literatures in the third candidate literature range is less than M, calculate the arithmetic mean and round of the literature sub-item evaluation scores based on expert review of each literature in the third candidate literature range as the literature sub-item evaluation score based on the large natural language model of the literature to be reviewed; Or when the number of literatures in the third candidate literature range is zero, set the literature sub-item evaluation score based on the large natural language model of the literature to be reviewed as the first preset value.

Images