Paper generation method and device based on large model

Abstract

The invention discloses a thesis generation method and device based on a large model. The method comprises the following steps: collecting original thesis data through network acquisition and official purchase channels; preprocessing the original paper data to obtain preprocessed data; based on a preset classification system and a metadata table, carrying out classification processing on the preprocessed data to obtain classification processing data; constructing a training sample based on the classification processing data, and outputting a corresponding bar content; a multi-task learning framework is adopted to carry out fine tuning on the pre-trained large language model, the multi-task learning framework comprises an outline generation task and a text generation task, and task types are distinguished through system prompt; and receiving a paper title and a keyword input by a user, sequentially generating a paper outline through the fine-tuned model, and generating text contents of each chapter in parallel based on the paper outline. According to the invention, the paper generation quality and efficiency can be improved, so that the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for generating papers based on a large model. Background Art

[0002] In the field of artificial intelligence and natural language processing, large model (LLM) technology has made significant progress in recent years, especially in the generation of long texts, and paper generation has become one of its important application directions.

[0003] Since the concept of big models emerged, it has developed rapidly. Big models generally refer to neural network models with a certain scale of parameters. Although there is no official definition of the parameter threshold, most of them adopt a generative structure and are trained through next token loss. When applied to paper generation tasks, common methods include direct application based on large language models, application based on fine-tuning of large language models, and application after secondary pre-training and then fine-tuning. Algorithm optimization effects such as human feedback alignment can also be added. Currently, many paper generation products on the market have similar processes. After the user enters the title and keywords, the product generates an outline. After confirmation by the user, the full text is generated and displayed paragraph by paragraph.

[0004] With the development of technology, the field of large language model paper generation has shown many trends. The scale of the model continues to expand, and the number of parameters has increased from hundreds of millions to hundreds of billions or even trillions, which has significantly improved the language understanding and generation capabilities, and can better meet the requirements of academic writing. The development of domain-based and customized development is prominent, and special corpora are fine-tuned for different disciplines to make the generated papers more professional and accurate. Technology is moving from assisted generation to fully automatic generation, and can also generate full-process content such as references, chart descriptions, etc. The integration with academic tools and platforms is also becoming increasingly in-depth, and the exploration of multimodal generation has gradually unfolded, meeting the diversified expression needs of academic writing.

[0005] However, the field still faces many problems. The generated content has authenticity and reliability risks, and is prone to "hallucination" phenomena, such as generating false references or experimental data. In terms of ethics and academic norms, it is easy to cause academic misconduct disputes, and the authorship and annotation norms of AI-generated papers are not yet clear. The generated content lacks logic and innovation, and it is difficult to propose new theories, methods or results. In high-depth subject areas, the model's understanding of knowledge is limited. Data privacy and copyright issues are also prominent. Training data may infringe copyright, and generated papers may leak sensitive information. At the same time, there is a lack of unified quality assessment standards, and it is difficult for users to judge the value of generated papers.

[0006] In the existing technical solutions, there is a lack of a mature solution for generating papers using commercial large language models. The common "input-outline-full text" generation framework has various defects. For example, when generating content paragraph by paragraph by invoking the domain model, problems such as content deviating from the theme, lacking a global perspective, and duplicate generation within the same chapter are likely to occur; moreover, the content style is not like that of a paper, and it is simple and empty, making it difficult to guarantee the quality of generating ultra-long texts.

[0007] Therefore, the existing large language model paper generation technology faces problems such as insufficient authenticity and logic in the generated content, and the content is prone to deviating from the theme required by users, resulting in a poor user experience. Summary of the Invention

[0008] The embodiments of the present invention provide a method and device for generating papers based on a large model. By optimizing data processing, model training, and generation processes, problems such as the generated content of large model papers not closely adhering to the theme, inconsistent style, and logical conflicts are solved, the quality and efficiency of paper generation are improved, and thus the user experience is enhanced.

[0009] An embodiment of the present invention provides a method for generating papers based on a large model, including:

[0010] S1. Data collection step: Collect original paper data through network acquisition and official purchase channels;

[0011] S2. Basic information processing step: Extract, classify, and tag the basic information of the original paper data to generate preprocessed data including subject classification, keyword annotation, and content tags;

[0012] S3. Classification processing step: Based on a preset classification system and metadata table, classify the preprocessed data to obtain classified processing data;

[0013] S4. Training sample construction step: Construct training samples based on the classified processing data. The input of the training sample includes the paper title, a rough outline of the paper, the detailed title of the chapter corresponding to the target section, and the title of the target section, and the output is the content of the corresponding section;

[0014] S5. Model fine-tuning step: Fine-tune the pre-trained large language model using a multi-task learning framework. The multi-task learning framework includes an outline generation task and a main text generation task, and the task types are distinguished through system prompts;

[0015] S6. Online generation step: Receive the paper title and keywords input by the user, and sequentially generate a paper outline through the fine-tuned model, and generate the main text content of each chapter in parallel based on the paper outline.

[0016] As an improvement to the above solution, in step S1, the preprocessing after obtaining the original paper data includes: performing text conversion on the PDF-format paper and performing non-text element cleaning operations on the converted text data.

[0017] As an improvement to the above solution, step S2 specifically includes:

[0018] Performing text segmentation on the original paper data to generate paragraph-level data units;

[0019] Performing keyword extraction and subject label matching operations on the paragraph-level data units;

[0020] Generating a three-level content label system based on the extracted keywords and matched subject labels.

[0021] As an improvement to the above solution, the classification processing in step S3 includes:

[0022] Establishing a subject classification hierarchy based on the Chinese Library Classification;

[0023] Establishing a keyword annotation system based on the paper keywords;

[0024] Establishing a content label classification based on the content structure characteristics of the paper.

[0025] As an improvement to the above solution, the input-output construction of the training samples in step S4 specifically includes:

[0026] The input layer includes the paper title encoding, the outline encoding sequence, and the chapter title feature vectors;

[0027] The output layer corresponds to the token sequence of the subsection content text;

[0028] An association is established between the input layer and the output layer through the attention mask mechanism.

[0029] As an improvement to the above solution, the implementation of the multi-task learning framework in step S5 includes:

[0030] Adopting a hierarchical prompting strategy for the outline generation task and injecting an outline generation instruction into the system prompt;

[0031] Adopting a chapter-level prompting strategy for the main text generation task and injecting a main text generation instruction into the system prompt;

[0032] Dynamically allocating weights to the loss functions of the two tasks through the task type identifier.

[0033] As an improvement to the above solution, the paper outline generation in step S6 includes:

[0034] Constructing outline tree structure data to generate a three-level title system of chapters, sections, and subsections;

[0035] Detect the semantic conflict between the keywords input by the user and the generated outline;

[0036] Dynamically adjust the outline hierarchy structure based on the conflict detection results.

[0037] As an improvement to the above solution, the body content generated in step S6 includes:

[0038] Perform format standardization processing on the text generated for each chapter;

[0039] Convert mathematical formulas and tabular data to LaTeX format;

[0040] Perform APA format verification on the cited literature data.

[0041] Another embodiment of the present invention correspondingly provides a paper generation device based on a large model, including:

[0042] A data collection module for collecting original paper data through the network and official purchase channels;

[0043] A basic information processing module for extracting, classifying, and tagging the basic information of the original paper data to generate preprocessed data including subject classification, keyword annotation, and content tags;

[0044] A classification processing module for classifying the preprocessed data based on a preset classification system and metadata table to obtain classified processing data;

[0045] A training sample construction module for constructing training samples based on the classified processing data, where the input of the training sample includes the paper title, the rough outline of the paper, the detailed title of the chapter corresponding to the target subsection, and the title of the target subsection, and the output is the content of the corresponding subsection;

[0046] A model fine-tuning module for fine-tuning a pre-trained large language model using a multi-task learning framework, where the multi-task learning framework includes an outline generation task and a body generation task, and the task types are distinguished through system prompts;

[0047] An online generation module for receiving the paper title and keywords input by the user, generating a paper outline in sequence through the fine-tuned model, and generating the body content of each chapter in parallel based on the paper outline.

[0048] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0049] In the embodiment of the present invention, the original paper data is first collected through the network and official purchase, and after basic information extraction, classification, and tagging, it is classified and processed based on a preset classification system and metadata table to construct training samples with a specific input-output format, and a pre-trained large language model is fine-tuned in a multi-task learning framework to distinguish between the outline and text generation tasks. In this way, the training samples in a specific format enable the model to master the global information, reduce the repetition in the same chapter, and learn the writing style of the paper; the multi-task fine-tuning promotes the generation of the outline and text with each other. During online generation, after receiving the user input, the fine-tuned model is used to generate the outline and generate the text in parallel in sequence. In summary, the solution of the embodiment of the present invention optimizes the data processing, model training, and generation processes, solves the problems such as the content of the large model paper generation not closely adhering to the theme, inconsistent style, and logical conflicts, improves the quality and efficiency of the paper generation, and thus improves the user experience. Description of the Drawings

[0050] Figure 1 is a schematic flowchart of a method for generating a paper based on a large model provided by an embodiment of the present invention;

[0051] Figure 2 is a schematic structural diagram of a device for generating a paper based on a large model provided by an embodiment of the present invention. Detailed Embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0053] See Figure 1 , which is a schematic flowchart of a method for generating a paper based on a large model provided by an embodiment of the present invention. The method for generating a paper based on a large model includes the following steps:

[0054] S1. Data collection step: Collect the original paper data through the network and official purchase channels;

[0055] S2. Basic information processing step: Perform basic information extraction, classification, and tagging on the original paper data to generate preprocessed data including subject classification, keyword annotation, and content tags;

[0056] S3. Classification processing step: Classify the preprocessed data based on a preset classification system and metadata table to obtain classified data;

[0057] S4. Training sample construction step: Construct training samples based on the classified processing data. The input of the training samples includes the paper title, the rough outline of the paper, the detailed title of the chapter corresponding to the target subsection, and the title of the target subsection, and the output is the content of the corresponding subsection.

[0058] S5. Model fine-tuning step: Fine-tune the pre-trained large language model using a multi-task learning framework. The multi-task learning framework includes an outline generation task and a main text generation task, and the task types are distinguished through system prompts.

[0059] S6. Online generation step: Receive the paper title and keywords input by the user, and use the fine-tuned model to generate the paper outline in sequence, and generate the main text content of each chapter in parallel based on the paper outline.

[0060] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:

[0061] In the embodiments of the present invention, first, the original paper data is collected through network acquisition and official purchase, and after basic information extraction, classification, and tagging, it is classified and processed based on a preset classification system and a metadata table to construct training samples with a specific input-output format, and a multi-task learning framework is used to fine-tune the pre-trained large language model to distinguish between the outline generation task and the main text generation task. In this way, the training samples in a specific format enable the model to master the global information, reduce the repetition in the same chapter, and learn the paper writing style; the multi-task fine-tuning promotes the mutual promotion of the outline generation and the main text generation. During online generation, after receiving the user input, the fine-tuned model is used to generate the outline in sequence and generate the main text in parallel. In summary, the solution of the embodiments of the present invention optimizes the data processing, model training, and generation processes, solves the problems of the content generated by the large model for papers not closely following the theme, inconsistent style, and logical conflicts, improves the quality and efficiency of paper generation, and thus improves the user experience.

[0062] In the first embodiment, in the step S1, the preprocessing after obtaining the original paper data includes: performing text conversion on the PDF format paper, and performing non-text element cleaning operations on the converted text data.

[0063] In this embodiment, specifically, in the data collection step S1, the original paper data may exist in PDF format. Since the PDF format is not conducive to subsequent data processing, it is necessary to first perform text conversion on it, extract the text information in the PDF file, and convert it into an editable text format to provide a basis for subsequent operations. After the text conversion is completed, the converted text data may contain non-text elements such as pictures, charts, and special format symbols. These elements will interfere with subsequent data processing. Therefore, a non-text element cleaning operation needs to be performed to remove these non-text elements and only retain the pure text content to obtain clean text data, providing high-quality data support for subsequent steps such as basic information extraction, classification, and tagging. This embodiment takes into account the diversity and complexity of the original data format. Converting the PDF format to text and cleaning non-text elements can remove interfering information, making subsequent data processing work more efficient and accurate. The text data processed in this way can better apply to subsequent data annotation, model training, and other links, improving the operation efficiency of the entire technical solution and the quality of the generated papers. In summary, after preprocessing in this embodiment, the data is purer, reducing the interference of non-text information on subsequent processing. This helps improve the accuracy of basic information extraction, making the classification and tagging work more precise, and thus providing high-quality data for model training, enhancing the learning effect of the model on the paper content, and finally generating papers that better meet the requirements.

[0064] As one example, the step S2 specifically includes:

[0065] Perform text segmentation on the original paper data to generate paragraph-level data units;

[0066] Perform keyword extraction and subject label matching operations on the paragraph-level data units;

[0067] Generate a three-level content label system based on the extracted keywords and matched subject labels.

[0068] In this embodiment, specifically, in the basic information processing step S2, the original paper data is first segmented. Since the original papers are usually quite long, dividing them into paragraph-level data units helps to analyze and process the text content more meticulously. Each paragraph serves as a relatively independent data unit, making it more convenient to extract key information. Then, keyword extraction and subject label matching operations are performed on these paragraph-level data units. Keyword extraction is to extract the words or phrases that can represent the core content of the paragraph from the paragraph text, and these keywords can highlight the key content of the paper. Subject label matching is to determine the subject category to which the paragraph belongs based on the paragraph content, so that the paper data can be classified according to the subject. Finally, a three-level content label system is generated based on the extracted keywords and the matched subject labels. This label system describes the characteristics of the paper data in more detail. For example, the first-level label can be the major subject category, the second-level label can be the research direction under this subject, and the third-level label can be the specific research topic or content characteristics, etc., providing more detailed and accurate annotation information for subsequent data classification and model training. By performing multi-level information processing on the original paper data and constructing a detailed label system in this embodiment, the model can better understand the internal characteristics and structure of the paper data. This refined processing method helps the model learn the characteristics and rules of papers in different subjects and different topics, so that when generating papers, it can more accurately grasp the content direction and improve the professionalism and accuracy of the generated papers. To sum up, in this embodiment, the generated three-level content label system provides a richer basis for data classification, facilitating the classification management and retrieval of a large amount of paper data. At the same time, more accurate annotation information enables the model to learn more detailed paper writing patterns during the training process, improving the quality of the papers generated by the model and making their content more in line with the subject norms and requirements.

[0069] As one example, the classification processing in the step S3 includes:

[0070] Establish a subject classification hierarchy based on the Chinese Library Classification;

[0071] Establish a keyword annotation system based on the paper keywords;

[0072] Establish a content label classification based on the content structure characteristics of the paper.

[0073] In this embodiment, specifically, in the classification processing step S3, a subject classification hierarchy is established based on the Chinese Library Classification. The Chinese Library Classification is a widely used subject classification system, which is systematic and scientific. Using it, the paper data can be systematically classified according to subjects. For example, papers in different fields such as natural science and social science can be distinguished, and each major category is further subdivided into minor categories, so that the paper data has a clear attribution at the subject level. A keyword annotation system is established based on the paper keywords. By extracting and organizing the paper keywords, key points are annotated for each paper, which facilitates the screening and retrieval of papers according to keywords and also helps the model understand the core content of the papers. A content label classification is established based on the content structure characteristics of the papers. The structure of the papers is analyzed, such as the organization method of parts like introduction, method, experimental results, conclusion, etc., and the logical relationship between each part. According to these characteristics, content labels are established to further refine the classification from the perspective of content structure. These three classification methods cooperate with each other to comprehensively and deeply classify the preprocessed data, making the characteristics of the data more explicit and more conducive to the subsequent construction of training samples and model training. This embodiment comprehensively uses a variety of classification methods to classify the data from three different dimensions: subject, key points, and content structure, and can comprehensively sort out the characteristics of the data. This multi-dimensional classification method can more accurately describe the characteristics of the paper data, provide more targeted data for the subsequent model training, enable the model to better learn the characteristics and laws of different types of papers, and improve the logic and academic standardization of the generated papers. In summary, through scientific and comprehensive classification processing in this embodiment, the classification of the data is more detailed and accurate, facilitating management and use. During the model training process, the model can better learn the characteristics of different types of papers, the generated papers are more reasonable in content structure, meet the logical requirements of academic papers, and are also more accurate in terms of subject professionalism and key point grasp, improving the overall quality of the papers.

[0074] As one example, the input-output construction of the training samples in the step S4 specifically includes:

[0075] The input layer includes a paper title encoding, an outline encoding sequence, and a chapter title feature vector;

[0076] The output layer corresponds to the token sequence of the subsection content text;

[0077] An association is established between the input layer and the output layer through an attention mask mechanism.

[0078] In this embodiment, specifically, in the training sample construction step S4, the input layer of the training sample includes a paper title encoding, an outline encoding sequence, and a chapter title feature vector. The paper title encoding converts the paper title into a digital form that the model can understand, highlighting the theme of the paper. The outline encoding sequence digitally represents the paper outline, showing the overall structural framework of the paper. The chapter title feature vector extracts the key features in the chapter title, providing information for the model at the chapter level. These encodings and vectors digitally represent the key information of the paper, enabling the model to process and learn more conveniently. The output layer corresponds to the token sequence of the subsection content text. The token sequence is a sequence obtained by splitting the subsection content text according to certain rules, representing the target content that the model needs to generate. An association is established between the input layer and the output layer through an attention mask mechanism. The attention mask mechanism allows the model to focus on the relationship between different parts of the input information and the output content during training. For example, it enables the model to know which information in the title, outline, and chapter title should be emphasized when generating a certain subsection content, so as to learn how to generate accurate outputs based on the input. This embodiment designs a specific input-output structure and utilizes the attention mask mechanism to enable the model to better learn the associations between various parts of the paper. By encoding and inputting the key information of the paper, clarifying the target form of the output, and then guiding the model to focus on key information with the help of the attention mask mechanism, the understanding and generation capabilities of the model for the paper structure and content can be enhanced, enabling the model to better grasp the overall logic and detailed content when generating a paper. In summary, through this design in this embodiment, the model can more effectively learn the mapping relationship between the input and the output during the training process, improving the training effect. When generating a paper, the content generated by the model better meets the input requirements, can better stick to the theme, reduce the problems of deviation from the theme and logical confusion in the generated content, improve the quality of paper generation, and make the generated paper more reasonable and coherent in terms of structure and content.

[0079] As one example, the implementation of the multi-task learning framework in step S5 includes:

[0080] Adopt a hierarchical prompting strategy for the outline generation task and inject an outline generation instruction into the system prompt;

[0081] Adopt a chapter-level prompting strategy for the main text generation task and inject a main text generation instruction into the system prompt;

[0082] Dynamically allocate weights to the loss functions of the two tasks through task type identifiers.

[0083] In this embodiment, specifically, in the model fine-tuning step S5, the multi-task learning framework adopts a hierarchical prompting strategy for the outline generation task and injects an outline generation instruction into the system prompt. For example, the prompting model generates an outline according to different logical levels. First, it determines the major chapter topics, and then gradually refines them to sub-section topics, guiding the model to learn how to generate a reasonable outline structure and content. For the main text generation task, a chapter-level prompting strategy is adopted, and a main text generation instruction is injected into the system prompt. For example, the prompting model generates the main text content of each chapter based on the chapter title and outline content, combined with the corresponding subject knowledge and writing style, enabling the model to focus on the generation of chapter content. The loss functions of the two tasks are dynamically weighted by task type identifiers. According to the difficulty, importance of the outline generation task and the main text generation task, as well as the performance of the model during training, the weights of the loss functions are dynamically adjusted. If the model performs poorly in the outline generation task, the weight of the loss function for the outline generation task is appropriately increased to make the model pay more attention to the training of outline generation, and vice versa, enabling the model to balance learning between different tasks. This embodiment adopts different prompting strategies and loss function weight allocation methods for the two different tasks of outline generation and main text generation, enabling the model to separately learn the skills and key points of different tasks. Through hierarchical prompting and chapter-level prompting, the model is guided to focus on different information in different tasks. At the same time, through dynamic weight allocation, the model balances the learning of the two tasks according to its own situation during training, improving the performance of the model at different stages of paper generation. This method improves the performance of the model in the outline generation and main text generation tasks. The generated outline is more logical and structured, providing a good framework for writing the main text. The main text content is more detailed, coherent, and has a stronger correlation with the outline, overall improving the quality and efficiency of paper generation and making the generated paper more in line with academic requirements and user needs.

[0084] As an example, the paper outline generation in step S6 includes:

[0085] Construct outline tree structure data and generate a three-level title system of chapters, sections, and sub-sections;

[0086] Detect the semantic conflict between the user input keywords and the generated outline;

[0087] Dynamically adjust the outline hierarchy structure based on the conflict detection result.

[0088] In this embodiment, specifically, in the paper outline generation link of the online generation step S6, a hierarchical tree structure data of the outline is constructed to generate a three-level title system of chapters, sections, and subsections. Starting from the overall framework, the main chapters of the paper are first determined, and then gradually broken down into sections and subsections to form a well-structured outline, providing a clear framework for the paper. The semantic conflict between the keywords input by the user and the generated outline is detected. The keywords input by the user and the generated outline titles are semantically analyzed to determine whether the outline content meets the user's requirements. If semantic conflicts are found, for example, the keywords input by the user emphasize a specific research direction that is not reflected or inaccurately reflected in the outline, the outline hierarchical structure is dynamically adjusted based on the conflict detection results. By adding, deleting, or modifying the outline titles, the outline better meets the user's expectations and the requirements of the paper theme, ensuring that the outline can accurately reflect the user's needs and the core content of the paper. In this embodiment, a structured outline is constructed and semantic conflicts are detected, and the outline is adjusted according to the conflicts, in order to ensure the accuracy and adaptability of the outline at the outline generation stage. By constructing a hierarchical tree structure outline system, a clear framework is provided for the paper; through semantic conflict detection and adjustment, the outline better meets the specific needs of the user, avoiding situations that do not match the user's needs during the subsequent main text generation process, and improving the efficiency and quality of paper generation. In summary, this embodiment improves the accuracy and adaptability of outline generation, and the generated outline better meets the user's needs and the paper theme. This helps to guide the generation of the main text, making the main text content more targeted, reducing paper generation problems caused by the mismatch between the outline and the user's needs, enhancing the user experience, and making the entire paper generation process smoother and more efficient.

[0089] As an example, the main text content generated in step S6 includes:

[0090] Perform format standardization processing on the text generated for each chapter;

[0091] Perform LaTeX format conversion on mathematical formulas and tabular data;

[0092] Perform APA format verification on the cited literature data.

[0093] In this embodiment, specifically, when generating the main text content in the online generation step S6, format standardization processing is performed on the text generated for each chapter. This includes unifying format settings such as font, font size, and line spacing, making the thesis more standardized and aesthetically pleasing in overall typesetting. Perform LaTeX format conversion on mathematical formulas and tabular data. LaTeX is a format specifically used for typesetting scientific and technical documents, which can accurately present mathematical formulas and tabular content, ensuring the accuracy of mathematical formulas and the standardization of tables in academic documents. Perform APA format verification on the cited literature data. APA format is a commonly used cited literature format in academic papers. By verification, ensure that the format of the cited literature meets the specifications, including the writing and arrangement of information such as author names, publication years, article titles, and journal names all meet the requirements of APA format, enhancing the professionalism and standardization of the thesis. In this embodiment, after the main text is generated, various format processing and verification are performed on it to make the thesis meet academic standards in terms of format. Through format standardization processing, LaTeX format conversion, and cited literature format verification, improve the quality of the thesis from aspects such as typesetting, formula presentation, and citation norms, making the thesis more in line with the requirements of academic publishing and research, enhancing the readability and credibility of the thesis. This embodiment makes the generated thesis meet academic requirements in terms of format, enhancing the standardization and professionalism of the thesis. A standardized format can improve the reading experience of readers for the thesis, and also helps the thesis to be recognized in the process of academic communication and publishing, improving the overall quality of the thesis and meeting the needs of academic research and academic communication.

[0094] For the understanding of the above embodiments, the following specific descriptions are made here:

[0095] The embodiment of the present invention alleviates the above problems by strengthening the guidance of the model to the overall writing logic and context content of the full text during the model training process. At the same time, during the model training process, through fine-grained data category division, the model is made to learn the writing styles of papers under different subject systems. The overall solution is divided into two parts: offline and online. The following is the offline processing flow:

[0096] 1. Data collection: Common data collection channels such as network acquisition and official purchase.

[0097] 2. Basic information extraction, classification, and tagging: Since some of the collected data lacks necessary basic information such as disciplines and keywords, and in addition, for each document, we also need to tag whether there is interview data and whether there are charts, etc., so here we will use a large model to annotate the collected data. After collecting a certain amount of annotated data, these data can be used to distill and train a small model, and further form a fixed data processing process, reducing the deployment and invocation cost of the large model.

[0098] 3. Classify and process data: After preliminary processing according to the pre-determined classification system, label system, and metadata table, we will further deeply process the data for different disciplines and sub-categories. For example, operations such as formula normalization are carried out. This step is usually carried out in cooperation with large models, small models, and rules.

[0099] 4. Construct training samples: The data output after the third step is basically clean and noise-free data. On this basis, we use large models, small models, and rules to extract the metadata required for training samples to construct training samples for subsequent model training. In this solution, the model generation follows the common process on the market of generating an outline first and then generating the text. In the process of generating the text from the outline, we adopt a paragraph-by-paragraph generation method. The specific inference process will be described in the subsequent online inference section. For the model training samples, specifically, its input and output are in the following format: The input includes the paper title, a rough outline, the detailed title of the chapter corresponding to the target section, and the title of the target section, and the output is the content corresponding to the target section. This input format has the following advantages:

[0100] (1) First, it increases the diversity of the samples, that is, the differential input. Compared with directly inputting the outline content, there are more differences between different samples. When directly inputting the outline, more than 80% of all samples corresponding to this outline are the same content. Because the input is "title + outline + xxx subtitle", the only difference between different samples is the subtitle, which will lead to insufficient model learning and poor task learning effect.

[0101] (2) Second, it increases the model's grasp of global information and improves the model's understanding of information in the same chapter. This is because a rough outline rather than a complete outline is provided in the input, which is equivalent to reducing the task difficulty while retaining the global information. In addition, the content of the corresponding chapter is emphasized in the input, which can reduce the probability of duplicate content generation between chapters for the model.

[0102] After fine-tuning the training data designed in this way, the model can not only learn the writing style of papers, but also learn to have a global perspective while reducing the generation repetition of content in the same chapter.

[0103] 5. Model multi-task fine-tuning: After constructing the training data in the previous step, we will further perform multi-task fine-tuning on the model by integrating the data for generating the outline and the data for generating the text. Different tasks are distinguished by the system prompt. The advantage of doing this is that the model can learn the writing styles of different types of papers while learning the task of generating the outline, and these two types of tasks have a positive effect of promoting each other.

[0104] In addition, during fine-tuning, considering that the model may have seen similar paper samples during the pre-training phase, we will evaluate the "freshness" of each type of sample under the existing category label system. For a certain sample, the so-called freshness refers to how much knowledge the base model has mastered for completing the tasks related to this sample. Here, we use the PPL of the decoding probability of the base model for the sample to measure. The lower the PPL value of a certain sample, the greater the model's grasp of the input. At this time, we will appropriately reduce the weight of this sample during the final gradient update. The loss calculation formula for specific model updates is as follows:

[0105] First, calculate the PPL of each sample:

[0106]

[0107] N represents there are N samples, and w is each decoded token. Then calculate the PPL_tag of each label system. The PPL_tag of each label system is the average value of the PPL of this type of set sample, that is:

[0108]

[0109] Finally, the overall loss value is:

[0110]

[0111] The loss weight of each sample is the PPL score of the corresponding category multiplied by a hyperparameter α.

[0112] The above content is the offline processing flow of this solution. The online part of this solution is as follows:

[0113] (1) User input: After the user inputs the title and keywords, we will perform basic processing on the input, such as basic natural language processing operations like typo correction and word segmentation.

[0114] (2) Generate an outline: As we have described in the offline part, both the outline generation and the main text generation are completed by one model. So here we will call the main text generation model and use different system prompts to distinguish, so that the model generates the corresponding writing outline.

[0115] (3) Generate the main text: From the construction of the training samples, it can be seen that although we generate paragraph by paragraph, all paragraphs can be generated in parallel, that is, when generating a certain paragraph, there is no need to wait for the previous paragraphs to be generated before calling the model to generate. This is because we have already told the model relevant information through the rough outline of the full text and the detailed outline of this chapter in the input content, which can ensure to a certain extent that the model generation does not deviate from the theme and there will be no context repetition. Generating all paragraphs in parallel can greatly improve the generation speed of the complete paper and improve the user experience.

[0116] See Figure 2 which is a schematic structural diagram of a paper generation device based on a large model provided by an embodiment of the present invention. The paper generation device based on a large model includes:

[0117] A data collection module 1, configured to collect original paper data through the network and official purchase channels;

[0118] A basic information processing module 2, configured to perform basic information extraction, classification, and tagging processing on the original paper data to generate preprocessed data including subject classification, keyword annotation, and content tags;

[0119] A classification processing module 3, configured to perform classification processing on the preprocessed data based on a preset classification system and a metadata table to obtain classified processing data;

[0120] A training sample construction module 4, configured to construct training samples based on the classified processing data, where the input of the training sample includes a paper title, a rough paper outline, the detailed title of the chapter corresponding to the target subsection, and the title of the target subsection, and the output is the content of the corresponding subsection;

[0121] A model fine-tuning module 5, configured to fine-tune a pre-trained large language model using a multi-task learning framework, where the multi-task learning framework includes an outline generation task and a main text generation task, and the task types are distinguished through system prompts;

[0122] An online generation module 6, configured to receive a paper title and keywords input by a user, and sequentially generate a paper outline through the fine-tuned model, and parallel generate the main text content of each chapter based on the paper outline.

[0123] Compared with the prior art, the embodiment of the present invention has the following beneficial effects:

[0124] In the embodiment of the present invention, original paper data is first collected through the network and official purchase, and after basic information extraction, classification, and tagging, it is classified based on a preset classification system and a metadata table, and training samples with a specific input-output format are constructed to fine-tune a pre-trained large language model using a multi-task learning framework to distinguish between the outline generation task and the main text generation task. In this way, the training samples in a specific format enable the model to master global information, reduce repetition in the same chapter, and learn the paper writing style; the multi-task fine-tuning promotes the mutual promotion of outline generation and main text generation. During online generation, after receiving the user input, the fine-tuned model is used to sequentially generate the outline and parallel generate the main text. In summary, the solution of the embodiment of the present invention optimizes the data processing, model training, and generation processes, solves the problems of the content of the large model paper generation not closely following the theme, inconsistent style, and logical conflicts, improves the quality and efficiency of paper generation, and thus improves the user experience.

[0125] It should be noted that the content of the device embodiments can be correspondingly referred to the content of the above method embodiments, which will not be elaborated here.

[0126] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present invention, the connection relationships between the modules indicate that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0127] The above is the preferred embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A paper generation method based on a large model, characterized in that: The following steps are involved: S1. Data collection steps: Collect original paper data through the Internet and official purchase channels; S2. Basic information processing step: extracting, classifying and labeling the original paper data to generate pre-processed data including subject classification, keyword annotation and content label; S3. Classification processing step: Based on a preset classification system and metadata table, the pre-processed data is classified to obtain classified processing data; S4. Training sample construction step: constructing training samples based on the classified processing data, wherein the input of the training samples includes the title of the paper, a rough outline of the paper, a detailed title of the chapter corresponding to the target subsection, and the title of the target subsection, and the output is the content of the corresponding subsection; S5. Model fine-tuning step: fine-tune the pre-trained large language model using a multi-task learning framework, wherein the multi-task learning framework includes an outline generation task and a text generation task, and the task type is distinguished by a system prompt; S6. Online generation step: receiving the paper title and keywords input by the user, generating the paper outline in sequence through the fine-tuned model, and generating the main text content of each chapter in parallel based on the paper outline.

2. The method for generating papers based on a large model according to claim 1, characterized in that: In the step S1, the preprocessing after the original paper data is acquired includes: converting the PDF format paper into text, and performing a non-text element cleaning operation on the converted text data.

3. The method for generating papers based on a large model according to claim 1, characterized in that: The step S2 specifically includes: Perform text segmentation on the original paper data to generate paragraph-level data units; Performing keyword extraction and subject label matching operations on the paragraph-level data unit; A three-level content labeling system is generated based on the extracted keywords and matching subject labels.

4. The method for generating papers based on a large model according to claim 1, characterized in that: The classification process in step S3 includes: Establish subject classification hierarchy based on China Library Classification; Establish a keyword annotation system based on the paper keywords; Establish content tag classification based on the content structure characteristics of the paper.

5. The method for generating papers based on a large model according to claim 1, characterized in that: The input and output structure of the training sample in step S4 specifically includes: The input layer contains the paper title encoding, outline encoding sequence and chapter title feature vector; The output layer corresponds to the token sequence of the subsection content text; The input layer and the output layer are connected through the attention mask mechanism.

6. The method for generating papers based on a large model according to claim 1, characterized in that: The implementation of the multi-task learning framework in step S5 includes: A hierarchical prompt strategy is adopted for the outline generation task, and outline generation instructions are injected into the system prompts; A chapter-level prompt strategy is adopted for the text generation task, and text generation instructions are injected into the system prompts; The loss functions of the two tasks are dynamically weighted using the task type identifier.

7. The method for generating papers based on a large model according to claim 1, characterized in that: The paper outline generation in step S6 includes: Construct outline tree structure data and generate a three-level title system of chapter, section and subsection; Detect semantic conflicts between user-input keywords and generated outlines; Dynamically adjust outline hierarchy based on conflict detection results.

8. The method for generating papers based on a large model according to claim 1, characterized in that: The text content generated in step S6 includes: Standardize the format of the text generated by each chapter; Convert mathematical formulas and table data into LaTeX format; Perform APA formatting check on citation data.

9. A paper generation device based on a large model, characterized in that: include: Data collection module, used to collect original paper data through online acquisition and official purchase channels; A basic information processing module is used to extract, classify and label the original paper data to generate pre-processed data including subject classification, keyword annotation and content label; A classification processing module, used for classifying the pre-processed data based on a preset classification system and metadata table to obtain classified processed data; A training sample construction module is used to construct a training sample based on the classification processing data, wherein the input of the training sample includes the title of the paper, a rough outline of the paper, the detailed title of the chapter corresponding to the target subsection and the target subsection title, and the output is the content of the corresponding subsection; A model fine-tuning module, which is used to fine-tune the pre-trained large language model using a multi-task learning framework, wherein the multi-task learning framework includes an outline generation task and a text generation task, and the task type is distinguished by a system prompt; The online generation module is used to receive the paper title and keywords input by the user, generate the paper outline in sequence through the fine-tuned model, and generate the main text content of each chapter in parallel based on the paper outline.

Images