A Document Generation Method and Device Based on Large Models

Abstract

The present invention provides a method and device for generating official documents based on large models, belonging to the technical field of intelligent document drafting. The method includes: extracting elements from the original official documents, clustering and analyzing the original official documents belonging to the same official document type according to the official document titles to form different sub-types, and establishing a vector index; constructing complete samples and MASK samples respectively according to the elements of the original official documents; fine-tuning the large model using the low-rank adaptation technology based on the complete samples and MASK samples; retrieving similar official documents using the vector index according to the official document elements input by the users of the target organization, and constructing prompt words according to the classification of the similar official documents to drive the fine-tuned large model to generate the final document drafting result. The present invention can realize the generation of the official document text in a specific style according to the official document elements input by the users, improving the adaptability and accuracy of the automatically generated official document content to the specific organizational style.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent document drafting, and in particular, to a method and device for generating official documents based on a large model. Background Art

[0002] In enterprises and government agencies, official document writing has always occupied an important position. In terms of content professionalism, format standardization, expression rigor, etc., it has relatively high standards compared to ordinary articles. Traditional official documents mainly rely on manual writing, so there are relatively high requirements for the professional skills, writing expression of staff, and understanding of the details of official document formats. The process is complicated, the learning cost is high, and problems such as detail errors and inaccurate expressions are likely to occur during the writing process, which affects the overall quality of official documents. Therefore, it is necessary to automate and intelligentize the process of manually writing official documents through intelligent means.

[0003] In recent years, the development of natural language generation large model technologies represented by ChatGPT has provided new ideas and technical support for the intelligent writing of official documents. Users can drive the large model to generate formal official documents with a certain degree of professionalism by inputting simple descriptive texts or outlines about the content of official documents and corresponding prompt words. However, since the training corpus of the large model mainly comes from publicly available text data on the Internet, although the generated official documents have a certain degree of standardization, they lack the writing styles and detail requirements of different industries, especially different enterprises and government agencies.

[0004] Therefore, it is urgent to carry out research on intelligent document drafting methods based on large models under specific enterprises and government agencies, so as to make the large model learn the styles and details of incoming and outgoing official documents within enterprises and agencies, make up for the above limitations of the large model, and improve the effect of intelligent document drafting. Summary of the Invention

[0005] The present invention provides a method and device for generating official documents based on a large model to solve the defect that the automatically generated content of official documents in the prior art lacks adaptability to specific organizational styles and detail accuracy.

[0006] In a first aspect, the present invention provides a method for generating official documents based on a large model, including:

[0007] Performing element extraction on the original official document, where the extracted elements include: the official document title and the official document body; for original official documents belonging to the same official document type, performing clustering analysis according to the official document title to form different sub-types, and establishing a mapping between the vector of the official document title and the preset elements and the sub-types to form a vector index;

[0008] Construct complete samples and MASK samples respectively according to the elements of the original official document. Among them, the input items of the complete sample are various elements of the original official document except the official document text and an abstract containing all key information of the official document text, and the output item is the official document text. The input items of the MASK sample are various elements of the original official document except the official document text and an abstract containing partial key information of the official document text, and the output item is the official document text with information supplement prompts.

[0009] Based on the complete sample and the MASK sample, construct a loss function, and use the low-rank adaptation technology to fine-tune the large model to learn the writing style and information completion ability of official documents in the samples.

[0010] According to the official document elements input by the user of the target institution, use vector indexing to retrieve similar official documents, and construct prompt words according to the classification of the similar official documents to drive the fine-tuned large model to generate the final document generation result.

[0011] In a second aspect, the present invention also provides an official document generation device based on a large model, including:

[0012] An official document preprocessing module for extracting elements from the original official document. Among them, the extracted elements include: the official document title and the official document text. For original official documents belonging to the same official document type, perform clustering analysis according to the official document title to form different sub-types, and establish a mapping between the vector of the official document title and the preset elements and the sub-types to form a vector index.

[0013] A sample generation module for constructing complete samples and MASK samples respectively according to the elements of the original official document. Among them, the input items of the complete sample are various elements of the original official document except the official document text and an abstract containing all key information of the official document text, and the output item is the official document text. The input items of the MASK sample are various elements of the original official document except the official document text and an abstract containing partial key information of the official document text, and the output item is the official document text with information supplement prompts.

[0014] A model fine-tuning module for constructing a loss function based on the complete sample and the MASK sample, and using the low-rank adaptation technology to fine-tune the large model to learn the writing style and information completion ability of official documents in the samples.

[0015] An official document generation module for using vector indexing to retrieve similar official documents according to the official document elements input by the user of the target institution, and constructing prompt words according to the classification of the similar official documents to drive the fine-tuned large model to generate the final document generation result.

[0016] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of any one of the above-mentioned large model-based official document generation methods are implemented.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned large model-based official document generation methods are implemented.

[0018] The large model-based official document generation method and device provided by the present invention have the following beneficial effects compared with the prior art:

[0019] The large model-based official document generation method and device provided by the present invention have the following several remarkable beneficial effects compared with the prior art:

[0020] (1) The present invention can generate the main body of an official document in a specific style according to the official document elements input by the user, improving the adaptability and accuracy of the automatically generated content of the official document to the specific organizational style.

[0021] (2) The present invention proposes a method for extracting the subdivision types based on historical official documents and retrieving the types and similar official documents according to the input official document element information, providing an effective reference template for the official document to be written.

[0022] (3) The present invention proposes a large model fine-tuning method based on the complete samples and MASK samples of historical official documents, enabling the large model to learn the writing style in the samples and generate a detailed main body of a specified style according to the brief description of the input official document content. Among them, the design of the MASK sample is particularly helpful for improving the performance of the model in information completion, ensuring that even if some information is missing, relevant content can be accurately predicted and supplemented, thereby improving the overall quality of the official document. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0024] Figure 1 is a framework schematic diagram of the technical solution of large model-based official document generation provided by the present invention;

[0025] Figure 2 is a schematic diagram of the user input interface provided by the present invention;

[0026] Figure 3 It is a schematic diagram of the output result provided by the present invention;

[0027] Figure 4 It is a schematic structural diagram of the electronic device provided by the present invention. Specific embodiments

[0028] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without making creative efforts fall within the scope of protection of the present invention.

[0029] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the element. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0030] The terms "first", "second", etc. in the present application are used to distinguish similar objects and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of the same type and do not limit the number of objects. For example, the first object can be one or multiple.

[0031] Figure 1 It is a schematic framework diagram of the technical solution of official document generation based on a large model provided by the present invention. The following combines Figure 1 to describe the official document generation method and device based on a large model provided by the present invention.

[0032] Step 101: Extract elements from the original official document (generally historical official documents). The extracted elements include: the official document title and the official document body. For the original official documents belonging to the same official document type, perform clustering analysis according to the official document title to form different sub-types, and establish a mapping between the vector of the official document title and the preset elements and the sub-types to form a vector index.

[0033] This step is the first part of the present invention and corresponds to the official document preprocessing module of the device. This module uses the official documents actually received and sent within an enterprise or government agency to extract the structured results of the official document elements, conducts cluster analysis based on the element texts to obtain the sub-types of official documents, and constructs the corresponding indexes. This module mainly includes the following steps:

[0034] (1)Extraction of official document elements

[0035] The original official documents within an enterprise or government agency are often stored unstructured in forms such as pdf. Therefore, it is first necessary to use technologies such as ocr to extract the text from the pdf documents.

[0036] Subsequently, for the extracted official document text, appropriate large model prompts are constructed, and the basic large model is used to extract various elements from the official document text and form a structured form of Json string. Among them, the specific elements include: official document title, main recipient unit, issuing unit, issuing time, carbon copy recipients, document number, official document body, etc. The large model used is an open-source large language model deployed locally, such as Qwen2.5-72B. The constructed prompts are centered around driving the large model to extract elements and generate a structured Json string. The following is an example:

[0037] “You are an assistant for extracting information from the text of an official document without any spaces and line breaks. Please generate the following elements of the official document: [document type], [title], [sender], [document number], [carbon copy recipients], [recipient], [body], [issuing time].

[0038] Requirements:

[0039] Please generate completely based on the content of the official document without adding any text by yourself.

[0040] Please only output the content of the above elements without any other redundant information.

[0041] The output body must be complete without deleting any content by yourself.

[0042] If a field has no value, please make sure to set it as an empty string when outputting.

[0043] Output example: {"document type":"[document type in the original text]","title":"[title content in the original text]","sender":"[sender in the original text]","document number":"[document number in the original text]","recipient":"[recipient in the original text]","carbon copy recipients":"[carbon copy recipients in the original text]","body":"[body content in the original text]","issuing time":"[issuing time in the original text]"}”

[0044] (2)Extraction of sub-types of official documents

[0045] In the official documents of enterprises and government agencies, there is a part of routine official documents with extremely similar writing styles. For example, in the official documents of water intake plan approvals, the titles are all in the form of "xxx's Reply on the Water Intake Plan for xx Month, xxxx", and the writing contents are highly similar. One of the contents of the present invention is to accurately identify this type of official document from historical official documents and enable the large model to learn its writing style in the subsequent process. In the identification stage introduced in this section, a clustering algorithm is used to extract sub-types, and the specific steps are as follows:

[0046] For the original official documents of the same type of official document, an embedding model, such as bge-large-zh-v1.5, is used to vectorize the titles of the official documents; all vectors are clustered through a preset clustering algorithm to obtain the clustering results. Specifically, the density clustering algorithm (DBSCAN) is used. According to the neighborhood radius ε and the minimum number of points MinPts, the neighborhood of each sample is found. If the number of samples in the neighborhood ≥ MinPts, it is marked as a core point, and it and the samples in its neighborhood are jointly promoted to form a cluster, and its center point is the cluster center. Note that not all official documents will be assigned to a certain cluster. Only official documents with strong generality will form a cluster. For the convenience of expression, the official documents in a certain cluster are called general official documents, and the official documents that do not belong to any cluster are called general official documents.

[0047] For each cluster, find the commonality of the themes of the official documents in the cluster and draw up a description of the sub-type. Such as, appointment notice, work transfer letter, water intake plan request, selection result approval, etc. Since the official documents in the same cluster are highly similar and the theme commonality is obvious, this step can directly name the sub-types manually.

[0048] After determining the sub-type, select an official document closest to the cluster center of the type as the representative official document of the type.

[0049] It should be noted that the sub-types in the present invention can be understood as the sub-types of official documents, and the relationships with official document elements such as the official document text and the official document title can be established.

[0050] (3)Construct a vector index

[0051] The role of the vector index is to retrieve the template official document (i.e., the similar official document) closest to the input content first when the user uses the intelligent official document generation system to assist the large model in generating.

[0052] Specifically, Faiss builds a vector index for the vectors corresponding to the titles of official documents and establishes the mappings of each vector to its true official document title, text, sub-type (if any), and other elements.

[0053] Step 102: Construct a complete sample and a MASK sample respectively according to the elements of the original official document; among them, the input items of the complete sample are various elements of the original official document except the official document text and an abstract containing all key information of the official document text, and the output item is the official document text; the input items of the MASK sample are various elements of the original official document except the official document text and an abstract containing partial key information of the official document text, and the output item is the official document text with information supplement prompts.

[0054] This step is the second part of the present invention, corresponding to the sample generation module of the device. It mainly includes two parts: the construction of the complete sample of the task and the construction of the MASK sample. Among them, the abstract containing all key information of the official document text and the abstract containing partial key information of the official document text are also generated according to the elements of the original official document.

[0055] It should be noted that the complete sample and the MASK sample in the present invention can be constructed by manual annotation (the final sample only needs to meet the above limitations on the sample content and sample structure), or can be automatically generated by natural language processing combined with a large model.

[0056] As a preferred embodiment, the implementation manner of automatically generating the complete sample and the MASK sample by using a large model will be described below.

[0057] (1) Construction of the complete sample

[0058] The complete sample contains all key information in the original official document. The input items of the complete sample are various elements of the original official document except the official document text and an abstract containing all key information of the official document text, and the output item is the official document text. The construction method of the complete sample is as follows:

[0059] Generate an abstract of the official document text according to the official document elements obtained in the previous steps. This abstract needs to contain all key information in the text, such as preset elements or preset information such as the matters, time, place, contacts, etc. described in the official document.

[0060] 1) Generation of the abstract containing all key information of the official document text

[0061] The steps for generating an abstract containing all key information based on the official document elements include: setting a corresponding text abstract for each representative official document of each subdivision type; constructing a specific prompt with element replacement rules to use the large model to generate an abstract containing all key information of the official document text; among them, the specific prompt includes the elements and text abstract of the representative official document, the general official document uses the elements of the representative official document of the subdivision type to which it belongs, and the general official document uses the elements of the representative official document closest to it.

[0062] Specifically, first, for each representative official document of a specific type of official document, a text abstract is formulated based on its various elements. Then, by constructing the elements of the representative official document and specific prompt words for the text abstract, and using a basic large model, the corresponding text abstracts for all official documents under this specific type are generated. The following are examples of the constructed prompt words:

[0063] "Please generate an abstract for the input official document according to the following example.

[0064] <Input>{"Document Type":"[Document Type]","Title":"[Title Content]","Sender":"[Sender]","Recipient":"[Recipient]","Text":"[Text Content]","Issuance Time":"[Issuance Time]"}< / Input>

[0065] <Output>[Abstract]< / Output>"

[0066] During the actual process of generating the abstract, the content such as [Document Type], [Title Content], [Sender], [Recipient], [Text Content], [Issuance Time], [Abstract], etc. in the prompt words are replaced: for general official documents, they are replaced with the corresponding elements in the representative official document of the cluster where the official document is located; for general official documents, they are replaced with the corresponding elements in the representative official document of the nearest cluster to the official document.

[0067] 2) Structuring of the complete sample elements

[0068] After generating the abstract containing all the key information of the official document text, in combination with other elements of the original official document, the "complete sample" in the model training sample is constructed. Specifically, the other elements of the original official document except the official document text, and the abstract containing all the key information of the official document text are structurally combined to form the input item (where the input item can also be supplemented with style descriptions), and the official document text of the original official document is used as the output item. The sample template is as follows:

[0069] Sample input item: "Please generate the text of the official document according to the following official document elements, requiring the writing style to be that of the internal official documents of the Yangtze River Water Resources Commission.

[0070] {"Document Type":"[Document Type]","Title":"[Title Content]","Sender":"[Sender]","Recipient":"[Recipient]","Issuance Time":"[Issuance Time]","Text":"[Text Content]"}"

[0071] Sample output item: "[Text Content]"

[0072] Among them, the content such as [Document Type], [Title Content], [Sender], [Recipient], [Issuance Time], [Abstract], [Text Content], etc. in the sample are all replaced with the corresponding elements in the sample official document.

[0073] In the input item, "style description" can be used to emphasize that the generated body text has a certain specific style, such as "Internal Official Documents of the Yangtze River Water Resources Commission". For general official documents, the style description should be further specified to its sub-types, such as: "Notice of Appointment of the Yangtze River Water Resources Commission". This description should be unified within the prompt words during model training and inference processes to distinguish it from the text generation function of the large model in a general context.

[0074] (2)MASK Sample Construction

[0075] Some key information in the original official document is hidden in the MASK sample. The input item of the MASK sample is various elements of the original official document except the body text of the official document and a summary containing some key information of the body text of the official document, and the output item is the body text of the official document with information supplement prompts. The purpose of using MASK is that when the information input by the user is incomplete, the model can still generate a complete body text framework and prompt the user to supplement information at the information missing places. The construction method of the MASK sample is as follows:

[0076] 1) MASK Processing of the Body Text of the Official Document

[0077] The steps to generate a summary containing some key information based on the official document elements include: determining the key information that needs to be MASK processed in the body text of the official document (such as some preset elements); performing MASK processing on the key information to be MASK processed in the official document text to generate the body text of the official document with information supplement prompts, that is, the MASK body text.

[0078] Specifically, first, it is necessary to determine the key information that needs to be MASK processed in the body text, that is, information with a clear and precise reference in addition to the matter described in the official document itself, such as the matter time, location, contact person, phone number, file name, etc. Replace the text related to the key information with fuzzy expressions and prompt texts such as "[Please enter the time]" or "[xxx - xxxxxxxx]". In an official document, different key information is randomly selected by the large model for MASK processing, and the MASKed body text is output. Example prompt words are as follows:

[0079] "Please follow the following example to perform fuzzy processing on the location information in the official document and output the processed body text content.

[0080] <Input>It is scheduled to hold a training session for newly recruited employees of the Network Information Center at 8:00 am on the 18th in Conference Room 326. Please ask all new colleagues to participate on time.< / Input>

[0081] <Output>It is scheduled to hold a training session for newly recruited employees of the Network Information Center at 8:00 am on the 18th in [[Please add the meeting location]] and please ask all new colleagues to participate on time.< / Output>

[0082] The above is a prompt template for MASK processing of location information. In actual operation, corresponding prompts are constructed for multiple types of information such as time, location, contact person, file name, etc., and for the same official document, one to multiple pieces of information are randomly processed respectively to ensure the diversity of samples.

[0083] 2) Generation of an abstract containing key information in the body part of the official document

[0084] The method of constructing the abstract is similar to the method of "generating the abstract of the full sample text" in the previous text. The difference is that the official document text used here is the official document text after MASK processing, that is, the MASK text. Other steps are the same as those in the previous text and will not be elaborated here.

[0085] The abstract produced in this step is the abstract with some key information hidden.

[0086] 3) Structuring of MASK sample elements

[0087] The method of structuring elements is similar to the method of "structuring elements of the full sample" in the previous text. There are two differences:

[0088] One is that the abstract used is the abstract summarized from the MASK text, that is, the abstract with some key information hidden.

[0089] The second is that the output item of the sample is the MASK text; the present invention can also set MASK tags in the input item, that is, the output item is the official document text with MASK tags and information supplement prompts.

[0090] Other steps are the same as those in the previous text and will not be elaborated here.

[0091] Step 103: Based on the full sample and the MASK sample, construct a loss function, and use the low-rank adaptation technology to fine-tune the large model to learn the writing style and information completion ability of official documents in the samples.

[0092] This step is the third part of the present invention, corresponding to the model fine-tuning module of the device. This module uses the constructed samples to fine-tune the basic large model, enabling the model to learn the writing style and information completion ability of official documents in the samples.

[0093] Adopt the fine-tuning method of the low-rank adaptation technology (Low-Rank Adaptation, LORA), and update the weights by introducing a low-rank matrix, so as to complete the fine-tuning of the model without comprehensively updating the parameters of the original large model.

[0094] Optionally, the loss function (which is a mixed loss function) includes a complete sample loss, a MASK sample loss, a model completion loss for measuring the ability of the model to generate complementary information based on the context, and a style loss for measuring the difference between the generated text of the official document by the model and the target writing style. Specifically, it is as follows:

[0095]

[0096]

[0097] Among them, represents the model parameters, represents the complete sample set, represents the MASK sample set, and S represents the entire sample set; represents the i-th sample in the complete sample set, represents the label of the i-th sample in the complete sample set; represents the j-th sample in the MASK sample set, represents the label of the j-th MASK sample in the MASK sample set; represents the k-th sample in the entire sample set, represents the label of the k-th sample in the entire sample set; represents the model prediction result of the i-th sample in the complete sample set; represents the model prediction result of the j-th sample in the MASK sample set; represents the model prediction result of the k-th sample in the entire sample set.

[0098] and represent the weights of the complete sample and the MASK sample, and represent the loss functions of the complete sample and the MASK sample, and are weight parameters; and represent the model completion loss function and the style loss function respectively.

[0099] These two loss functions can use the cross-entropy loss function, or use the mean squared error loss function. The calculation method of the loss function is as follows: First, convert and into vectors using the embedding model, and then calculate the cosine similarity distance between the two vectors as the value of the loss function.

[0100] Step 104: According to the official document elements input by the user of the target organization, use vector indexing to retrieve similar official documents, and construct prompt words based on the classification of the similar official documents to drive the fine-tuned large model to generate the final official document drafting result.

[0101] Optionally, when multiple retrieved most similar official documents belong to the same target sub-type, construct prompt words based on the target sub-type. For example, add a style description of the target sub-type to the prompt words to drive the fine-tuned large model to generate the final official document drafting result.

[0102] Optionally, construct prompt words based on the target sub-type, including: determining a representative official document based on the target sub-type; constructing prompt words based on the representative official document of the target sub-type. For example, add the representative official document of the target sub-type to the prompt words as an example to drive the fine-tuned large model to generate the final official document drafting result.

[0103] Optionally, when multiple retrieved most similar official documents do not belong to the same sub-type, only retain the style description of the target organization as a prompt word constraint for constructing prompt words. The following is a specific description of this part.

[0104] This step is the fourth part of the present invention, corresponding to the official document generation module of the device. The module receives inputs from the user, including information such as the type, title, issuing unit, receiving unit, and abstract of the official document. The interface is as Figure 2 shown Figure 2 and is a schematic diagram of the user input interface provided by the present invention.

[0105] The module generates the final official document text through steps such as retrieval and large model driving. The specific steps are as follows:

[0106] First, convert the input official document title into a vector using an embedded model, and retrieve similar official documents in the title index library of the corresponding official document type constructed above to obtain the top k similar official documents and scores (such as similarity).

[0107] (1) If the scores of the top k similar official documents are all higher than the threshold θ and all belong to the same sub-type, it means that this official document is a general official document of this sub-type, and then construct prompt words according to this sub-type. For example, if the title of the official document input by the user is "Request for the Electricity Usage Plan in January 2025", and the top k similar official documents retrieved are all requests for monthly electricity usage plans in 2024, all belonging to the "Electricity Usage Plan" sub-type under the "Request" official document type, then construct the following prompt words:

[0108] "Please refer to the following examples and generate the official document text of the electricity usage plan with the internal official document style of the Yangtze River Water Resources Commission according to the input official document elements.

[0109] <Example Input>[Template input for electricity consumption plan]< / Example Input>

[0110] <Example Output>[Template output for electricity consumption plan]< / Example Output>

[0111] (2) If the top k similar official documents do not belong to the same subdivision type (belong to different types, or some official documents have no corresponding subdivision type), or the scores of some official documents are lower than the threshold θ, it indicates that this official document is a general official document and is not used. Only use the following prompt words to drive the large model:

[0112] "Please generate the official document text with the internal official document style of the Yangtze River Water Resources Commission according to the following official document elements:

[0113] {"Document type":"[Document type]","Title":"[Title content]","Sender":"[Sender]","Receiver":"[Receiver]","Issuance time":"[Issuance time]","Abstract":"[Abstract]"}"

[0114] Finally, input the constructed prompt words into the large model, and the output result is used as the output of the intelligent official document generation device. As Figure 3 shown, Figure 3 is the schematic diagram of the output result provided by the present invention.

[0115] In summary, the present invention has the following beneficial effects compared with the prior art:

[0116] (1) The present invention can realize the generation of official document text with a specific style according to the official document elements input by the user, improving the adaptability and accuracy of the automatically generated official document content to the specific organizational style.

[0117] (2) The present invention proposes a method for extracting the subdivision type based on historical official documents and retrieving the type and similar official documents according to the input official document element information, providing an effective reference template for the official document to be written.

[0118] (3) The present invention proposes a large model fine-tuning method based on the complete samples and MASK samples of historical official documents, enabling the large model to learn the writing style in the samples and generate a detailed text of the specified style according to the brief description of the input official document content. Among them, the design of the MASK sample is particularly helpful for improving the performance of the model in information completion, ensuring that even if some information is missing, relevant content can be accurately predicted and supplemented, thereby improving the overall quality of the official document.

[0119] Figure 4 is the schematic diagram of the structure of the electronic device provided by the present invention. As Figure 4As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440. Among them, the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 may invoke the logical instructions in the memory 430 to execute the official document generation method based on the large model.

[0120] In addition, when the logical instructions in the above-mentioned memory 430 are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, etc., which can store program codes.

[0121] On the other hand, the present invention also provides a computer program product. The computer program product includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the official document generation method provided in the above-mentioned various embodiments.

[0122] On yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is implemented to execute the official document generation method provided in the above-mentioned various embodiments.

[0123] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the technical solution, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disks, optical discs, etc., and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A document generation method based on a large model, characterized in that, Including: Extract elements from the original official document. Among them, the extracted elements include: the document title and the document body. For the original official documents belonging to the same document type, perform clustering analysis based on the document title to form different sub-types, and establish a mapping between the vector of the document title, the preset elements, and the sub-types to form a vector index; Construct a complete sample and a MASK sample respectively according to the elements of the original official document. Among them, the input item of the complete sample is various elements of the original official document except the document body and an abstract containing all the key information of the document body, and the output item is the document body; the input item of the MASK sample is various elements of the original official document except the document body and an abstract containing partial key information of the document body, and the output item is the document body with information supplement prompts; Based on the complete sample and the MASK sample, construct a loss function, and use the low-rank adaptation technology to fine-tune the large model to learn the document writing style and information completion ability in the sample; According to the document elements input by the user of the target institution, use the vector index to retrieve similar official documents, and construct prompt words according to the classification of the similar official documents to drive the fine-tuned large model to generate the final drafted document result.

2. The official document generation method based on a large model according to claim 1, wherein, Perform clustering analysis based on the document title to form different sub-types, including: Use a preset embedding model to convert the document title into a vector; Cluster all the vectors through a preset clustering algorithm to obtain the clustering result; Name the sub-type for each cluster, and use the document closest to the cluster center as the representative document of the sub-type; The official documents in a certain cluster are called general official documents, and the official documents that do not belong to any cluster are called general official documents.

3. The method for generating official documents based on a large model according to claim 1, wherein The loss function includes a complete sample loss, a MASK sample loss, a model completion loss used to measure the ability of the model to generate complementary information according to the context, and a style loss used to measure the difference between the text generated by the model for official documents and the target writing style.

4. The method for generating official documents based on a large model according to claim 3, wherein, The loss function Specifically: Among them, represents the model parameters, represents the complete sample set, represents the MASK sample set, and S represents the entire sample set; represents the \(i\)-th sample in the complete sample set, represents the label of the \(i\)-th sample in the complete sample set; represents the \(j\)-th sample in the MASK sample set, represents the label of the \(j\)-th MASK sample in the MASK sample set; represents the \(k\)-th sample in the entire sample set, represents the label of the \(k\)-th sample in the entire sample set; represents the model prediction result of the \(i\)-th sample in the complete sample set; represents the model prediction result of the \(j\)-th sample in the MASK sample set; represents the model prediction result of the \(k\)-th sample in the entire sample set; and represent the weights of the complete sample and the MASK sample, and represent the loss functions of the complete sample and the MASK sample, and are the weight parameters; and represent the model completion loss function and the style loss function respectively.

5. The official document generation method based on the large model according to claim 2, wherein, Use the vector index to retrieve similar official documents, and construct prompt words according to the classification of the similar official documents, including: When multiple retrieved most similar official documents belong to the same target sub-type, construct prompt words based on the target sub-type; When multiple retrieved most similar official documents do not belong to the same sub-type, only retain the style description of the target institution as a prompt word constraint for constructing prompt words.

6. The method for generating official documents based on a large model according to claim 5, wherein Construct prompt words based on the target sub-type, including: Determine the representative document based on the target sub-type; Construct prompt words based on the representative document of the target sub-type.

7. The method for generating official documents based on a large model according to claim 2, wherein The preset clustering algorithm is a density clustering algorithm.

8. An official document generation device based on a large model, characterized in that, Including: A document preprocessing module for extracting elements from the original official document. Among them, the extracted elements include: the document title and the document body. For the original official documents belonging to the same document type, perform clustering analysis based on the document title to form different sub-types, and establish a mapping between the vector of the document title, the preset elements, and the sub-types to form a vector index; A sample generation module, configured to construct a complete sample and a MASK sample respectively according to the elements of the original official document; wherein, the input items of the complete sample are various elements of the original official document except the official document text and an abstract containing all key information of the official document text, and the output item is the official document text; the input items of the MASK sample are various elements of the original official document except the official document text and an abstract containing partial key information of the official document text, and the output item is the official document text with information supplement prompts. A model fine-tuning module, configured to construct a loss function based on the complete sample and the MASK sample, and use the low-rank adaptation technique to fine-tune the large model to learn the official document writing style and information completion ability in the samples. An official document generation module, configured to retrieve similar official documents by using vector indexing according to the official document elements input by the user of the target institution, and construct prompt words according to the classification situation of the similar official documents to drive the fine-tuned large model to generate the final official document generation result.

9. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the official document generation method based on a large model according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium, having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the official document generation method based on a large model according to any one of claims 1 to 7.

Images