Big model-based official document processing method and device and official document generating and checking all-in-one machine

Abstract

The invention discloses an official document processing method and device based on a large model and an official document generating and checking all-in-one machine. The official document processing method based on the large model can quickly respond to a user request, automatically match and call a pre-trained official document generation sub-model corresponding to a literary and sports category by automatically identifying an official document title and an official document keyword, and automatically process the official document in combination with knowledge base data of a corresponding application field. According to the method, the large language model technology is utilized to quickly and intelligently generate the official document file with high quality, official document processing which is specially refined in various official document styles and is combined with the user industry can be realized, and the official document processing efficiency is greatly improved. According to the document generating and checking all-in-one machine, software and hardware are perfectly combined, the problems of document format accurate generation, large model illusion, safety risk, computing power support and the like can be solved at the same time, safe and efficient operation of the all-in-one machine in a user intranet environment is achieved, and the difficulty and cost of building and maintaining a complex system by a unit are reduced.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, and in particular relates to a document processing method, device and document generation and review integrated machine based on a large model. Background Technology

[0002] With the comprehensive advancement of digital office work, government agencies, public institutions, and state-owned enterprises are increasingly demanding higher efficiency in document processing. However, due to the strict formatting standards and specific language style requirements of official documents, current general-purpose models struggle to accurately generate content that meets these requirements. For example, different document types, such as notices, reports, and requests, have clearly defined rules from title format to body wording, and the content generated by general-purpose models often contains formatting errors or imprecise language. Summary of the Invention

[0003] This application provides a document processing method, apparatus, and document generation and review integrated machine based on a large model, which can at least solve the problem that general models in related technologies are difficult to accurately generate document content that meets the requirements.

[0004] In a first aspect, embodiments of this application provide a document processing method based on a large model, including:

[0005] Receive document generation requests sent by the user client;

[0006] Parse the document generation request to obtain the document title and keywords;

[0007] Based on the document title and keywords, determine the target document type corresponding to the document generation request;

[0008] Call the target document generation sub-model in the pre-trained target large language model that matches the target document style category. The target large language model includes multiple document generation sub-models corresponding to multiple style categories. The multiple document generation sub-models are obtained by training multiple pre-set large language models in parallel using a multi-task learning framework based on multiple historical document data and the style category labels and key elements corresponding to each historical document data.

[0009] The system queries a pre-built knowledge base that matches the document title and keywords. The knowledge base is constructed based on policy and regulatory documents and industry standard information, and includes structured knowledge base data.

[0010] The document title and keywords are input into the target document generation sub-model, and the target document file is obtained based on the knowledge base data.

[0011] Secondly, embodiments of this application provide a document processing apparatus based on a large model, the apparatus comprising:

[0012] The receiving module is configured to receive document generation requests sent by the user client;

[0013] The parsing module is configured to parse the document generation request to obtain the document title and keywords;

[0014] The determination module is configured to determine the target document type corresponding to the document generation request based on the document title and document keywords.

[0015] The calling module is configured to call the target document generation sub-model in the pre-trained target large language model that matches the target document style category. The target large language model includes multiple document generation sub-models corresponding to multiple style categories. The multiple document generation sub-models are obtained by training multiple preset large language models in parallel using a multi-task learning framework based on multiple historical document data and the style category labels and key elements corresponding to each historical document data.

[0016] The query module is configured to query knowledge base data that matches the document title and document keywords from a pre-built knowledge base, which is constructed based on policy and regulatory documents and industry standard information, including structured knowledge base data.

[0017] The input module is configured to input the document title and keywords into the target document generation sub-model, and obtain the target document file based on the knowledge base data.

[0018] Thirdly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the steps of the document processing method based on a large model as described in any embodiment of the first aspect.

[0019] Fourthly, embodiments of this application provide a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the document processing method based on a large model as provided in the first aspect of embodiments of this application.

[0020] Fifthly, embodiments of this application provide an integrated machine for generating and reviewing official documents. The integrated machine is deployed locally on the user's internal network. The integrated machine includes: a computing server, a memory, and an official document processing device based on a large model as described in any embodiment of the second aspect.

[0021] The computing server is configured for multi-core parallel computing and to connect to the user terminal's internal network via a preset dedicated network interface to transmit document data from the large-model-based document processing device to the user terminal's internal network, including a CPU processor and / or a GPU processor.

[0022] The memory is configured to store localized model data, knowledge base data, and official document data.

[0023] The document processing method, apparatus, and document generation and review integrated machine based on a large model in this application embodiment can quickly respond to user requests. By automatically recognizing document titles and keywords, it automatically matches and calls the pre-trained document generation sub-model of the corresponding document type, and combines the knowledge base data of the corresponding application field to quickly, intelligently, and with high quality generate document documents. It can achieve document processing that is specialized in various document types and combined with the user's industry, and greatly improves the efficiency of document processing. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a document processing method based on a large model provided in an embodiment of this application;

[0026] Figure 2 This is a flowchart illustrating the target large language model training method provided in the embodiments of this application;

[0027] Figure 3 This is a flowchart illustrating the specific implementation of S203 in the target large language model training method provided in this application embodiment;

[0028] Figure 4 This is a schematic diagram of the structure of a document processing device based on a large model provided in an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0030] Figure 6 This is an architecture diagram of an integrated document generation and review machine provided in an embodiment of this application.

[0031] Figure label:

[0032] The document processing device 400 based on a large model includes a receiving module 401, a parsing module 402, a determining module 403, a calling module 404, a query module 405, and an input module 406.

[0033] Electronic device 500, processor 501, memory 502, communication interface 503, bus 510. Detailed Implementation

[0034] The features and exemplary embodiments of various aspects of this application will be described in detail below. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain this application and not to limit it. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples.

[0035] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0036] With the comprehensive advancement of digital office work, government agencies, public institutions, and state-owned enterprises are increasingly demanding higher efficiency in document processing. However, due to the strict formatting standards and specific language style requirements of official documents, current general-purpose models struggle to accurately generate content that meets these requirements. For example, different document types, such as notices, reports, and requests, have clearly defined rules from title format to body wording, and the content generated by general-purpose models often contains formatting errors or imprecise language.

[0037] To address the issues in related technologies, this application provides a method, apparatus, and integrated document generation and review machine based on a large model for official document processing. Involving natural language processing, it can be deployed locally through a combination of computer hardware and software for document drafting scenarios in government agencies, public institutions, and state-owned enterprises.

[0038] The document processing method based on a large model provided in this application will be described in detail below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0039] Figure 1 A flowchart illustrating a document processing method 100 based on a large model, according to an embodiment of this application, is shown. Figure 1 As shown, the document processing method 100 based on a large model may specifically include the following steps:

[0040] S101. Receive a document generation request sent by the user client;

[0041] S102. Parse the document generation request to obtain the document title and document keywords;

[0042] S103. Determine the target document type corresponding to the document generation request based on the document title and document keywords;

[0043] S104. Call the target document generation sub-model in the pre-trained target large language model that matches the target document style category. The target large language model includes multiple document generation sub-models corresponding to multiple style categories. The multiple document generation sub-models are obtained by training multiple pre-set large language models in parallel using a multi-task learning framework based on multiple historical document data and the style category labels and key elements corresponding to each historical document data.

[0044] S105. Query the knowledge base data that matches the title and keywords of the official document from the pre-built knowledge base. The knowledge base is constructed based on policy and regulatory documents and industry standard information, including structured knowledge base data.

[0045] S106. Input the document title and document keywords into the target document generation sub-model, and obtain the target document file based on the knowledge base data.

[0046] As an optional embodiment, in S101, when a user uses the target large language model of this application embodiment to write official documents, they need to input the document title and keywords (e.g., content prompts) to send a document generation request; then, in S103, the document title and keywords in the request parsed in S102 are identified, and after similarity analysis, a classification head is added to accurately classify the document writing requirement, i.e., determine which type of document it belongs to (e.g., a notice); in S104, the document generation sub-model of the corresponding document type is called; further, in S105, it is determined which industry knowledge of the application field it involves (e.g., the large model used in the Transportation Commission should be combined with transportation field knowledge); thus, in S106, by combining industry knowledge parameters and preset entity attributes (e.g., leader's name, unit name, etc.), the document generation sub-model can generate an official document that meets the user's needs.

[0047] In other words, by adding a classification header design mechanism, the system automatically extracts the official document title and content prompts entered by the user, identifies its document type, and automatically calls the corresponding sub-model. Combined with other parameters and data, it generates high-quality official document content.

[0048] Therefore, it can quickly respond to user requests, automatically identify document titles and keywords, automatically match and call the pre-trained document generation sub-model of the corresponding document type, and combine the knowledge base data of the corresponding application field to generate official documents quickly, intelligently and with high quality. It can achieve document processing that is proficient in various document types and combined with the user's industry, and greatly improve the efficiency of document processing.

[0049] In addition, current large language models are prone to "illusion" problems when generating content, that is, generating information that seems reasonable but is actually inconsistent with the facts or lacks evidence, which is absolutely not allowed in official documents.

[0050] Therefore, to address the "illusion" problem prevalent in large models, this application proposes a trustworthy control architecture for large models, reducing the possibility of models generating false or erroneous information from the source. Specifically, it includes a data layer, a rule layer, and a training layer.

[0051] In some embodiments, at the data layer, a trusted official document database is built, and the data is rigorously screened, organized, and labeled to ensure its authenticity, accuracy, and authority.

[0052] In some embodiments, a series of constraint rules are introduced at the rule layer, including context consistency detection, semantic similarity constraints, and domain rule constraints. Specifically, for context consistency detection: by introducing a logical rule base, such as but not limited to chronological order and causal relationships, the generated content is ensured to be logically consistent with the context, avoiding contradictory statements. For semantic similarity constraints: semantic similarity calculation methods are used to constrain the semantic similarity between the generated content and the context, ensuring that the generated content is highly relevant to the context's topic. For domain rule constraints: specific rules for the official document domain are set, including but not limited to document format, terminology standards, and rules prohibiting user-defined settings, ensuring that the generated content conforms to the standardized requirements of official document writing and avoiding situations such as large model overload or illusions.

[0053] In some embodiments, at the training layer, in view of the diversity of government documents, a pre-classified and reliable document database is used to adopt a targeted training strategy. For specific document types, corresponding datasets are used for specialized training to form a dedicated "expert mini-model" for a specific document type, that is, multiple document generation sub-models that are matched with multiple document style categories respectively.

[0054] Specifically, prior to S104, a target large language model is constructed and trained. The target large language model includes multiple document generation sub-models, each of which corresponds one-to-one with a document style category, in order to improve the accuracy and applicability of the target large language model in different document scenarios.

[0055] It should be noted that, in the embodiments of this application, the document types include at least notices, requests for instructions, letters, correspondence, orders, and reports.

[0056] refer to Figure 2 This is a flowchart illustrating the target large language model training method 200 according to an embodiment of this application. Figure 2 As shown, the training method 200 for the target large language model may specifically include the following steps:

[0057] S201. Obtain multiple historical official document data and the document type category label and key elements corresponding to each historical official document data. The document type category includes at least notice, request for instructions, letter, correspondence, order, and report.

[0058] S202. Logical structure features are extracted from multiple historical official document data to obtain general feature data of official documents and multiple special feature data, wherein the special feature data corresponds one-to-one with the document category label;

[0059] S203. Using a multi-task learning framework, multiple pre-set large language models are trained in parallel based on general feature data of official documents and multiple special feature data to obtain multiple document generation sub-models corresponding to each of the multiple document categories.

[0060] S204. Integrate multiple document generation sub-models into a target large language model.

[0061] The specific implementation methods for each of the above steps are described below.

[0062] In some embodiments, in S201, a large amount of authentic historical official document data is obtained through cooperation with relevant government departments, public institutions, and central and state-owned enterprises. This historical document data is then labeled to obtain corresponding document category labels and key elements. Specifically, it can be meticulously classified according to dimensions such as document style, theme, and time, and the format characteristics and key information of different documents are labeled. For example, notification documents are labeled according to the issuing department, the recipient, and the notification subject, providing clear data labels for subsequent model training. In other words, a large number of different types of document samples are collected and preprocessed, including requests, reports, and notifications, and the samples are labeled and classified to clarify the logical structure and key elements of each type of document.

[0063] In some optional embodiments, various policy and regulatory documents, internal rules and regulations, and industry standards can be collected. An external knowledge base can then be built based on these documents and standards, and the data can be structured to facilitate rapid model retrieval and retrieval. For example, policies and regulations can be categorized by topic, and key clauses and their scope of application can be extracted and stored in an easily searchable format.

[0064] In this way, by collecting reliable government document data, meticulously classifying and labeling the document data, and integrating external knowledge base data, a fundamental guarantee is provided for the accuracy and reliability of the model.

[0065] Furthermore, in some embodiments, in S202, the general feature data of official documents includes at least: font, font size, layout position, paragraph spacing, indentation, etc.; while the special feature data corresponds one-to-one with the document category label, for example, for official documents, it includes the font, color, size and position of the issuing authority's logo, etc.

[0066] In practice, a large number of official document samples are analyzed to extract the general logical framework of official documents, such as the introduction (background and reasons), body (specific content), and conclusion (request or summary), and the learning of the general logical framework is strengthened. For each type of official document, its specific logical structure and expression characteristics are extracted. For example, a request for instructions should first explain the background and reasons, then present the specific content, and finally express the desire for approval; a report should first summarize the background and purpose, then describe the content, and finally make suggestions or a summary; a notification should first explain the matters, then describe the requirements and arrangements, and finally state the deadline and contact information.

[0067] In this way, a multi-task learning framework is adopted, which allows the model to learn the logical structure and expression characteristics of different document types at the same time. By sharing the underlying parameters, the general logical features of the documents are extracted, and a specific output layer is set for each document type to capture its unique expression characteristics.

[0068] As an optional embodiment, the specific implementation of S203 can be referred to Figure 3 This means that execution can be performed in parallel for various text types. Figure 3 The steps shown are: S2031 to S2034.

[0069] S2031. Call the pre-built knowledge base, use the document category labels, key elements, general feature data of official documents, and special feature data as input to the preset large language model, and use the historical document data as output to train the preset large language model.

[0070] It should be noted that prior to S2031, a general-purpose large language model with good scalability and powerful language processing capabilities was selected as the basic framework. The model was pre-trained on publicly available large-scale text data to enable it to initially possess language understanding and generation capabilities, master basic language grammar and semantic structure, and thus obtain the pre-set large language model.

[0071] S2032. Determine whether the preset large language model meets the preset training stopping condition. If not, if the attention layer of the preset large language model has a first low-rank matrix and a second low-rank matrix, create a third low-rank matrix, wherein the third low-rank matrix contains user-localized parameters.

[0072] S2033. Adjust the parameters of the third low-rank matrix to obtain an updated third low-rank matrix, and determine the weight change of the initial weight matrix of the preset large language model by multiplying the updated third low-rank matrix with the first low-rank matrix, the second low-rank matrix and the preset scaling factor.

[0073] S2034. Train the adjusted preset large language model until the preset training stop condition is met to obtain the document generation sub-model.

[0074] Optionally, the preset training stopping conditions include logical consistency constraints, which include semantic similarity constraints, logical rule constraints, and domain rule constraints, to ensure the logic and coherence of the generated content.

[0075] In some embodiments, by adjusting the model's loss function and hyperparameters, the model's ability to learn the logical structure and expression characteristics of official documents can be further optimized, and the model's logic and accuracy can be evaluated through verification and testing to ensure that the model can generate official documents with clear logic and complete content.

[0076] In practice, a dynamic learning rate adjustment strategy is adopted to enable the model to learn quickly in the early stages and focus more on detailed optimization in the later stages. Regularization techniques, such as adaptive regularization and multi-task regularization, are introduced to prevent overfitting. Reinforcement learning is used to evaluate and provide feedback on the content generated by the model in real time, further improving the credibility and accuracy of the generated content. Through these optimization methods, the model pays more attention to factual evidence in the data, improving the credibility of the generated content. In this way, training the model based on credible government document data and strengthening its ability to learn and generate real and accurate information by adjusting key parameters such as model weights, biases, learning rate, and regularization parameters can reduce the "illusion" phenomenon.

[0077] In another embodiment, through contrastive learning and reinforcement learning, the model learns the difference between logically clear and logically chaotic documents. That is, positive and negative samples are input into the preset model, and the logical consistency of the generated content is evaluated to optimize the model's generation capability.

[0078] Furthermore, it should be understood that localized deployment has limited computing power. However, in traditional model training schemes, the initial weight matrix of the model is usually updated directly, resulting in a large number of fine-tuning parameters, which is difficult for localized deployment to support. Current low-rank adaptation (LoRA) methods, while keeping the initial weight matrix unchanged during fine-tuning, introduce two low-rank small matrices A and B to calculate the adaptive weight matrix, which can reduce the number of training parameters. However, the number of parameters in matrices A and B is still relatively large, failing to solve the problem of insufficient computing power for localized deployment.

[0079] For example, let's define the initial weight matrix of the model as W0, with a dimension of d×k. We introduce a d×r matrix A and an r×k matrix B, where the rank r is an integer much smaller than both d and k (e.g., r = 8). Assuming the initial weight matrix W0 has a dimension of 32768×32768, fine-tuning it directly would update over a billion parameters (1,073,741,824 parameters). When we add a 32768×8 matrix A (262,144 parameters) and an 8×32768 matrix B (262,144 parameters), we need to train 262,144 + 262,144 = 524,288 parameters. Although this reduces the number of parameters required for training, it is still difficult to train the model locally when computing power is extremely limited.

[0080] Therefore, in this embodiment, to address the insufficient computing power of localized deployments and considering the user's need for fine-tuning with a small number of newly added parameters locally, the low-rank adaptation method is improved, i.e., S2032-S2034 are executed. Specifically, a third low-rank matrix W1 of e*h is created, and it should be noted that the parameters in the third low-rank matrix W1 are all professional knowledge extracted from local data or user-defined proprietary information. Then, during the fine-tuning process, the adaptive weight matrix w is calculated as follows:

[0081]

[0082] Where ΔW=AB, This indicates that the effect of the preset scaling factor, i.e., the low-rank adjustment of the initial weight matrix W0, is achieved through the preset scaling factor. To take control.

[0083] Furthermore, parameter fine-tuning is performed only on W1. Since the small matrix W1 obtained by e*h consists of a small number of local parameters, the number of parameters participating in the fine-tuning is very small. For example, adding a dimension e*h = 8 × 1024 to matrix W1 (8192 parameters) significantly reduces the number of parameters. Moreover, a preset scaling factor can be used. Control is then implemented. Therefore, even with limited computing power, this embodiment can achieve localized parameter fine-tuning by incorporating user-defined information, meeting the user's need for self-adjustment.

[0084] In some optional embodiments, smaller matrices can be created corresponding to industry knowledge parameters, unit roster parameters, and special official document terminology parameters, etc., to support fine-tuning of a model with over a billion records using minimal computing power.

[0085] Therefore, this paper proposes specific methods and parameter settings for fine-tuning large language models under limited computing power. By adding multiple small parameter matrices, the parameters of large models can be fine-tuned while greatly reducing the number of parameters. The final result is a document generation sub-model that can be fine-tuned using user-localized parameters in the user environment.

[0086] In summary, through S2031 to S2034, high-quality and reliable official document data are classified and labeled according to document type (such as notices, letters, correspondence, orders, reports, etc.). Based on different document types, multiple document generation sub-models are constructed for each type. The classified and labeled document data is used for specialized training to construct unique parameters for different document types. By optimizing the low-rank adaptation method, multiple low-rank small matrices are introduced for fine-tuning to reduce the number of fine-tuning parameters. This enables the final trained target large language model to grasp the logical structure and expression characteristics of different types of official documents and accurately generate official document documents that meet the format requirements.

[0087] The above describes the specific implementation of constructing and training a large language model according to the embodiments of this application. Multiple optimization measures play a crucial role in addressing the "illusion" problem of large models: the credibility of the data source ensures that the knowledge learned by the model is authentic and reliable, reducing the possibility of generating false information at its source; model optimization makes the generated official document content more logically rigorous and reasonable, effectively avoiding contradictions or unreasonable expressions, and improving the quality and credibility of the official documents; the external knowledge base provides the model with real-time and accurate information support such as policies, regulations, and industry standards, ensuring that the content of the official documents conforms to relevant regulations and actual conditions, and enhancing the authority of the official documents.

[0088] Back Figure 1 The document processing method 100 based on a large model is shown.

[0089] In some embodiments, in S106, the document title and document keywords are input into the target document generation sub-model to generate an outline framework corresponding to the target document type; the outline framework is filled according to the knowledge base data to obtain the document text; the target typesetting rule corresponding to the target document type is searched from the pre-set typesetting rule library, and the document text is typeset according to the target typesetting rule to obtain the target document file.

[0090] In practice, when a user initiates a document drafting request and selects a document type on their local client, the model quickly generates a corresponding outline framework based on its learning of the document's style, format, and content structure. The system automatically fills in the appropriate locations in the outline based on key information input by the user, such as the meeting topic, time, and location in a meeting notice. For example, for a work summary report, the outline generated by the model might include a work overview, work results, problems encountered, solutions, and future work plans, and, based on the work content input by the user, initially constructs an outline with specific objectives.

[0091] In practice, the model combines the organization's business characteristics, historical document data, and information from an external knowledge base to populate the content of the generated outline. Taking a government department's report on a livelihood project as an example, the model extracts specific data from the organization's internal database regarding the project's implementation, such as the number of beneficiaries and the use of funds. Simultaneously, it retrieves relevant policy information from the knowledge base and utilizes the standardized official document language learned during fine-tuning to organize this information into text content that meets the report's requirements. During the content population process, the model queries the knowledge base in real time to ensure that the cited policies, regulations, and other information are accurate and consistent with the current project's actual situation.

[0092] In specific implementation, according to the typesetting standards preset for different official document types, the model automatically typesets the generated official documents, analyzes the content of the official documents using natural language processing technology, and identifies different elements such as titles, texts, charts, etc. Call the preset typesetting rule library, and retrieve the corresponding typesetting rules according to the official document type (such as ordinary official documents or red-headed documents), covering parameter settings such as font, font size, line spacing, and page margins. For ordinary official documents, set the font (such as Song typeface, Fangsong typeface), font size (title in No. 2 font, text in No. 3 font, etc.), line spacing (usually a fixed value of about 28 points), and page margins and other parameters according to the standard. For red-headed documents, strictly follow the regulations to generate the issuing agency logo, ensure that its font, color, and position meet the standards, and accurately arrange elements such as the document number and dividing line. At the same time, the model will automatically adjust the page layout according to the amount of content in the official document to ensure the overall beauty and standardization of the official document.

[0093] In this way, the outline generation algorithm is customized according to the stylistic characteristics of official documents. By analyzing the theme and requirements of official documents using natural language processing technology, extracting key information, and combining with the preset official document structure template, a logical and standardized outline is generated; the content filling algorithm combines the application method of unit business data, extracts relevant information from data such as the unit's historical official documents, business processes, policies and regulations, etc. through data mining and knowledge graph construction, and fills it into each part of the official document to improve the accuracy and practicality of the generated content; the automatic typesetting algorithm follows national standards, and through text analysis and format recognition technology, automatically adjusts parameters such as the font, font size, line spacing, and page margins of the official document to ensure the standardization and beauty of the official document format.

[0094] In addition, in some embodiments, after S106, the method further includes: correcting typos in the target official document file using a pre-constructed character-level deep learning model; using a keyword interception mechanism to identify whether there are preset illegal keywords in the target official document file, and in response to the existence of preset illegal keywords, intercept and correct them.

[0095] In specific implementation, by comparing with the personnel information database within the unit, check whether key information such as the names of leaders is accurate. For typo checking, the model uses deep learning algorithms to analyze each word in the text and judges whether there are typos in combination with the context semantics. For example, it can accurately identify situations where "large language model" is miswritten as "large prediction model", or the leader's name is miswritten as other homophonic words.

[0096] In specific implementation, in terms of detecting inappropriate language, the model identifies expressions such as colloquialisms and Internet buzzwords that do not conform to the norms in the text according to the pre-set formal official document language specification library, and provides appropriate replacement suggestions, such as replacing "make things happen" with "carry out activities", to ensure the formality and solemnity of the official document language style.

[0097] In practice, a keyword interception mechanism is used to monitor the model-generated results in real time. When predefined keywords that may indicate false, erroneous, or inappropriate information appear, the system automatically triggers interception and prompts the user to check and correct the content, or regenerate the content. In this way, the keyword interception mechanism in the generated results acts as a last line of defense, which can promptly detect and prevent potentially "illusory" content, further ensuring the authenticity and accuracy of official document content.

[0098] As can be seen, after the official document is generated, a comprehensive review of its content is conducted using various technologies, including natural language processing, deep learning algorithms, rule matching and semantic analysis, keyword interception mechanisms, and machine learning algorithms. A character-level deep learning model identifies and corrects typos, identifies non-standard expressions in the text based on a pre-defined library of official document language standards, and provides replacement suggestions. Simultaneously, a keyword interception mechanism monitors the model's results in real time. When predefined keywords that may indicate false, erroneous, or inappropriate information appear, the system automatically triggers interception and prompts the user to check and correct them. This ensures the accuracy and standardization of the official document.

[0099] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0100] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides a document processing device 400 based on a large model.

[0101] like Figure 4 As shown, the document processing device 400 based on a large model may include:

[0102] The receiving module 401 is configured to receive document generation requests sent by the user terminal;

[0103] The parsing module 402 is configured to parse the document generation request to obtain the document title and document keywords;

[0104] The determination module 403 is configured to determine the target document type corresponding to the document generation request based on the document title and document keywords.

[0105] The calling module 404 is configured to call the target document generation sub-model in the pre-trained target large language model that matches the target document style category. The target large language model includes multiple document generation sub-models corresponding to multiple style categories. The multiple document generation sub-models are obtained by training multiple preset large language models in parallel using a multi-task learning framework based on multiple historical document data and the style category labels and key elements corresponding to each historical document data.

[0106] The query module 405 is configured to query knowledge base data that matches the document title and document keywords from a pre-built knowledge base. The knowledge base is constructed based on policy and regulatory documents and industry standard information, including structured knowledge base data.

[0107] The input module 406 is configured to input the document title and document keywords into the target document generation sub-model, and obtain the target document file based on the knowledge base data.

[0108] In some embodiments, the input module 406 is specifically configured to: input the document title and document keywords into the target document generation sub-model to generate an outline framework corresponding to the target document type; fill the outline framework according to the knowledge base data to obtain the document text; search for the target typesetting rule corresponding to the target document type from a pre-set typesetting rule library, and typeset the document text according to the target typesetting rule to obtain the target document file.

[0109] In some embodiments, the large-model-based document processing apparatus 400 further includes a training module ( Figure 4 (Not shown in the image). Specifically, the training module includes the following units:

[0110] The acquisition unit is configured to acquire multiple historical official document data and the document category label and key elements corresponding to each historical official document data. The document category includes at least notice, request for instructions, letter, correspondence, order, and report.

[0111] The extraction unit is configured to extract logical structure features from multiple historical official document data to obtain general feature data of official documents and multiple special feature data, wherein the special feature data corresponds one-to-one with the document category label;

[0112] The training unit is configured to use a multi-task learning framework to train multiple preset large language models in parallel based on general feature data of official documents and multiple special feature data, so as to obtain multiple document generation sub-models corresponding to each of the multiple document categories.

[0113] The integration unit is configured to integrate multiple document generation sub-models into a target large language model.

[0114] In some optional embodiments, the training unit is specifically configured to perform the following steps in parallel for each document category: call a pre-built knowledge base, using the document category labels, key elements, general document feature data, and specific feature data as input to the preset large language model, and the historical document data as output to train the preset large language model; determine whether the preset large language model meets a preset training stopping condition; if not, create a first low-rank matrix, a second low-rank matrix, and a third low-rank matrix in the attention layer of the preset large language model, wherein the third low-rank matrix contains user-localized parameters; adjust the parameters of the third low-rank matrix to obtain an updated third low-rank matrix, and determine the weight change of the initial weight matrix of the preset large language model by multiplying the updated third low-rank matrix with the first low-rank matrix, the second low-rank matrix, and a preset scaling factor; train the adjusted preset large language model until the preset training stopping condition is met, thus obtaining a document generation sub-model.

[0115] Optionally, the preset training stopping condition includes a logical consistency constraint, which includes semantic similarity constraints, logical rule constraints, and domain rule constraints.

[0116] In some embodiments, the large-model-based document processing apparatus 400 further includes a proofreading module. Figure 4 (Not shown in the image), the proofreading module is configured to use a pre-built character-level deep learning model to correct typos in the target document; and to use a keyword interception mechanism to identify whether there are preset illegal keywords in the target document, and to intercept and correct them in response to the presence of preset illegal keywords.

[0117] It should be noted that, for ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0118] The apparatus described above is used to implement the corresponding large-model-based document processing method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0119] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides an electronic device.

[0120] Figure 5 A schematic diagram of a more specific electronic device hardware structure provided in this embodiment is shown.

[0121] The electronic device 500 may include a processor 501 and a memory 502 storing computer program instructions.

[0122] Specifically, the processor 501 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0123] Memory 502 may include mass storage for data or instructions. For example, and not limitingly, memory 502 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 502 may include removable or non-removable (or fixed) media. Where appropriate, memory 502 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 502 is non-volatile solid-state memory.

[0124] In certain embodiments, the memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Thus, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this application.

[0125] The processor 501 reads and executes computer program instructions stored in the memory 502 to implement any of the document processing methods based on a large model in the above embodiments.

[0126] In some examples, the electronic device 500 may also include a communication interface 503 and a bus 510. For example, Figure 5 As shown, the processor 501, memory 502, and communication interface 503 are connected through bus 510 and complete communication with each other.

[0127] The communication interface 503 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0128] Bus 510 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not as a limitation, bus 510 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 510 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0129] For example, the electronic device 500 can be a mobile phone, tablet computer, laptop computer, handheld computer, in-vehicle electronic device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0130] Based on the same technical concept, corresponding to any of the methods in the above embodiments, this application also provides a non-transitory computer-readable storage medium. This computer-readable storage medium stores computer program instructions; when executed by a processor, these computer program instructions implement any of the large-model-based document processing methods in the above embodiments. Examples of computer-readable storage media include non-transitory computer-readable storage media such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0131] Based on the same technical concept, corresponding to any of the above embodiments, this application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer to cause the computer and / or the processors to perform the large-model-based document processing method. Corresponding to the execution entity for each step in each embodiment of the large-model-based document processing method, the processor executing the corresponding step can belong to the corresponding execution entity.

[0132] In addition, this application also provides an integrated machine for generating and reviewing official documents.

[0133] It is understandable that most large language models in related technologies are deployed in the cloud, posing security risks to data transmission and storage. For government agencies, public institutions, and state-owned enterprises, official documents often contain sensitive information, such as policy details and internal work arrangements. Uploading this data to the cloud for processing may lead to threats such as data leaks and malicious attacks, seriously affecting the normal operation and information security of the organization.

[0134] Furthermore, regarding computing power support, existing document processing solutions based on large language models, when deployed on ordinary office computers, suffer from slow processing speeds due to the massive computational demands of the models, failing to meet the needs of rapid document generation in practical work. Building a dedicated computing server is not only costly but also requires a professional operations and maintenance team for management, posing significant technical and economic challenges for many organizations.

[0135] Therefore, this application provides an integrated machine for generating and reviewing official documents, which combines large language model software specifically optimized for official document scenarios with a supporting computing server, and enables local deployment to the user's intranet environment.

[0136] It should be noted that the document generation and review all-in-one machine in this application embodiment is deployed locally on the user's internal network. In terms of security, by deploying the all-in-one machine locally within the user's intranet environment, the security risks associated with uploading data to the cloud are completely eliminated. Sensitive document data from government agencies, public institutions, and state-owned enterprises are processed and stored within their internal network environment, without needing to be uploaded to the cloud. This effectively prevents data leakage and malicious attacks during transmission and storage, greatly ensuring the security and confidentiality of the organization's information. This allows organizations to confidently use the device for document processing without worrying about information security issues.

[0137] refer to Figure 6 This is an architectural diagram of the document generation and review integrated machine according to an embodiment of this application. Figure 6 As shown, the all-in-one machine includes: a computing server, a storage device, and a document processing device 400 based on a large model.

[0138] The computing server is configured for multi-core parallel computing and connects to the user's internal network via a pre-defined dedicated network interface to transmit document data from the large model-based document processing device to the user's internal network, including a CPU processor and / or a GPU processor; the memory is configured to store localized model data, knowledge base data, and document data.

[0139] Optionally, as the hardware core of the all-in-one machine, the computing server adopts high-performance CPU and GPU processors with multi-core parallel computing capabilities to meet the needs of complex calculations of large language models. For example, a professional server processor with powerful floating-point operation capabilities is selected to ensure rapid response when processing large-scale official document data and complex model calculations.

[0140] Optionally, the server is equipped with high-capacity, high-speed memory to ensure rapid data retrieval and storage during model operation, avoiding computational lag due to insufficient memory. Simultaneously, high-speed, high-capacity storage devices are configured to store localized model data, user-accessed materials, and generated official documents. The storage devices employ redundant array technology to enhance data storage security and reliability, preventing data loss.

[0141] Thus, from a hardware computing power perspective, the supporting computing servers provide powerful computational capabilities for the large language model. High-performance processors, large-capacity high-speed memory, and high-speed storage devices enable the model to quickly respond to user requests and handle complex computational tasks, significantly shortening the time for document generation, typesetting, and proofreading, and substantially improving document processing efficiency. Staff no longer need to wait for long system responses, enabling them to complete work tasks more efficiently and improving overall office efficiency.

[0142] Optionally, the server is equipped with a dedicated network interface that connects to the user's intranet, enabling data exchange with the user's internal network through secure network protocols to ensure data transmission security.

[0143] Therefore, the document generation and review integrated machine of this application perfectly combines software and hardware, simultaneously solving multiple problems such as accurate document format generation, the "illusion" problem, security risks, and computing power support. It enables the integrated machine to operate securely and efficiently within the user's intranet environment, reducing the difficulty and cost for organizations to build and maintain complex systems independently. Organizations do not need to invest significant funds and technical resources to build professional computing servers and complex software systems; they can quickly establish a secure and efficient document processing platform simply by introducing the integrated machine of this application, offering extremely high cost-effectiveness and scalability.

[0144] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0145] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0146] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0147] The aspects of this application have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0148] The above description is merely a specific implementation of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A document processing method based on a large model, characterized in that, include: Receive document generation requests sent by the user client; Parse the document generation request to obtain the document title and keywords; Based on the document title and keywords, determine the target document type corresponding to the document generation request; Call the target document generation sub-model in the pre-trained target large language model that matches the target document style category. The target large language model includes multiple document generation sub-models corresponding to multiple style categories. The multiple document generation sub-models are obtained by training multiple pre-set large language models in parallel using a multi-task learning framework based on multiple historical document data and the style category labels and key elements corresponding to each historical document data. The system queries a pre-built knowledge base that matches the document title and keywords. The knowledge base is constructed based on policy and regulatory documents and industry standard information, and includes structured knowledge base data. The document title and keywords are input into the target document generation sub-model, and the target document file is obtained based on the knowledge base data.

2. The method according to claim 1, characterized in that, Before invoking the target document generation sub-model in the pre-trained target large language model that matches the target document style category, the method further includes: Acquire multiple historical official document data and the corresponding document type category tags and key elements for each historical official document data. The document type categories include at least notices, requests for instructions, letters, correspondence, orders, and reports. Logical structure features are extracted from multiple historical official document data to obtain general document feature data and multiple special feature data. The special feature data corresponds one-to-one with the document category label. A multi-task learning framework is adopted, and multiple pre-set large language models are trained in parallel based on general feature data of official documents and multiple special feature data to obtain multiple document generation sub-models corresponding to each of the multiple document categories. Multiple document generation sub-models are integrated into a target large language model.

3. The method according to claim 2, characterized in that, The aforementioned multi-task learning framework trains multiple pre-set large language models in parallel based on general document feature data and multiple specialized feature data, resulting in multiple document generation sub-models corresponding to each document type, including: For each genre, the following steps are performed in parallel: The pre-built knowledge base is invoked, and the text category labels, key elements, general feature data of official documents, and special feature data of official documents are used as inputs to the preset large language model, and the historical official document data is used as the output of the preset large language model to train the preset large language model. Determine whether the preset large language model meets the preset training stopping condition. If it does not meet the condition, and if the attention layer of the preset large language model has a first low-rank matrix and a second low-rank matrix, create a third low-rank matrix, wherein the third low-rank matrix contains user-localized parameters. The parameters of the third low-rank matrix are adjusted to obtain an updated third low-rank matrix, and the product of the updated third low-rank matrix, the first low-rank matrix, the second low-rank matrix, and the preset scaling factor is determined as the weight change of the initial weight matrix of the preset large language model. The preset large language model is trained and adjusted until the preset training stopping condition is met, thus obtaining the document generation sub-model.

4. The method according to claim 3, characterized in that, The preset training stop condition includes a logical consistency constraint, which includes semantic similarity constraints, logical rule constraints, and domain rule constraints.

5. The method according to claim 1, characterized in that, The step of inputting the document title and keywords into the target document generation sub-model, and obtaining the target document file based on the knowledge base data, includes: Input the document title and keywords into the target document generation sub-model to generate an outline framework corresponding to the target document document type. Based on the knowledge base data, the outline framework is filled in to obtain the official document text; The target typesetting rule corresponding to the target document type is found in the pre-defined typesetting rule library, and the document text is typed according to the target typesetting rule to obtain the target document file.

6. The method according to claim 1, characterized in that, After inputting the document title and keywords into the target document generation sub-model and obtaining the target document file based on the knowledge base data, the method further includes: The typos in the target official document are corrected using a pre-built character-level deep learning model; The keyword interception mechanism is used to identify whether there are preset illegal keywords in the target document. In response to the existence of preset illegal keywords, the keywords are intercepted and corrected.

7. A document processing device based on a large model, characterized in that, The device includes: The receiving module is configured to receive document generation requests sent by the user client; The parsing module is configured to parse the document generation request to obtain the document title and keywords; The determination module is configured to determine the target document type corresponding to the document generation request based on the document title and document keywords. The calling module is configured to call the target document generation sub-model in the pre-trained target large language model that matches the target document style category. The target large language model includes multiple document generation sub-models corresponding to multiple style categories. The multiple document generation sub-models are obtained by training multiple preset large language models in parallel using a multi-task learning framework based on multiple historical document data and the style category labels and key elements corresponding to each historical document data. The query module is configured to query knowledge base data that matches the document title and document keywords from a pre-built knowledge base, which is constructed based on policy and regulatory documents and industry standard information, including structured knowledge base data. The input module is configured to input the document title and keywords into the target document generation sub-model, and obtain the target document file based on the knowledge base data.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when invoked by a processor, implement the document processing method based on a large model as described in any one of claims 1-6.

9. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device performs the document processing method based on a large model as described in any one of claims 1-6.

10. A document generation and review integrated machine, characterized in that, The all-in-one machine is deployed locally on the user's internal network, and the all-in-one machine includes: a computing server, a storage device, and the document processing device based on a large model as described in claim 7. The computing server is configured for multi-core parallel computing and to connect to the user terminal's internal network via a preset dedicated network interface to transmit document data from the large-model-based document processing device to the user terminal's internal network, including a CPU processor and / or a GPU processor. The memory is configured to store localized model data, knowledge base data, and official document data.

Images