Manuscript verification system and method based on ChatGLM model

Abstract

The invention is suitable for the technical field of information, and provides a review decision-making system and method based on high-risk system operation. In the embodiment of the invention, before processing manuscript data, a system collects historical sample manuscripts, then determines a plurality of manuscript rules and logic relationships among the manuscript rules through the historical sample manuscripts by using a ChatGLM model, and constructs a corresponding manuscript rule engine library. When the manuscript data is processed, the manuscript data of different file formats are converted into the uniform format, then the manuscript data are segmented through a preset logic structure or configuration requirements, then the segmented data are sent to different computer nodes for manuscript correction, and finally a manuscript correction report is generated. Therefore, automatic and intelligent document checking of the official document is realized, and the processing efficiency is improved; and meanwhile, the comprehensiveness and the accuracy of verification can be improved by utilizing deep learning of the model on the document format and the language.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to a document verification system and method based on a ChatGLM model. Background Art

[0002] With the rapid development of the information age, official document processing has become an indispensable part of the daily work of enterprises and institutions. However, the existing technology has several obvious defects in the verification of official document drafts. First, the traditional manual verification method is inefficient and difficult to meet the needs of large-scale official document processing. Secondly, manual verification is easily affected by subjective factors, resulting in inconsistency and high error rate in the verification results. Furthermore, the verification of official document formats and language specifications by existing technologies is often not comprehensive and accurate enough, making it difficult to ensure the quality of official documents. In addition, the lack of intelligent auxiliary tools in the process of official document verification makes the verification work cumbersome and error-prone. Finally, with the continuous increase in the number of official documents, existing technologies are difficult to cope with the processing and analysis of massive data. Summary of the invention

[0003] In view of this, the present application provides a document verification system and method based on the ChatGLM model to realize the automation and intelligent verification of official documents and improve processing efficiency; at the same time, by using the model's deep learning of the document format and language, the comprehensiveness and accuracy of the verification can be improved.

[0004] The first aspect of the present application provides a document verification system based on the ChatGLM model, the system comprising a multi-format compatible document intelligent reading module, a document rule engine formulation module, a metadata extraction module, a distributed verification module and a proofreading result output module;

[0005] The multi-format compatible document intelligent reading module is used to identify and read target document data in at least one different file format, and convert the target document data into target data in a unified format;

[0006] The target data is segmented according to a preset logical structure or configuration requirement through the ChatGLM model to generate a plurality of segmented data, the segmented data is stored in the memory of the computer device according to preset conditions, and the state of the segmented data is set to be verified;

[0007] The document rule engine formulation module is used to collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document characteristics of the historical sample documents, and determine a number of document rules and the logical relationship between the document rules according to the input business requirements and the document characteristics, and construct a document rule engine library through the document rules and the corresponding logical relationship, wherein the document rules include identifiers, descriptions, conditions, and actions;

[0008] The metadata extraction module is used to extract key metadata from the target data and store the key metadata as an index to facilitate subsequent processing and retrieval;

[0009] The distributed verification module is used to generate a document verification task according to the segmented data, wherein the document verification task includes the size of the segmented data and a verification method, and distributes multiple document verification tasks to multiple computing nodes for parallel processing according to the size of the segmented data, so that the computer nodes generate proofreading results according to the document proofreading task;

[0010] Receiving proofreading results of the computer nodes, and after confirming that all computer nodes have returned manuscripts, summarizing the proofreading results of each computer node to generate a proofreading report of the manuscript data;

[0011] The proofreading result output module is used to output the proofreading report in the form of a file or a web page, and provide a download link for the proofreading report.

[0012] Optionally, the system further includes a file preprocessing module;

[0013] The file preprocessing module is used to preprocess the document data before converting the document data into target data in a unified format. The preprocessing includes removing invalid data in the document data and correcting coding errors of the document data.

[0014] Optionally, the system further includes a security protection module;

[0015] The security protection module is used to ensure the security and privacy of document data, including data encryption, access control and audit log recording.

[0016] Optionally, after the manuscript rule engine formulation module constructs a manuscript rule engine library through the manuscript rules and corresponding logical relationships, it further includes:

[0017] The document features are updated through the target document data, document rules and corresponding logical relationships are redetermined through the updated document features, and the document rule engine library is updated through the redetermined document rules and corresponding logical relationships.

[0018] Optionally, the computer node generates a proofreading result according to the file proofreading task, including:

[0019] According to the proofreading method in the file proofreading task, the corresponding document rules are determined from the document rule engine library, and the segmented data in the file proofreading task are formatted and semantically analyzed using the document rules to determine the defects, errors or warning contents in the segmented data, and generate corresponding proofreading results.

[0020] The second aspect of the present application provides a manuscript verification method based on the ChatGLM model, the method comprising:

[0021] Collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document characteristics of the historical sample documents, and determine a number of document rules and the logical relationship between the document rules according to the input business requirements and the document characteristics, and construct a document rule engine library through the document rules and the corresponding logical relationship, wherein the document rules include identifiers, descriptions, conditions, and actions;

[0022] Identify and read at least one target document data in different file formats through the ChatGLM model, and convert the target document data into target data in a unified format;

[0023] Segmenting the target data according to a preset logical structure or configuration requirement to generate a plurality of segmented data, storing the segmented data in a memory of a computer device according to preset conditions, and setting the status of the segmented data to be verified;

[0024] Generate a document verification task according to the segmented data, wherein the document verification task includes the size of the segmented data and a verification method, and distribute multiple document verification tasks to multiple computing nodes for parallel processing according to the size of the segmented data, so that the computer nodes generate proofreading results according to the document proofreading task;

[0025] Receiving proofreading results of the computer nodes, and after confirming that all computer nodes have returned manuscripts, summarizing the proofreading results of each computer node to generate a proofreading report of the manuscript data;

[0026] The proofreading report is output in the form of a file or a web page, and a download link for the proofreading report is provided.

[0027] Optionally, before converting the document data into target data in a unified format, the method further includes:

[0028] The document data is preprocessed, wherein the preprocessing includes removing invalid data in the document data and performing coding error correction on the document data.

[0029] Optionally, after constructing a manuscript rule engine library through the manuscript rules and corresponding logical relationships, the method further includes:

[0030] The document features are updated through the target document data, document rules and corresponding logical relationships are redetermined through the updated document features, and the document rule engine library is updated through the redetermined document rules and corresponding logical relationships.

[0031] Optionally, the computer node generates a proofreading result according to the file proofreading task, including:

[0032] According to the proofreading method in the file proofreading task, the corresponding document rules are determined from the document rule engine library, and the segmented data in the file proofreading task are formatted and semantically analyzed using the document rules to determine the defects, errors or warning contents in the segmented data, and generate corresponding proofreading results.

[0033] In the embodiment provided by the present application, before processing the manuscript data, the system first collects historical sample manuscripts, and then uses the ChatGLM model to determine a number of manuscript rules and the logical relationship between each manuscript rule through the historical sample manuscripts, and then builds the corresponding manuscript rule engine library. When processing the manuscript data, the manuscript data of each different file format is first converted into a unified format, and then segmented according to the preset logical structure or configuration requirements, and then the segmented data is sent to different computer nodes for proofreading, and finally a proofreading report is generated. The technical effects achieved by the present application are as follows:

[0034] 1: Use the ChatGLM large model to read manuscript files, support all file formats, so that the file format is no longer a limitation for manuscript verification, achieve a unified entry at the input end, and support comprehensive data file types.

[0035] 2: The customized document rule engine is comprehensive and accurate in terms of personalization and professionalism. It supports automatic upgrade of the document rule engine library and further iterates and optimizes the proofreading model.

[0036] 3: Supporting massive document verification, the proofreading service application adopts distributed service deployment, which can proofread different files and paragraphs at the same time for formatting check and semantic analysis, greatly improving the proofreading processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 A system module diagram provided for an embodiment of the present application;

[0038] Figure 2 A flow chart of the method provided in the embodiment of the present application;

[0039] Figure 3A schematic diagram of the internal structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0040] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0043] The present application provides a document verification system and method based on the ChatGLM model to realize the automation and intelligent verification of official documents and improve processing efficiency.

[0044] like Figure 1 As shown, this is a module diagram of a document verification system based on the ChatGLM model provided by the present application, the system comprising: a multi-format compatible document intelligent reading module, a document rule engine formulation module, a metadata extraction module, a distributed verification module and a proofreading result output module.

[0045] The functions of each module are described below:

[0046] 1. Multi-format compatible document intelligent reading module. This module is used to identify and read target document data in at least one different file format, and convert the target document data into target data in a unified format.

[0047] The target document data may include but is not limited to text files, PDF files, Word documents, PPT slides, and Excel spreadsheets. First, the file type is identified through a component that can identify different file formats, such as HTML5File API, WebAssembly, etc. Then, a reading interface is developed or integrated for each file format, and then these interfaces are used to open files of different formats and extract the content. The content is then converted into target data in a unified format that can be processed by the ChatGLM model.

[0048] Then the target data is input into the ChatGLM model and divided into several segmented data according to the preset logical structure, such as paragraph, title level, timestamp, specific keyword or user-defined tag. The target data can also be divided into several segmented data according to the preset configuration requirements, such as the length of the text, the location of the keyword, the natural language structure (such as sentence or paragraph), timestamp, specific tag or encoding. And the status of each segmented data is set to be saved after verification.

[0049] 2. Document rule engine formulation module. This module is used to build a document rule engine library for recording document rules, so that when the computer node is proofreading, it can determine the corresponding document rules from the document rule engine library for formatting check and semantic analysis check. The construction process is as follows:

[0050] First, collect historical sample documents from various relevant platforms or databases based on the file types that need to be verified later, such as securities, sports, etc.

[0051] Secondly, the historical sample document is input into the ChatGLM model, and the document characteristics of the historical sample document are determined through feature extraction, data analysis, characteristic induction and other steps. Then, according to the input business requirements and the document characteristics, several document rules and the logical relationship between each document rule are determined, such as sequential execution, parallel execution or conditional triggering. The document rules include identifiers, descriptions, conditions, actions and other components.

[0052] Finally, by saving the document rules and the logical relationships between the document rules, a document rule engine library can be obtained.

[0053] In another embodiment, after constructing the document rule engine library, the document features can also be updated through the target document data, and the document rules and corresponding logical relationships can be redetermined through the updated document features, and then the document rule engine library can be updated through the redetermined document rules and corresponding logical relationships.

[0054] 3. Metadata extraction module: This module is used to extract key metadata from the target data and store the key metadata as an index for subsequent processing and retrieval.

[0055] In this module, we first determine the key metadata types that need to be extracted, such as title, author, keywords, abstract, creation date, modification date, etc. Then we use text processing techniques, such as regular expressions and natural language processing (NLP) techniques, to identify and extract metadata. Then we use the extracted metadata to build an index for fast retrieval.

[0056] 4. Distributed verification module. This module consists of a task allocation node and several computer nodes that perform verification tasks. After the task allocation node obtains each segmented data in the state of being verified, it will generate multiple document verification tasks according to the size of the segmented data and the verification method. And it is assigned to each computing node for processing. For example, according to the size of the segmented data and the load of each computer node, the document verification task with small segmented data is assigned to the computer node with a larger load, and the document verification task with large segmented data is assigned to the computing node with a smaller load. In this way, the time when each computing node returns the proofreading results is relatively close.

[0057] After receiving the proofreading results from each computer node, the task allocation node can determine whether the feedback from all computing nodes has been received from the computing node ID carried in the feedback result. If it is determined, the proofreading results of each are summarized to generate a proofreading report for the above document data.

[0058] The computer node generates proofreading results based on the file proofreading task, including:

[0059] According to the proofreading methods in the document proofreading task, such as grammatical error checking, spelling error checking, terminology format checking, etc., the corresponding document rules are determined from the document rule engine library, and the segmented data in the document proofreading task is formatted and semantically analyzed through the document rules to determine the defects, errors or warnings in the segmented data, and generate the corresponding proofreading results.

[0060] 5. A proofreading result output module, which is used to output the proofreading report in the form of a file or a web page, and provide a download link for the proofreading report.

[0061] For example, an interactive interface may be provided to the user, allowing the user to view and download the proofreading report through the interactive interface.

[0062] So far, completed Figure 1 Functional description of each module in .

[0063] In the embodiment of the present application, before processing the manuscript data, the system first collects historical sample manuscripts, and then uses the ChatGLM model to determine a number of manuscript rules and the logical relationship between each manuscript rule through the historical sample manuscripts, and then builds the corresponding manuscript rule engine library. When processing the manuscript data, the manuscript data of each different file format is first converted into a unified format, and then segmented according to the preset logical structure or configuration requirements, and then the segmented data is sent to different computer nodes for proofreading, and finally a proofreading report is generated. The technical effects achieved by this application are as follows:

[0064] 1: Use the ChatGLM large model to read manuscript files, support all file formats, so that the file format is no longer a limitation for manuscript verification, achieve a unified entry at the input end, and support comprehensive data file types.

[0065] 2: The customized document rule engine is comprehensive and accurate in terms of personalization and professionalism. It supports automatic upgrade of the document rule engine library and further iterates and optimizes the proofreading model.

[0066] 3: Supporting massive document verification, the proofreading service application adopts distributed service deployment, which can proofread different files and paragraphs at the same time for formatting check and semantic analysis, greatly improving the proofreading processing efficiency.

[0067] In another embodiment, the above system further includes a file preprocessing module;

[0068] The file preprocessing module is used to preprocess the document data before converting the document data into target data in a unified format. The preprocessing includes removing invalid data in the document data and correcting coding errors of the document data.

[0069] This module can improve the accuracy and reliability of manuscript data to enhance the effectiveness of subsequent work.

[0070] In another embodiment, the above system further includes a security protection module;

[0071] The security protection module is used to ensure the security and privacy of document data, including data encryption, access control and audit log recording.

[0072] This module can prevent data leakage and ensure data integrity, thus protecting the system from data security threats.

[0073] like Figure 2 As shown, the present application also provides a manuscript verification method based on the ChatGLM model, the method comprising:

[0074] Step S201, collecting historical sample documents, inputting the historical sample documents into the ChatGLM model, determining the document characteristics of the historical sample documents, and determining a number of document rules and the logical relationship between the document rules according to the input business requirements and the document characteristics, and constructing a document rule engine library through the document rules and the corresponding logical relationship, wherein the document rules include identifiers, descriptions, conditions, and actions;

[0075] Step S202, identifying and reading target document data in at least one different file format, and converting the target document data into target data in a unified format;

[0076] Step S203, segmenting the target data according to a preset logical structure or configuration requirement through the ChatGLM model to generate a plurality of segmented data, storing the segmented data in a memory of a computer device according to preset conditions, and setting the status of the segmented data to be verified;

[0077] Step S204, generating a document verification task according to the segmented data, wherein the document verification task includes the size of the segmented data and a verification method, and allocating multiple document verification tasks to multiple computing nodes for parallel processing according to the size of the segmented data, so that the computer nodes generate proofreading results according to the document proofreading task;

[0078] Step S205, receiving the proofreading results of the computer nodes, and after confirming that all computer nodes have returned manuscripts, summarizing the proofreading results of each computer node to generate a proofreading report of the manuscript data;

[0079] Step S206: output the proofreading report in the form of a file or a web page, and provide a download link for the proofreading report.

[0080] In another embodiment, before converting the document data into target data in a unified format, the method further includes:

[0081] The document data is preprocessed, wherein the preprocessing includes removing invalid data in the document data and performing coding error correction on the document data.

[0082] In another embodiment, after constructing the manuscript rule engine library through the manuscript rules and corresponding logical relationships, the method further includes:

[0083] The document features are updated through the target document data, document rules and corresponding logical relationships are redetermined through the updated document features, and the document rule engine library is updated through the redetermined document rules and corresponding logical relationships.

[0084] In another embodiment, the computer node generates a proofreading result according to the file proofreading task, including:

[0085] According to the proofreading method in the file proofreading task, the corresponding document rules are determined from the document rule engine library, and the segmented data in the file proofreading task are formatted and semantically analyzed using the document rules to determine the defects, errors or warning contents in the segmented data, and generate corresponding proofreading results.

[0086] The above embodiment of the present invention provides a document verification system based on the ChatGLM model, and based on the system provides a document verification method based on the ChatGLM model. Through the above system and method, the automatic and intelligent verification of official documents can be realized, thereby improving the processing efficiency.

[0087] This embodiment also discloses a computer device, such as Figure 3 As shown, the computer device includes a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement any of the above-mentioned document verification methods based on the ChatGLM model.

[0088] In addition, in the above-mentioned example implementation of the document verification system based on the ChatGLM model, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the document verification system based on the ChatGLM model is divided into different program modules to complete all or part of the functions described above.

[0089] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A document verification system based on the ChatGLM model, characterized in that: The system includes a multi-format compatible document intelligent reading module, a document rule engine formulation module, a metadata extraction module, a distributed verification module and a proofreading result output module; The multi-format compatible document intelligent reading module is used to identify and read target document data in at least one different file format, and convert the target document data into target data in a unified format; The target data is segmented according to a preset logical structure or configuration requirement through the ChatGLM model to generate a plurality of segmented data, the segmented data is stored in the memory of the computer device according to preset conditions, and the state of the segmented data is set to be verified; The document rule engine formulation module is used to collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document characteristics of the historical sample documents, and determine a number of document rules and the logical relationship between the document rules according to the input business requirements and the document characteristics, and construct a document rule engine library through the document rules and the corresponding logical relationship, wherein the document rules include identifiers, descriptions, conditions, and actions; The metadata extraction module is used to extract key metadata from the target data and store the key metadata as an index to facilitate subsequent processing and retrieval; The distributed verification module is used to generate a document verification task according to the segmented data, wherein the document verification task includes the size of the segmented data and a verification method, and distributes multiple document verification tasks to multiple computing nodes for parallel processing according to the size of the segmented data, so that the computer nodes generate proofreading results according to the document proofreading task; Receiving proofreading results of the computer nodes, and after confirming that all computer nodes have returned manuscripts, summarizing the proofreading results of each computer node to generate a proofreading report of the manuscript data; The proofreading result output module is used to output the proofreading report in the form of a file or a web page, and provide a download link for the proofreading report.

2. The system according to claim 1, characterized in that The system also includes a file preprocessing module; The file preprocessing module is used to preprocess the document data before converting the document data into target data in a unified format. The preprocessing includes removing invalid data in the document data and correcting coding errors of the document data.

3. The system according to claim 1, characterized in that The system also includes a safety protection module; The security protection module is used to ensure the security and privacy of document data, including data encryption, access control and audit log recording.

4. The system according to claim 1, characterized in that After the manuscript rule engine formulation module constructs a manuscript rule engine library through the manuscript rules and corresponding logical relationships, it also includes: The document features are updated through the target document data, document rules and corresponding logical relationships are redetermined through the updated document features, and the document rule engine library is updated through the redetermined document rules and corresponding logical relationships.

5. The system according to claim 1, characterized in that The computer node generates a proofreading result according to the file proofreading task, including: According to the proofreading method in the file proofreading task, the corresponding document rules are determined from the document rule engine library, and the segmented data in the file proofreading task are formatted and semantically analyzed using the document rules to determine the defects, errors or warning contents in the segmented data, and generate the corresponding proofreading results.

6. A manuscript verification method based on the ChatGLM model, characterized in that: The method comprises: Collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document characteristics of the historical sample documents, and determine a number of document rules and the logical relationship between the document rules according to the input business requirements and the document characteristics, and construct a document rule engine library through the document rules and the corresponding logical relationship, wherein the document rules include identifiers, descriptions, conditions, and actions; Identifying and reading target document data in at least one different file format, and converting the target document data into target data in a unified format; The target data is segmented according to a preset logical structure or configuration requirement through the ChatGLM model to generate a plurality of segmented data, the segmented data is stored in the memory of the computer device according to preset conditions, and the state of the segmented data is set to be verified; Generate a document verification task according to the segmented data, wherein the document verification task includes the size of the segmented data and a verification method, and distribute multiple document verification tasks to multiple computing nodes for parallel processing according to the size of the segmented data, so that the computer nodes generate proofreading results according to the document proofreading task; Receiving proofreading results of the computer nodes, and after confirming that all computer nodes have returned manuscripts, summarizing the proofreading results of each computer node to generate a proofreading report of the manuscript data; The proofreading report is output in the form of a file or a web page, and a download link for the proofreading report is provided.

7. The method according to claim 6, characterized in that Before converting the document data into target data in a unified format, the method further includes: The document data is preprocessed, wherein the preprocessing includes removing invalid data in the document data and performing coding error correction on the document data.

8. The method according to claim 6, characterized in that After constructing the manuscript rule engine library through the manuscript rules and corresponding logical relationships, the method further includes: The document features are updated through the target document data, document rules and corresponding logical relationships are redetermined through the updated document features, and the document rule engine library is updated through the redetermined document rules and corresponding logical relationships.

9. The method according to claim 6, characterized in that The computer node generates a proofreading result according to the file proofreading task, including: According to the proofreading method in the file proofreading task, the corresponding document rules are determined from the document rule engine library, and the segmented data in the file proofreading task are formatted and semantically analyzed using the document rules to determine the defects, errors or warning contents in the segmented data, and generate the corresponding proofreading results.

Images