A document proofreading system and method based on a ChatGLM model

Abstract

The application belongs to the field of information technology, and provides a review decision system and method based on high-risk system operation. In the embodiment, the system collects historical sample documents before processing document data, and then uses a ChatGLM model to determine a plurality of document rules and the logical relationship between the document rules through the historical sample documents, and then constructs a corresponding document rule engine library. When processing the document data, the document data in different file formats is first converted into a unified format, and then segmented through a preset logical structure or configuration requirement, and then the segmented data is sent to different computer nodes for proofreading, and finally a proofreading report is generated. This realizes the automatic and intelligent proofreading of official documents, improves the processing efficiency; at the same time, the deep learning of the model on the official document format and language can improve the comprehensiveness and accuracy of the verification.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, in particular to a document checking system and method based on ChatGLM model. BACKGROUND

[0002] With the rapid development of the information age, document processing has become an indispensable part of daily work in enterprises and institutions. However, the existing technology has many obvious defects in document checking. First, the traditional manual checking method is inefficient and difficult to meet the needs of large-scale document processing. Second, manual checking is easily influenced by subjective factors, resulting in inconsistency and high error rate of checking results. Third, the existing technology for checking document format and language specifications is often not comprehensive and accurate enough to ensure the quality of the document. In addition, the lack of intelligent auxiliary tools in the document checking process makes the checking work tedious and prone to errors. Finally, with the increasing number of documents, the existing technology is difficult to handle and analyze massive data. SUMMARY

[0003] Therefore, the present application provides a document checking system and method based on ChatGLM model, which realizes automatic and intelligent checking of documents and improves processing efficiency. At the same time, the model learns the format and language of the document deeply, which can improve the comprehensiveness and accuracy of the checking.

[0004] The first aspect of the present application provides a document checking system based on ChatGLM model, which comprises a multi-format compatible document intelligent reading module, a document rule engine development module, a metadata extraction module, a distributed checking module and a checking result output module.

[0005] The multi-format compatible document intelligent reading module is used to identify and read at least one target document data in different file formats, and convert the target document data into a unified format target data.

[0006] The ChatGLM model is used to segment the target data according to the preset logical structure or configuration requirements, generate a plurality of segmented data, store the segmented data in the memory of the computer device according to the preset conditions, and set the state of the segmented data to be checked.

[0007] The document rule engine development module is used to collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document characteristics of the historical sample documents, and determine a plurality of document rules and the logical relationship between each document rule according to the input business requirements and the document characteristics, and construct a document rule engine library through the document rules and the corresponding logical relationship, wherein the document rules include identifier, description, condition, action.

[0008] The metadata extraction module is configured to extract key metadata from the target data and store the key metadata as an index for subsequent processing and retrieval.

[0009] The distributed proofreading module is configured to generate a manuscript proofreading task according to the segmented data, the manuscript proofreading task including the size of the segmented data and a proofreading method, and distribute a plurality of manuscript proofreading tasks to a plurality of computing nodes according to the size of the segmented data for parallel processing, so that the computing nodes generate proofreading results according to the manuscript proofreading tasks.

[0010] The distributed proofreading module is configured to generate a manuscript proofreading task according to the segmented data, the manuscript proofreading task including the size of the segmented data and a proofreading method, and distribute a plurality of manuscript proofreading tasks to a plurality of computing nodes according to the size of the segmented data for parallel processing, so that the computing nodes generate proofreading results according to the manuscript proofreading tasks.

[0011] The proofreading result output module is configured to output the proofreading report in the form of a file or a webpage and provide a download link of the proofreading report.

[0012] Optionally, the system further comprises a file preprocessing module.

[0013] The file preprocessing module is configured to preprocess the manuscript data before converting the manuscript data into target data in a unified format, the preprocessing including removing invalid data in the manuscript data and correcting encoding errors of the manuscript data.

[0014] Optionally, the system further comprises a security protection module.

[0015] The security protection module is configured to ensure the security and privacy of the manuscript data, including data encryption, access control and audit log recording.

[0016] Optionally, after the manuscript rule engine formulation module formulates the manuscript rule engine library by using the manuscript rules and the corresponding logical relationships, the manuscript rule engine formulation module further comprises:

[0017] The manuscript rule engine formulation module formulates the manuscript rule engine library by using the manuscript rules and the corresponding logical relationships, and the manuscript rule engine formulation module further comprises:

[0018] Optionally, the computing nodes generate proofreading results according to the manuscript proofreading tasks include:

[0019] According to the proofreading method in the file proofreading task, corresponding document rules are determined from the document rule engine library, the segmented data in the file proofreading task is subjected to format checking and semantic analysis checking through the document rules, defects, errors or warning contents in the segmented data are determined, and corresponding proofreading results are generated.

[0020] The second aspect of the application provides a document verification method based on a ChatGLM model, which comprises:

[0021] Historical sample documents are collected, the historical sample documents are input into the ChatGLM model, document characteristics of the historical sample documents are determined, a plurality of document rules and logical relationships between the document rules are determined according to input business requirements and the document characteristics, and a document rule engine library is constructed through the document rules and the corresponding logical relationships, wherein the document rules comprise an identifier, a description, a condition and an action;

[0022] The ChatGLM model is used to identify and read target document data in at least one or more different file formats, and the target document data is converted into target data in a unified format;

[0023] The target data is segmented according to a preset logical structure or configuration requirement, a plurality of segmented data are generated, the segmented data are stored in the memory of a computer device according to a preset condition, and the state of the segmented data is set to be verified;

[0024] A document verification task is generated according to the segmented data, the document verification task comprises the size of the segmented data and a verification method, a plurality of document verification tasks are distributed to a plurality of computing nodes for parallel processing according to the size of the segmented data, and the computing nodes generate proofreading results according to the file proofreading task;

[0025] The proofreading results of the computing nodes are received, after all the computing nodes return the proofreading results, the proofreading results are summarized to generate a proofreading report of the document data;

[0026] The proofreading report is output in the mode of a file or a webpage, and a download link of the proofreading report is provided.

[0027] Optionally, before the document data is converted into target data in a unified format, the method further comprises:

[0028] The document data is preprocessed, and the preprocessing comprises removing invalid data in the document data and correcting coding errors of the document data.

[0029] Optionally, after the document rule engine library is constructed by the document rules and the corresponding logical relationship, the method further comprises:

[0030] The document features are updated by the target document data, the document rules and the corresponding logical relationship are re-determined by the updated document features, and the document rule engine library is updated by the re-determined document rules and the corresponding logical relationship.

[0031] Optionally, the computer node generates the proofreading result according to the file proofreading task comprises:

[0032] According to the proofreading method in the file proofreading task, the corresponding document rule is determined from the document rule engine library, the segmented data in the file proofreading task is subjected to format checking and semantic analysis checking by the document rule, the defects, errors or warning contents in the segmented data are determined, and the corresponding proofreading result is generated.

[0033] In the embodiments provided in the application, before processing document data, the system first collects historical sample documents, and then uses a ChatGLM model to determine a plurality of document rules and logical relationships between the document rules by the historical sample documents to construct a corresponding document rule engine library. When processing document data, the document data in different file formats are first converted into a unified format, then segmented by a preset logical structure or configuration requirement, and then sent to different computer nodes for proofreading to finally generate a proofreading report. The technical effects achieved by the application are as follows:

[0034] 1: The ChatGLM large model is used to read document files, all file formats are supported, the file format is no longer a limitation for document verification, a unified entrance is achieved at the input end, and the data file types are fully supported.

[0035] 2: The document rule engine is customized, and the individuality and professionalism are fully and accurately achieved, the document rule engine library is automatically upgraded, and the proofreading model is further iteratively optimized.

[0036] 3: A large amount of document verification is supported, the proofreading service application adopts distributed service deployment, different files and different paragraphs can be proofread at the same time for format checking and semantic analysis, and the proofreading processing efficiency is greatly improved. DETAILED DESCRIPTION

[0037] Figure 1 The system module diagram provided for the embodiments of the application;

[0038] Figure 2 The method flowchart provided for the embodiments of the application;

[0039] Figure 3A computer device internal structure schematic diagram provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] The exemplary embodiments will be described in detail herein with reference to the attached drawings. In the following description, the same numbers are used to indicate the same or similar components. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the present application and the appended claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0042] It will be understood that, although the terms first, second, third, etc. can be used herein to describe various information, these terms are not intended to denote a temporal or chronological order. Rather, these terms are used merely as labels to identify particular information. For example, a first information can be termed a second information, and similarly, a second information can be termed a first information, without departing from the scope of the present application. Depending on the context, the word "if' as used herein can be interpreted to mean "when" or "in response to determining" or "in response to a determination".

[0043] The present application provides a document proofreading system and method based on ChatGLM model to realize automatic and intelligent document proofreading and improve processing efficiency.

[0044] As shown in Figure 1 A module diagram of a document proofreading system based on ChatGLM model provided by the present application, the system comprises: a multi-format compatible document intelligent reading module, a document rule engine formulation module, a metadata extraction module, a distributed proofreading module and a proofreading result output module.

[0045] The functions of each module are described as follows:

[0046] 1. Multi-format compatible document intelligent reading module. This module is used to identify and read at least one or more different file formats of target document data, and convert the target document data into a unified format of target data.

[0047] The target document data can include, but is not limited to, text files, PDF files, Word documents, PowerPoint presentations, and Excel spreadsheets. First, a component capable of recognizing different file formats, such as the HTML5 File API or WebAssembly, is used to identify the file type. Then, reading interfaces are developed or integrated for each file format, and these interfaces are used to open the files of different formats and extract their content. Finally, this content is converted into target data in a unified format that the ChatGLM model can process.

[0048] The target data is then input into the ChatGLM model and divided into several segments according to a preset logical structure, such as by paragraph, heading level, timestamp, specific keywords, or user-defined tags. Alternatively, it can be divided into segments according to preset configuration requirements, such as text length, keyword placement, natural language structure (e.g., sentences or paragraphs), timestamps, specific tags, or encodings. Each segment is then set to "pending verification" before being saved.

[0049] 2. Document Rule Engine Definition Module. This module is used to build a document rule engine library for recording document rules. This allows computer nodes to determine the corresponding document rules from the document rule engine library for formatting checks and semantic analysis during proofreading. The construction process is as follows:

[0050] First, based on the type of document that needs to be verified later, such as securities or sports documents, historical sample documents are collected from various relevant platforms or databases.

[0051] Secondly, the historical sample document is input into the ChatGLM model. Through feature extraction, data analysis, and characteristic summarization, the document characteristics of the historical sample document are determined. Then, based on the input business requirements and the document characteristics, several document rules and the logical relationships between these rules are determined, such as sequential execution, parallel execution, or conditional triggering. Document rules include components such as identifiers, descriptions, conditions, and actions.

[0052] Finally, by saving the document rules and the logical relationships between them, you can obtain the document rule engine library.

[0053] In another embodiment, after constructing the document rule engine library, the document features can be updated using the target document data, the document rules and corresponding logical relationships can be redefined using the updated document features, and the document rule engine library can be updated using the redefined document rules and corresponding logical relationships.

[0054] 3, Metadata extraction module. This module is used to extract key metadata from the target data and store the key metadata as an index for subsequent processing and retrieval.

[0055] In this module, first determine the type of key metadata that needs to be extracted, such as title, author, keywords, abstract, creation date, modification date, etc. Then through text processing technology, such as using regular expressions, natural language processing (NLP) technology and other methods to identify and extract metadata. Then use the extracted metadata to build an index to facilitate fast retrieval.

[0056] 4, Distributed verification module. This module consists of a task allocation node and several computer nodes that perform verification tasks. The task allocation node obtains each segmented data in the state of waiting for verification, generates multiple manuscript verification tasks through the size of the segmented data and the verification method, and distributes them to each computer node for processing, such as according to the size of the segmented data and the load of each computer node. The manuscript verification task with small segmented data is allocated to the computer node with large load, and the manuscript verification task with large segmented data is allocated to the computer node with small load. Thus, the time for each computer node to return the manuscript result is relatively close.

[0057] After the task allocation node receives the manuscript results from each computer node, it can determine whether all computer nodes have returned the manuscript from the computer node ID carried in the manuscript result. When determined, the manuscript results are summarized to generate the manuscript report of the above manuscript data.

[0058] The computer node generates a manuscript result according to the file manuscript task, which includes:

[0059] According to the manuscript method in the file manuscript task, such as syntax error verification, spelling error verification, term format verification, etc., determine the corresponding manuscript rules from the manuscript rule engine library, and perform format checking and semantic analysis checking on the segmented data in the file manuscript task through the manuscript rules. Determine the defects, errors or warning content in the segmented data, and generate the corresponding manuscript result.

[0060] 5, Manuscript result output module, this module is used to output the manuscript report through the mode of file or web page, and provides the download link of the manuscript report.

[0061] For example, an interactive interface can be provided to the user, so that the user can view and download the manuscript report through the interactive interface.

[0062] Thus, the functions of each module in Figure 1 are completed.

[0063] In the embodiments of the present application, before processing the manuscript data, the system first collects historical sample manuscripts, and then uses the ChatGLM model to determine a plurality of manuscript rules and the logical relationship between each manuscript rule through the historical sample manuscripts, and then constructs a corresponding manuscript rule engine library. When processing the manuscript data, first convert the manuscript data of each different file format into a unified format, then segment through the preset logical structure or configuration requirements, then send the segmented data to different computer nodes for proofreading, and finally generate a proofreading report. The technical effects realized by the present application are as follows:

[0064] 1: Use ChatGLM large model to read manuscript files, support all file formats, let file format no longer become the limitation of manuscript verification, realize unified entrance at the input end, and support comprehensive data file type.

[0065] 2: Custom manuscript rule engine, realize comprehensiveness and accuracy in personalization and professionalism, support automatic upgrading of manuscript rule engine library, and further iterate and optimize the proofreading model.

[0066] 3: Support mass manuscript verification, the proofreading service application adopts distributed service deployment, can check different files and different paragraphs at the same time for formatting check and semantic analysis, greatly improving the proofreading processing efficiency.

[0067] In another embodiment, the above system further comprises a file preprocessing module;

[0068] The file preprocessing module is used to preprocess the manuscript data before converting the manuscript data into target data in a unified format, which includes removing invalid data in the manuscript data and correcting encoding errors of the manuscript data.

[0069] Through this module, the accuracy and reliability of the manuscript data can be improved to improve the effect of subsequent work.

[0070] In another embodiment, the above system further comprises a security protection module;

[0071] The security protection module is used to ensure the security and privacy of the manuscript data, including data encryption, access control and audit log recording.

[0072] Through this module, data leakage can be prevented, the integrity of the data can be ensured, and the system can be protected from data security threats.

[0073] As Figure 2 shown, the present application also provides a manuscript verification method based on a ChatGLM model, which comprises:

[0074] Step S201, collect historical sample manuscripts, input the historical sample manuscripts into the ChatGLM model, determine the manuscript characteristics of the historical sample manuscripts, and determine a plurality of manuscript rules and logical relationships between the manuscript rules according to the input business requirements and the manuscript characteristics, and construct a manuscript rule engine library through the manuscript rules and the corresponding logical relationships, wherein the manuscript rules include an identifier, a description, a condition, and an action;

[0075] Step S202, identify and read target manuscript data in at least one of different file formats, and convert the target manuscript data into target data in a unified format;

[0076] Step S203, segment the target data according to a preset logical structure or configuration requirement through the ChatGLM model, generate a plurality of segmented data, store the segmented data in the memory of the computer device according to a preset condition, and set the state of the segmented data to be checked;

[0077] Step S204, generate a manuscript checking task according to the segmented data, wherein the manuscript checking task includes the size of the segmented data and a checking method, and distribute a plurality of manuscript checking tasks to a plurality of computing nodes for parallel processing according to the size of the segmented data, so that the computing nodes generate a manuscript checking result according to the manuscript checking task;

[0078] Step S205, receive the manuscript checking result of the computing node, determine that all computing nodes have returned the manuscript, and then aggregate the manuscript checking results to generate a manuscript checking report of the manuscript data;

[0079] Step S206, output the manuscript checking report in a file or webpage mode, and provide a download link of the manuscript checking report.

[0080] In another embodiment, before converting the manuscript data into target data in a unified format, the method further comprises:

[0081] preprocessing the manuscript data, wherein the preprocessing includes removing invalid data in the manuscript data and correcting coding errors of the manuscript data.

[0082] In another embodiment, after constructing the manuscript rule engine library through the manuscript rules and the corresponding logical relationships, the method further comprises:

[0083] updating the manuscript characteristics through the target manuscript data, re-determining the manuscript rules and the corresponding logical relationships through the updated manuscript characteristics, and updating the manuscript rule engine library through the re-determined manuscript rules and the corresponding logical relationships.

[0084] In another embodiment, the computer node generating the proofreading result according to the file proofreading task comprises:

[0085] According to the proofreading method in the file proofreading task, the corresponding manuscript rule is determined from the manuscript rule engine library, the segmented data in the file proofreading task is checked for format and semantic analysis, defects, errors or warning contents in the segmented data are determined, and corresponding proofreading results are generated.

[0086] The above embodiments of the present application provide a manuscript verification system based on a ChatGLM model, and a manuscript verification method based on the ChatGLM model. Through the above system and method, automatic and intelligent manuscript review can be realized, and processing efficiency can be improved.

[0087] The present embodiment also discloses a computer device, as shown in Figure 3 The computer device includes a processor and a memory, and the memory stores at least one instruction, which is loaded and executed by the processor to implement the ChatGLM model-based manuscript verification method described above.

[0088] In addition, in the implementation of the above example of the ChatGLM model-based manuscript verification system, the logical division of each program module is only an example, and in actual application, the above functions can be completed by different program modules according to needs, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. The internal structure of the ChatGLM model-based manuscript verification system is divided into different program modules to complete all or part of the functions described above.

[0089] The above is only a preferred embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included within the scope of protection of the present application.

Claims

1. A manuscript verification system based on the ChatGLM model, characterized in that, The system includes a multi-format compatible intelligent document reading module, a document rule engine formulation module, a metadata extraction module, a distributed verification module, and a proofreading result output module. The multi-format compatible intelligent document reading module is used to identify and read target document data in at least one or more different file formats, and convert the target document data into target data in a unified format; The target data is segmented according to a preset logical structure or configuration requirements using the ChatGLM model, generating several segmented data. The segmented data is stored in the memory of a computer device according to preset conditions, and the status of the segmented data is set to pending verification. The document rule engine formulation module is used to collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document features of the historical sample documents, and determine several document rules and the logical relationships between each document rule based on the input business requirements and the document features. The document rule engine library is constructed through the document rules and the corresponding logical relationships. The document rules include identifiers, descriptions, conditions, and actions. The metadata extraction module is used to extract key metadata from the target data and store the key metadata as an index for subsequent processing and retrieval. The distributed verification module is used to generate a document verification task based on the segmented data. The document verification task includes the size of the segmented data and the verification method. Multiple document verification tasks are distributed to multiple computer nodes for parallel processing based on the size of the segmented data, so that the computer nodes generate proofreading results based on the document verification task. After receiving the proofreading results from the computer nodes and confirming that all computer nodes have returned the manuscript, the proofreading results are summarized to generate a proofreading report of the manuscript data. The proofreading result output module is used to output the proofreading report in the form of a file or webpage, and provide a download link for the proofreading report.

2. The system according to claim 1, characterized in that, The system also includes a file preprocessing module; The document preprocessing module is used to preprocess the document data before converting it into target data in a unified format. The preprocessing includes removing invalid data from the document data and correcting encoding errors in the document data.

3. The system according to claim 1, characterized in that, The system also includes a security protection module; The security protection module is used to ensure the security and privacy of the manuscript data, including data encryption, access control, and audit log recording.

4. The system according to claim 1, characterized in that, After the document rule engine formulation module constructs the document rule engine library based on the document rules and corresponding logical relationships, it also includes: The document features are updated using the target document data. The document rules and corresponding logical relationships are then redefined using the updated document features. Finally, the document rule engine library is updated using the redefined document rules and corresponding logical relationships.

5. The system according to claim 1, characterized in that, The computer node generates proofreading results based on the document verification task, including: Based on the proofreading method in the document verification task, the corresponding document rules are determined from the document rule engine library. The document rules are then used to perform formatting checks and semantic analysis checks on the segmented data in the document verification task to identify defects, errors, or warnings in the segmented data and generate corresponding proofreading results.

6. A manuscript verification method based on the ChatGLM model, characterized in that, The method includes: Collect historical sample documents, input the historical sample documents into the ChatGLM model, determine the document features of the historical sample documents, and determine several document rules and the logical relationships between each document rule based on the input business requirements and the document features. Then, construct a document rule engine library through the document rules and the corresponding logical relationships. The document rules include identifiers, descriptions, conditions, and actions. Identify and read target document data in at least one or more different file formats, and convert the target document data into target data in a unified format; The target data is segmented according to a preset logical structure or configuration requirements using the ChatGLM model, generating several segmented data. The segmented data is stored in the memory of a computer device according to preset conditions, and the status of the segmented data is set to pending verification. A document verification task is generated based on the segmented data. The document verification task includes the size of the segmented data and the verification method. Multiple document verification tasks are distributed to multiple computer nodes for parallel processing based on the size of the segmented data, so that the computer nodes generate proofreading results based on the document verification task. After receiving the proofreading results from the computer nodes and confirming that all computer nodes have returned the manuscript, the proofreading results are summarized to generate a proofreading report of the manuscript data. The proofreading report will be output as a file or webpage, and a download link for the proofreading report will be provided.

7. The method according to claim 6, characterized in that, Before converting the document data into target data in a uniform format, the method further includes: The document data is preprocessed, including removing invalid data from the document data and correcting encoding errors in the document data.

8. The method according to claim 6, characterized in that, After constructing the document rule engine library using the document rules and corresponding logical relationships, the method further includes: The document features are updated using the target document data. The document rules and corresponding logical relationships are then redefined using the updated document features. Finally, the document rule engine library is updated using the redefined document rules and corresponding logical relationships.

9. The method according to claim 6, characterized in that, The computer node generates proofreading results based on the document verification task, including: Based on the proofreading method in the document verification task, the corresponding document rules are determined from the document rule engine library. The document rules are then used to perform formatting checks and semantic analysis checks on the segmented data in the document verification task to identify defects, errors, or warnings in the segmented data and generate corresponding proofreading results.

Images