Data processing method and device, readable storage medium and data processing system
By using multiple modules of the data processing system to perform type conversion, structure conversion, and review of oil and gas business documents, the problem of poor data standardization and uniformity in oil and gas business has been solved, and the standardization and uniformity of document processing has been achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- CHINA NAT PETROLEUM CORP
- Filing Date
- 2025-01-22
- Publication Date
- 2026-07-24
AI Technical Summary
In existing technologies, oil and gas business documents and data are often stored in a scattered manner, have inconsistent formats, and vary in quality, resulting in poor data standardization and uniformity.
By utilizing the source file parsing interface, format parsing configuration module, standard format update module, source file management module, and data quality verification module in the data processing system, document type conversion, data structure conversion, storage, and auditing processing are achieved, thereby improving data standardization and uniformity.
It achieves standardized processing and unified data structure for different documents, improves the data standardization and uniformity of documents, and reduces the workload and error rate of manual processing.
Smart Images
Figure CN122450896A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, readable storage medium, and data processing system. Background Technology
[0002] Currently, overseas oil and gas operations involve multiple upstream oil and gas businesses. These different upstream businesses involve a large amount of different document data, which faces problems such as scattered storage, inconsistent formats, and varying quality. Therefore, existing data processing methods suffer from technical problems such as poor data standardization and inconsistent data. Summary of the Invention
[0003] This application provides a data processing method, apparatus, readable storage medium, and data processing system to solve technical problems such as poor data standardization and poor data uniformity in the prior art.
[0004] A first aspect of this application provides a data processing method applied to a data processing system. The data processing system includes a source file parsing interface, a format parsing configuration module, a standard format update module, a source file management module, and a data quality verification module. The method includes:
[0005] Obtain the first document to be processed through the source file parsing interface;
[0006] The format parsing configuration module performs document type conversion on the first document to obtain the second document;
[0007] The second document is processed by the standard format update module to transform its data structure and obtain the third document.
[0008] The source file management module stores third-party documents in the document database;
[0009] The data quality verification module is used to review third-party documents in the document database to obtain the review results.
[0010] The data processing method in this embodiment improves the data standardization and data uniformity of documents by utilizing the source file parsing interface, format parsing configuration module, standard format update module, source file management module, and data quality verification module in the data processing system.
[0011] A second aspect of this application provides a data processing apparatus applied to a data processing system. The data processing system includes a source file parsing interface, a format parsing configuration module, a standard format update module, a source file management module, and a data quality verification module. The apparatus includes:
[0012] The processing unit is used to obtain the first document to be processed through the source file parsing interface;
[0013] The processing unit is also used to perform document type conversion processing on the first document through the format parsing configuration module to obtain the second document;
[0014] The processing unit is also used to perform data structure transformation processing on the second document through the standard format update module to obtain the third document;
[0015] The processing unit is also used to store third-party documents in the document database through the source file management module;
[0016] The processing unit is also used to review third-party documents in the document database through the data quality verification module to obtain the review results of the third-party documents.
[0017] The data processing device in this embodiment improves the data standardization and data uniformity of documents through the source file parsing interface, format parsing configuration module, standard format update module, source file management module, and data quality verification module in the data processing system.
[0018] A third aspect of this application provides another data processing apparatus, including a processor and a memory. The memory stores a computer program, which, when executed by the processor, implements the steps of the data processing method as described in any of the above embodiments. Therefore, this data processing apparatus possesses all the beneficial effects of the data processing method in any of the above embodiments, and will not be elaborated further here.
[0019] A fourth aspect of this application provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the data processing method as described in any of the above embodiments. Therefore, this readable storage medium possesses all the beneficial effects of the data processing method in any of the above embodiments, which will not be elaborated further here.
[0020] According to a fifth aspect of the present invention, a data processing system is provided, comprising: a data processing apparatus as defined in the second aspect above, or a data processing apparatus as defined in the third aspect above, and / or a readable storage medium as defined in the fourth aspect above, thereby having all the beneficial technical effects of the data processing apparatus as defined in the second aspect above, or the data processing apparatus as defined in the third aspect above, and / or the readable storage medium as defined in the fourth aspect above, which will not be elaborated further here. Attached Figure Description
[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0022] Figure 1 A flowchart illustrating the data processing method provided in the embodiments of this application;
[0023] Figure 2 Functional block diagram of the data processing apparatus provided in the embodiments of this application;
[0024] Figure 3 This is a structural block diagram of the data processing apparatus provided in the embodiments of this application. Detailed Implementation
[0025] To better understand the technical solutions provided in the embodiments of this specification, the technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this specification and the specific features in the embodiments are detailed descriptions of the technical solutions of the embodiments of this specification, rather than limitations on the technical solutions of this specification. In the absence of conflict, the embodiments of this specification and the technical features in the embodiments can be combined with each other.
[0026] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The term "two or more" includes two or more cases.
[0027] In some embodiments, such as Figure 1 As shown, an embodiment of this application provides a data processing method, including:
[0028] Step S101: Obtain the first document to be processed through the source file parsing interface;
[0029] Step S102: The first document is converted into a second document through the format parsing configuration module.
[0030] Step S103: The second document is processed by the standard format update module to perform data structure transformation to obtain the third document;
[0031] Step S104: Store the third document in the document database through the source file management module;
[0032] Step S105: The data quality verification module is used to review the third-party documents in the document database to obtain the review results of the third-party documents.
[0033] In this embodiment, a data processing method is proposed and applied to a data processing system. The data processing system is a system for processing document data and includes a source file parsing interface, a format parsing configuration module, a standard format update module, a source file management module, and a data quality verification module.
[0034] Specifically, the source file parsing interface is the program interface for receiving files and other data; the format parsing configuration module is the data processing module for handling document types; the standard format update module is the data processing module for handling data structures; the source file management module is the data processing module for storing data; and the data quality verification module is the data processing module for verifying data.
[0035] The first document to be processed is read through the source file parsing interface. The first document is the document data that the data processing system needs to process.
[0036] For example, the source file parsing interface can achieve the following functions: the data formats of each project file include multiple types, such as Excel (a file format), Access (a file format), PDF (a file format), etc. The interface program can parse out the key information of various data formats to ensure that the data is displayed normally and that there are no display errors due to format issues.
[0037] The format parsing configuration module performs document type conversion on the first document to obtain the second document, which is the document data of the first document after document type conversion.
[0038] For example, the format parsing configuration module can unify different types of documents into a standard document type.
[0039] The second document is processed by the standard format update module to transform its data structure, resulting in a third document, which is the document data of the second document after the data structure has been transformed.
[0040] For example, the standard format update module can unify data in different documents into data with a standard structure.
[0041] The source file management module stores third-party documents in the document database, which is a database that stores document data.
[0042] For example, the document database can be a database in the cloud.
[0043] The data quality verification module reviews third-party documents in the document database to obtain the review results of the third-party documents.
[0044] For example, the audit result can be either "document data is normal" or "document data is abnormal".
[0045] It should be noted that this embodiment, through a data processing system, can standardize the processing of different projects and different types of documents, unify the file types of different documents, and unify the data structure of different documents, thereby improving the data standardization and uniformity of documents. In addition, it solves the technical problems of large workload and easy error in manual data processing.
[0046] The data processing method in this embodiment improves the data standardization and data uniformity of documents by utilizing the source file parsing interface, format parsing configuration module, standard format update module, source file management module, and data quality verification module in the data processing system.
[0047] In some embodiments, this application provides a data processing method that performs document type conversion processing on a first document through a format parsing configuration module to obtain a second document, including:
[0048] Step S201: Determine the document type of the first document through the format parsing configuration module;
[0049] Step S202: The document type is converted to a preset type through the format parsing configuration module, so that the first document is converted into the second document.
[0050] In this embodiment, the document type of the first document is determined by the format parsing configuration module, wherein the document type is the document type of the first document.
[0051] For example, the document type can be Excel, Access, PDF, etc.
[0052] The format parsing configuration module converts the document type to a preset type, so that the first document can be converted into the second document. The preset type is a pre-defined document type.
[0053] For example, the format parsing configuration module's functions include: after reading the correct information for each type of format, it needs to make individual settings for file reading and input based on the characteristics of each project and each type of file, and generate standard type format files based on the configured file information, facilitating subsequent processing and unified storage into the database system. Each type of file in each project exists independently, and the configuration can be adjusted at any time according to the subsequent source file format, ensuring the correctness and effectiveness of data reading.
[0054] In some embodiments, this application provides a data processing method that uses a standard format update module to perform data structure transformation on a second document to obtain a third document, including:
[0055] Step S301: Read the data from the second document through the standard format update module to obtain the document data;
[0056] Step S302: The document data structure is converted into a preset structure through the standard format update module, so that the second document is converted into the third document.
[0057] In this embodiment, the standard format update module reads data from the second document to obtain document data, wherein the document data is the content data of the second document.
[0058] The standard format update module converts the document data structure into a preset structure, enabling the second document to be converted into a third document.
[0059] For example, the standard format update module's functions include: the standard format file generated by the format parsing configuration module can be recognized by the standard format update module and uniformly updated using a pre-configured update configuration file. The module comes with a unified data update standard, which clearly defines each field and its unit corresponding to the standard format file and establishes a fixed update mapping relationship, allowing the standard format file to correspond to the standard database tables.
[0060] In some embodiments, this application provides a data processing method that stores a third document in a document database through a source file management module, including:
[0061] Step S401: Determine the storage path corresponding to the third document through the source file management module;
[0062] Step S402: Based on the storage path, store the third document in the document database.
[0063] In this embodiment, the storage path corresponding to the third document is determined by the source file management module, wherein the storage path is the directory path where the third document is stored.
[0064] Based on the storage path, the third document is stored in the document database.
[0065] For example, the source file management module includes the following functions: after the source files are parsed and stored in the database, the system automatically saves them to the respective project directories according to certain naming rules for unified management. This ensures that if data is lost or erroneous later, the source files can be used to update and complete the data at any time.
[0066] In some embodiments, this application provides a data processing method that uses a data quality verification module to review third documents in a document database to obtain the review results of the third documents, including:
[0067] Step S501: The data in the third document is processed for quality audit by the data quality verification module to obtain the quality audit result;
[0068] Step S502: The data in the third document is subjected to continuity audit processing through the data quality verification module to obtain the continuity audit result;
[0069] Step S503: The data range of the data in the third document is reviewed by the data quality verification module to obtain the range review result.
[0070] In this embodiment, the audit results include quality audit results, continuity audit results, and scope audit results. The quality audit results represent the data quality of the third document, the continuity audit results represent the data continuity of the third document, and the scope audit results represent the data scope of the third document.
[0071] The data quality verification module performs quality auditing on the data in the third-party document to obtain the quality audit results.
[0072] The data quality verification module performs continuity audit processing on the data in the third document to obtain the continuity audit results.
[0073] The data quality verification module is used to perform data range verification on the data in the third document in order to obtain the range verification results.
[0074] For example, the data quality verification module's functions include: after the source file is parsed, processed, and stored in the database, the module can verify the data quality to ensure data accuracy. The module first automatically audits the fields already stored to ensure there is no abnormally formatted data; then it judges the data continuity to ensure there are no missing or lost data; finally, it judges the stored data based on historical data patterns, identifies abnormal data that significantly exceeds the normal range, and reminds the user to verify and check, ensuring data accuracy.
[0075] In some embodiments of this application, a data processing method is provided, which further includes:
[0076] Step S601: If the audit result indicates that the data is abnormal, output the data abnormality information through the data quality verification module.
[0077] In this embodiment, if the audit result indicates that the data is abnormal, it means that the data in the third document is abnormal. The data quality verification module outputs data abnormality information, which indicates that the data in the third document is abnormal.
[0078] For example, the audit result can be either "data is abnormal" or "data is normal".
[0079] In some embodiments, such as Figure 2 As shown, an embodiment of this application provides a data processing apparatus 700, including:
[0080] Processing unit 702 is used to obtain the first document to be processed through the source file parsing interface;
[0081] The processing unit 702 is also used to perform document type conversion processing on the first document through the format parsing configuration module to obtain the second document;
[0082] The processing unit 702 is also used to perform data structure transformation processing on the second document through the standard format update module to obtain the third document;
[0083] The processing unit 702 is also used to store the third document in the document database through the source file management module;
[0084] The processing unit 702 is also used to review the third document in the document database through the data quality verification module to obtain the review result of the third document.
[0085] In this embodiment, a data processing device 700 is proposed and applied to a data processing system. The data processing system is a system for processing document data and includes a source file parsing interface, a format parsing configuration module, a standard format update module, a source file management module, and a data quality verification module.
[0086] Specifically, the source file parsing interface is the program interface for receiving files and other data; the format parsing configuration module is the data processing module for handling document types; the standard format update module is the data processing module for handling data structures; the source file management module is the data processing module for storing data; and the data quality verification module is the data processing module for verifying data.
[0087] The first document to be processed is read through the source file parsing interface. The first document is the document data that the data processing system needs to process.
[0088] For example, the source file parsing interface can achieve the following functions: the data formats of each project file include multiple types, such as Excel (a file format), Access (a file format), PDF (a file format), etc. The interface program can parse out the key information of various data formats to ensure that the data is displayed normally and that there are no display errors due to format issues.
[0089] The format parsing configuration module performs document type conversion on the first document to obtain the second document, which is the document data of the first document after document type conversion.
[0090] For example, the format parsing configuration module can unify different types of documents into a standard document type.
[0091] The second document is processed by the standard format update module to transform its data structure, resulting in a third document, which is the document data of the second document after the data structure has been transformed.
[0092] For example, the standard format update module can unify data in different documents into data with a standard structure.
[0093] The source file management module stores third-party documents in the document database, which is a database that stores document data.
[0094] For example, the document database can be a database in the cloud.
[0095] The data quality verification module reviews third-party documents in the document database to obtain the review results of the third-party documents.
[0096] For example, the audit result can be either "document data is normal" or "document data is abnormal".
[0097] It should be noted that this embodiment, through a data processing system, can standardize the processing of different projects and different types of documents, unify the file types of different documents, and unify the data structure of different documents, thereby improving the data standardization and uniformity of documents. In addition, it solves the technical problems of large workload and easy error in manual data processing.
[0098] The data processing device 700 in this embodiment improves the data standardization and data uniformity of documents through the source file parsing interface, format parsing configuration module, standard format update module, source file management module, and data quality verification module in the data processing system.
[0099] In some embodiments of this application, a data processing apparatus 700 is provided, further comprising:
[0100] The processing unit 702 is also used to determine the document type of the first document through the format parsing configuration module;
[0101] The processing unit 702 is also used to convert the document type into a preset type through the format parsing configuration module, so that the first document is converted into the second document.
[0102] In some embodiments of this application, a data processing apparatus 700 is provided, further comprising:
[0103] The processing unit 702 is also used to read data from the second document through the standard format update module to obtain document data;
[0104] The processing unit 702 is also used to convert the structure of the document data into a preset structure through the standard format update module, so as to convert the second document into a third document.
[0105] In some embodiments of this application, a data processing apparatus 700 is provided, further comprising:
[0106] The processing unit 702 is also used to determine the storage path corresponding to the third document through the source file management module;
[0107] The processing unit 702 is also used to store the third document in the document database based on the storage path.
[0108] In some embodiments of this application, a data processing apparatus 700 is provided, further comprising:
[0109] The processing unit 702 is also used to perform quality audit processing on the data in the third document through the data quality verification module to obtain the quality audit result;
[0110] The processing unit 702 is also used to perform continuity audit processing on the data in the third document through the data quality verification module to obtain the continuity audit result;
[0111] The processing unit 702 is also used to perform data range verification processing on the data in the third document through the data quality verification module to obtain the range verification result.
[0112] In some embodiments of this application, a data processing apparatus 700 is provided, further comprising:
[0113] The processing unit 702 is also used to output data abnormality information through the data quality verification module when the audit result is that the data is abnormal.
[0114] In some embodiments, such as Figure 3 As shown, a data processing apparatus 800 is proposed. The data processing apparatus 800 includes a processor 802 and a memory 804. The memory 804 stores a computer program, which, when executed by the processor 802, implements the steps of the data processing method as described in any of the above embodiments. Therefore, the data processing apparatus 800 possesses all the beneficial effects of the data processing method in any of the above embodiments, which will not be elaborated further here.
[0115] In some embodiments, a readable storage medium is provided on which a program is stored, which, when executed by a processor, implements the steps of the data processing method as described in any of the above embodiments, and thus has all the beneficial technical effects of the data processing method in any of the above embodiments.
[0116] In some embodiments, a data processing system is proposed, comprising: a data processing apparatus as described in any of the above embodiments, and / or a readable storage medium as described in any of the above embodiments, thus having all the beneficial technical effects of the data processing apparatus and / or the readable storage medium as described in any of the above embodiments, which will not be elaborated further here.
[0117] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.
[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-readable program code.
[0119] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0120] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0121] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0122] This application also provides a computer program product, which includes computer software instructions that, when executed on a processing device, cause the processing device to perform a data processing method.
[0123] A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0125] In the several embodiments provided in this application, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.
[0126] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0129] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
[0130] Although preferred embodiments have been described in this specification, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this specification.
[0131] Obviously, those skilled in the art can make various modifications and variations to this specification without departing from its spirit and scope. Therefore, if such modifications and variations fall within the scope of the claims and their equivalents, this specification is also intended to include such modifications and variations.
Claims
1. A data processing method, characterized in that, Applied to a data processing system, the data processing system includes a source file parsing interface, a format parsing configuration module, a standard format update module, a source file management module, and a data quality verification module, and the method includes: The first document to be processed is obtained through the source file parsing interface. The format parsing configuration module performs document type conversion on the first document to obtain the second document; The second document is processed by the standard format update module to perform data structure transformation to obtain the third document; The source file management module stores the third document in the document database; The data quality verification module is used to review the third document in the document database to obtain the review result of the third document.
2. The method according to claim 1, characterized in that, The step of performing document type conversion processing on the first document through the format parsing configuration module to obtain the second document includes: The document type of the first document is determined through the format parsing configuration module; The format parsing configuration module converts the document type into a preset type, thereby converting the first document into the second document.
3. The method according to claim 1, characterized in that, The step of performing data structure transformation on the second document through the standard format update module to obtain the third document includes: The standard format update module reads data from the second document to obtain document data. The standard format update module converts the structure of the document data into a preset structure, thereby converting the second document into the third document.
4. The method according to claim 1, characterized in that, The step of storing the third document in the document database through the source file management module includes: The storage path corresponding to the third document is determined through the source file management module; Based on the storage path, the third document is stored in the document database.
5. The method according to claim 1, characterized in that, The audit results include quality audit results, continuity audit results, and scope audit results. The process of auditing the third document in the document database through the data quality verification module to obtain the audit results for the third document includes: The data quality verification module performs quality auditing on the data in the third document to obtain the quality audit result. The data quality verification module performs continuity verification on the data in the third document to obtain the continuity verification result. The data quality verification module performs a data range audit on the data in the third document to obtain the range audit result.
6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: If the audit result indicates that the data is abnormal, the data quality verification module will output data anomaly information.
7. A data processing apparatus, characterized in that, The device is applied to a data processing system, which includes a source file parsing interface, a format parsing configuration module, a standard format update module, a source file management module, and a data quality verification module. The device includes: The processing unit is used to obtain the first document to be processed through the source file parsing interface; The processing unit is further configured to perform document type conversion processing on the first document through the format parsing configuration module to obtain the second document; The processing unit is further configured to perform data structure conversion processing on the second document through the standard format update module to obtain a third document; The processing unit is also used to store the third document in the document database through the source file management module; The processing unit is further configured to review the third document in the document database through a data quality verification module to obtain the review result of the third document.
8. A data processing apparatus, characterized in that, include: processor; A memory, which stores programs or instructions, wherein a processor, when executing programs or instructions in the memory, implements the steps of the data processing method as claimed in any one of claims 1 to 6.
9. A readable storage medium, characterized in that, A program or instructions are stored on a readable storage medium, which, when executed by a processor, implement the steps of the data processing method as described in any one of claims 1 to 6.
10. A data processing system, characterized in that, include: The data processing apparatus as described in claim 7 or 8; and / or The readable storage medium as described in claim 9.