Data standard establishment checking method and system

Abstract

The invention relates to a data standard establishment checking method and system. The method comprises the steps of establishing a checking rule list of a data standard based on compilation management requirements of the data standard; packaging the inspection rule into an inspection tool according to the inspection rule list; generating a data standard table according to compilation management requirements of data standards; and selecting the corresponding data standard table by utilizing an inspection tool to inspect the data standard, and matching the inspection rule to obtain an inspection result table. According to the method, tedious work covered by manual checking can be remarkably reduced, meanwhile, tiny problems which are prone to being ignored in manual checking are sensitively and rapidly found, and therefore the overall quality and working efficiency of basic data standard checking are greatly improved, rapid checking is achieved, and the checking efficiency is improved. And the checking content can be continuously expanded through dynamic updating of the checking rule, and the checking result is perfected.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a data standard establishment and verification method and system. Background Art

[0002] With data now recognized as the fifth major factor of production, it has gradually become a crucial engine driving socioeconomic development, and its importance has been widely recognized. To enhance the standardization, consistency, and accuracy of data, various organizations are actively promoting the standardization of basic data from the perspective of data governance. This effort aims to clarify the business, technical, and management attributes of basic data, thereby promoting the accurate and standardized collection, storage, and use of data.

[0003] Currently, the primary approach to establishing data standards in the industry is to fill in each attribute of the standard one by one according to the standard definition template and submit an application. A standard review team then conducts a manual review to determine the standard's uniqueness, completeness, accuracy, and standardization of the attribute content. Key technical approaches currently employed in the standard review process include: determining the uniqueness of the standard by comparing the Chinese characters of the standard's name; identifying incomplete non-null attribute definitions by filtering for null values in a certain attribute column of the standard definition template; and identifying problematic standards with identical business definitions by highlighting duplicate values in the business definition column of the standard definition template.

[0004] Basic data standards are characterized by large numbers and multiple attributes, and in order to meet the time requirements of system development and construction, the time window for review is often very short. Auditors use traditional manual methods to complete the item-by-item review of a large amount of standard content in a short period of time, and may overlook some minor issues, which has a certain impact on the efficiency and quality of the standard review. For example, the method of breaking down Chinese names one by one is inefficient and requires a high level of personal ability from the reviewers, which can easily lead to uneven review results. When filtering for null values in a single column, it is difficult to judge null values under complex conditions. When filtering for duplicate values, only completely duplicate content can be identified, and semantically similar or mostly duplicate content cannot be identified. Summary of the Invention

[0005] In order to address the deficiencies of the prior art, the present invention provides a verification method for basic data standard establishment. By dynamically updating the verification rules, the verification content is continuously expanded and the verification results are improved. It can quickly verify and discover problems such as incomplete and non-standard filling of basic data standard attributes.

[0006] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0007] A data standard establishment and verification method, characterized by comprising:

[0008] S1. Establish a list of inspection rules for data standards based on the compilation and management requirements of data standards;

[0009] S2. Encapsulate the inspection rules into an inspection tool according to the inspection rule list;

[0010] S3. Generate a data standard table based on the compilation and management requirements of data standards;

[0011] S4. Use the inspection tool to select the corresponding data standard table to check the data standard, match the inspection rules, and obtain the inspection result table.

[0012] Furthermore, the inspection rule list includes compilation rule requirements for data standard related attributes, wherein the data standard related attributes include data item content, business logic, and code encoding.

[0013] Furthermore, the inspection rules are packaged into an inspection tool according to the inspection rule list. The inspection tool is packaged and designed based on a modular architecture, specifically including:

[0014] Define a rule language to write specific rules, including data item content, business logic, and code encoding;

[0015] Design a rule engine to parse and execute inspection rules;

[0016] Introducing a semantic recognition module to identify repeated or similar content;

[0017] Added a user interface for selecting inspection objects and viewing inspection results.

[0018] Furthermore, the data standard table includes a data item list, a code table, and a data item classification framework.

[0019] Furthermore, the inspection result table includes data item inspection results and code inspection results.

[0020] Furthermore, the method further comprises:

[0021] Locate errors and problems, and mark cells that failed the check in the check result table;

[0022] Modify the data standard based on the error prompt in the inspection result table, wherein the error prompt is a description based on the inspection rule content;

[0023] Obtain data that meets the required standards.

[0024] The present invention also relates to a data standard establishment and verification system, which is characterized by comprising:

[0025] A checklist creation module is used to create a checklist of data standard rules based on the compilation and management requirements of data standards;

[0026] A tool packaging module is used to package the inspection rules into inspection tools according to the inspection rule list;

[0027] Table generation module, used to generate data standard tables according to the compilation and management requirements of data standards;

[0028] The data checking module is used to use the checking tool to select the corresponding data standard table to check the data standard and obtain the checking result table.

[0029] The present invention also relates to a computer-readable storage medium, characterized in that a computer program is stored on the storage medium, and the computer program implements the above method when executed by a processor.

[0030] The present invention also relates to an electronic device, characterized in that it comprises a processor and a memory;

[0031] The memory is used to store program instructions;

[0032] The processor is used to execute the above method by calling program instructions.

[0033] The present invention also relates to a computer program product, comprising a computer program and / or instructions, characterized in that the computer program and / or instructions implement the steps of the above method when executed by a processor.

[0034] The beneficial effects of the present invention are:

[0035] The use of a data standard establishment and verification method and system described in the present invention can significantly reduce the tedious work involved in manual review, while sensitively and quickly discovering relatively minor errors that are easily overlooked in manual review, thereby greatly improving the overall quality and work efficiency of basic data standard review. If the invention is used to check the basic data standards of the previous manual review, minor errors that do not meet the requirements of the verification rule list can be found in tens of thousands of existing standards within a few minutes, such as special characters in the data item names, duplication of code values between different data items, non-class errors in data types, etc., to achieve rapid verification, and the dynamic update of the verification rules can continuously expand the verification content and improve the verification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0037] Figure 1 This is a flow chart of a data standard establishment and verification method according to the present invention;

[0038] Figure 2This is a structural diagram of a data standard establishment and verification system according to the present invention;

[0039] Figure 3 The figure is a schematic structural diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. It should be understood that the described embodiments are only part of the embodiments of this application, not all of the embodiments, and these embodiments are only used to illustrate this application and are not used to limit the scope of this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0041] like Figure 1 As shown, the embodiment of the present invention discloses a method for establishing and verifying data standards, the steps of which include:

[0042] Step S1: Establish a list of inspection rules for data standards based on the compilation management requirements of data standards.

[0043] In this embodiment, the inspection rule list includes compilation requirements for data standard-related attributes, including data item content, business logic, and code encoding. For example, data item names must not use adverbs, prepositions, pronouns, or conjunctions, and the data length must be consistent with the length specified in the business rules. These rules can detect subtle errors that are difficult to detect through manual inspection, improving review efficiency and precision.

[0044] Step S2: Encapsulate the inspection rules into an inspection tool according to the inspection rule list.

[0045] In this embodiment, the inspection rules are packaged into an inspection tool according to the inspection rule list. The inspection tool is packaged based on a modular architecture and specifically includes:

[0046] Define a rule language to write specific rules, including data item content, business logic, and code encoding;

[0047] Design a rule engine to parse and execute inspection rules;

[0048] Introducing a semantic recognition module to identify repeated or similar content;

[0049] Add a user interface for selecting inspection objects and viewing inspection results. The user interface can visualize inspection results.

[0050] Preferably, the inspection tool includes a modular architecture design, which can support flexible addition or modification of different inspection rules and facilitate subsequent expansion and maintenance.

[0051] Preferably, a rule language is defined for writing inspection rules so that non-technical languages can also define and modify rules, define data standards, and determine the format, type, uniqueness, etc. of data.

[0052] Preferably, the semantic recognition module can utilize a machine learning model to identify semantically similar texts, first using natural language processing technology to extract text features, and then training a classification model to classify whether the texts are similar.

[0053] The semantic recognition module can also adopt semantic analysis method, first using word embedding technology to convert the text into vector representation, and then calculating cosine similarity to evaluate semantic similarity.

[0054] Step S3: Generate a data standard table according to the compilation management requirements of the data standard.

[0055] In this embodiment, the data standard table includes a data item list, a code table, and a data item classification framework.

[0056] The data standard table can be an Excel table. Based on the compilation and management requirements of data standards, the data standard table can include three sheet pages: "Data Item List", "Code Table", and "Data Item Classification Framework": The "Data Item List" sheet page identifies the header from the second row and fills in the data item content to be verified from the third row; the "Code Table" sheet page identifies the header from the first row and fills in the code content from the second row. The header must include the Chinese name of the code item, code value, and code value description.

[0057] Step S4: Use the inspection tool to select the corresponding data standard table to check the data standard, match the inspection rules, and obtain an inspection result table, which includes data item inspection results and code inspection results.

[0058] Click the "Browse" button in the inspection tool user interface, select the Excel table address to be checked, click the "Execute Check" button, and the execution is completed when the window appears "Complete! Inspection results are output to file: xxxx".

[0059] The inspection results are automatically stored as a spreadsheet file in the same directory as the file to be inspected. The file name is the original file with the suffix "-InspectionResult" appended. The inspection results output contains two sheets: "Data Item Inspection Results" corresponds to the "Data Item List" in the standard data sheet to be inspected; "Code Inspection Results" corresponds to the "Code Table" in the standard data sheet to be inspected.

[0060] In an embodiment of the present invention, the method further includes:

[0061] Locate errors and problems, and mark cells that failed the check in the check result table;

[0062] Modify the data standard based on the error prompt in the inspection result table, wherein the error prompt is a description based on the inspection rule content;

[0063] Obtain data that meets the required standards.

[0064] In this embodiment, cells that fail the check are marked in red, which facilitates rapid and accurate location of standard errors in the basic data.

[0065] A specific embodiment of the present invention is given below. The "business rules" of the "payment system bank number" item in the data standard table are:

[0066] A 12-digit 3+4+4+1 hierarchical code is used: Digits 1-3 from the left represent the bank's type, consisting of a one-digit category code and a two-digit sequence code. Digits 4-7 are the region code, encoded using the National Clearing Center code (GB13497-92) for the city or county where the bank is located. Digits 8-11 are the bank's branch number, assigned sequentially from "0001" by each bank for its branches within the city (county). The last digit is a check digit, derived using a modulo-10 / 11 algorithm. To support the 14-digit financial institution code required by the "Financial Institution Coding Standards" issued by the People's Bank of China, the "Participant Bank Number" field in the second-generation payment system message standard is set to a maximum length of 14 characters. This supports both 12-digit payment system bank numbers and 14-digit financial institution codes. Currently, the 12-digit payment system bank number is used.

[0067] The corresponding "data length" of this item is: 14.

[0068] Use the inspection tool to check, match the inspection rules, output the inspection result "

Rule 30

[0069] Based on the inspection results, the data standard for the "Payment System Bank Number" was adjusted to unify the data length with the length described in the business rules.

[0070] The above embodiment only provides verification of a specific item. The verification tool contains multiple verification rules that can be used for verification. The verification rules in the verification tool can be dynamically updated and adjusted, and the present invention does not limit the verification rules. The present invention can use the verification tool to verify multiple data standards simultaneously, greatly reducing labor costs, avoiding manual review errors, and improving the efficiency of data standard verification, providing a good foundation for the application of data standards in the next step.

[0071] The above problems are difficult to be discovered during the data standard compilation process, or require a lot of manpower to check. However, through the design and verification matching of the inspection rules of the present invention, these problems can be quickly discovered and located, and the content that needs to be modified or adjusted can be pointed out in a targeted manner, which can greatly improve the efficiency of data standard review and the quality of the content.

[0072] like Figure 2 As shown, the embodiment of the present invention discloses a data standard establishment and verification system, the structure of which includes:

[0073] A checklist creation module is used to create a checklist of data standard rules based on the compilation and management requirements of data standards;

[0074] A tool packaging module is used to package the inspection rules into inspection tools according to the inspection rule list;

[0075] Table generation module, used to generate data standard tables according to the compilation and management requirements of data standards;

[0076] The data checking module is used to use the checking tool to select the corresponding data standard table to check the data standard and obtain the checking result table.

[0077] By using this system, the above-mentioned calculation and processing method can be executed and the corresponding technical effects can be achieved.

[0078] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all steps of the method in the above embodiment. The computer-readable storage medium stores a computer program that implements all steps of the method in the above embodiment when executed by a processor.

[0079] The embodiment of the present invention further provides an electronic device for executing the above method, as an implementation device of the method, whose structure is as follows: Figure 3As shown, the electronic device 300 includes: a processor 301 and a memory 303. The processor 301 and the memory 303 are connected, such as through a bus 302. Optionally, the electronic device 300 may further include a transceiver 304. It should be noted that in actual applications, the transceiver 304 is not limited to one. It is understandable that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the specific structure of the electronic device 300. In other embodiments of the present application, the electronic device 300 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware. Optionally, the electronic device may further include a display screen 305 for displaying images, or receiving user operation instructions when necessary.

[0080] The processor 301 is used in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 304 may include a receiver and a transmitter. The transceiver 304 is used in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0081] The processor 301 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. The processor 301 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0082] The processor 301 may also include one or more processing units. For example, the processor 301 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). The different processing units may be independent devices or integrated into one or more processors. The controller may serve as the nerve center and command center of the electronic device 300. The controller may generate operation control signals based on instruction opcodes and timing signals to control instruction fetching and execution. The processor 301 may also include memory for storing instructions and data. In some embodiments, the memory in the processor 301 is a high-speed cache memory. This memory can store instructions or data that have just been used or are being recycled by the processor 301. If the processor 301 needs to use the instruction or data again, it can directly retrieve it from the memory. This avoids duplicate accesses, reduces the processor 301's waiting time, and thus improves system efficiency.

[0083] The processor 301 can execute the method provided in the embodiments of the present application. The processor 301 can include different devices. For example, when the processor 301 integrates a CPU and a GPU, the CPU and the GPU can cooperate to execute the method provided in the embodiments of the present application. For example, part of the algorithm in the method is executed by the CPU, and another part of the algorithm is executed by the GPU to achieve faster processing efficiency.

[0084] The bus 302 may include a path for transmitting information between the above components. The bus 302 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 302 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0085] The memory 303 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory), or may include a high-speed random access memory. It may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash storage (UFS), or other optical disk storage, optical disk storage (including compact discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0086] Optionally, the memory 303 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 301. The processor 301 is used to execute the application code stored in the memory 303 to implement the method provided in any embodiment of the present application.

[0087] The memory 303 can be used to store computer executable program code, which includes instructions. The processor 301 executes various functional applications and data processing of the electronic device 300 by running the instructions stored in the memory 303. The memory 303 may include a program storage area and a data storage area. Among them, the program storage area can store the code of the operating system and application programs, etc. The data storage area can store data created during the use of the electronic device 300 (such as images and videos collected by the camera application, etc.).

[0088] The memory 303 may also store one or more computer programs corresponding to the methods provided in the embodiments of the present application. The one or more computer programs are stored in the memory 303 and configured to be executed by the one or more processors 301. The one or more computer programs include instructions that can be used to perform the various steps in the above-mentioned corresponding embodiments.

[0089] Of course, the code of the method provided in the embodiment of the present application can also be stored in an external memory. In this case, the processor 301 can run the code of the method stored in the external memory through the external memory interface, and the processor 301 can control the running process.

[0090] The display screen 305 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, a quantum dot light-emitting diode (QLED), or the like. In some embodiments, the electronic device 300 may include one or N display screens 305, where N is a positive integer greater than 1. The display screen 305 can be used to display information input by a user or information provided to a user, as well as various graphical user interfaces (GUIs). For example, the display screen 305 can display photos, videos, web pages, or files.

[0091] The electronic device provided in the embodiment of the present application is applicable to any embodiment of the above method. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method provided above, and will not be repeated here.

[0092] It will be understood by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or boxes.

[0094] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction system that is implemented in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The present invention is described in detail below. ...

[0096] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention are intended to be covered by the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A data standard establishment and verification method, characterized in that: include: S1. Establish a list of inspection rules for data standards based on the compilation and management requirements of data standards; S2. Encapsulate the inspection rules into an inspection tool according to the inspection rule list; S3. Generate a data standard table based on the compilation and management requirements of data standards; S4. Use the inspection tool to select the corresponding data standard table to check the data standard, match the inspection rules, and obtain the inspection result table.

2. The method according to claim 1, wherein The inspection rule list includes compilation rule requirements for data standard related attributes, wherein the data standard related attributes include data item content, business logic, and code encoding.

3. The method according to claim 1, wherein The inspection rules are encapsulated into an inspection tool according to the inspection rule list. The inspection tool is encapsulated and designed based on a modular architecture, specifically including: Define a rule language to write specific rules, including data item content, business logic, and code encoding; Design a rule engine to parse and execute inspection rules; Introducing a semantic recognition module to identify repeated or similar content; Added a user interface for selecting inspection objects and viewing inspection results.

4. The method according to claim 1, wherein The data standard table includes a data item list, a code table, and a data item classification framework.

5. The method according to claim 1, wherein The inspection result table includes data item inspection results and code inspection results.

6. The method according to claim 1, wherein The method further comprises: Locate errors and problems, and mark cells that failed the check in the check result table; Modify the data standard based on the error prompt in the inspection result table, wherein the error prompt is a description based on the inspection rule content; Obtain data that meets the required standards.

7. A data standard establishment and verification system, characterized in that: include: A checklist creation module is used to create a checklist of data standard rules based on the compilation and management requirements of data standards; A tool packaging module is used to package the inspection rules into inspection tools according to the inspection rule list; Table generation module, used to generate data standard tables according to the compilation and management requirements of data standards; The data checking module is used to use the checking tool to select the corresponding data standard table to check the data standard and obtain the checking result table.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, which implements the method according to any one of claims 1 to 6 when executed by a processor.

9. An electronic device, characterized in that: including processor and memory; The memory is used to store program instructions; The processor is configured to execute the method according to any one of claims 1 to 6 by calling program instructions.

10. A computer program product comprising a computer program and / or instructions, characterized in that When the computer program and / or instructions are executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images