Rule-Base Multi-Type Data Reporting Verification Method, Device, and Storage Medium

Abstract

An embodiment of the present invention discloses a multi-type data reporting verification method, device and storage medium based on a rule library. The method includes: in response to a data reporting verification instruction, if the database to be verified is a newly added database, obtaining the attribute information of the database to be verified and the target reporting verification standard for the database to be verified; obtaining a target data verification rule that matches the database to be verified, where the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to a target data verification rule template, so as to verify the data to be verified in the database to be verified based on the target data verification rule and obtain a data verification result. The technical solution of the embodiment of the present invention simplifies the user operation by reusing rules when verifying data in a newly added database without manual rule configuration by the user.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of computer application technologies, and in particular, to a multi-type data reporting verification method, device, and storage medium based on a rule library. Background Art

[0002] Currently, data quality control software is used to solve data quality problems in the operation of business systems, the construction of data warehouses, and the process of data governance. A common solution is to configure rules for the database to generate a data verification report. However, the disadvantage of the above solution is that the configuration is troublesome. Every time a new database to be verified is added, the rules often need to be reconfigured, resulting in cumbersome user operations and a large workload. Summary of the Invention

[0003] The embodiments of the present invention provide a multi-type data reporting verification method, device, and storage medium based on a rule library, which solve the problem of cumbersome user operations when verifying data for a newly added database.

[0004] In a first aspect, the embodiments of the present invention provide a multi-type data reporting verification method based on a rule library, which may include:

[0005] In response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified;

[0006] Obtain a target data verification rule that matches the database to be verified, and based on the target data verification rule, verify the data to be verified in the database to be verified to obtain a data verification result;

[0007] Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to a target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0008] In a second aspect, the embodiments of the present invention further provide a multi-type data reporting verification device based on a rule library, which may include:

[0009] A standard acquisition module, configured to, in response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified;

[0010] A data verification module, configured to obtain a target data verification rule that matches the database to be verified, and verify the data to be verified in the database to be verified based on the target data verification rule, so as to obtain a data verification result;

[0011] Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to attribute information and a target reporting verification standard, or a data verification rule generated according to a target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to attribute information and a target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0012] Thirdly, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the multi-type data reporting verification method based on a rule library provided in any embodiment of the present invention.

[0013] The technical solution of the embodiment of the present invention is as follows: in response to a data reporting verification instruction, when the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified; furthermore, obtain a target data verification rule that matches the database to be verified. The target data verification rule can be a data verification rule determined from each generated historical data verification rule according to attribute information and a target reporting verification standard, or a data verification rule generated according to a target data verification rule template, so as to verify the data to be verified in the database to be verified based on the target data verification rule and obtain a data verification result. The above technical solution does not require the user to manually configure rules when verifying a newly added database, but can directly obtain from each generated historical data verification rule or automatically generate according to each generated general data verification rule template, reducing the complexity of user operations through rule reuse, thereby reducing the workload of the user. Description of the Drawings

[0014] Figure 1 is a flowchart of a multi-type data reporting verification method based on a rule library in the first embodiment of the present invention;

[0015] Figure 2 is a flowchart of a multi-type data reporting verification method based on a rule library in the second embodiment of the present invention;

[0016] Figure 3 is a flowchart of a multi-type data reporting verification method based on a rule library in the third embodiment of the present invention;

[0017] Figure 4 It is a flowchart of a multi-type data reporting verification method based on a rule library in the fourth embodiment of the present invention;

[0018] Figure 5 It is a structural block diagram of a multi-type data reporting verification device based on a rule library in the fifth embodiment of the present invention;

[0019] Figure 6 It is a schematic structural diagram of a multi-type data reporting verification device based on a rule library in the sixth embodiment of the present invention. Detailed implementation manners

[0020] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that for the convenience of description, only parts related to the present invention are shown in the drawings rather than all the structures.

[0021] Embodiment 1

[0022] Figure 1 It is a flowchart of a multi-type data reporting verification method based on a rule library provided in the first embodiment of the present invention. This embodiment is applicable to the situation of data verification for a newly added database to be verified, especially applicable to the situation of data verification for a newly added database to be verified without the need for the user to reconfigure the data verification rules. This method can be executed by the multi-type data reporting verification device provided in the embodiment of the present invention. The device can be implemented in software and / or hardware, and the device can be integrated on a multi-type data reporting verification device based on a rule library. The device can be various user terminals or servers.

[0023] See Figure 1 , the method of the embodiment of the present invention specifically includes the following steps:

[0024] S110. In response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified.

[0025] Among them, the data reporting verification instruction can be an instruction triggered when there is a need to verify data in a certain database, which can be called the database to be verified. In response to this data reporting verification instruction, when the database to be verified is a newly added database, the attribute information of the database to be verified and the target reporting verification standard for the database to be verified are obtained. Specifically, a newly added database is a database that was not involved when the user pre-configured the historical data verification rules. When the verified database is a newly added database, this indicates that there may or may not be a target data verification rule that matches the database to be verified among the historical data verification rules. Therefore, the attribute information of the database to be verified and the target reporting verification standard for the database to be verified are obtained, so as to subsequently determine whether to obtain the target data verification rule from the historical data verification rules or to regenerate the target data verification rule according to the attribute information and the target reporting verification standard.

[0026] The above attribute information can be the information of the database to be verified itself, such as the name of the data table in the database to be verified, the structure and field type of the data in the data table, the server address of the target server where the database to be verified is located, etc., which are not specifically limited here. The above target reporting verification standard can indicate based on what standard to conduct data reporting verification. In practical applications, optionally, it can be a standard recognized by a certain object, and this object can be at least one of objects at the national level, regional level, hospital level, and each medical system level within the hospital, etc. Correspondingly, the above standards can also be of multiple types, such as reporting verification standards at the national level, regional level, hospital level, and each medical system level within the hospital for different objects, and can also include multiple types of reporting verification standards for the same object and different data, that is, the target reporting verification standard for data reporting verification of the database to be verified can be any one or more of the above multiple types of reporting verification standards. For example, the target reporting verification standard can be the standard issued by the General Office of the National Health Commission, the standard in the National Hospital Data Reporting Management Plan - Medical Operations (Trial), the standard in the National Hospital Data Reporting Management Plan - Operational Operations (Trial), the standard in the National Hospital Data Reporting Management Plan - Interface Technical Specification (Trial), the standard in the National Hospital Data Reporting Management Plan - Medical Data Dictionary (Trial), the standard in the National Hospital Data Reporting Management Plan - Operational Data Dictionary (Trial), the standard recognized by the clinical data center of a certain hospital (so that each medical system within the hospital conducts medical data verification based on this standard and uploads the verified medical data to the clinical data center), etc., which are not specifically limited here.

[0027] S120. Obtain a target data verification rule that matches the database to be verified, and verify the data to be verified in the database to be verified based on the target data verification rule to obtain a data verification result. Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to the target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0028] Among them, the data to be verified can be the data to be verified in the database to be verified. The specific verification content can be to verify the quality of the data to be verified, whether it is standard (such as whether male is represented by 1 and female is represented by 2, etc.), format, etc., which are not specifically limited here. The target data verification rule can be a data verification rule that matches the database to be verified, which can indicate based on what rules to verify the data to be verified, such as verifying whether each field in table A in the database to be verified is non-null, numerical type, numerical range, enumeration value, etc., which are not specifically limited here. Verify the data to be verified in the database to be verified based on the target data verification rule to obtain a data verification result. It should be noted that the data verification process can be executed by a multi-type data reporting verification device or by the database to be verified (such as sending the target data verification rule to the database to be verified so that the database to be verified performs data verification based on the received target data verification rule), which are not specifically limited here.

[0029] The target data verification rule can be obtained in any of the following ways. One is to determine it from each of the generated historical data verification rules according to the attribute information and the target reporting verification standard. The other is to automatically generate it according to the attribute information and the target reporting verification standard. Specifically, for the first way, the historical data verification rule can be a rule configured according to the historical information (i.e., its attribute information) of a certain historical database and the data reporting verification standard for that historical database, which matches the historical database. Therefore, it can be determined whether there is a target data verification rule in each of the historical data verification rules according to the attribute information and the target reporting verification standard. If it exists, the target data verification rule can be directly determined from each of the historical data verification rules, so that there is no need for the user to manually configure the target data verification rule, thus reducing the workload of the user. In practical applications, optionally, the attribute information can also play the following role: when mapping the data table in the target reporting verification standard to the data table in the database to be verified, there may be a situation where the two data tables are essentially the same but have different names / formats / structures, etc. At this time, the data table in the database to be verified can be corresponded to the data table in the target reporting verification standard through the attribute information, which ensures that the subsequent verification of the data to be verified in the database to be verified can be effectively carried out based on the obtained target data verification rule.

[0030] For the second way, when there is no target data verification rule in each of the historical data verification rules, the target data verification rule can be automatically generated. Specifically, obtain various types of data reporting verification models and the model data reporting management requirements corresponding to each data reporting verification model respectively. The data reporting verification model can be understood as the data table in a certain data reporting verification standard, such as the basic information table, electrocardiogram table, nuclear magnetic resonance table, etc. The model data reporting management requirement can indicate what data in the data reporting verification model should be reported according to what requirements. In practical applications, optionally, each data reporting verification standard can include at least one data reporting verification model; the same data reporting verification model can exist in at least one data reporting verification standard. For each data reporting verification model, generate a corresponding general data verification rule template according to the data reporting verification model and the model data reporting management requirement corresponding to the data reporting verification model, that is, a template that matches the data reporting verification model, such as a template for setting what rules for what fields in the data reporting verification model; furthermore, determine the target data verification rule template (i.e., the general data verification rule template that matches the database to be verified) from each of the general data verification rule templates according to the attribute information and the target reporting verification standard. Thus, without the need for the user to manually configure, the target data verification rule can be automatically generated according to the target data verification rule template, thus reducing the workload of the user.

[0031] It should be noted that in practical applications, the above historical data verification rules and general data verification rule templates can be stored in the rule library. Among them, the historical data verification rules can be rules pre-generated before the first data reporting verification or / and rules generated during the historical data reporting verification process. The general data verification rule templates can be templates pre-generated before the first data reporting verification. These pre-generated rules and templates can be directly stored in the rule library, and the historically generated rules can also be added to the rule library after generation. Therefore, the process of obtaining the above target data verification rules can be achieved based on the historical data verification rules obtained from the rule library or / and based on the processing result, where the processing result can be the result after processing the general data verification rule templates obtained from the rule library. This can be understood as the process of obtaining the target data verification rules based on the rule library.

[0032] In the technical solution of the embodiment of the present invention, in response to a data reporting verification instruction, when the database to be verified is a newly added database, the attribute information of the database to be verified and the target reporting verification standard for the database to be verified are obtained. Furthermore, a target data verification rule matching the database to be verified is obtained. The target data verification rule can be a data verification rule determined from the generated historical data verification rules according to the attribute information and the target reporting verification standard or a data verification rule generated according to the target data verification rule template, so as to verify the data to be verified in the database to be verified based on the target data verification rule and obtain a data verification result. In the above technical solution, when verifying the data of a newly added database, there is no need for the user to manually configure the rules, but the rules can be directly obtained from the generated historical data verification rules or automatically generated according to the generated general data verification rule templates. By means of rule reuse, the complexity of user operations is reduced, thereby reducing the workload of the user.

[0033] An optional technical solution is that the data verification rule templates to be generated in each general data verification rule template can be pre-generated through the following steps: obtaining the corresponding reporting verification model corresponding to the data verification rule template to be generated in each data reporting verification model and the corresponding data reporting management requirements corresponding to the data verification rule template to be generated in each model data reporting management requirement; parsing the model information of the corresponding reporting verification model and the corresponding data reporting management requirements, and generating the data verification rule template to be generated according to the parsing result; or using the candidate data verification rule template that matches the model information and the corresponding data reporting management requirements among the generated candidate data verification rule templates as the data verification rule template to be generated; or generating the data verification rule template to be generated in response to a template configuration instruction, where the template configuration instruction is an instruction triggered when an operation is performed on a visual verification rule configuration interface.

[0034] Among them, the data verification rule template to be generated can be one of the general data verification rule templates, and it can be pre-generated through the following steps: Since there is a corresponding relationship between the general data verification rule template and the data reporting verification model and the model data reporting management requirements, the corresponding reporting verification model corresponding to the data verification rule template to be generated can be obtained from each data reporting verification model, and the corresponding data reporting management requirements corresponding to the data verification rule template to be generated can be obtained from each model data reporting management requirements. Further, the data verification rule template to be generated can be generated in any of the following ways: Analyze the model information of the corresponding reporting verification model and the corresponding data reporting management requirements, and generate according to the analysis results. It should be noted that the model information and the attribute information are essentially the same. Here, different names are only used to distinguish the information of the corresponding reporting verification model itself and the information of the target reporting verification model itself, and it is not a specific limitation on the essential content; The candidate data verification rule template that matches the model information and the corresponding data reporting management requirements among the generated candidate data verification rule templates is used as the data verification rule template to be generated, that is, the matching candidate data verification rule template is directly obtained from the generated candidate data verification rule templates and used as the data verification rule template to be generated; In response to the template configuration instruction, the data verification rule template to be generated is generated, where the template configuration instruction can be an instruction triggered when the visual verification rule configuration interface is operated, such as an instruction triggered when the user configures the template in the visual verification rule configuration interface. It should be noted that as described above, the data verification rule template to be generated is one of the general data verification rule templates, which means that the generation method of the data verification rule template to be generated is the generation method of a certain general data verification rule template, and the generation methods of these general data verification rule templates can be the same or different, and no specific limitation is made here. The above technical solution achieves the effect of automatic generation of each general data verification rule template.

[0035] Another alternative technical solution is that the target reporting verification standard includes a target reporting verification model and target data reporting management requirements corresponding to the target reporting verification model. The target reporting verification model is derived from various types of data reporting verification models. The above multi-type data reporting verification method may further include: determining a target data verification rule template from each general data verification rule template based on the attribute information, the target reporting verification model, and the target data reporting management requirements. Among them, since the same data reporting verification model may exist in at least one data reporting verification standard, and the model data reporting management requirements for the same data reporting verification model in different data reporting verification standards may be different, this means that the same data reporting verification model may correspond to at least one model data reporting management requirement, and thus it means that the same data reporting verification model may correspond to at least one general data verification rule template. Therefore, a target data verification rule template can be determined from each general data verification rule template according to the attribute information (used to indicate what database it is), the target reporting verification model, and the target data reporting management requirements, thereby achieving the effect of automatically generating the target data verification rule template.

[0036] Embodiment 2

[0037] Figure 2 It is a flowchart of a multi-type data reporting verification method based on a rule library provided in Embodiment 2 of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, verifying the data to be verified in the database to be verified based on the target data verification rule to obtain a data verification result may include: determining a target server on which the database to be verified is deployed according to the attribute information, and sending the target data verification rule to the database to be verified deployed on the target server, so that the database to be verified verifies the data to be verified in the database to be verified according to the received target data verification rule and returns a data verification result; receiving the data verification result. Among them, the explanations of the same or corresponding terms in the above embodiments will not be repeated here.

[0038] See Figure 2 , the method of this embodiment may specifically include the following steps:

[0039] S210. In response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified.

[0040] S220. Obtain a target data verification rule that matches the database to be verified, and determine a target server on which the database to be verified is deployed according to the attribute information. Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reported verification standard, or a data verification rule generated according to a target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reported verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0041] Among them, since the attribute information may include the server address of the target server on which the database to be verified is deployed, the target server where the database to be verified is located can be determined according to the attribute information.

[0042] S230. Send the target data verification rule to the database to be verified deployed on the target server, so that the database to be verified verifies the data to be verified in the database to be verified according to the received target data verification rule, and returns a data verification result.

[0043] Among them, after determining the target server, the multi-type data reporting verification device can send the target data verification rule to the database to be verified deployed on the target server, so that the database to be verified verifies the data to be verified according to the received target data verification rule, and returns the data verification result to the multi-type data reporting verification device.

[0044] S240. Receive the data verification result.

[0045] The technical solution of the embodiment of the present invention determines the target server on which the database to be verified is deployed through the attribute information, so that the target data verification rule can be sent to the database to be verified deployed on the target server, so that the database to be verified performs data verification, thereby achieving the effect of effective data verification.

[0046] An alternative technical solution is to verify the data to be verified in the database to be verified according to the received target data verification rule, which may include: parsing the received target data verification rule to obtain the correspondence between the target data verification rule and the data to be verified in the database to be verified; constructing a data verification model according to the correspondence, and inputting the read data to be verified into the data verification model to verify the data to be verified; and / or generating a data verification statement according to the correspondence, and executing the data verification statement to verify the data to be verified. Among them, the correspondence can be the relationship of verifying what data to be verified based on what rules obtained after parsing the received target data verification rule. This correspondence can also be understood as the binding information between the target data verification rule and the database to be verified. For example, performing non-null verification and absolute value range verification on the a field in the data table A in the database to be verified, and performing association verification on the b field in the data table b. On this basis, data verification can be performed in at least one of the following ways according to the correspondence: constructing a data verification model according to the correspondence. The application of the data verification model enables all the data to be verified to be obtained only by reading from the database to be verified once, without frequent data reading. Then, the read data to be verified is directly input into the data verification model, and the data verification result can be obtained according to the output result of the data verification model, thus avoiding the situation of excessive resource occupation of the database to be verified caused by repeated data reading; generating a data verification statement according to the correspondence, and executing the data verification statement to verify the data to be verified, thereby improving the convenience of data verification.

[0047] Embodiment 3

[0048] Figure 3 It is a flowchart of a multi-type data reporting verification method based on a rule library provided in Embodiment 3 of the present invention. This embodiment is optimized based on the above technical solutions. In this embodiment, optionally, the above multi-type data reporting verification method may further include: obtaining the data verification frequency of the database to be verified, and generating a data verification strategy according to the target data verification rule and the data verification frequency; verifying the data to be verified in the database to be verified based on the target data verification rule, which may include: verifying the data to be verified in the database to be verified based on the data verification strategy; after obtaining the data verification result, the above multi-type data reporting verification method may further include: determining a frequency adjustment rule, adjusting the data verification frequency according to the frequency adjustment rule, and regenerating a data verification strategy according to the adjusted data verification frequency and the target data verification rule, so as to verify the data to be verified based on the regenerated data verification strategy. Among them, the explanations of the same or corresponding terms as those in the above embodiments are not repeated here.

[0049] SeeFigure 3 , the method of this embodiment may specifically include the following steps:

[0050] S310. In response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified.

[0051] S320. Obtain the target data verification rule that matches the database to be verified and the data verification frequency of the database to be verified. Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to the target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0052] Among them, the data verification frequency can indicate how often the data of the database to be verified is verified.

[0053] S330. Generate a data verification strategy according to the target data verification rule and the data verification frequency, and verify the data to be verified in the database to be verified based on the data verification strategy to obtain a data verification result.

[0054] Among them, the data verification strategy can indicate at what time and based on what rule the data to be verified in the database to be verified is verified. Therefore, after obtaining the data verification strategy, the data can be verified based on the data verification strategy to obtain a data verification result.

[0055] S340. Determine a frequency adjustment rule, adjust the data verification frequency according to the frequency adjustment rule, and regenerate the data verification strategy according to the adjusted data verification frequency and the target data verification rule, so as to verify the data to be verified based on the regenerated data verification strategy.

[0056] Among them, the frequency adjustment rule can indicate the rule based on which the data verification frequency is adjusted, which can be expressed by the frequency adjustment direction, the frequency adjustment amplitude, etc., and can be determined by the data verification results obtained in the previous data verification, the operating parameters of the target server, etc. The data verification frequency is adjusted according to the frequency adjustment rule so that the adjusted data verification frequency is consistent with the actual data verification requirements, and the data verification requirements can reflect the frequency based on which the verified data needs to be or can be verified; further, the data verification strategy is regenerated according to the adjusted data verification frequency and the target data verification rule, so that the verified data is verified based on the regenerated data verification strategy, that is, the data verification is performed based on the data verification strategy that matches the actual data verification requirements. In actual applications, optionally, when performing data verification on the verified database, the data verification tasks of a certain data table or a certain field in a certain data table in the verified database can be determined in a targeted manner according to the adjusted data verification frequency and target data verification rules, and then the data verification strategy is determined based on these data verification tasks, thereby achieving the effect of conducting targeted data verification on each data table and / or each field in the verified database.

[0057] The technical solution of the embodiment of the present invention generates a data verification strategy through target data verification rules and the data verification frequency of the obtained database to be verified, so as to perform data verification based on the data verification strategy, and further adjust the data verification frequency according to the frequency adjustment rule to regenerate a data verification strategy that matches the actual data verification demand, so as to perform data verification based on the regenerated data verification strategy. In this way, compared with scheduled or real-time data verification, data verification is performed through a data verification strategy that matches the actual data verification demand, without repeatedly running a large number of scripts and frequently reading the verified data, thereby reducing the resource consumption and load pressure of the target server and ensuring the effective operation of the target server.

[0058] An optional technical solution for determining frequency adjustment rules may include: obtaining a knowledge base, and based on the mapping relationship between each general verification result in the knowledge base and each general adjustment rule, determining the frequency adjustment rule corresponding to the data verification result in each general adjustment rule, the general verification result including the verification result corresponding to the general data verification rule template; and / or, obtaining the operating parameters of the target server where the verified database is deployed, and determining the frequency adjustment rule based on the operating parameters.

[0059] Among them, the knowledge in the knowledge base can represent the mapping relationship between each general verification result and each general adjustment rule, that is, what kind of general verification result can map out what kind of general adjustment rule, and this knowledge can be summarized based on the previous data verification situation. In other words, the general verification result can be the data verification result obtained previously (that is, verified based on the historical data verification rule), and the historical data verification rule can be the data verification rule generated based on the general data verification rule template. For example, when the general verification result shows a high vacancy rate, the corresponding general adjustment rule can be to increase the data verification frequency, and this is a piece of knowledge; another example is that when the general verification result shows that the value exceeds the range value, the corresponding general adjustment rule can be to increase the data verification frequency, and this is a piece of knowledge; another example is that when the general verification result shows that the data quality is poor, the corresponding general adjustment rule can be to increase the data verification frequency to improve the real-time feedback of data problems, and this is a piece of knowledge; and so on. Therefore, after obtaining the data verification result, the general verification result that matches the data verification result can be determined from each general verification result, and then the general adjustment rule corresponding to the matching general verification result can be used as the frequency adjustment rule, thereby achieving the effect of accurately and quickly determining the frequency adjustment rule based on the knowledge base.

[0060] The operating parameters can be the relevant parameters of the target server during operation, such as network card traffic, number of network connections, the usage rate of its own Central Processing Unit (CPU), memory occupancy, etc., which are not specifically limited here. According to the operating parameters, the occupancy of the server resources of the target server can be determined. Therefore, in order to ensure the effective operation of the target server, the frequency adjustment rule can be determined according to the operating parameters. For example, when the operating parameters indicate a relatively high occupancy ratio of the server resources, the frequency adjustment rule can be to reduce the data verification frequency; when the operating parameters indicate a relatively low occupancy ratio of the server resources, the frequency adjustment rule can be to increase the data verification frequency, so that more data verification can be performed when the target server is idle and less data verification can be performed when the target server is busy, thus ensuring both the effective operation of the target server and the effective execution of data verification.

[0061] Another alternative technical solution for the above multi-type data reporting verification method may further include: obtaining sampling data from the data to be verified, verifying the sampling data based on the target data verification rules to obtain a sampling verification result; determining the initial verification frequency of the database to be verified according to the sampling verification result; correspondingly, obtaining the data verification frequency of the database to be verified may include: determining the data verification frequency of the database to be verified based on the initial verification frequency. Among them, the sampling data may be a part of the data to be verified, and the sampling verification result may be a data verification result obtained by verifying the sampling data based on the target data verification rules. Compared with the data to be verified, since the amount of the sampling data is smaller, the data verification can be completed with less server resources occupied. Furthermore, the initial verification frequency is determined according to the sampling verification result, and then the data verification frequency is determined according to the initial verification frequency, such as directly using the initial verification frequency as the data verification frequency; or determining a frequency adjustment rule according to the initial verification frequency, and then adjusting the frequency based on the frequency adjustment rule to obtain the data verification frequency; and so on. The above technical solution reduces the load pressure on the target server from the very beginning through sampling verification, and then determines the data verification frequency through the obtained sampling verification result to further reduce the load pressure on the target server, so that the data verification can be completed with as little server resources occupied as possible.

[0062] Embodiment 4

[0063] Figure 4 It is a flowchart of a multi-type data reporting verification method based on a rule library provided in Embodiment 4 of the present invention. This embodiment is optimized based on the technical solutions in the above Embodiment 3, combining the knowledge base and sampling verification. Among them, the explanations of the same or corresponding terms as those in the above embodiments will not be repeated here.

[0064] See Figure 4 , the method of this embodiment may specifically include the following steps:

[0065] S410. Obtain various types of data reporting verification models and the model data reporting management requirements corresponding to each data reporting verification model respectively.

[0066] S420. For each data reporting verification model, generate a general data verification rule template corresponding to the data reporting verification model and the model data reporting management requirements according to the data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0067] S430. In response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified.

[0068] S440. Obtain a target data verification rule that matches the database to be verified, and obtain sampled data from the data to be verified, so as to verify the sampled data based on the target data verification rule to obtain a sampled verification result. Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reported verification standard, or a data verification rule generated according to the target data verification rule template. The target data verification rule template includes a data verification rule template determined from each general data verification rule template according to the attribute information and the target reported verification standard.

[0069] S450. Determine the initial verification frequency of the database to be verified according to the sampled verification result, and use the initial verification frequency as the data verification frequency of the database to be verified. Generate a data verification strategy according to the target data verification rule and the data verification frequency, so as to verify the data to be verified in the database to be verified based on the data verification strategy to obtain a data verification result.

[0070] Among them, in practical applications, optionally, the initial verification frequency can be determined according to the knowledge base. That is, in addition to storing the knowledge described above in the knowledge base, it can also store knowledge representing the mapping relationship between the sampled verification result and the initial verification frequency. Therefore, after obtaining the sampled verification result, the initial verification frequency can be obtained in combination with the knowledge base. Use the initial verification frequency as the data verification frequency to generate a data verification strategy. The data verification strategy at this time can also be called the initial verification strategy.

[0071] S460. Obtain the knowledge base, and determine the frequency adjustment rule corresponding to the data verification result in each general adjustment rule based on the mapping relationship between each general verification result and each general adjustment rule in the knowledge base. Among them, the general verification result includes the verification result corresponding to the general data verification rule template.

[0072] S470. Adjust the data verification frequency according to the frequency adjustment rule, and regenerate the data verification strategy according to the adjusted data verification frequency and the target data verification rule, so as to verify the data to be verified based on the regenerated data verification strategy.

[0073] The technical solution of the embodiment of the present invention combines the knowledge base and the sampled verification, thereby achieving the effect of reducing the occupation ratio of server resources from multiple angles when performing data verification.

[0074] Embodiment Five

[0075] Figure 5The block diagram of the multi-type data reporting verification device based on a rule library provided in the fifth embodiment of the present invention. This device is used to execute the multi-type data reporting verification method based on a rule library provided in any of the above embodiments. This device and the multi-type data reporting verification method based on a rule library in each of the above embodiments belong to the same inventive concept. For the details not described in detail in the embodiments of this device, reference can be made to the embodiments of the multi-type data reporting verification method based on a rule library. Refer to Figure 5 , specifically, this device may include: a standard acquisition module 510 and a data verification module 520. Among them,

[0076] The standard acquisition module 510 is configured to, in response to a data reporting verification instruction, if the database to be verified is a newly added database, acquire the attribute information of the database to be verified and the target reporting verification standard for the database to be verified;

[0077] The data verification module 520 is configured to acquire a target data verification rule that matches the database to be verified, and verify the data to be verified in the database to be verified based on the target data verification rule to obtain a data verification result;

[0078] Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to a target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0079] Optionally, the to-be-generated data verification rule template in each general data verification rule template is pre-generated through the following modules:

[0080] The corresponding data reporting management requirement acquisition module is configured to acquire the corresponding reporting verification model corresponding to the to-be-generated data verification rule template in each data reporting verification model and the corresponding data reporting management requirement corresponding to the to-be-generated data verification rule template in each model data reporting management requirement;

[0081] The first generation module of the to-be-generated data verification rule template is configured to parse the model information of the corresponding reporting verification model and the corresponding data reporting management requirement, and generate the to-be-generated data verification rule template according to the parsing result; or

[0082] The second generation module of the to-be-generated data verification rule template is used to use the candidate data verification rule templates that match the model information and the corresponding data reporting management requirements in the generated candidate data verification rule templates as the to-be-generated data verification rule templates; or,

[0083] The third generation module of the to-be-generated data verification rule template is used to generate the to-be-generated data verification rule template in response to a template configuration instruction, where the template configuration instruction is an instruction triggered when an operation is performed on the visual verification rule configuration interface.

[0084] Optionally, the target reporting verification standard includes a target reporting verification model and the corresponding target data reporting management requirements for the target reporting verification model. The target reporting verification model is derived from various types of data reporting verification models. The above multi-type data reporting verification device may further include:

[0085] The target data verification rule template determination module is used to determine the target data verification rule template from each general data verification rule template based on the attribute information, the target reporting verification model, and the target data reporting management requirements.

[0086] Optionally, the data verification module 520 may include:

[0087] The data verification result return unit is used to determine the target server on which the database to be verified is deployed according to the attribute information, and send the target data verification rule to the database to be verified deployed on the target server, so that the database to be verified verifies the data to be verified in the database to be verified according to the received target data verification rule, and returns the data verification result;

[0088] The data verification result receiving unit is used to receive the data verification result.

[0089] On this basis, optionally, the data verification result return unit may include:

[0090] The corresponding relationship obtaining subunit is used to parse the received target data verification rule to obtain the corresponding relationship between the target data verification rule and the data to be verified in the database to be verified;

[0091] The data verification subunit is used to construct a data verification model according to the corresponding relationship, and input the read data to be verified into the data verification model to verify the data to be verified; and / or generate a data verification statement according to the corresponding relationship and execute the data verification statement to verify the data to be verified.

[0092] Optionally, the above multi-type data reporting verification device may further include:

[0093] A data verification strategy generation module, configured to obtain the data verification frequency of the database to be verified, and generate a data verification strategy according to the target data verification rule and the data verification frequency;

[0094] The data verification module 520 may include:

[0095] A data verification unit, configured to verify the data to be verified in the database to be verified based on the data verification strategy;

[0096] The above multi-type data reporting and verification device may further include:

[0097] A data re-verification module, configured to determine a frequency adjustment rule after obtaining the data verification result, adjust the data verification frequency according to the frequency adjustment rule, and regenerate a data verification strategy according to the adjusted data verification frequency and the target data verification rule, so as to verify the data to be verified based on the regenerated data verification strategy.

[0098] On this basis, optionally, the data re-verification module may include:

[0099] A first frequency adjustment rule determination unit, configured to obtain a knowledge base, and determine the frequency adjustment rule corresponding to the data verification result in each general adjustment rule based on the mapping relationship between each general verification result and each general adjustment rule in the knowledge base, where the general verification result includes the verification result corresponding to the general data verification rule template;

[0100] And / or,

[0101] A second frequency adjustment rule determination unit, configured to obtain the operation parameters of the target server where the database to be verified is deployed, and determine the frequency adjustment rule according to the operation parameters.

[0102] Optionally again, the above multi-type data reporting and verification device may further include:

[0103] A sampling verification result obtaining module, configured to obtain sampling data from the data to be verified, so as to verify the sampling data based on the target data verification rule and obtain a sampling verification result;

[0104] An initial verification frequency determination module, configured to determine the initial verification frequency of the database to be verified according to the sampling verification result;

[0105] Correspondingly, the data verification strategy generation module may include:

[0106] A data verification frequency obtaining unit, configured to determine the data verification frequency of the database to be verified based on the initial verification frequency.

[0107] The multi-type data reporting verification device based on a rule library provided in the fifth embodiment of the present invention, through a standard acquisition module, in response to a data reporting verification instruction, when the database to be verified is a newly added database, obtains the attribute information of the database to be verified and the target reporting verification standard for the database to be verified; furthermore, through a data verification module, obtains a target data verification rule that matches the database to be verified, and the target data verification rule may include a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to a target data verification rule template, so as to verify the data to be verified in the database to be verified based on the target data verification rule and obtain a data verification result. The above device, when verifying a newly added database, does not require the user to manually configure rules, but can directly obtain from each generated historical data verification rule or automatically generate according to each generated general data verification rule template, reducing the complexity of user operations through the method of rule reuse, thereby reducing the workload of the user.

[0108] The multi-type data reporting verification device based on a rule library provided in the embodiments of the present invention can execute the multi-type data reporting verification method based on a rule library provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution of the method.

[0109] It should be noted that in the embodiments of the above multi-type data reporting verification device based on a rule library, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0110] Embodiment Six

[0111] Figure 6 FIG. is a schematic structural diagram of a multi-type data reporting verification device based on a rule library provided in the sixth embodiment of the present invention, as Figure 6 shown, the device includes a memory 610, a processor 620, an input device 630, and an output device 640. The number of processors 620 in the device may be one or more, Figure 6 taking one processor 620 as an example; the memory 610, the processor 620, the input device 630, and the output device 640 in the device may be connected through a bus or other means, Figure 6 taking connection through a bus 650 as an example.

[0112] The memory 610, being a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the rule-base-based multi-type data reporting and verification method in the embodiments of the present invention (for example, the standard acquisition module 510 and the data verification module 520 in the rule-base-based multi-type data reporting and verification device). The processor 620 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 610, thereby implementing the above-mentioned rule-base-based multi-type data reporting and verification method.

[0113] The memory 610 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the device, etc. In addition, the memory 610 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 610 may further include a memory remotely set relative to the processor 620, and these remote memories can be connected to the device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The input device 630 can be used to receive input digital or character information and generate key signal inputs related to the user settings and function controls of the device. The output device 640 may include a display device such as a display screen.

[0115] Embodiment Seven

[0116] Embodiment Seven of the present invention provides a storage medium containing computer-executable instructions, and the computer-executable instructions are used to execute a rule-base-based multi-type data reporting and verification method when executed by a computer processor. The method includes:

[0117] In response to a data reporting and verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting and verification standard for the database to be verified;

[0118] Obtain a target data verification rule that matches the database to be verified, and based on the target data verification rule, verify the data to be verified in the database to be verified to obtain a data verification result;

[0119] Among them, the target data verification rules include data verification rules determined from each of the generated historical data verification rules according to the attribute information and the target reporting verification criteria, or data verification rules generated according to the target data verification rule template. The target data verification rule template includes a data verification rule template determined from each of the generated general data verification rule templates according to the attribute information and the target reporting verification criteria, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

[0120] Certainly, for a storage medium containing computer-executable instructions provided by an embodiment of the present invention, the computer-executable instructions are not limited to the method operations described above, and can also execute relevant operations in the multi-type data reporting verification method based on a rule library provided by any embodiment of the present invention.

[0121] From the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software and necessary general hardware. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a floppy disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FLASH), a hard disk, or an optical disc of a computer, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0122] Note that the above is only a preferred embodiment of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein. Various obvious changes, re-adjustments, and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the inventive concept, it can also include more other equivalent embodiments, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for verifying multi-type data reporting based on a rule library, characterized in that, Including: In response to a data reporting verification instruction, if the database to be verified is a newly added database, obtain the attribute information of the database to be verified and the target reporting verification standard for the database to be verified; Obtain sampling data from the data to be verified in the database to be verified, and verify the sampling data based on the target data verification rule matching the database to be verified to obtain a sampling verification result; Determine the initial verification frequency of the database to be verified according to the sampling verification result, determine the data verification frequency of the database to be verified based on the initial verification frequency, generate a data verification strategy according to the target data verification rule and the data verification frequency, and verify the data to be verified in the database to be verified based on the data verification strategy to obtain a data verification result; Determine a frequency adjustment rule, adjust the data verification frequency according to the frequency adjustment rule, and regenerate the data verification strategy according to the adjusted data verification frequency and the target data verification rule, so as to verify the data to be verified based on the regenerated data verification strategy; Among them, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to a target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

2. The method according to claim 1, characterized in that, The to-be-generated data verification rule template in each general data verification rule template is pre-generated through the following steps: Obtain the corresponding reporting verification model corresponding to the to-be-generated data verification rule template in each data reporting verification model, and the corresponding data reporting management requirements corresponding to the to-be-generated data verification rule template in each model data reporting management requirement; Analyze the model information of the corresponding reporting verification model and the corresponding data reporting management requirements, and generate the to-be-generated data verification rule template according to the analysis result; Or, use the candidate data verification rule template that matches the model information and the corresponding data reporting management requirements among the generated candidate data verification rule templates as the to-be-generated data verification rule template; Or, in response to a template configuration instruction, generate the to-be-generated data verification rule template, where the template configuration instruction is an instruction triggered when an operation is performed on a visual verification rule configuration interface.

3. The method according to claim 1, characterized in that, The target reporting verification standard includes a target reporting verification model and target data reporting management requirements corresponding to the target reporting verification model. The target reporting verification model is derived from each type of data reporting verification model. The method further includes: Based on the attribute information, the target reporting verification model, and the target data reporting management requirements, determine the target data verification rule template from the general data verification rule templates.

4. The method according to claim 1, characterized in that, Verifying the data to be verified in the database to be verified based on the data verification strategy, and the obtained data verification result includes: Determine the target server on which the database to be verified is deployed according to the attribute information, and send the data verification strategy to the database to be verified deployed on the target server, so that the database to be verified verifies the data to be verified in the database to be verified according to the received data verification strategy, and returns a data verification result; Receive the data verification result.

5. The method according to claim 4, characterized in that, Verifying the data to be verified in the database to be verified according to the received data verification strategy includes: Analyze the received data verification strategy to obtain the corresponding relationship between the data verification strategy and the data to be verified in the database to be verified; Construct a data verification model according to the corresponding relationship, and input the read data to be verified into the data verification model to verify the data to be verified; and / or, generate a data verification statement according to the corresponding relationship, and execute the data verification statement to verify the data to be verified.

6. The method according to claim 1, characterized in that, The determination of the frequency adjustment rule includes: Obtain a knowledge base, and based on the mapping relationship between the general verification results and the general adjustment rules in the knowledge base, determine the frequency adjustment rule corresponding to the data verification result among the general adjustment rules, where the general verification results include the verification results corresponding to the general data verification rule template; and / or, Obtain the operation parameters of the target server on which the database to be verified is deployed, and determine the frequency adjustment rule according to the operation parameters.

7. A multi-type data reporting verification device based on a rule library, characterized in that, including: A standard acquisition module, configured to, in response to a data reporting verification instruction, if the database to be verified is a newly added database, acquire the attribute information of the database to be verified and the target reporting verification standard for the database to be verified; A data verification module, configured to acquire a target data verification rule that matches the database to be verified; A sampling verification result acquisition module, configured to acquire sampling data from the data to be verified in the database to be verified, and verify the sampling data based on the target data verification rule to obtain a sampling verification result; An initial verification frequency determination module, configured to determine the initial verification frequency of the database to be verified according to the sampling verification result; A data verification strategy generation module, configured to determine the data verification frequency of the database to be verified based on the initial verification frequency, and generate a data verification strategy according to the target data verification rule and the data verification frequency; The data verification module is further configured to verify the data to be verified in the database to be verified based on the data verification strategy to obtain a data verification result; A data re-verification module, configured to determine a frequency adjustment rule, adjust the data verification frequency according to the frequency adjustment rule, and regenerate the data verification strategy according to the adjusted data verification frequency and the target data verification rule, so as to verify the data to be verified based on the regenerated data verification strategy; Wherein, the target data verification rule includes a data verification rule determined from each generated historical data verification rule according to the attribute information and the target reporting verification standard, or a data verification rule generated according to a target data verification rule template. The target data verification rule template includes a data verification rule template determined from each generated general data verification rule template according to the attribute information and the target reporting verification standard, and each general data verification rule template is generated according to each type of data reporting verification model and the model data reporting management requirements corresponding to the data reporting verification model.

8. A computer-readable storage medium, on which a computer program is stored, characterized in that,When the computer program is executed by a processor, it implements the rule library-based multi-type data reporting verification method according to any one of claims 1-6.

Images