Verification rule generation method and device, storage medium and electronic equipment

Abstract

The embodiment of the present specification provides a generation method and device of a check rule, a storage medium and an electronic device, the generation method of the check rule comprises the following steps: extracting an original check rule corresponding to a target business scenario from a rule library, and acquiring traffic data corresponding to the target business scenario; then, generating a target check rule under the target business scenario according to the traffic data corresponding to the target business scenario, i.e. scene traffic data; then, comparing the original check rule and the target check rule for differences to obtain difference data; and finally, supplementing the original check rule according to the difference data to generate an updated check rule corresponding to the target business scenario.

Description

Technical Field

[0001] This specification relates to the field of data verification technology, and in particular to a method, apparatus, storage medium, and electronic device for generating verification rules. Background Technology

[0002] Verification, as a crucial part of the quality control system, is one of the core methods for detecting financial risks. To promptly identify financial risks, related technologies often employ verification rules deployed on data verification platforms to verify data.

[0003] Currently, there are two methods for writing verification rules: manual writing, where rules are written by individuals based on their familiarity with business data, and platform generation, where rich online data is used to balance rules using genetic algorithms, resulting in verification rules generated on the platform. Statistical analysis of online issues in business scenarios, such as insurance, over a period of time reveals that approximately 43% of online issues can be detected promptly by verification rules. However, 50% of these issues fail to be detected in a timely manner due to missing or incorrect verification rules. Therefore, reducing the failure to detect online issues promptly due to missing or incorrect verification rules has become a pressing technical challenge.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this specification, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This specification provides a method, apparatus, storage medium, and electronic device for generating verification rules, which can reduce the failure to detect online problems in a timely manner due to missing or incorrect verification rules, and help reduce the impact of online problems.

[0006] Other features and advantages of the embodiments described herein will become apparent from the following detailed description, or may be learned in part by practice of this specification.

[0007] Firstly, a method for generating verification rules is provided. The method includes: extracting original verification rules corresponding to a target business scenario from a rule base; obtaining traffic data corresponding to the target business scenario; generating target verification rules for the target business scenario based on the traffic data; comparing the original verification rules with the target verification rules to obtain difference data; and supplementing the original verification rules based on the difference data to generate updated verification rules corresponding to the target business scenario.

[0008] In conjunction with the first aspect, in some possible implementations, generating the target verification rules for the target business scenario based on the traffic data includes: determining the inter-table relationships between the related data tables corresponding to the target business scenario based on the traffic data; filtering the fields in the related data tables based on the valid fields corresponding to the target business scenario to obtain the valid fields in the related data tables; determining the relationships between the valid fields in the related data tables based on the inter-table relationships and the valid fields in the related data tables; and generating the target verification rules for the target business scenario based on the relationships between the valid fields in the related data tables.

[0009] In conjunction with the first aspect, in some possible implementations, the above-mentioned generation of target verification rules for the target business scenario based on the association relationships between valid fields in the aforementioned associated data tables includes: determining the target data table corresponding to the aforementioned associated data table in the data warehouse based on the table name mapping relationship and the name of the aforementioned associated data table; performing data cleaning on the aforementioned target data table to obtain a wide table corresponding to the aforementioned target business scenario; filtering the association relationships between the aforementioned valid fields based on the data values ​​in the aforementioned wide table to obtain target association relationships; wherein the correlation between the valid fields corresponding to the aforementioned target association relationships is greater than a preset correlation; and generating target verification rules for the aforementioned target business scenario based on the aforementioned target association relationships.

[0010] In conjunction with the first aspect, in some possible implementations, the process of filtering the relationships between the valid fields based on the data values ​​in the wide table to obtain a target relationship includes: determining the mutual information between the valid fields based on the data values ​​in the wide table; filtering the relationships between the valid fields based on the mutual information to obtain candidate relationships; wherein the mutual information between the valid fields corresponding to the candidate relationships is greater than a first preset threshold; determining the conditional mutual information between the valid fields corresponding to the candidate relationships based on the data values ​​in the wide table; filtering the candidate relationships based on the conditional mutual information to obtain a target relationship; wherein the conditional mutual information between the valid fields corresponding to the target relationship is greater than a second preset threshold.

[0011] In conjunction with the first aspect, in some possible implementations, the above-mentioned filtering of the relationships between the effective fields based on the data values ​​in the wide table to obtain the target relationship includes: deduplicating the data values ​​in the wide table based on the effective fields to obtain a deduplicated wide table; and filtering the relationships between the effective fields based on the data values ​​in the deduplicated wide table to obtain the target relationship.

[0012] In conjunction with the first aspect, in some possible implementations, after cleaning the target data table to obtain the wide table corresponding to the target business scenario, the method further includes: generating general verification rules corresponding to the target business scenario based on the wide table and a preset general rule template; generating target verification rules under the target business scenario based on the target association includes: generating refined verification rules under the target business scenario based on the target association and a refined rule template; and merging the general verification rules and the refined verification rules to generate the target verification rules under the target business scenario.

[0013] In conjunction with the first aspect, in some possible implementations, the aforementioned target verification rules include rule data in multiple dimensions; the aforementioned comparison of the differences between the original verification rules and the target verification rules to obtain difference data includes: decomposing the original verification rules according to the aforementioned multiple dimensions to obtain rule data of the original verification rules in the aforementioned multiple dimensions; comparing the differences between the original verification rules and the target verification rules in the same dimension to obtain difference data corresponding to the rule data in the aforementioned multiple dimensions respectively.

[0014] In conjunction with the first aspect, in some possible implementations, the above method further includes: atomically decomposing the rule data of the target verification rule in the aforementioned multiple dimensions to obtain atomic rule data of the target verification rule in the aforementioned multiple dimensions; atomically decomposing the rule data of the original verification rule in the aforementioned multiple dimensions to obtain atomic rule data of the original verification rule in the aforementioned multiple dimensions; and comparing the differences between the rule data of the original verification rule and the target verification rule in the same dimension to obtain the difference data corresponding to the rule data in the aforementioned multiple dimensions, including: comparing the differences between the atomic rule data of the original verification rule and the target verification rule in the same dimension to obtain the difference data corresponding to the atomic rule data in the aforementioned multiple dimensions.

[0015] In conjunction with the first aspect, in some possible implementations, the above-mentioned supplementation of the original verification rules based on the aforementioned difference data to generate updated verification rules corresponding to the aforementioned target business scenario includes: if the aforementioned difference data includes first difference data, then generating a rule supplement set based on the aforementioned first difference data, and supplementing the aforementioned original verification rules based on the aforementioned rule supplement set to generate updated verification rules corresponding to the aforementioned target business scenario; wherein, the aforementioned first difference data exists in the aforementioned target verification rules but does not exist in the aforementioned original verification rules.

[0016] In conjunction with the first aspect, in some possible implementations, if the aforementioned difference data includes second difference data, then the missing rule template is determined based on the aforementioned second difference data, and the aforementioned missing rule template is added to the aforementioned refined rule template; wherein, the aforementioned second difference data exists in the aforementioned original verification rule but does not exist in the aforementioned target verification rule.

[0017] In conjunction with the first aspect, in some possible implementations, the aforementioned multiple dimensions include: tables, fields, preconditions, and verification content.

[0018] Secondly, a verification rule generation apparatus is provided, comprising: an original verification rule extraction module for extracting original verification rules corresponding to a target business scenario from a rule base; a traffic data acquisition module for acquiring traffic data corresponding to the target business scenario; a target verification rule generation module for generating target verification rules for the target business scenario based on the traffic data; a rule comparison module for comparing the original verification rules with the target verification rules to obtain difference data; and a rule supplementation module for supplementing the original verification rules based on the difference data to generate updated verification rules corresponding to the target business scenario.

[0019] Thirdly, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for generating the verification rules as described above.

[0020] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer or processor, cause the computer or processor to perform the method for generating the verification rules as described above.

[0021] According to another aspect of this specification, a computer program product containing instructions is provided that, when the computer program product is run on a computer or processor, causes the computer or processor to perform the method for generating the verification rules as described above.

[0022] The methods, apparatus, storage media, and electronic devices for generating verification rules provided in the embodiments of this specification have the following technical effects:

[0023] The solution provided in the embodiments of this specification acquires traffic data corresponding to the target business scenario from the perspective of scenario traffic, and generates target verification rules under the target business scenario based on the traffic data. This helps to adapt the target verification rules to dynamic scenario traffic data. By comparing the differences between the original verification rules and the target verification rules, it is helpful to analyze the rules that are missing or incorrectly written in the original verification rules. Then, based on the difference data, the original verification rules are supplemented to generate updated verification rules corresponding to the target business scenario. This helps to supplement the rules that are missing or incorrectly written in the original verification rules, thereby reducing the failure to detect online problems in a timely manner due to missing or incorrectly written verification rules, and helping to reduce the impact of online problems.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this specification and, together with the description, serve to explain the principles of this specification. It is obvious that the drawings described below are merely some embodiments of this specification, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0026] Figure 1 A flowchart illustrating the method for generating verification rules provided in the embodiments of this specification is shown.

[0027] Figure 2 A schematic diagram illustrating the implementation principle of the verification rule generation method provided in the embodiments of this specification is shown.

[0028] Figure 3 A schematic diagram of the structure of the verification rule generation apparatus provided in the embodiments of this specification is shown;

[0029] Figure 4 A schematic diagram of the electronic device in an embodiment of this specification is shown. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this specification clearer, the embodiments of this specification will be described in further detail below with reference to the accompanying drawings.

[0031] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0032] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this specification more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of the embodiments described herein. However, those skilled in the art will recognize that the technical solutions described herein may be practiced with one or more of the specific details mentioned above omitted, or other methods, components, apparatus, steps, etc., may be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this specification.

[0033] Furthermore, the accompanying drawings are merely illustrative diagrams of this specification and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0034] Verification, as a core quality defense line in risk detection, plays a crucial role in the funding chain of insurance products. It is one of the core means of detecting financial risks; a robust verification system can help businesses promptly identify financial security risks, and combined with relevant risk mitigation capabilities, the impact can be reduced. To promptly detect financial risks, related technologies often employ verification rules deployed on a data verification platform to verify data.

[0035] Currently, there are two ways to write verification rules: one is manual writing, which is based on the familiarity with business link data to manually write verification rules; the other is generation by the ZhiRen platform (intelligent verification platform), which uses rich online data to balance rules based on genetic algorithms and thus generates verification rules on the ZhiRen platform.

[0036] The inventors of this application, through statistical analysis of online issues in business scenarios such as insurance over a period of time, found that approximately 43% of online issues could be detected promptly by verification rules. However, 50% of these issues were not detected in a timely manner due to missing or incorrect verification rules. Manually written verification rules are inherently prone to omissions and errors; that is, the omission or miswriting of the verification rules used to detect these online issues makes them difficult to detect in a timely manner. Therefore, to address the technical problem of untimely detection of online issues, embodiments of this specification provide a method, apparatus, storage medium, and electronic device for generating verification rules. By combining existing manual verification rules with human capabilities, the method recommends and supplements verification rules based on existing manual verification rules, thereby reducing the failure to detect online issues in a timely manner due to missing or incorrect verification rules and mitigating the impact of online issues.

[0037] The following is a description of an embodiment of the method for generating verification rules involved in the embodiments of this specification:

[0038] The method for generating verification rules in the embodiments of this specification is applied to an electronic device, which can be a server. Specifically, the server can be a backend business server capable of data transmission, data processing, and other functions. Specifically, the server can be an electronic device with data processing, storage, and network interaction capabilities; or it can be a software program running on the electronic device that provides support for data processing, storage, and network interaction. In this embodiment, the number of servers is not specifically limited. The server can be a single server, several servers, or a server cluster formed by several servers.

[0039] It should be noted that the methods in the embodiments of this specification can be executed by a single electronic device, such as a computer or server, or they can be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the methods in the embodiments of this specification, and these multiple devices will interact with each other to complete the above-mentioned method for generating verification rules.

[0040] In one scenario example, the method for generating verification rules provided in the embodiments of this specification can automatically generate update verification rules for various business scenarios. The generated update verification rules can be deployed on the verification platform.

[0041] This allows us to use updated verification rules to verify tasks in various business scenarios, obtain verification results, and promptly identify potential online issues.

[0042] A flowchart illustrating the method for generating verification rules in the embodiments of this specification can be found in [reference needed]. Figure 1 ,include:

[0043] Step 102: Extract the original verification rules corresponding to the target business scenario from the rule base.

[0044] Step 104: Obtain traffic data corresponding to the target business scenario.

[0045] Step 106: Generate target verification rules for the target business scenario based on traffic data.

[0046] Step 108: Compare the original verification rules with the target verification rules to obtain the difference data.

[0047] Step 110: Based on the difference data, supplement the original verification rules and generate updated verification rules corresponding to the target business scenario.

[0048] exist Figure 1 In the illustrated embodiment, from the perspective of scenario traffic, traffic data corresponding to the target business scenario is obtained, and target verification rules under the target business scenario are generated based on the traffic data. This helps to adapt the target verification rules to dynamic scenario traffic data. By comparing the differences between the original verification rules and the target verification rules, it is helpful to analyze the rules that are missing or incorrectly written in the original verification rules. Then, based on the difference data, the original verification rules are supplemented to generate updated verification rules corresponding to the target business scenario. This helps to supplement the rules that are missing or incorrectly written in the original verification rules, thereby reducing the failure to detect online problems in a timely manner due to missing or incorrect verification rules, and helping to reduce the impact of online problems.

[0049] The following is about Figure 1 The specific implementation methods of each step in the illustrated embodiment are explained below:

[0050] In step 102, the rule base may include original verification rules corresponding to various business scenarios. These original verification rules can be manually written, for example, verification rules applicable to different business scenarios can be manually written by R&D personnel based on their understanding of those scenarios, and these manually written verification rules can be stored in the rule base. The rule base can be deployed on the verification platform or other platforms that communicate with the verification platform, so that the verification platform can call the corresponding original verification rules in the rule base to obtain the verification results when performing verification tasks. In this step, the target business scenario can be any one of the above-mentioned multiple business scenarios. For example, the multiple business scenarios involved in insurance business may include: insurance application scenario, insurance renewal scenario, renewal business scenario, and insurance surrender scenario, so the target business scenario can be any one of the above four business scenarios.

[0051] It should be noted that the various business scenarios in this embodiment are only examples of the four specific business scenarios involved in the above-mentioned insurance business. In actual implementation, the business scenarios are not limited to the above examples.

[0052] For example, the original verification rules may include TH verification rules and T+1 verification rules. TH verification rules are hourly verification rules, and T+1 verification rules are daily verification rules. In this step, the extracted original verification rules corresponding to the target business scenario may include the TH verification rules and T+1 verification rules corresponding to the target business scenario.

[0053] In step 104, traffic data corresponding to the target business scenario can be obtained, that is, traffic data generated under the target business scenario. Traffic data can refer to behavioral record data under the target business scenario, such as browsing history, click history, payment history, exit history, etc.

[0054] In step 106, verification rules can be mined and derived based on the traffic data corresponding to the target business scenario to obtain the target verification rules for the target business scenario. Specifically, from the perspective of scenario traffic, the code fields involved in the traffic data corresponding to the target business scenario can be refined and processed to generate highly correlated relationships. Finally, verification rules are intelligently generated based on these highly correlated relationships, i.e., the target verification rules are generated. Here, highly correlated relationships can be understood as relationships with a correlation greater than a preset correlation threshold.

[0055] In an exemplary embodiment, step 106 described above may be implemented in the following ways: S1 to S7:

[0056] S1: Based on traffic data, determine the inter-table relationships between the relevant data tables corresponding to the target business scenario.

[0057] Specifically, different business scenarios typically involve different data tables. The related data tables corresponding to the target business scenario can be understood as multiple data tables with interrelationships within that target business scenario. Generally, if a row and column of data in one data table is identical to a row and column of data in another data table, it indicates a relationship between the two tables. In this embodiment, after obtaining the traffic data corresponding to the target business scenario, the inter-table relationships between the related data tables corresponding to that target business scenario can be obtained through bytecode tracing. Besides obtaining the inter-table relationships, bytecode tracing can also retrieve the key-value pairs corresponding to these relationships.

[0058] For example, if the target business scenario involves related data tables including table a, table b, and table c, after obtaining the traffic data corresponding to the target business scenario, the inter-table relationships between tables a and b, and between tables a and c, can be found through bytecode tracing. In practice, this bytecode tracing can be achieved by using taint analysis on the traffic data.

[0059] For example, after obtaining traffic data, dynamic code analysis can be performed based on the traffic data to obtain the inter-table relationships and the scene field values ​​corresponding to the target business scenario. The scope of scene data can be limited by the traffic data corresponding to the target business scenario.

[0060] S3: Based on the valid fields corresponding to the target business scenario, filter the fields in the associated data table to obtain the valid fields in the associated data table.

[0061] In this context, the effective fields corresponding to the target business scenario can be understood as fields that are valuable to that specific business scenario. Because the number of fields involved between tables is enormous, some fields are undefined at the data level, such as ID and time, which are not valuable to the business scenario and can be disregarded. Therefore, this step primarily aims to reduce the number of fields involved between tables after establishing the inter-table relationships. This minimizes the range of field relationships mapped to the inter-table relationships, preventing an excessively large data volume when generating the target verification rules and thus reducing the efficiency of rule generation.

[0062] Specifically, based on the valid fields corresponding to the target business scenario, all fields in the associated data table can be filtered to remove fields that are not valid fields and to select the valid fields, thus obtaining the valid fields in the associated data table.

[0063] For example, data cleaning can be used to identify which fields are considered worthless in a given business scenario. This can be achieved through data cleaning algorithms. For instance, if some fields in a given business scenario lack a clear, ordered relationship with other fields (i.e., their relationship is weak), these fields can be considered less meaningful and can be removed. In an insurance business scenario, for example, valid fields could include those related to funds.

[0064] S5: Determine the relationships between valid fields in the related data tables based on the relationships between tables and the valid fields in the related data tables.

[0065] It is understandable that inter-table relationships can be specifically reflected in the relationships between fields in the data tables. In this step, we can filter out the valid field relationships from all the relationships between the related data tables.

[0066] S7: Generate target verification rules for the target business scenario based on the relationships between valid fields in the associated data table.

[0067] Specifically, rule mining can be performed based on the relationships between valid fields to obtain the target verification rules under the target business scenario.

[0068] In this embodiment, by filtering all fields of the associated data table, that is, by reducing the fields in the associated data table through field pruning, the effective relationship between the fields in the associated data table is obtained, so as to avoid the data volume being too large when generating the target verification rules, which would affect the generation efficiency of the target verification rules.

[0069] In an exemplary embodiment, the implementation of S7 above may include the following S72 to S78:

[0070] S72: Based on the table name mapping relationship and the name of the associated data table, determine the target data table corresponding to the associated data table in the data warehouse.

[0071] In this context, table names can be used to distinguish different data tables. The data warehouse can be ODPS (Open Data Processing Service), the related data tables can be tables in a database (DB), and the target data table can be a table in ODPS. Generally, ODPS is used for storing dynamic data, essentially extracting the desired fields from the DB data using an aggregation method. Considering that the names of tables in the DB and ODPS may differ—for example, a table in the DB might correspond to multiple tables in ODPS, which could be categorized as daily, hourly, or full tables—the target data table in ODPS can be determined based on a pre-established table name mapping relationship and the names of the related data tables in the DB.

[0072] S74: Perform data cleaning on the target data table to obtain a wide table corresponding to the target business scenario.

[0073] Among them, a wide table is a database table with a large number of fields. It usually refers to a data table that links together business-related metrics, attributes, and maintenance. By putting related fields from multiple tables into the same table, it reduces table joins and can improve the efficiency of data calculation.

[0074] In this embodiment, based on the aforementioned table relationships, data cleaning can be performed on the target data table to obtain a wide table corresponding to the target business scenario. For example, the aforementioned table relationships can be assembled into SQL (Structured Query Language) statements. Data can be queried from the target data table in ODPS based on these SQL statements. The data retrieved based on the SQL statements is used as the cleaned data obtained from data cleaning of the target data table. Then, a wide table corresponding to the target business scenario can be created based on this cleaned data. The data in this wide table may include the data retrieved from ODPS based on the aforementioned SQL statements.

[0075] S76: Based on the data values ​​in the wide table, filter the relationships between valid fields to obtain the target relationships; where the correlation between the valid fields corresponding to the target relationships is greater than the preset correlation.

[0076] Specifically, based on the data values ​​in the wide table, the relevance of each valid field relationship can be calculated. This relevance effectively characterizes the strength of the association between the valid fields involved in the relationship. This step aims to filter out strong relationships and eliminate weak ones. The preset relevance can be set according to actual needs, indicating that if the relevance between the valid fields corresponding to the target relationship is greater than the preset relevance, then the target relationship is considered a strong relationship.

[0077] In an exemplary embodiment, the implementation of S76 above may include: deduplicating data values ​​in the wide table based on valid fields to obtain a deduplicated wide table; and filtering the relationships between valid fields based on the data values ​​in the deduplicated wide table to obtain target relationships. Here, valid fields can be considered as keys, and data values ​​in the wide table as values, thus creating a key-value mapping. If a key corresponds to multiple identical values, then value deduplication can be performed to obtain a deduplicated wide table. Performing data deduplication helps reduce interference when measuring the correlation between valid fields.

[0078] For example, the implementation of S76 above may include the following S760 to S766:

[0079] S760: Determine the mutual information between valid fields based on the data values ​​in the wide table.

[0080] Mutual information is a concept used to measure the correlation between two random variables. It can be used to measure the predictive power of information in one random variable X for another random variable Y. Mutual information is defined as follows: I(X;Y) = H(X) - H(X|Y). Here, H(X) is the information entropy of random variable X, and H(X|Y) is the conditional entropy of X given Y. Mutual information represents the amount of information X provides about Y given Y.

[0081] In this embodiment, mutual information can characterize the correlation between two valid fields, namely the correlation between the two random variables X and Y. The correlation between two valid fields can be reflected by the data values ​​corresponding to these two valid fields. Therefore, when it is necessary to determine the mutual information between two valid fields, the data values ​​corresponding to these two valid fields can be queried from the data values ​​in the wide table, and then the mutual information between the valid fields can be determined based on the data values ​​corresponding to these two valid fields retrieved from the wide table.

[0082] For example, the wide table in S760 above can be the deduplicated wide table in the example above. Determining the mutual information between valid fields based on the data values ​​in the deduplicated wide table helps to avoid inaccurate mutual information determination due to duplicate data values.

[0083] S762: Filter the association relationships between valid fields based on mutual information to obtain candidate association relationships; wherein, the mutual information between the valid fields corresponding to the candidate association relationships is greater than a first preset threshold.

[0084] It is understandable that mutual information measures the relevance between valid fields corresponding to a relationship; in essence, it is a value. Therefore, the mutual information between valid fields corresponding to each relationship can be compared with a first preset threshold, and relationships with mutual information greater than the first preset threshold can be selected as candidate relationships. The first preset threshold can be set according to actual needs, and this embodiment does not limit it.

[0085] Considering that while mutual information can identify candidate relationships, the scope of these relationships is still relatively coarse, potentially filtering out fields with weak connections, this embodiment adds a step for further filtering based on conditional mutual information. This more refined, targeted filtering based on mutual information can obtain the desired target relationships as accurately as possible.

[0086] S764: Based on the data values ​​in the wide table, determine the conditional mutual information between the valid fields corresponding to the candidate associations.

[0087] Conditional mutual information is an extension of mutual information; it measures the correlation between X and Y given a third random variable Z. The definition of conditional mutual information is as follows: I(X;Y|Z) = H(X|Z) - H(X|Y,Z); where H(X|Z) is the conditional entropy of X given Z, and H(X|Y,Z) is the conditional entropy of X given Y and Z. Conditional mutual information represents the correlation between X and Y given Z.

[0088] In this embodiment, conditional mutual information can characterize the correlation between the three valid fields, namely, the correlation between the three random variables X, Y, and Z. The larger the conditional mutual information, the stronger the dependency between the valid fields. The correlation between the three valid fields can be reflected by the data values ​​corresponding to these three valid fields. Therefore, when it is necessary to determine the mutual information between the three valid fields, the data values ​​corresponding to these three valid fields can be queried from the data values ​​in the wide table, and then the conditional mutual information between the valid fields can be determined based on the data values ​​corresponding to these three valid fields retrieved from the wide table.

[0089] For example, the wide table in S764 above can be the deduplicated wide table in the example above. Determining the conditional mutual information based on the data values ​​in the deduplicated wide table helps to avoid inaccurate conditional mutual information due to duplicate data values.

[0090] S766: Filter candidate associations based on conditional mutual information to obtain target associations; wherein, the conditional mutual information between the valid fields corresponding to the target association is greater than a second preset threshold.

[0091] It is understandable that conditional mutual information measures the correlation between three valid fields corresponding to a relationship; in essence, it is a single value. Therefore, the conditional mutual information between the valid fields corresponding to each relationship can be compared with a second preset threshold, and relationships with conditional mutual information greater than the second preset threshold can be selected as candidate relationships. The second preset threshold can be set according to actual needs, and this embodiment does not limit it.

[0092] For example, the target association can be a refined association under the target business scenario. This refined association can include: scenario-consistent association and business semantic causal association. Scenario-consistent association can include, for example, equality, inclusion, and enumeration relationships. The business semantic causal association can be obtained by: firstly, filtering invalid associations based on mutual information, i.e., filtering out associations with weak correlation, to obtain candidate associations; then, further pruning the candidate associations based on conditional mutual information to obtain the target association.

[0093] S78: Generate target verification rules for the target business scenario based on the target association relationship.

[0094] Specifically, rule mining can be performed on target relationships to obtain target verification rules under the target business scenario. For example, target relationships can be input into the ZhiRen rule platform to generate target verification rules.

[0095] For example, the generated target verification rule can be represented as a quadruple, specifically including: table, field, preconditions, and verification content. The table in the quadruple can be understood as: which data tables are involved in the target verification rule, specifically represented by the table names. The field in the quadruple can be understood as: which fields are involved in the target verification rule. The preconditions can be understood as: the prerequisites for the target verification rule to be true, ensuring that the verification content in the quadruple is true when the preconditions are true. For example, attribute fields and status fields from related data tables can be used as preconditions, which can identify the target business scenario. The verification content in the quadruple can be understood as a verification formula, which can be defined as an equation, including fields and operators. The verification formula refers to the equality relationship between fields in the data table. Verification formulas are divided into two categories: computational formulas (i.e., equations where the variables are numeric fields or constants and contain arithmetic operators) and consistency formulas (equations where the variables are non-numeric fields and only compare whether two values ​​are equal).

[0096] The mining of consistency formulas can be understood as follows: First, compare whether the field types are the same pairwise. If the field types are the same, then compare whether the corresponding field values ​​are the same. The proportion of records with the same values ​​to the total number of records is the proportion of the formula to be true. If the field types are different, the formula is considered to be semantically incorrect and is directly filtered out.

[0097] In this embodiment, rule mining can be performed on the target association to generate the aforementioned four-tuples. Each four-tuple can uniquely define a verification rule. The target verification rule can be obtained by assembling the tables, fields, preconditions, and verification content involved in the aforementioned four-tuples.

[0098] In an exemplary embodiment, after S74 cleans the target data table to obtain a wide table corresponding to the target business scenario, the method further includes: generating general verification rules corresponding to the target business scenario based on the wide table and a preset general rule template. Correspondingly, S78 generates target verification rules for the target business scenario based on the target association relationship, including: generating refined verification rules for the target business scenario based on the target association relationship and a refined rule template; and merging the general verification rules and the refined verification rules to generate target verification rules for the target business scenario.

[0099] For example, a wide table can be injected into the ZhiRen platform. The ZhiRen platform can have built-in general rule templates, allowing it to output general verification rules based on the data values ​​in the wide table and the general rule templates. A general rule template can be understood as a general rule formula template, involving fewer variables, such as two. General verification rules typically only find unique relationships between strongly dependent fields, such as a = b, because this is achieved by balancing a + (-1 * b) = 0. That is, if a field can be balanced, it's considered a relationship. However, complex business scenarios often don't involve such a balance; typically, a = T, b = U, c = R, which cannot be balanced. Therefore, further refined verification rules are generated.

[0100] For example, target relationships can be matched against refined rule templates. Based on the successfully matched target relationships and refined rule templates, refined verification rules are obtained. A refined rule template can be understood as a refined rule formula template, involving multiple variables, such as at least three. The matching method can be as follows: using the valid fields involved in the target relationship as keys, determining all values ​​corresponding to the keys, and then substituting the keys and values ​​into the refined rule formula template to determine if the key and value still make the refined rule formula valid. If the refined rule formula is valid, the match is considered successful; if the refined rule formula is invalid, the match is considered unsuccessful.

[0101] After obtaining the general verification rules and the refined verification rules, the general verification rules and the refined verification rules can be merged to obtain the target verification rules for the target business scenario, so that the target verification rules for the target business scenario can include the general verification rules and the refined verification rules for the target business scenario.

[0102] Step 108: Compare the original verification rules with the target verification rules to obtain the difference data between them. This difference data can be the data outside the intersection of the original and target verification rules. For example, the difference data may include first difference data and / or second difference data, where the first difference data exists in the target verification rule but not in the original verification rule, and the second difference data exists in the original verification rule but not in the target verification rule.

[0103] In an exemplary embodiment, the aforementioned target verification rules include rule data across multiple dimensions, which can be custom dimensions defined during rule mining. For example, these multiple dimensions may include the four dimensions mentioned above: table, field, preconditions, and verification content. The implementation of step 108 may include the following steps S20-S22:

[0104] S20: Decompose the original verification rules according to multiple dimensions to obtain rule data of the original verification rules in multiple dimensions.

[0105] Specifically, the original verification rules can be analyzed based on the aforementioned multiple dimensions to extract rule data from them across various dimensions. The original verification rules, which are manually written rules, are typically SQL statements. These SQL statements can be parsed into an AST (Abstract Syntax Tree) format, thus breaking them down into rule data across multiple dimensions. For example, the SQL statement can be decomposed into an AST to obtain rule data for the original verification rules in four dimensions: table, field, preconditions, and verification content.

[0106] S22: Compare the differences between the original verification rules and the target verification rules in the same dimension to obtain the difference data corresponding to the rule data in multiple dimensions.

[0107] For example, taking the aforementioned multiple dimensions including tables, fields, preconditions, and verification content as an example, this step allows for the comparison of differences across multiple dimensions, thereby obtaining table difference data, field difference data, precondition difference data, and verification content difference data.

[0108] Specifically, the differences between the original and target verification rules at the table dimension can be compared to obtain the corresponding difference data, i.e., table difference data. Similarly, the differences between the original and target verification rules at the field dimension can be compared to obtain the corresponding difference data, i.e., field difference data. The differences between the original and target verification rules at the precondition dimension can also be compared to obtain the corresponding difference data, i.e., precondition difference data. Finally, the differences between the original and target verification rules at the verification content dimension can be compared to obtain the corresponding difference data, i.e., verification content difference data.

[0109] In this embodiment, by comparing the differences between the original verification rules and the target verification rules in multiple dimensions, the difference data between the original verification rules and the target verification rules in multiple dimensions can be obtained, so as to accurately measure the differences between the original verification rules and the target verification rules in the target business scenario.

[0110] In an exemplary embodiment, after obtaining the target verification rule, the rule data of the target verification rule in multiple dimensions can be atomically decomposed to obtain atomic rule data of the target verification rule in multiple dimensions. After obtaining the original verification rule, the rule data of the original verification rule in multiple dimensions can be atomically decomposed to obtain atomic rule data of the original verification rule in multiple dimensions. Correspondingly, the implementation of S22 above may include: comparing the differences between the atomic rule data of the original verification rule and the target verification rule in the same dimension to obtain the difference data corresponding to the atomic rule data in multiple dimensions respectively.

[0111] The aforementioned atomization decomposition can be a minimum atomization decomposition. For example, after obtaining the rule data of the target verification rule and the original verification rule in multiple dimensions, it can be determined whether the rule data of the target verification rule in multiple dimensions satisfies the definition of minimum atomization, and whether the rule data of the original verification rule in multiple dimensions satisfies the definition of minimum atomization. Then, minimum atomization decomposition is performed on the rule data in dimensions that do not satisfy the definition of minimum atomization. Minimum atomization decomposition can be understood as breaking down the rule data into rule statements in units of OR. For example, rule data in the content dimension is usually a composite rule statement. Suppose this composite rule statement is a = T and b = U or c = R. "a = T and b = U or c = R" is actually a single rule statement. However, because this rule statement includes "or", it cannot be considered to meet the definition of minimum atomization. Therefore, we can perform minimum atomization decomposition on the rule statement "a = T and b = U" and the rule statement "c = R" to obtain the rule statement "a = T and b = U" and the rule statement "c = R". In other words, minimum atomization decomposition breaks down the original single rule statement into two rule statements.

[0112] Suppose that the original verification rule uses "a=T and b=U or c=R" as a rule statement, and the target verification rule uses "a=T and b=U" as verification rule 1 and "c=R" as another verification rule 2. During difference comparison, it's easy to conclude that "a=T and b=U or c=R" is inconsistent with both verification rule 1 ("a=T and b=U") and verification rule 2 ("c=R"). However, the combined meaning of verification rules 1 and 2 is essentially the same as the meaning of "a=T and b=U or c=R" in the original verification rule. Therefore, in this embodiment, by atomizing the rules, atomic rule data in multiple dimensions can be obtained for both the original and target verification rules. Comparing differences based on this atomic rule data improves the accuracy of the difference comparison and avoids misjudgments.

[0113] In step 110: Based on the differences between the original verification rules and the target verification rules, the original verification rules can be supplemented to generate updated verification rules corresponding to the target business scenario. For example, verification rules that exist in the target verification rules but not in the original verification rules can be added to the original verification rules to obtain updated verification rules. These updated verification rules can also be understood as the union of the original verification rules and the target verification rules.

[0114] In an exemplary embodiment, the implementation of step 110 may include: if the difference data includes the first difference data that exists in the target verification rule but not in the original verification rule, then a rule supplement set is generated based on the first difference data, and the original verification rule is supplemented based on the rule supplement set to generate an updated verification rule corresponding to the target business scenario.

[0115] For example, after obtaining the supplementary rule set, it can be pruned to obtain a pruned supplementary rule set. Then, the supplementary rules in the pruned supplementary rule set are classified for validity to obtain a set of valid rules. This set of valid rules is then added to the original verification rules to obtain the updated verification rules. The updated verification rules include the set of valid rules and the original verification rules.

[0116] In an exemplary embodiment, if the difference data includes second difference data, then the missing rule template is determined based on the second difference data, and the missing rule template is added to the refined rule template; wherein, the second difference data exists in the original verification rule but does not exist in the target verification rule.

[0117] It is understandable that the number of refined rule templates involved in rule mining based on scene traffic is difficult to exhaust. Therefore, in this embodiment, the missing rule templates can be determined based on the second difference data, and the missing rule templates can be added to the refined rule templates to continuously supplement the refined rule templates. As dynamic scene traffic is distributed, rule mining can continue to be performed based on the supplemented refined rule templates to improve the accuracy of rule mining.

[0118] For example, the second difference data may be any one or more of the following: difference data on the table dimension, difference data on the field dimension, difference data on the precondition dimension, and difference data on the verification content dimension. Regardless of the dimension, the difference data can essentially be understood as a difference in field relationships. In this embodiment, expert experience can be used to explain why the target verification rule did not produce the aforementioned field relationships. If it is determined that the reason is the absence of a certain rule template, the missing rule template can be generated based on the missing field relationship, thereby supplementing the missing rule template into the refined rule template.

[0119] For example, the principle block diagram of the embodiments in this specification can be referred to. Figure 2 .pass Figure 2 As can be seen, the core steps of the embodiments in this specification may include: establishing table relationships, extracting preconditions, field pruning, rule mining, and difference comparison. The following will combine... Figure 2 Each will be introduced separately:

[0120] 1. Establish table relationships and extract prerequisites, including: bytecode tracing, key-value pair retrieval, table relationship matching, and table name mapping.

[0121] Bytecode tracing, as mentioned above, involves finding the relationships between tables by tracing the traffic data corresponding to the target business scenario, obtaining the key-value pairs involved in the relationships between tables, and extracting the preconditions between tables, i.e., the aforementioned prerequisites.

[0122] Table relationship matching refers to the inter-table relationships found above. Table name mapping can be understood as the table name mapping relationship mentioned in S72 above, so that data can be cleaned in ODPS based on the table name mapping relationship.

[0123] 2. Field trimming, including: table relationship establishment, custom filtering, feature exploration, and field cleaning.

[0124] Considering that the fields involved between tables are very large, and some fields are completely irregular at the data level, such as id and time, which are valueless fields that do not need to be paid attention to or are basically unrelated to the business scenario, the fields can be reduced after the table relationship is established, i.e., field pruning.

[0125] Custom filtering can be understood as defining fields that are valuable to the business scenario, i.e., effective fields. Feature exploration can be understood as exploring the fields involved between tables based on the effective fields. Field cleaning can be understood as identifying the effective and invalid fields among the fields involved between tables after feature exploration, and then filtering out invalid fields that have no value, such as id and time, while retaining the effective fields among the fields involved between tables.

[0126] 3. Rule mining, including: data aggregation / deduplication, information entropy relationship matching, mutual information to define the relationship range, conditional mutual information to define the precise relationship sequence, rule aggregation matching, scenario rule generation, ODPS data cleaning, data backflow to new tables, addition of wide tables, templated rule cleaning, and fusion of refined rules and general rules.

[0127] Data aggregation / deduplication can be understood as follows: based on the valid fields, the data values ​​in the wide table are deduplicated to obtain the deduplicated wide table.

[0128] Information entropy relationship matching: Information entropy is the mathematical expectation of information content. Based on information entropy, the association relationship between effective fields can be obtained.

[0129] The mutual information-based definition of the relationship range can be understood as follows: based on the mutual information, the relationships between valid fields are filtered to obtain candidate relationships.

[0130] For example, it can be done through Figure 2 The following formula in the table calculates the mutual information used to measure the magnitude of the association between two valid fields:

[0131]

[0132] Here, X and Y are two random variables, and their joint probability density function is P(x,y), with marginal probability density functions P(x) and P(y), respectively. Mutual information I(X;Y) is the relative entropy between the joint distributions P(x,y) and P(x)P(y). In this embodiment, X and Y can represent two valid fields, and I(X;Y) can represent the mutual information between these two valid fields.

[0133] The exact relation sequence of conditional mutual information can be understood as: the above-mentioned screening of candidate associations based on conditional mutual information to obtain the target association.

[0134] For example, it can be done through Figure 2 The following formula calculates the conditional mutual information used to measure the degree of association between three valid fields:

[0135]

[0136] The aforementioned conditional mutual information can be understood as: if there exists a known random variable, then the conditional mutual information of random variables X and Y when Z is known. In this embodiment, X, Y, and Z can be three valid fields.

[0137] Rule template matching can be understood as matching the target association relationship with the refined rule template as described above.

[0138] The generation of scenario rules can be understood as follows: after matching rule templates, refined verification rules are obtained based on the successfully matched target associations and refined rule templates.

[0139] ODPS data cleaning can be understood as: performing data cleaning on the target data table in ODPS as described above to obtain cleaned data.

[0140] The new data return table can be understood as a table used to hold the cleaned data obtained after the ODPS data cleaning process. After obtaining the new data return table, relevant fields can be renamed according to actual needs.

[0141] The newly added wide table can be understood as the wide table corresponding to the target business scenario obtained above. This newly added wide table can be the same table as the new table for data return.

[0142] Templated rule cleaning can be understood as: inputting a wide table into the Homo sapiens platform to obtain universal verification rules.

[0143] The fusion of refined and general rules can be understood as combining the above refined and general verification rules into a target verification rule, which can be represented as a verification quadruple.

[0144] 4. Difference comparison, including: extraction of original verification rules, AST tree generation, minimum atomization decomposition, difference comparison, expert experience, and supplementary verification.

[0145] Specifically, after extracting the original verification rules, the original verification rules are parsed to generate an AST tree. Based on the AST tree, the original verification rules are decomposed into verification quadruples. Then, the verification quadruples are further decomposed into minimal atomization, and the differences between the original verification rules and the target verification rules are compared based on the verification quadruples after minimal atomization.

[0146] The results of the difference comparison include: if there is a first difference data that is in the target verification rule but not in the original verification rule, then a rule supplement set is generated based on the first difference data, and the original verification rule is supplemented based on the rule supplement set.

[0147] The results of the discrepancy comparison include: if there are second discrepancies that exist in the original verification rules but not in the target verification rules, then the reasons for the existence of the second discrepancies are analyzed based on expert experience, and a missing rule template is generated based on the second discrepancy. This missing rule template is then fed back into the ZhiRen platform to achieve [the desired outcome]. Figure 2 The feedback optimization process will continuously expand the rule templates in the Zhiren platform to better mine rules based on scene traffic.

[0148] In the embodiments of this specification, data correlation is dynamically analyzed from a traffic perspective to establish target verification rules for corresponding business scenarios, thereby supplementing the existing original verification rules and upgrading risk control. This is beneficial for adapting to refined business scenarios, that is, online problems can be discovered in a timely manner even in refined business scenarios.

[0149] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0150] It should be noted that the above figures are merely illustrative of the processes included in the methods according to exemplary embodiments of this specification, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may, for example, be executed synchronously or asynchronously in multiple modules.

[0151] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0152] This specification also provides an apparatus for generating verification rules, such as... Figure 3 As shown, it includes:

[0153] The original verification rule extraction module 301 is used to extract the original verification rules corresponding to the target business scenario from the rule base. The traffic data acquisition module 302 is used to acquire the traffic data corresponding to the target business scenario. The target verification rule generation module 303 is used to generate target verification rules for the target business scenario based on the traffic data. The rule comparison module 304 is used to compare the original verification rules with the target verification rules to obtain difference data. The rule supplementation module 305 is used to supplement the original verification rules based on the difference data to generate updated verification rules corresponding to the target business scenario.

[0154] For example, the target verification rule generation module 303 includes: an inter-table relationship determination unit, used to determine the inter-table relationship between related data tables corresponding to the target business scenario based on the traffic data; a valid field filtering unit, used to filter the fields in the related data tables based on the valid fields corresponding to the target business scenario to obtain the valid fields in the related data tables; a valid field relationship determination unit, used to determine the relationship between the valid fields in the related data tables based on the inter-table relationship and the valid fields in the related data tables; and a target verification rule generation unit, used to generate target verification rules under the target business scenario based on the relationship between the valid fields in the related data tables.

[0155] For example, the target verification rule generation unit includes: a target data table determination unit, used to determine the target data table corresponding to the associated data table in the data warehouse according to the table name mapping relationship and the name of the associated data table; a data cleaning unit, used to clean the target data table to obtain a wide table corresponding to the target business scenario; a relationship filtering unit, used to filter the relationship between the effective fields according to the data values ​​in the wide table to obtain the target relationship; wherein the correlation between the effective fields corresponding to the target relationship is greater than a preset correlation; and a target verification rule generation subunit, used to generate the target verification rule under the target business scenario according to the target relationship.

[0156] For example, the association filtering unit includes: a mutual information determination unit, used to determine the mutual information between the valid fields based on the data values ​​in the wide table; a candidate association determination unit, used to filter the associations between the valid fields based on the mutual information to obtain candidate associations; wherein the mutual information between the valid fields corresponding to the candidate associations is greater than a first preset threshold; a conditional mutual information determination unit, used to determine the conditional mutual information between the valid fields corresponding to the candidate associations based on the data values ​​in the wide table; and a target association filtering unit, used to filter the candidate associations based on the conditional mutual information to obtain target associations; wherein the conditional mutual information between the valid fields corresponding to the target associations is greater than a second preset threshold.

[0157] For example, the association filtering unit is specifically used to: deduplicate the data values ​​in the wide table according to the effective fields to obtain a deduplicated wide table; and filter the association relationships between the effective fields according to the data values ​​in the deduplicated wide table to obtain the target association relationship.

[0158] For example, the above-mentioned generation device further includes: a general verification rule generation unit, used to generate general verification rules corresponding to the target business scenario based on the wide table and a preset general rule template; the association relationship filtering unit is specifically used to: generate refined verification rules under the target business scenario based on the target association relationship and the refined rule template; and merge the general verification rules and the refined verification rules to generate target verification rules under the target business scenario.

[0159] For example, the target verification rule includes rule data in multiple dimensions, and the rule comparison module 304 includes: a dimension decomposition unit, used to decompose the original verification rule according to the multiple dimensions to obtain the rule data of the original verification rule in the multiple dimensions; and a difference comparison unit, used to compare the difference of the rule data in the same dimension between the original verification rule and the target verification rule to obtain the difference data corresponding to the rule data in the multiple dimensions respectively.

[0160] For example, the above-mentioned generation device further includes: an atomization decomposition unit, used to atomize the rule data of the target verification rule in the multiple dimensions to obtain atomized rule data of the target verification rule in the multiple dimensions; to atomize the rule data of the original verification rule in the multiple dimensions to obtain atomized rule data of the original verification rule in the multiple dimensions; and the above-mentioned difference comparison unit, specifically used to compare the differences between the original verification rule and the target verification rule in the same dimension of atomized rule data to obtain difference data corresponding to the atomized rule data in the multiple dimensions respectively.

[0161] For example, the rule supplementation module is specifically used to: if the difference data includes first difference data, generate a rule supplementation set based on the first difference data, and supplement the original verification rule based on the rule supplementation set to generate an updated verification rule corresponding to the target business scenario; wherein, the first difference data exists in the target verification rule but does not exist in the original verification rule.

[0162] For example, the above-mentioned generation device further includes: a rule template supplementation module, configured to: if the difference data includes second difference data, determine the missing rule template based on the second difference data, and supplement the missing rule template into the refined rule template; wherein the second difference data exists in the original verification rule but does not exist in the target verification rule.

[0163] For example, the multiple dimensions include: tables, fields, preconditions, and verification content.

[0164] It should be noted that the verification rule generation apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the verification rule generation apparatus and the verification rule generation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0165] This specification also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0166] Figure 4 A structural diagram of the electronic device in an embodiment of this specification is shown. Please refer to... Figure 4 As shown, the electronic device 400 includes a processor 401 and a memory 402.

[0167] In this embodiment, processor 401 is the control center of the computer system and can be a processor of a physical machine or a processor of a virtual machine. Processor 701 may include one or more processing cores, such as a 4-core processor or an 8-core processor. Processor 401 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). Processor 401 may also include a main processor and a coprocessor; the main processor is used to process data in the wake-up state, and the coprocessor is a low-power processor used to process data in the standby state.

[0168] Memory 402 may include one or more computer-readable storage media, which may be non-transitory. Memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments of this specification, the non-transitory computer-readable storage media in memory 402 are used to store at least one instruction for execution by processor 401 to implement the methods in the embodiments of this specification.

[0169] In some embodiments, the electronic device 400 further includes a peripheral device interface 403 and at least one peripheral device. The processor 401, memory 402, and peripheral device interface 403 are connected via a bus or signal line. Each peripheral device is connected to the peripheral device interface 403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of a display screen 404, a camera 405, and an audio circuit 406.

[0170] Peripheral device interface 403 can be used to connect at least one input / output (I / O) related peripheral device to processor 401 and memory 402. In some embodiments of this specification, processor 401, memory 402, and peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments of this specification, any one or two of processor 401, memory 402, and peripheral device interface 403 can be implemented on separate chips or circuit boards. This specification does not specifically limit the embodiments in this regard.

[0171] Display screen 404 is used to display a user interface (UI). The UI may include graphics, text, icons, videos, and any combination thereof. When display screen 404 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 401 for processing. In this case, display screen 404 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments of this specification, there may be one display screen 404, which serves as the front panel of the electronic device 400; in other embodiments, there may be at least two display screens 404, respectively disposed on different surfaces of the electronic device 400 or in a folded design; in still other embodiments, display screen 404 may be a flexible display screen, disposed on a curved or folded surface of the electronic device 400. Furthermore, display screen 404 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 404 may be made of materials such as Liquid Crystal Display (LCD) or Organic Light-Emitting Diode (OLED).

[0172] Camera 405 is used to capture images or videos. Optionally, camera 405 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the electronic device, and the rear-facing camera is located on the back of the electronic device. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, virtual reality (VR) shooting, or other fusion shooting functions. In some embodiments of this specification, camera 405 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cool light flash, which can be used for light compensation at different color temperatures.

[0173] The audio circuit 406 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 401 for processing. For stereo sound acquisition or noise reduction purposes, there may be multiple microphones, each located in a different part of the electronic device 400. The microphone may also be an array microphone or an omnidirectional microphone.

[0174] Power supply 407 is used to supply power to various components in electronic device 400. Power supply 407 can be alternating current, direct current, a disposable battery, or a rechargeable battery. When power supply 407 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0175] The block diagrams of the electronic device shown in the embodiments of this specification do not constitute a limitation on the electronic device 400. The electronic device 400 may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0176] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of these terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0177] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the above embodiments. If the constituent modules of the above workflow engine are implemented as software functional units and sold or used as independent products, they can be stored in the above-described computer-readable storage medium.

[0178] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The aforementioned available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0179] It should be noted that the above description describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims may be performed in a different order than that shown in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0180] The above description is merely a specific embodiment of this specification, but the scope of protection of this specification is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this specification should be included within the scope of protection of this specification. Therefore, equivalent variations made in accordance with the claims of this specification are still within the scope of this specification.

Claims

1. A method for generating verification rules, wherein, The method includes: Extract the original verification rules corresponding to the target business scenario from the rule base; Obtain the traffic data corresponding to the target business scenario; Based on the traffic data, generate the target verification rules for the target business scenario; The original verification rules are compared with the target verification rules to obtain the difference data; Based on the difference data, the original verification rules are supplemented to generate updated verification rules corresponding to the target business scenario; The step of generating target verification rules for the target business scenario based on the traffic data includes: Based on the traffic data, determine the inter-table relationships between the associated data tables corresponding to the target business scenario; Based on the valid fields corresponding to the target business scenario, the fields in the associated data table are filtered to obtain the valid fields in the associated data table; Based on the inter-table relationships and the valid fields in the associated data tables, determine the relationships between the valid fields in the associated data tables; Based on the relationships between valid fields in the associated data table, target verification rules for the target business scenario are generated.

2. The method according to claim 1, wherein, The step of generating target verification rules for the target business scenario based on the relationships between valid fields in the associated data table includes: Based on the table name mapping relationship and the name of the associated data table, determine the target data table corresponding to the associated data table in the data warehouse; The target data table is cleaned to obtain a wide table corresponding to the target business scenario; Based on the data values ​​in the wide table, the relationships between the valid fields are filtered to obtain target relationships; wherein the correlation between the valid fields corresponding to the target relationships is greater than a preset correlation. Based on the target association, generate target verification rules for the target business scenario.

3. The method according to claim 2, wherein, The step of filtering the relationships between the valid fields based on the data values ​​in the wide table to obtain the target relationships includes: Based on the data values ​​in the wide table, determine the mutual information between the valid fields; The association relationships between the valid fields are filtered based on the mutual information to obtain candidate association relationships; wherein the mutual information between the valid fields corresponding to the candidate association relationships is greater than a first preset threshold. Based on the data values ​​in the wide table, determine the conditional mutual information between the valid fields corresponding to the candidate associations; The candidate associations are filtered based on the conditional mutual information to obtain the target association; wherein the conditional mutual information between the valid fields corresponding to the target association is greater than a second preset threshold.

4. The method according to claim 2, wherein, The step of filtering the relationships between the valid fields based on the data values ​​in the wide table to obtain the target relationships includes: Based on the valid fields, the data values ​​in the wide table are deduplicated to obtain the deduplicated wide table. Based on the data values ​​in the deduplicated wide table, the relationships between the valid fields are filtered to obtain the target relationships.

5. The method according to claim 2, wherein, After cleaning the target data table to obtain the wide table corresponding to the target business scenario, the method further includes: Based on the wide table and the preset general rule template, generate general verification rules corresponding to the target business scenario; The step of generating target verification rules under the target business scenario based on the target association includes: Based on the target association and refined rule template, generate refined verification rules for the target business scenario; The general verification rules and the refined verification rules are merged to generate the target verification rules for the target business scenario.

6. The method according to any one of claims 1 to 5, wherein, The target verification rules include rule data in multiple dimensions; The step of comparing the original verification rules with the target verification rules to obtain difference data includes: The original verification rules are decomposed according to the multiple dimensions to obtain the rule data of the original verification rules in the multiple dimensions; The original verification rules and the target verification rules are compared in the same dimension to obtain the difference data corresponding to the rule data in the multiple dimensions.

7. The method according to claim 6, wherein, The method further includes: The target verification rule is atomically decomposed into rule data in the multiple dimensions to obtain atomic rule data of the target verification rule in the multiple dimensions; The original verification rules are atomically decomposed into rule data in the multiple dimensions to obtain atomic rule data of the original verification rules in the multiple dimensions. The step of comparing the differences in rule data along the same dimension between the original verification rule and the target verification rule yields the difference data corresponding to the rule data in various dimensions, including: The original verification rule and the target verification rule are compared in the same dimension to obtain the difference data corresponding to the atomic rule data in the multiple dimensions.

8. The method according to any one of claims 1 to 5, wherein, The step of supplementing the original verification rules based on the difference data to generate updated verification rules corresponding to the target business scenario includes: If the difference data includes first difference data, then a supplementary rule set is generated based on the first difference data, and the original verification rules are supplemented based on the supplementary rule set to generate the updated verification rules corresponding to the target business scenario; wherein, the first difference data exists in the target verification rules but does not exist in the original verification rules.

9. The method according to claim 5, wherein, The method further includes: if the difference data includes second difference data, then determining the missing rule template based on the second difference data, and supplementing the missing rule template into the refined rule template; wherein the second difference data exists in the original verification rule but does not exist in the target verification rule.

10. The method according to claim 6, wherein, The various dimensions include: tables, fields, preconditions, and verification content.

11. An apparatus for generating verification rules, wherein, The device includes: The original verification rule extraction module is used to extract the original verification rules corresponding to the target business scenario from the rule base; The traffic data acquisition module is used to acquire traffic data corresponding to the target business scenario; The target verification rule generation module is used to generate target verification rules for the target business scenario based on the traffic data. The rule comparison module is used to compare the original verification rules with the target verification rules to obtain difference data; The rule supplementation module is used to supplement the original verification rules based on the difference data, and generate updated verification rules corresponding to the target business scenario; The target verification rule generation module includes: The inter-table relationship determination unit is used to determine the inter-table relationship between the associated data tables corresponding to the target business scenario based on the traffic data. The effective field filtering unit is used to filter the fields in the associated data table according to the effective fields corresponding to the target business scenario, so as to obtain the effective fields in the associated data table; The effective field association determination unit is used to determine the association relationship between the effective fields in the associated data table based on the inter-table association relationship and the effective fields in the associated data table; The target verification rule generation unit is used to generate target verification rules for the target business scenario based on the association relationships between valid fields in the associated data table.

12. A computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform the method as claimed in any one of claims 1 to 10.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 10.

Images