Data analysis method and device, electronic equipment and storage medium

Abstract

The invention discloses a data analysis method and device, electronic equipment and a storage medium, and belongs to the technical field of data analysis, in the method, according to field description information of each field in a service table, a normal form corresponding to each field is determined in a normal form library, the field description information comprises a field category and a value expression of the field, and the value expression of the field is determined; when the normal form corresponding to any field comprises a conclusion normal form, the value expression of the field is used as a conclusion of the decision tree, the value expressions of the candidate fields are used as decision sources of the decision tree, the decision tree is constructed according to a decision tree construction rule, and the normal form corresponding to each candidate field comprises a condition normal form; and generating value association rules among the fields according to the decision tree, and storing the value association rules in a native rule base, thereby providing a scheme for automatically analyzing the association relationship among the data. The value diversity of the field can be simplified by means of the normal form, the decision-making difficulty of the decision-making tree is reduced, the method is suitable for mass data, and effective rules can be learned when resources are limited.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, and in particular to a data analysis method, device, electronic device and storage medium. Background Art

[0002] At present, data has become a valuable asset for an enterprise or organization. Taking the financial industry as an example, the quality of data not only affects the accuracy of data analysis and business decisions, but also affects the compliance and credibility of the enterprise.

[0003] In order to monitor the quality of data, it is necessary to know the data value characteristics such as the field value rules in advance. In related technologies, the field value rules can be known through pre-defined metadata information. However, the actual situation is that most businesses do not have a complete metadata information management method, and can only rely on manual business experience to define the relationship between tables and fields. In this way, relying on manual experience configuration is not only inefficient but also prone to errors under massive data.

[0004] It can be seen that how to analyze the correlation between data in related technologies is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The embodiments of the present application provide a data analysis method, device, electronic device and storage medium, which are used to provide a solution for analyzing the relationship between data.

[0006] In a first aspect, an embodiment of the present application provides a data analysis method, comprising:

[0007] According to the field description information of each field in the business table, the paradigm corresponding to each field is determined in the paradigm library, and the corresponding relationship between the field description information and the paradigm is stored in the paradigm library, wherein the field description information of a field includes the field category and the field value expression;

[0008] If the paradigm corresponding to any field includes a conclusion paradigm, the value expression of the field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision sources of the decision tree. According to the set decision tree construction rules, a decision tree is constructed, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and value association rules between fields are generated according to the decision tree;

[0009] Store each value association rule in the native rule base.

[0010] In some embodiments, generating value association rules between fields according to the decision tree includes:

[0011] Selecting a target decision condition that meets a preset condition from each decision condition of the decision tree, wherein each decision condition includes a value expression of at least one candidate field;

[0012] Each target decision condition is combined with the value expression of the field to obtain a value association rule between the fields.

[0013] In some embodiments, the paradigms in the paradigm library include first-type paradigms and / or second-type paradigms, the first-type paradigms are generated according to configuration rules, and the second-type paradigms are generated according to multiple structured query language SQL statements.

[0014] In some embodiments, when the paradigms in the paradigm library include the second type of paradigms, the second type of paradigms are generated according to the following steps:

[0015] Performing semantic analysis on each of the SQL statements;

[0016] Extracting a field value expression from the SQL statement according to the semantic analysis result;

[0017] The second type of paradigm is obtained by performing paradigm conversion according to the value expression of the extracted field and the field type of the field.

[0018] In some embodiments, the native rule base also stores value rules for a single field. After storing the value association rules in the native rule base, the following is further included:

[0019] Eliminate the rules that fail the grayscale verification in the native rule library, wherein the rules include value association rules and value rules;

[0020] The rules that have not been eliminated in the native rule base are merged to obtain an updated native rule base.

[0021] In some embodiments, it also includes:

[0022] Based on the monitoring quality requirements of each field, the rules in the native rule base are inverted to obtain a monitoring rule base;

[0023] Based on the monitoring rule base, quality monitoring is performed on the values ​​of each field.

[0024] In a second aspect, an embodiment of the present application provides a data analysis device, including:

[0025] A determination module, used to determine the paradigm corresponding to each field in the paradigm library according to the field description information of each field in the business table, wherein the paradigm library stores the correspondence between the field description information and the paradigm, wherein the field description information of a field includes the field category and the field value expression;

[0026] A generation module, for, if the paradigm corresponding to any field includes a conclusion paradigm, taking the value expression of the field as the conclusion of the decision tree and the value expressions of multiple candidate fields as the decision sources of the decision tree, and building a decision tree according to a set decision tree building rule, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and generating value association rules between fields according to the decision tree;

[0027] The storage module is used to store each value association rule in the native rule base.

[0028] In a third aspect, an embodiment of the present application provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein:

[0029] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by the at least one processor so that the at least one processor can perform any of the above-mentioned data analysis methods.

[0030] In a fourth aspect, an embodiment of the present application provides a storage medium. When a computer program in the storage medium is executed by a processor of an electronic device, the electronic device can execute any of the above-mentioned data analysis methods.

[0031] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which implements any of the above-mentioned data analysis methods when executed by a processor.

[0032] In the embodiment of the present application, according to the field description information of each field in the business table, the paradigm corresponding to each field is determined in the paradigm library, wherein the field description information of a field includes the field category and the value expression of the field, and the corresponding relationship between the field description information and the paradigm is stored in the paradigm library. If the paradigm corresponding to any field includes a conclusion paradigm, the value expression of this field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision source of the decision tree. According to the set decision tree construction rules, a decision tree is constructed, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and the value association rules between fields are generated according to the decision tree, and each value association rule is stored in the native rule library, providing a solution for automatically analyzing the association relationship between data. Moreover, with the help of the paradigm, the value diversity of the field itself can be simplified, the decision difficulty of the decision tree can be reduced, and thus the learning difficulty of the value association rules between fields can be reduced. It is not only applicable to massive data, but also effective value association rules can be learned even in the case of limited resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0034] Figure 1 A flowchart of a data analysis method provided in an embodiment of the present application;

[0035] Figure 2 A flowchart of another data analysis method provided in an embodiment of the present application;

[0036] Figure 3 A schematic diagram of a data processing process provided by an embodiment of the present application;

[0037] Figure 4 A schematic diagram of a process for preprocessing training data provided in an embodiment of the present application;

[0038] Figure 5 A schematic diagram of a decision tree provided in an embodiment of the present application;

[0039] Figure 6 A schematic diagram of a rule merging optimization process provided in an embodiment of the present application;

[0040] Figure 7 A schematic diagram of the structure of a data analysis device provided in an embodiment of the present application;

[0041] Figure 8 A schematic diagram of the hardware structure of an electronic device for implementing a data analysis method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to provide a solution for analyzing the relationship between data, the embodiments of the present application provide a data analysis method, device, electronic device and storage medium.

[0043] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments and features in the embodiments of the present application may be combined with each other if there is no conflict.

[0044] In order to facilitate the understanding of this application, the technical terms involved in this application are:

[0045] Field type, such as enumeration field, date field, numeric field, etc. Among them, the value of the enumeration field is such as field equal to a value, field not equal to a value; the value of the date field is such as field is empty, field is not empty; the value of the numeric field is such as field greater than Z, field less than Z, field greater than or equal to Z, field less than or equal to Z, etc., Z is an arbitrary number.

[0046] A paradigm refers to a set of relationship patterns that conform to a certain level.

[0047] A decision tree, also called a decision tree model, is a structured language representation that uses a tree structure to describe the relationship learning process, and graphically expresses the relationship between decision conditions and conclusions. The leaf nodes at the bottom of the decision tree correspond to the conclusions, and the judgment conditions of the nodes from the root node to the bottom leaf node in the decision tree are the decision conditions for obtaining the corresponding conclusions.

[0048] In order to monitor the quality of data, it is necessary to know in advance the data value characteristics such as the field value rules, the relationship between fields, etc. In related technologies, the field value rules can be known based on the pre-defined metadata information, and some simple and general monitoring rules can be generated based on the field value rules. However, the maintenance and management costs of metadata are high, and the metadata information is limited, while the relationship between fields is more implicit in the data content. Therefore, relying on metadata information can only cover the relationship of a single field, and the coverage is limited, which is not completely applicable to data quality monitoring of massive data.

[0049] In addition, the actual situation is that most businesses do not have a complete metadata information management method, and can only rely on manual business experience to define the relationship between tables and fields. In this way, relying on manual experience configuration is not only inefficient but also prone to errors under massive data.

[0050] To this end, the present application embodiment provides a solution for automatically analyzing the relationship between data without relying on metadata. Figure 1 , Figure 1 A flowchart of a data analysis method provided in an embodiment of the present application includes the following steps.

[0051] In step 101, based on the field description information of each field in the business table, the paradigm corresponding to each field is determined in the paradigm library. The paradigm library stores the correspondence between the field description information and the paradigm. The field description information of a field includes the field category and the field value expression.

[0052] Among them, the field category of the field is such as enumeration type, date type, numeric type, etc. The field value expression can be determined according to the field value. For example, if the value of the enumeration type field is S1, S2, the field value expression may include: field == S1, field! = S1, field == S2, and field! = S2, where "==" means equal to, and "! = " means not equal to. If the value of the date type field is NULL (null value or non-existent value) or NOT NULL (neither null value nor non-existent value), the field value expression may include: field is NULL, field NOT NULL. If the value of the numeric field is greater than 0, the field value expression may include: field value > 0, field value ≤ 0.

[0053] In practical applications, the paradigm library can store multiple paradigms corresponding to each field type, and the value expressions of the fields corresponding to these multiple paradigms are in different forms. See Table 1, which is an example of a paradigm in a paradigm library provided in an embodiment of the present application.

[0054] Table 1

[0055]

[0056] In Table 1, each paradigm has a paradigm code (i.e., paradigm identifier). Each paradigm also has categories such as conditional paradigm and conclusion paradigm, and a paradigm can be only a conditional paradigm, only a conclusion paradigm, or both a conditional paradigm and a conclusion paradigm.

[0057] Then, for each field in the business table, according to the field type of the field, a paradigm (one or more) that matches the expression of the field can be determined from the multiple paradigms corresponding to the field type in the paradigm library, and used as the paradigm corresponding to the field.

[0058] The following is an introduction to the paradigm of the paradigm library.

[0059] In actual applications, the paradigms in the paradigm library may include the first type of paradigm and / or the second type of paradigm. The first type of paradigm can be generated based on field value rules, such as field A cannot be empty, field B must be greater than 0, etc. The second type of paradigm can be generated based on the target SQL statement, which is used to describe the relationship between fields. It is generally written by R&D personnel based on business summaries and has a strong correlation with business logic.

[0060] In specific implementation, semantic analysis can be performed on each target SQL statement, and based on the semantic analysis result, the field value expression is extracted from the target SQL statement. Then, a paradigm shift is performed based on the extracted field value expression and the field type of the field to obtain the second type of paradigm.

[0061] See Table 2, which is an example of an SQL statement extraction paradigm provided in an embodiment of the present application. The paradigm abstraction in Table 2 belongs to the second type of paradigm.

[0062] Table 2

[0063]

[0064]

[0065] In step 102, if the paradigm corresponding to any field includes a conclusion paradigm, the value expression of this field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision sources of the decision tree. According to the set decision tree construction rules, a decision tree is constructed, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and value association rules between fields are generated according to the decision tree.

[0066] In the specific implementation, the value expression of this field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision sources of the decision tree. According to the set decision tree, an attempt is made to build a decision tree. In this process, only the conclusion of the decision tree (corresponding to the leaf nodes of the decision tree, that is, the nodes at the bottom of the decision tree) is determined, and the judgment conditions corresponding to the non-leaf nodes in the decision tree can be updated. The source of the judgment conditions is the value expressions of multiple candidate fields. In addition, it should be noted that the decision tree construction rules are used to describe what kind of decision tree to build, such as the depth of the decision tree, the node splitting rules, etc.

[0067] Moreover, after generating a decision tree, a target decision condition (one or more) that meets the preset conditions can be selected from the various decision conditions of the decision tree, and then each target decision condition and the field value expression are merged to obtain a value association rule between fields. For example, taking each target decision condition as a premise and the field value expression as a conclusion, a value association rule between fields is obtained.

[0068] A decision condition refers to a set of all judgment conditions from the root node to a leaf node of the decision tree, which may include one or more candidate field value expressions. The preset condition may include that the loss value is a first preset value such as 0, and the sample size supporting the decision condition is greater than a second preset value such as 1000.

[0069] In step 103, each value association rule is stored in a native rule base.

[0070] In some embodiments, before storing each value association rule in the native rule base, each value association rule can be verified with the help of a verification set. The value association rules that pass the verification are stored in the native rule base, and the value association rules that fail the verification will be discarded.

[0071] The embodiment of the present application provides a solution for automatically analyzing the association relationship between data. With the help of the paradigm, the diversity of the values ​​of the field itself can be simplified, the decision difficulty of the decision tree can be reduced, and the learning difficulty of the value association rules between the fields can be reduced. It is not only applicable to massive data, but also effective value association rules can be learned when learning resources are limited.

[0072] See also Figure 2 , Figure 2 A flowchart of another data analysis method provided in an embodiment of the present application includes the following steps.

[0073] In step 201, based on the field description information of each field in the business table, the paradigm corresponding to each field is determined in the paradigm library. The paradigm library stores the correspondence between the field description information and the paradigm. The field description information of a field includes the field category and the field value expression.

[0074] The implementation of this step can refer to step 101 and will not be described in detail here.

[0075] In step 202, if the paradigm corresponding to any field includes a conclusion paradigm, the value expression of this field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision sources of the decision tree. According to the set decision tree construction rules, a decision tree is constructed, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and value association rules between fields are generated according to the decision tree.

[0076] The implementation of this step can refer to step 102, which will not be described in detail here.

[0077] In step 203, each value association rule is stored in a native rule library, and the native rule library also stores the value rules of a single field.

[0078] Among them, the determination of the value rule of a single field is relatively simple and can be obtained through simple data analysis instead of decision tree learning, so as to reduce the solution space of decision tree learning.

[0079] In step 204, the rules in the native rule library that fail the grayscale verification are removed, and the rules include value association rules between fields and value rules for a single field.

[0080] In step 205, the rules that have not been eliminated in the native rule base are merged to obtain an updated native rule base.

[0081] The merge includes merging fields with the same conditions and conclusions, merging rules of different fields in the same table, and merging multi-field rules, i.e. rules with one or more condition fields, etc. The updated native rule base can also be called the merged rule base.

[0082] Since the number of rules is reduced after the rules are merged, the number of queries on the data in the table can be reduced during monitoring. Therefore, the software and hardware resources consumed by publishing subsequent rules to the monitoring platform for monitoring can be reduced.

[0083] In step 206, the rules in the native rule base are inverted to obtain a monitoring rule base.

[0084] Since the rules in the native rule base are positive logic rules, they need to be converted into judgment logic rules for abnormal data required for monitoring and inspection. The core logic of the conversion is that the condition field remains unchanged and the conclusion field is negated. For example, the conclusion field date IS NOT NULL is negated to date IS NULL. The original rule is Head By Body1 and Body2 and…, and the converted rule is NOT Head By Body1 and Body2 and…. Among them, Body is the condition field and Head is the conclusion field.

[0085] In step 207, quality monitoring is performed based on the monitoring rule base and the monitoring quality requirements of each field.

[0086] Among them, the monitoring quality requirements of the field are such as the monitoring level of the field. Generally, the higher the monitoring level, the lower the tolerance for alarms when monitoring the value of the field. Quality monitoring is performed, such as quality monitoring of the values ​​of newly generated fields in subsequent tables.

[0087] It should be noted that, in the above process, steps 204-205 are optional, that is, they may be executed or not.

[0088] The following introduces the scheme of the embodiment of the present application by taking data in the financial field as an example.

[0089] See also Figure 3 , Figure 3 A schematic diagram of a data processing process provided for an embodiment of the present application includes data access, preprocessing training data, paradigm learning, rule generation learning, rule merging optimization, and rule deployment, covering the entire process of data quality monitoring. Each process is introduced below.

[0090] 1. Data access.

[0091] In specific implementation, the accessed data sources may include relational database table data and distributed hive library table data. Both types of data sources are structured storage data sources, which can improve the accuracy of accessed data. Among them, hive is a data warehouse tool based on Hadoop, which is used for data extraction, transformation and loading. When accessing any data source, you only need to configure the library name and table name, without configuring field information, so the access is relatively simple.

[0092] Generally, the total sample size accessed is positively correlated with the machine resource memory size, the number of table fields, and the amount of table data. That is, the larger the machine resource memory size, the number of table fields, and the amount of table data, the larger the total sample size accessed. Taking different financial loan data domains in complex data scenarios as an example, the average maximum total sample size that can be carried by a single machine with 32G memory is about 10 million. The configuration for data access is shown in Table 3.

[0093] Table 3

[0094] Library Name Table name Dynamic total sample size ratio Recently processed batches hduser03db TM_ACCOUNT 0.9 20250201 hduser03db TM_LOAN 20240202

[0095] Among them, when the dynamic total sample ratio is not empty, the training sample size can be obtained according to the value of the maximum total sample size × the dynamic total sample size ratio; when the configuration is empty, the training sample size can be obtained according to the maximum total sample size.

[0096] 2. Preprocess training data.

[0097] The purpose of preprocessing training data is to obtain the data required for model (referring to decision tree model) training. The frequency of model training depends on the machine resources provided. If the machine resources provided exceed the number of daily configuration tables, daily polling can be performed according to the configuration table to update the model output results (referring to the value association rules between fields).

[0098] Assuming that the model training frequency is to update the model once a day, the synchronization rules in the configuration table can be updated every day. Figure 4 This paper introduces the process of preprocessing training data.

[0099] Step 1: Get the list of tables to be processed today based on the configuration table of the access data source. Assuming that a single instance (referring to a single virtual machine) can train m tables every day, the number of instances (referring to the number of virtual machines) is N, and the total number of configuration tables is T, the polling calculation for obtaining the number of tables to be processed today can be simply processed as follows:

[0100] The number of table lists to be processed on the same day = if(m * N < T, T, top(m * N)). That is, when m * N < T, the number of table lists to be processed on the same day = T; when m * N ≥ T, the number of table lists to be processed on the same day = top(m * N). Among them, top(m * N) means to take m * N tables in reverse order according to the most recent processing batch in Table 1 to ensure that all tables can be trained as soon as possible through polling.

[0101] Step 2: Determine whether there is a read permission for the tables in the table list. If there is, proceed to Step 3; otherwise, perform permission application processing until the read permission is obtained, and then proceed to Step 3. Generally, the tables configured in Table 1 already have read permissions. To prevent errors caused by permission changes, a judgment is made again here.

[0102] Step 3: Obtain the structured features of the tables in the table list, including field names, field types, field defined lengths, field default values, and field descriptions. If the table has more comprehensive metadata information, the metadata information can also be utilized; otherwise, the original table structured features are used.

[0103] Step 4: Classify the field types of the fields in the table according to the structured features of the table. First, perform a rough classification according to the field type to obtain three major categories: numeric type, date type, and character enumeration type. Refer to Table 4. Since the string type is relatively special, technicians can write it as a numeric type, a date type, or a character enumeration type. To better differentiate the string type, a small number of samples in the table can be selected for further classification. The further classification can include first using a general strategy for screening, and then performing differential screening according to the field type for those that meet the general strategy to obtain the final classification result. Assume that the screening strategies for the numeric type, date type, and character enumeration type are Strategy 1, Strategy 2, and Strategy 3, respectively.

[0104] Among them, the general strategy is that the field type is not within the original feature range or the field type is string and the field defined length is less than 100;

[0105] Strategy 1 is that the field type is (int, bigint, decimal) or the character type is string and the converted numeric type is within the bigint range;

[0106] Strategy 2 is that the field type is (date) and the character type is string and the character value matches the date format;

[0107] Strategy 3 is that the number of distinct character fields is less than 100 and the proportion is less than 0.01.

[0108] Table 4

[0109] Field Category Classification Original Features Screening strategy Numeric int, bigint, decimal, string Strategy 1 Date Type date、string Strategy 2 Character enumeration type char, varchar, string Strategy 3

[0110] Step 5: Determine whether the fields in the table match the field screening strategy. If so, add the category features of the field to the original field features. If not, skip the field, i.e. discard the field. The field screening rule is used to describe which fields to remove, which are generally fields that are not helpful for rule learning.

[0111] Step 6: Get the data samples in the table. The sample size is obtained according to the dynamic total sample ratio in Table 1. The data samples are then split into training samples (i.e., training set) and validation samples (i.e., validation set) according to the preset ratio. For example, the training samples account for 0.8 of the total samples, and the validation samples account for 0.2 of the total samples. The training samples and validation samples can be stored in the Hadoop Distributed File System (HDFS) respectively.

[0112] 3. Paradigm learning.

[0113] In order to avoid learning invalid and redundant rules under massive data, improve the effectiveness of rule learning, and reduce the dimension of the solution space of rule learning, in the embodiment of the present application, before rule learning, a rule paradigm library is generated. The generation process of the paradigm library is introduced below.

[0114] In the specific implementation, the paradigm analysis of the data quality monitoring rules defined by experts can be found that the data quality monitoring rules are essentially a combination of single-field paradigms, that is, the data quality monitoring rules are derived from one or more conditional paradigms to derive the conclusion paradigm of another field. To this end, the single-field paradigm can be extracted based on the existing data quality monitoring rules to form a paradigm library of rules.

[0115] In practical applications, data quality monitoring rules mainly include two categories: field value rules and SQL semantic rules (i.e. target SQL statements).

[0116] Generally, field value rules are usually relatively standardized, such as field A cannot be empty, field B must be greater than 0, etc. The field pattern extraction and learning of such rules are relatively simple. By performing statistical analysis on the semantics of the configuration, high-frequency rule patterns can be obtained, and the conditional pattern and conclusion pattern can be extracted and stored in the pattern library.

[0117] SQL semantic rules are mainly written and deployed by R&D personnel based on business conclusions. The rules are strongly related to business logic, and the paradigm for summarizing such rules is relatively complex. It is necessary to first perform an abstract syntax tree (AST) semantic analysis on the SQL statement, split out the library, table, and conditional field involved in the SQL statement, and then extract the field paradigm. If it is a single table, extract the conditional paradigm and conclusion paradigm. If it is multiple tables, the association condition paradigm between multiple tables can also be extracted. Finally, the three types of paradigms proposed are stored in the paradigm library. See Table 5, which is an example of an SQL statement extraction paradigm provided in an embodiment of the present application.

[0118] Table 5

[0119]

[0120]

[0121]

[0122] See Table 6, which is an example of a paradigm in a paradigm library provided in an embodiment of the present application.

[0123] Table 6

[0124]

[0125] In Table 6, <> means not equal to, for example, ENUM1<>'xx' means ENUM1 is not equal to 'xx', and ENUM1<>ENUM2 means ENUM1 is not equal to ENUM2.

[0126] 4. Rule generation learning.

[0127] Through the previous steps, we have obtained the data samples required for model learning and the paradigm library that reduces the difficulty of model learning. The next step is to train the value association rules between the model output fields.

[0128] In the embodiment of the present application, an algorithm combining an improved decision tree and a strongly constrained regression is adopted, and value association rules between fields are constructed and output through a decision tree. During the training process, the output value association rules are simultaneously verified in real time with a strongly constrained verification set. At the same time, multiple rounds of grayscale verification feedback of newly added samples are performed to improve the accuracy of the value association rules output by the model.

[0129] The following is an introduction to the process of learning association rules between field values.

[0130] First, simple rules are analyzed and verified in the validation set based on the field characteristics of the training samples, and simple rules are learned first and stored in the native rule library to reduce the solution space of subsequent decision tree learning. Simple rules include: for all fields: whether they are all empty / not empty; for numeric fields: whether they are all >0, <0, =0; for enumeration fields: whether they are all equal to a specific value.

[0131] Then, according to the paradigm library, a field-level decision tree is constructed table by table. Specifically, for each table, the paradigm (one or more) corresponding to the field can be determined in the paradigm library according to the field description information of each field in the table, such as the field category and the field value expression (one or more), and the fields of the corresponding paradigm including the conditional paradigm are used as candidate fields. Then, for each field of the corresponding paradigm including the conclusion paradigm, the value expression of this field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision sources of the decision tree. According to the set decision tree construction rules, a decision tree is constructed, and then, according to the decision tree, the value association rules between this field and multiple candidate fields (i.e., the value association rules between fields) are generated. Among them, the decision tree construction rules include tree depth, split stop conditions, etc.

[0132] Finally, the verification set is used to verify each value association rule, and the value association rules that pass the verification can be stored in the native rule library.

[0133] The above process is explained below with reference to a specific example.

[0134] Refer to Table 7, which is a partial data example of the TM_LOAN table provided in an embodiment of the present application, wherein the field REGISTER_ID represents the registration ID, the field LOAN_BAL_FEE represents the loan fee balance, the field AGE_CD represents the account age, the field PRIN_PAID represents the principal has been repaid, the field INT_PAID represents the interest has been repaid, the field FEE_PAID represents the fee has been repaid, the field FEE_RATE represents the fee rate, and the field ACQ_CHANNEL represents the source channel.

[0135] Table 7

[0136]

[0137]

[0138] Step 1: According to the field description information of each field, determine the paradigm corresponding to each field in the paradigm library. The field description information of a field includes the field category and the field value expression.

[0139] For example, if the field type of a field is an enumeration type and has n discrete values, you can generate 2n value expressions for the field according to ==, <> (indicating inequality). Then, from the paradigms corresponding to the enumeration type fields in the paradigm library, find the paradigm that matches the value expression of the field and use it as the paradigm corresponding to the field.

[0140] Take the enumeration field ACQ_CHANNEL in Table 5 as an example. ACQ_CHANNEL has two enumeration values: C1 and C2. Four value expressions can be generated for the ACQ_CHANNEL field: ACQ_CHANNEL == 'C1', ACQ_CHANNEL == 'C2', ACQ_CHANNEL <> 'C1', and ACQ_CHANNEL <> 'C2'. According to Table 4, the paradigms corresponding to the ACQ_CHANNEL field in the paradigm library are paradigm E1 and paradigm E2.

[0141] For another example, if the field category of a field is a numeric field and there are t thresholds X1, X2, ..., Xt, the value of the field is divided into t+1 ranges: {less than or equal to X1, greater than X1 and less than or equal to X2, ..., greater than Xt}, each range defines a value expression, and the field has t+1 value expressions.

[0142] Take the numerical field FEE_RATE in Table 5 as an example. Assuming that the division is performed according to the threshold 0, the value expression of FEE_RATE is: FEE_RATE<=0, FEE_RATE>0. According to Table 4, the paradigm corresponding to the FEE_RATE field in the paradigm library is paradigm N1.

[0143] Step 2: Build a decision tree according to the paradigm corresponding to each field and the value expression of each field and the set decision tree construction rules.

[0144] Among them, the parameters involved in the decision tree construction rules include the depth of the decision tree, node type, etc. The depth of the decision tree can be adjusted as needed. Node types include conditional nodes, conclusion nodes, etc. Generally, when the paradigm corresponding to the field includes the conditional paradigm, the value expression of the field can be used as a conditional node, and when the paradigm corresponding to the field includes the conclusion paradigm, the value expression of the field can be used as a conclusion node.

[0145] Take the FEE_PAID field in Table 5 as the conclusion field to build a decision tree as an example. The FEE_PAID field is a numerical field. According to the threshold of 0, the value of the FEE_PAID field is expressed as: FEE_PAID==0, FEE_PAID!=0 (i.e., FEE_PAID is not equal to 0). Taking the conclusion of the decision tree as FEE_PAID==0 as an example, the value expressions of each field in the corresponding paradigm including the conditional paradigm are used as the judgment condition source of each leaf node in the decision tree, and try to generate a decision tree with a maximum depth of 3.

[0146] See also Figure 5 , Figure 5 This is a schematic diagram of the final decision tree. Figure 5 The leaf node of the decision tree (referring to the bottom-level node) represents the conclusion field FEE_PAID==0, and the non-leaf node such as INT_PAID==0 is the condition field. log_loss represents the loss function value of the decision tree, samples represents the number of samples that meet the conditions, value={a, b}, where a is a counterexample (i.e., the number of samples that do not meet the judgment conditions), and b is a positive example (i.e., the number of samples that meet the judgment conditions).

[0147] Step 3: According to the decision conditions of the decision tree, select the value association rules between fields.

[0148] For example, a path whose length leading to a leaf node is not greater than a set value is selected, the leaf node corresponds to the conclusion, the non-leaf node corresponds to the condition, and the value association rule can be selected in combination with the loss function value and the number of positive and negative examples.

[0149] Take the example of requiring the loss function value to be 0 and the number of positive examples to be >1000.

[0150] Starting from the root node INT_PAID==0, the total sample size is 21824, the number of counterexamples, that is, the number of samples that meet INT_PAID!=0 is 665, and the number of positive examples, that is, the number of samples that meet INT_PAID==0 is 21159; then the field used for the left node decision is FEE_RATE==0, the total sample size is 2129, the number of counterexamples FEE_RATE!=0 is 649, and the number of positive examples FEE_RATE==0 is 1480; then continue to make decisions to the leaf node, when INT_PAID!=0 and FEE_RATE!=0 conditions are met, the total sample size of the leaf node FEE_RATE==0 is 649, all of which are counterexamples, the number of positive examples is 0, and the loss function is 0, and this path is marked as value association rule 1. Since the minimum sample size of value association rule 1 is less than 1000, the candidate conclusion is not passed.

[0151] When INT_PAID! = 0 and FEE_RATE == 0 is established, the sample size of the leaf node FEE_RATE == 0 is 1480, and the number of counterexamples is 0, the number of positive examples is 1480, and the loss function is 0. This path is identified as value association rule 2. Value association rule 2 meets the condition that the loss function is 0 and the sample size is greater than 1000, so the candidate conclusion is passed.

[0152] Starting from the root node INT_PAID==0, the total sample size is 21824, the number of counterexamples that meet INT_PAID!=0 is 665, and the number of positive examples that meet INT_PAID==0 is 21159; then the field used for the right node decision is PRIN_PADI>0, the total sample size is 19695, the number of counterexamples PRIN_PADI≤0 is 16, and the number of positive examples PRIN_PADI>0 is 19679; the field used for the left node decision in the next layer is LOAN_BAL_FEE==0, the total sample size is 2129, the number of counterexamples LOAN_BAL_FEE!=0 is 649, and the number of positive examples LOAN_BAL_FEE==0 is 1480. Continue to make decisions to the leaf node. When INT_PAID==0, PRIN_PADI≤0, and LOAN_BAL_FEE==0, the total number of samples of the leaf node FEE_RATE==0 is 15961, the number of counterexamples is 0, the number of positive examples is 15961, the loss function is 0, and the path is identified as value association rule 3. Since the minimum sample size of value association rule 3 is greater than 1000 and the loss function is 0, the candidate conclusion is passed.

[0153] The field used for the next right node decision is AGE_CD=9, the total number of samples is 577, the number of samples of the counterexample AGE_CD!=9 is 11, and the number of samples of the positive example AGE_CD=9 is 566. Continue to make decisions to the leaf node. When INT_PAID==0, PRIN_PADI is greater than 0, and AGE_CD!=9 is established, the total number of samples of the leaf node FEE_RATE==0 is 348, the number of counterexamples is 0, the number of positive examples is 348, the loss function is 0, and this path is identified as value association rule 4. Since the minimum sample size of value association rule 4 is less than 1000, the candidate conclusion is not passed.

[0154] In summary, the candidate conclusions shown in Table 8 are finally output.

[0155] Table 8

[0156]

[0157] The BY before the rule indicates the conclusion of the rule, and the BY after the rule indicates the condition of the rule.

[0158] Step 4: Verify the value association rules on the verification set.

[0159] All value association rules are verified on the validation set, and the value association rules with counterexamples are removed and fed back to the model for output filtering.

[0160] The above process can be repeated. After a preset number of rounds, the value association rules that pass the verification set verification are the value association rules output by the model and can be stored in the native rule library.

[0161] 5. Rule merging and optimization.

[0162] In specific implementation, the rules that fail the grayscale verification in the native rule base (including the value association rules between fields and the value rules of the fields) can be eliminated, and the rules that are not eliminated in the native rule base can be merged and verified on the test set. The rules that pass the verification are stored in the merged rule base (that is, the updated native rule base). Since the number of rules is reduced after the rules are merged, the number of queries on the data in the table can be reduced during monitoring, so the subsequent rules can be published to the monitoring platform and the software and hardware resources consumed for monitoring can be reduced.

[0163] See also Figure 6 , Figure 6 A schematic diagram of a rule merging optimization process provided in an embodiment of the present application, Figure 6 This is an example of combining and optimizing the rules corresponding to the fields of status (status), date (date), currency (curr_cd), interest rate (int_rate), and account opening date (a_date) in the TM_ACCOUNT table.

[0164] a. Global field merging strategy.

[0165] Fields with the same conditions and conclusions are merged.

[0166] like Figure 6 The enumeration values ​​of the status field are: OPEN, APPROVED and CLOSED, and the enumeration values ​​of the date field are: NULL and NOT NULL. There are three rules for the value association between these two fields:

[0167] Rule 1: date IS NULL by status == OPEN;

[0168] Rule 2: date IS NULL by status==APPROVED;

[0169] Rule 3: date IS NOT NULL by status==CLOSED.

[0170] According to the global field merging strategy, the rules are merged as follows:

[0171] Rule 1′: date IS NULL by status IN (OPEN, APPROVED);

[0172] Rule 2′: date IS NOT NULL by status==CLOSED.

[0173] b. Table merging strategy.

[0174] Different fields in the same table are merged according to rules.

[0175] For example, the rules for multiple single fields in a table are:

[0176] Rule 4: curr_cd = 156;

[0177] Rule 5: int_rate!=0;

[0178] Rule 6: a_date IS NOT NULL;

[0179] After merging, it becomes rule 3′: curr_cd=156OR int_rate!=0OR a_date IS NOT NULL, which is actually a single-field rule.

[0180] c. Layered merging strategy.

[0181] For multi-field rules, that is, rules with one or more conditional fields, hierarchical merging is performed.

[0182] For example, after layering and merging the above rule 1′ and rule 2′, the rule is: (status in (OPEN, APPROVED) AND date IS NULL) OR (status in (CLOSE) AND date IS NOT NULL), which is actually a multi-field rule.

[0183] In addition, after the rules are merged, they can be verified on the test set. If the verification fails, they can be eliminated, and the rules that are not eliminated are retained to form a merged rule base.

[0184] 6. Rules and controls.

[0185] The merged rule base can be converted into the monitoring rule base required by the data quality monitoring platform, deployed to the production or test environment, and monitor whether the incremental data is abnormal. When there is an abnormality, monitoring alarms can be issued to ensure the accuracy of business data. In order to ensure the effectiveness and freshness of the monitoring rules in the monitoring rule base and reduce the probability of false alarms in monitoring alarms, multiple rounds of grayscale verification can be performed before deployment. Monitoring rules that pass the grayscale verification can be officially released, and then routine monitoring checks can be started.

[0186] The above process is described in detail below.

[0187] a. First, perform rule conversion. The rules in the merged rule base are positive logic rules, which need to be converted into judgment logic rules for abnormal data required for monitoring and inspection. The core logic of the conversion is as follows:

[0188] ①The condition field remains unchanged and the conclusion field is reversed.

[0189] For example, the conclusion field date IS NOT NULL is negated to date IS NULL. The original rule is Head ByBody1 and Body2 and…, and the converted rule is NOT Head By Body1 and Body2 and…. Among them, Body1 is the condition field, and Head is the conclusion field.

[0190] ②Convert the conversion rules into SQL statements and determine the number of abnormal data.

[0191] Taking the converted rule of NOT Head By Body1 and Body2 and… as an example, the converted SQL statement is: SELECT COUNT(1)FROM DB.TableName WHERE NOT Head By Body1 and Body2 and….

[0192] ③Supplement the monitoring deployment platform with the monitoring alarm manager, monitoring rule name, monitoring rule ID, and other ancillary information.

[0193] b. The monitoring deployment of rules is first deployed to the grayscale verification project for grayscale verification. After the grayscale verification passes, it is fed back to the model for formal release and merging. The number of grayscale verification rounds can be configured. For example, if it is configured to 3, when all three grayscale verifications pass, the model is fed back as passed, otherwise it is failed.

[0194] c. The results of the grayscale verification are fed back to the model. When the model outputs the corresponding rules, the rules that fail the verification will be filtered out, and the rules that pass the verification will be deployed to the formal project after conversion.

[0195] In the embodiment of the present application, the fields are divided into types such as numerical values, characters, and enumerations, and different paradigms are designed for the value conditions of different types of fields. The basic paradigms are designed based on a single field, and the logical rules between fields can be obtained based on the combination of any basic paradigms. On the one hand, compared with the need to specify the metadata information of each field when generating rules using metadata information, model learning only requires the input of a limited number of paradigms, which is not only applicable to massive data, but also covers richer rules. On the other hand, the abstracted paradigm can reduce the solution space of the model learning objective, especially the solution space between multiple fields, so that the model can learn more accurate and effective logical rules between fields under limited resource conditions.

[0196] In addition, in order to prevent the model from learning too many invalid and redundant rules, the data samples are divided into training sets and validation sets. During the training process, the decision rule output is verified in real time using the validation set through an improved decision tree and strong constraint regression combined algorithm. In order to further improve the effectiveness of the rules, the output logic rules also undergo multiple rounds of grayscale verification feedback, and grayscale verification model feedback is timely provided on newly added data samples to improve the accuracy of the rules learned by the model.

[0197] Based on the same technical concept, the embodiment of the present application also provides a data analysis device. The principle of solving the problem by the data analysis device is similar to that of the above-mentioned data analysis method. Therefore, the implementation of the data analysis device can refer to the implementation of the data analysis method, and the repeated parts will not be repeated.

[0198] Figure 7 A schematic diagram of the structure of a data analysis device provided in an embodiment of the present application includes:

[0199] The determination module 701 is used to determine the paradigm corresponding to each field in the paradigm library according to the field description information of each field in the business table, wherein the paradigm library stores the correspondence between the field description information and the paradigm, wherein the field description information of a field includes the field category and the field value expression;

[0200] A generating module 702 is used for, if the paradigm corresponding to any field includes a conclusion paradigm, taking the value expression of the field as the conclusion of the decision tree and the value expressions of multiple candidate fields as the decision sources of the decision tree, and building a decision tree according to a set decision tree building rule, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and generating a value association rule between fields according to the decision tree;

[0201] The storage module 703 is used to store each value association rule in the native rule library.

[0202] In some embodiments, the generating module 702 is specifically configured to:

[0203] Selecting a target decision condition that meets a preset condition from each decision condition of the decision tree, wherein each decision condition includes a value expression of at least one candidate field;

[0204] Each target decision condition is combined with the value expression of the field to obtain a value association rule between the fields.

[0205] In some embodiments, the paradigms in the paradigm library include first-type paradigms and / or second-type paradigms, wherein the first-type paradigms are generated based on field value rules, and the second-type paradigms are generated based on multiple target structured query language SQL statements, each of which is used to describe the association relationship between fields.

[0206] In some embodiments, when the paradigms in the paradigm library include the second type of paradigms, the second type of paradigms are generated according to the following steps:

[0207] Performing semantic analysis on each of the target SQL statements;

[0208] Extracting a field value expression from the target SQL statement according to the semantic analysis result;

[0209] The second type of paradigm is obtained by performing paradigm conversion according to the value expression of the extracted field and the field type of the field.

[0210] In some embodiments, the native rule base also stores the value rules of a single field, and also includes:

[0211] A removal module 704 is used to remove the rules that fail the grayscale verification in the native rule base after storing each value association rule in the native rule base, wherein the rules include value association rules and value rules;

[0212] The merging module 705 is used to merge the rules that have not been eliminated in the native rule base to obtain an updated native rule base.

[0213] In some embodiments, a monitoring module 706 is further included for:

[0214] Reverse the rules in the native rule base to obtain a monitoring rule base;

[0215] Quality monitoring is performed based on the monitoring quality requirements of each field in the monitoring rule base.

[0216] The division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, the functional modules in the embodiments of the present application may be integrated into a processor, or may exist physically alone, or two or more modules may be integrated into one module. The coupling between the modules may be achieved through some interfaces, which are usually electrical communication interfaces, but it is not excluded that they may be mechanical interfaces or other forms of interfaces. Therefore, the modules described as separate components may or may not be physically separated, and may be located in one place or distributed to different locations of the same or different devices. The above-mentioned integrated modules may be implemented in the form of hardware or in the form of software functional modules.

[0217] After introducing the data analysis method and device according to the exemplary embodiment of the present application, next, an electronic device according to another exemplary embodiment of the present application is introduced.

[0218] Refer to the following Figure 8 The electronic device 130 implemented according to this embodiment of the present application is described. Figure 8 The electronic device 130 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0219] like Figure 8 As shown, the electronic device 130 is in the form of a general electronic device. The components of the electronic device 130 may include but are not limited to: the at least one processor 131, the at least one memory 132, and a bus 133 connecting different system components (including the memory 132 and the processor 131).

[0220] Bus 133 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a processor, or a local bus using any of a variety of bus architectures.

[0221] The memory 132 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 1321 and / or a cache memory 1322 , and may further include a read-only memory (ROM) 1323 .

[0222] The memory 132 may also include a program / utility 1325 having a set (at least one) of program modules 1324, such program modules 1324 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0223] The electronic device 130 may also communicate with one or more external devices 134 (e.g., keyboards, pointing devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 130, and / or communicate with any device that enables the electronic device 130 to communicate with one or more other electronic devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 135. Furthermore, the electronic device 130 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 136. As shown, the network adapter 136 communicates with other modules for the electronic device 130 via a bus 133. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 130, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0224] In an exemplary embodiment, the electronic device of the present application may include at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores a computer program executable by the at least one processor, and when the computer program is executed by the at least one processor, the at least one processor may perform the steps of any data analysis method provided in the embodiments of the present application.

[0225] In an exemplary embodiment, a storage medium is also provided, and when a computer program in the storage medium is executed by a processor of an electronic device, the electronic device can perform any of the above data analysis methods. Optionally, the storage medium can be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0226] In an exemplary embodiment, a computer program product is also provided. When the computer program product is executed by an electronic device, the electronic device can implement any exemplary method provided in the present application.

[0227] It should be noted that, although several modules or submodules of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided into multiple modules to be embodied.

[0228] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0229] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0230] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0231] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application also includes these modifications and variations.

Claims

1. A data analysis method, characterized in that: include: According to the field description information of each field in the business table, the paradigm corresponding to each field is determined in the paradigm library, and the corresponding relationship between the field description information and the paradigm is stored in the paradigm library, wherein the field description information of a field includes the field category and the field value expression; If the paradigm corresponding to any field includes a conclusion paradigm, the value expression of the field is used as the conclusion of the decision tree, and the value expressions of multiple candidate fields are used as the decision sources of the decision tree. According to the set decision tree construction rules, a decision tree is constructed, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and value association rules between fields are generated according to the decision tree; Store each value association rule in the native rule base.

2. The method according to claim 1, characterized in that According to the decision tree, the value association rules between the fields are generated, including: Selecting a target decision condition that meets a preset condition from each decision condition of the decision tree, wherein each decision condition includes a value expression of at least one candidate field; Each target decision condition is combined with the value expression of the field to obtain a value association rule between the fields.

3. The method according to claim 1, characterized in that The paradigms in the paradigm library include first-class paradigms and / or second-class paradigms, wherein the first-class paradigms are generated according to field value selection rules, and the second-class paradigms are generated according to multiple target structured query language SQL statements, each of which is used to describe the association relationship between fields.

4. The method according to claim 3, characterized in that When the paradigms in the paradigm library include the second type of paradigm, the second type of paradigm is generated according to the following steps: Performing semantic analysis on each of the target SQL statements; Extracting a field value expression from the target SQL statement according to the semantic analysis result; The second type of paradigm is obtained by performing paradigm conversion according to the value expression of the extracted field and the field type of the field.

5. The method according to claim 1, characterized in that The native rule base also stores the value rules of a single field. After storing the value association rules in the native rule base, it also includes: Eliminate the rules that fail the grayscale verification in the native rule library, wherein the rules include value association rules and value rules; The rules that have not been eliminated in the native rule base are merged to obtain an updated native rule base.

6. The method according to any one of claims 1 to 5, characterized in that: Also includes: Reverse the rules in the native rule base to obtain a monitoring rule base; Quality monitoring is performed based on the monitoring quality requirements of each field in the monitoring rule base.

7. A data analysis device, characterized in that: include: A determination module, used to determine the paradigm corresponding to each field in the paradigm library according to the field description information of each field in the business table, wherein the paradigm library stores the correspondence between the field description information and the paradigm, wherein the field description information of a field includes the field category and the field value expression; A generation module, for, if the paradigm corresponding to any field includes a conclusion paradigm, taking the value expression of the field as the conclusion of the decision tree and the value expressions of multiple candidate fields as the decision sources of the decision tree, and building a decision tree according to a set decision tree building rule, wherein the paradigm corresponding to each candidate field includes a conditional paradigm, and generating value association rules between fields according to the decision tree; The storage module is used to store each value association rule in the native rule base.

8. An electronic device, characterized in that: include: at least one processor, and a memory communicatively connected to the at least one processor, wherein: The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: When the computer program in the storage medium is executed by a processor of an electronic device, the electronic device can execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images