A data screening method and device based on reinforcement learning

Abstract

The present application provides a data screening method and device based on reinforcement learning, which is used to screen target data and determine whether the screened target data is valid data. The method includes obtaining the target data and determining the data attributes of the target data; determining a set of candidate predicates and a selected predicate set according to the data attributes; building a predicate classification model according to the candidate predicates and the selected predicate set, and determining the valid predicates in the candidate predicates through the predicate classification model; adding the valid predicates to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the target data with the valid predicates is determined to be the valid data. In the process of data screening, meaningless predicate combinations are screened out, thereby improving the execution efficiency of data screening.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data screening method and device based on reinforcement learning. Background Art

[0002] Typical rule mining algorithms are based on depth-first or breadth-first search, and perform rule mining by enumerating predicates. All possible permutations and combinations of predicates may constitute a rule. Therefore, to perform rule mining in large-scale data, it is necessary to traverse all combinations of predicates, which is a time-consuming and labor-intensive process. Summary of the invention

[0003] In view of the above problems, the present application is proposed to provide a data screening method based on reinforcement learning and a device thereof that overcomes the above problems or at least partially solves the above problems, including:

[0004] A data screening method based on reinforcement learning is used to mine rules for target data and determine whether the rules mined are valid rules, including:

[0005] Acquire the target data, and determine a data attribute of the target data, wherein the data attribute is a predicate in the target data;

[0006] Determining a candidate predicate set and a selected predicate set according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set;

[0007] constructing a predicate classification model according to the candidate predicates and the selected predicate set, and determining valid predicates among the candidate predicates through the predicate classification model;

[0008] The valid predicate is added to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the target data with the valid predicate is determined to be the valid data.

[0009] Furthermore, the step of acquiring the target data and determining a data attribute of the target data, wherein the data attribute is a predicate in the target data, includes:

[0010] Acquiring the target data;

[0011] Determining a corresponding database according to the target data;

[0012] generating a data relationship table according to the database and the target data;

[0013] The data attribute corresponding to the target data is determined in the data relationship table.

[0014] Furthermore, the step of constructing a predicate classification model based on the candidate predicates and the selected predicate set, and determining valid predicates among the candidate predicates by using the predicate classification model includes:

[0015] Constructing a predicate classification model through a feedforward neural network according to the candidate predicates and the selected predicate set;

[0016] The effective predicate is determined according to the predicate classification model, the candidate predicates and the selected predicate set.

[0017] Furthermore, the step of constructing a predicate classification model through a feedforward neural network based on the candidate predicates and the selected predicate set includes:

[0018] Performing deep reinforcement learning processing according to the candidate predicates and the selected predicate set to construct a reinforcement learning strategy;

[0019] generating training data according to the reinforcement learning strategy, the candidate predicates and the selected predicate set;

[0020] The predicate classification model is constructed according to the training data and the reinforcement learning strategy.

[0021] Furthermore, the step of performing deep reinforcement learning processing on the basis of the candidate predicates and the selected predicate set to construct a reinforcement learning strategy includes:

[0022] Constructing a current network and a target network, marking the candidate predicate as an execution action and marking the selected predicate set as a current state;

[0023] generating a reward value according to the current network, the current state, and the executed action;

[0024] The reinforcement learning strategy is constructed according to the reward value, wherein the reinforcement learning strategy includes determining that the candidate predicate and the selected predicate set are related when the reward value is positive; or determining that the candidate predicate and the selected predicate set are not related when the reward value is negative.

[0025] Furthermore, the step of determining the effective predicate according to the predicate classification model, the candidate predicates and the selected predicate set includes:

[0026] determining an estimated value of the candidate predicate according to the predicate classification model and the candidate predicate;

[0027] When the estimated value of the candidate predicate is greater than or equal to a preset threshold, the candidate predicate is determined to be the valid predicate.

[0028] Further, when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the step of determining the target data with the valid predicate as the valid data includes:

[0029] generating state information according to the predicate classification model and the valid predicate, wherein the state information includes correct and incorrect;

[0030] Determining the relevance information of the effective predicate according to the predicate classification model, the state information and the effective predicate, wherein the relevance information includes relevance and irrelevance;

[0031] When the state information returned by the predicate classification model according to the current valid predicate is correct, the correlation information of the current valid predicate is determined to be relevant, and the target data with the valid predicate is determined to be the valid data.

[0032] The present application also discloses a data screening device based on reinforcement learning, which is used to perform rule mining on target data and determine whether the rules mined are valid rules, including:

[0033] an acquisition module, configured to acquire the target data and determine a data attribute of the target data, wherein the data attribute is a predicate in the target data;

[0034] A first determination module is used to determine a candidate predicate set and a selected predicate set according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set;

[0035] A second determination module is used to construct a predicate classification model according to the candidate predicates and the selected predicate set, and determine the valid predicates among the candidate predicates through the predicate classification model;

[0036] A determination module is configured to add the valid predicate to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determine that the target data with the valid predicate is the valid data.

[0037] The present application also discloses a device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the data screening method based on reinforcement learning as described above are implemented.

[0038] The present application also discloses a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data screening method based on reinforcement learning as described above are implemented.

[0039] This application has the following advantages:

[0040] In an embodiment of the present application, the target data is obtained and the data attribute of the target data is determined, wherein the data attribute is a predicate in the target data; a candidate predicate set and a selected predicate set are determined according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set; a predicate classification model is constructed according to the candidate predicates and the selected predicate set, and valid predicates in the candidate predicates are determined by the predicate classification model; the valid predicates are added to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the target data with the valid predicate is determined to be the valid data. By using a correlation classifier M corr (P sel ,p,e), filter out meaningless predicate combinations during data screening. M corr (P sel ,p,e) essentially utilizes the DQN approach to reduce the search space for data ,screening, and achieves the purpose of accelerating the discovery process; ,it can efficiently discover cross-table REE rules defined by multiple ,tuple variables, thereby improving the execution efficiency of BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solution of the present application, the drawings required for use in the description of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0042] Figure 1 It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0043] Figure 2It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0044] Figure 3 It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0045] Figure 4 It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0046] Figure 5 It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0047] Figure 6 It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0048] Figure 7 It is a flowchart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0049] Figure 8 It is a structural block diagram of a data screening device based on reinforcement learning provided by an embodiment of the present application;

[0050] Fig. 9 It is a structural schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the objects, features and advantages of the present application more obvious and understandable, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0052] It should be noted that the rules used in the present invention are entity enhancing rules (REE for short). The basic component of REE is the predicate p, which is defined as follows:

[0053] p:=R(t)|tA◎c|tA◎sB|M(tA,sB)

[0054] Among them, ◎ is an operator, which can be equal or not equal; R(t) means t is a tuple variable in the relational table R; tA represents the A attribute of variable t; M is a machine learning model, if tA and sB are related, then this model returns true (correct), otherwise it returns false (wrong). tA◎c with a constant is called a constant predicate; tA◎sB without a constant is called a variable predicate; M(tA,sB) is called a machine learning predicate.

[0055] Based on predicates, REE is defined as: X->e, where (1) X is the combination of multiple predicates, which is called the condition of this REE; (2) e is a predicate, which is called the result of this REE.

[0056] A specific REE example is as follows:

[0057] Express(t)∧Express(s)∧t.Recipient=s.Recipient∧t.Address=“Shenzhen City, Guangdong Province”->s.Postal Code=“510000”

[0058] The scenario described by this REE is that if the recipients of courier t and courier s are the same person, and the address of courier t is in "Shenzhen City, Guangdong Province", then the postal code of courier s must be "510000".

[0059] Typical rule mining algorithms are based on depth-first or breadth-first search to discover REE rules in relational data. Specifically, for each REE result e, these methods store two predicate sets: P sel and P re . Among them, P sel What is stored is the predicate that has been selected to form the REE condition, and P re The predicates to be selected are stored. sel is an empty set, P re is the set of all possible predicates. Then the rule mining algorithm traverses the search space in a depth-first or breadth-first manner and iteratively selects P re The predicate in P sel until one of the following conditions is met: (1) P re becomes an empty set; or, (2)P sel →e is a valid REE.

[0060] Reference Figure 1 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0061] The method is used to perform rule mining on target data and determine whether the rules mined are valid rules, including:

[0062] S110, acquiring the target data, and determining a data attribute of the target data, wherein the data attribute is a predicate in the target data;

[0063] S120, determining a candidate predicate set and a selected predicate set according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates, and the selected predicate set is an empty set;

[0064] S130, constructing a predicate classification model according to the candidate predicates and the selected predicate set, and determining valid predicates among the candidate predicates through the predicate classification model;

[0065] S140: Add the valid predicate to the selected predicate set. When all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determine that the target data with the valid predicate is the valid data.

[0066] In an embodiment of the present application, the target data is obtained and the data attribute of the target data is determined, wherein the data attribute is a predicate in the target data; a candidate predicate set and a selected predicate set are determined according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set; a predicate classification model is constructed according to the candidate predicates and the selected predicate set, and valid predicates in the candidate predicates are determined by the predicate classification model; the valid predicates are added to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the target data with the valid predicate is determined to be the valid data. By using a correlation classifier M corr (P sel , p, e) is the predicate classification model, which filters out meaningless predicate combinations in the process of rule mining. corr (P sel ,p,e) In essence, the DQN approach is used to reduce the search space of rule mining ,and achieves the purpose of accelerating the discovery process; it can efficiently discover ,cross-table

[0067] Next, a data screening method based on reinforcement learning in this exemplary embodiment will be further described.

[0068] As described in step S110, the target data is acquired, and a data attribute of the target data is determined, wherein the data attribute is a predicate in the target data.

[0069] In an embodiment of the present invention, the specific process of "obtaining the target data and determining the data attribute of the target data, wherein the data attribute is a predicate in the target data" in step S110 can be further explained in combination with the following description.

[0070] Reference Figure 2 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0071] As described in the following steps,

[0072] S210, acquiring the target data;

[0073] S220, determining a corresponding database according to the target data;

[0074] S230, generating a data relationship table according to the database and the target data;

[0075] S240: Determine the data attribute corresponding to the target data in the data relationship table.

[0076] It should be noted that the corresponding database is determined according to the target data; the database from which the target data comes is determined by the target data;

[0077] It should be noted that a data relationship table is generated based on the database and the target data; and a data relationship table corresponding to the target data exists in the database.

[0078] It should be noted that the data attribute corresponding to the target data is determined in the data relationship table; the data attribute of the target data is determined through the data relationship table.

[0079] As described in step S130, a predicate classification model is constructed according to the candidate predicates and the selected predicate set, and valid predicates among the candidate predicates are determined through the predicate classification model.

[0080] In an embodiment of the present invention, the specific process of "building a predicate classification model according to the candidate predicates and the selected predicate set, and determining the valid predicates among the candidate predicates by using the predicate classification model" in step S130 can be further explained in combination with the following description.

[0081] Reference Figure 3 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0082] As described in the following steps,

[0083] S310, constructing a predicate classification model through a feedforward neural network according to the candidate predicates and the selected predicate set;

[0084] S320: Determine the effective predicate according to the predicate classification model, the candidate predicates and the selected predicate set.

[0085] It should be noted that a predicate classification model is constructed by a feedforward neural network based on the candidate predicates and the selected predicate set; and a predicate classification model for determining the relevance of the candidate predicates is constructed by a feedforward neural network.

[0086] It should be noted that the effective predicate is determined according to the predicate classification model, the candidate predicates and the selected predicate set; the effective predicates among the candidate predicates can be determined through the predicate classification model, and the effective predicates are stored in the selected predicate set.

[0087] As described in step S310, a predicate classification model is constructed through a feedforward neural network according to the candidate predicates and the selected predicate set;

[0088] In an embodiment of the present invention, the specific process of "building a predicate classification model through a feedforward neural network according to the candidate predicates and the selected predicate set" in step S310 can be further explained in combination with the following description.

[0089] Reference Figure 4 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0090] As described in the following steps,

[0091] S410, performing deep reinforcement learning processing according to the candidate predicate and the selected predicate set to construct a reinforcement learning strategy;

[0092] S420, generating training data according to the reinforcement learning strategy, the candidate predicates and the selected predicate set;

[0093] S430: construct the predicate classification model according to the training data and the reinforcement learning strategy.

[0094] It should be noted that a reinforcement learning strategy is generated by performing deep reinforcement learning processing based on the candidate predicates and the selected predicate set; training data is generated by adopting deep reinforcement learning (Deep Q-learning, DQN). The cost of labels for the training data generated by deep reinforcement learning increases exponentially.

[0095] As an example, deep reinforcement learning includes a Q network and a target Q network, where the target Q network is fixed, the Q network will output a predicted value, and the target Q network will output a target value. The gap between the predicted value and the target value can be defined by a quantitative indicator loss, so that the predicted value and the target value are as close as possible, that is, the loss is as small as possible. The calculated loss can be used to update the Q network, and the Q network will also be regularly updated to the target Q network. According to the output of the Q network, an action a is sampled to interact with the environment, and the environment will return the reward r for executing this action and a new state s; this information will be stored in the replay memory; the information learned in the replay memory will be used to calculate the quantitative indicator loss, thereby updating the Q network.

[0096] It should be noted that the training data is generated according to the reinforcement learning strategy, the candidate predicates and the selected predicate set; the training data is generated for the candidate predicates and the selected predicate set by the reinforcement learning strategy.

[0097] It should be noted that the predicate classification model is constructed based on the training data and the reinforcement learning strategy; the predicate classification model is constructed by the reinforcement learning strategy and a number of training data.

[0098] As described in step S410, deep reinforcement learning is performed based on the candidate predicate and the selected predicate set to generate a reinforcement learning strategy;

[0099] In an embodiment of the present invention, the specific process of "constructing a reinforcement learning strategy by performing deep reinforcement learning processing according to the candidate predicate and the selected predicate set" in step S410 can be further explained in combination with the following description.

[0100] Reference Figure 5 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0101] As described in the following steps,

[0102] S510, constructing the current network and the target network, marking the candidate predicate as an execution action and marking the selected predicate set as a current state;

[0103] S520, generating a reward value according to the current network, the current state and the executed action;

[0104] S530: construct the reinforcement learning strategy according to the reward value, wherein the reinforcement learning strategy includes determining that the candidate predicate and the selected predicate set are related when the reward value is positive; or determining that the candidate predicate and the selected predicate set are not related when the reward value is negative.

[0105] It should be noted that a reward value is generated based on the current network, the current state and the executed action; the higher the reward value, the better the executed action; in reinforcement learning, the predicate classification model will gradually learn how to execute actions to maximize the reward.

[0106] In a specific implementation, the current selection P sel As the current state, the next candidate predicate p to be added is used as the execution action, and all candidate predicates p are added to P sel The confidence difference before and after is used as the reward value. In DQN, when the candidate predicate p is added to P sel When the post-rule exceeds the maximum length or the rule is invalid, the candidate predicate p has a negative reward, that is, the reward value is negative, indicating that the corresponding predicate combination is hopeless, that is, this predicate combination will not produce a valid REE rule.

[0107] Initially, use s 1 The first P sel is an empty state. In the i-th state s i , DQN needs to determine the next action p to apply i , p i Add to P sel , and s i Convert to s i+1 , use s i → pi s i+1 express.

[0108] To determine the next action (e.g., from s i Convert to s i+1 ), DQN uses two networks: the current network and the target network; the current network is implemented as a feedforward network with two hidden layers, and the current network converts the current state P sel and the execution action p as input, and output a predicted value as the estimated reward value for taking the action, i.e., Q^ value; this Q^ value is learned and updated at each state transition. A larger Q^ value means a greater likelihood of applying the corresponding action. For the target network, its parameters are only determined by the latest state of the parameters in the Q network; specifically, by copying the parameters of the Q network in the previous state, the next learning target is generated; the Q network is updated as P sel As P increases, its parameters are gradually learned. sel i is a set of predicates of size i; given an empty P sel 0 , P sel 0will be gradually expanded by ΔL predicates to obtain a sequence S generated by the alternation of execution actions and current states seq , sequence S seq The Q* value is:

[0109]

[0110] Where ξ is the environment, i.e. the rule mining function, and is the discount rate. In DQN, the Q^ value is used to learn the Q* value output by the target network, so that Q*≈Q^. The learning method is similar to DQN, and its loss function is:

[0111]

[0112] where ρ is the behavior distribution, θ i represents the parameters of the Q network in the i-th step. However, unlike the traditional DQN, the execution action is not selected from a fixed action set; instead, the execution action selected by the present invention must not belong to the current state P sel predicate; so for a given P sel , when obtaining the sequence of execution actions and current state generation, the action set is constantly shrinking. After learning the DQN strategy, generate some M corr For example, for P sel , by using DQN to select a new predicate with the maximum Q value until the composed rule satisfies the REE result e, then the training data has a label of 1. Otherwise, the training data has a label of 0. Finally, use these training data to perform M corr That is, the training of the predicate classification model. corr That is, after the predicate classification model is trained, it can be used with M corr That is, the predicate classification model is used to filter out irrelevant predicates in rule mining.

[0113] As described in step S320, determining the effective predicate according to the predicate classification model, the candidate predicate and the selected predicate set;

[0114] In an embodiment of the present invention, the specific process of "determining the effective predicate according to the predicate classification model, the candidate predicates and the selected predicate set" in step S320 can be further explained in combination with the following description.

[0115] Reference Figure 6 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0116] As described in the following steps,

[0117] S610, determining an estimated value of the candidate predicate according to the predicate classification model and the candidate predicate;

[0118] S620: When the estimated value of the candidate predicate is greater than or equal to a preset threshold, determine that the candidate predicate is the valid predicate.

[0119] It should be noted that the estimated value of the candidate predicate is determined based on the predicate classification model and the candidate predicate; corr (P sel , p, e) that is, the predicate classification model determines the estimated value of the predicate to be selected;

[0120] It should be noted that when the estimated value of the candidate predicate is greater than or equal to a preset threshold, the candidate predicate is determined to be the valid predicate; by judging whether the estimated value of the candidate predicate is greater than or equal to the preset threshold, whether the candidate predicate is a valid predicate is determined;

[0121] In a specific implementation, in order to avoid erroneous exclusion of related predicates, a preset threshold is adopted and a heuristic method is used to filter only predicates whose confidence is not less than (ie, greater than or equal to) the preset threshold.

[0122] As described in step S140, the valid predicate is added to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the target data with the valid predicate is determined to be the valid data;

[0123] In one embodiment of the present invention, the specific process of "adding the valid predicate to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determining that the target data with the valid predicate is the valid data" in step S140 can be further explained in combination with the following description.

[0124] Reference Figure 7 , showing a step flow chart of a data screening method based on reinforcement learning provided by an embodiment of the present application;

[0125] As described in the following steps,

[0126] S710, generating status information according to the predicate classification model and the valid predicate, wherein the status information includes correct and incorrect;

[0127] S720, determining the correlation information of the effective predicate according to the predicate classification model, the state information and the effective predicate, wherein the correlation information includes correlation and irrelevance;

[0128] S730: When the status information returned by the predicate classification model according to the current valid predicate is correct, it is determined that the correlation information of the current valid predicate is relevant, and the target data with the valid predicate is determined to be the valid data.

[0129] It should be noted that when the status information returned by the predicate classification model according to the current valid predicate is the error, the correlation information of the current valid predicate is determined to be irrelevant, and the target data with the valid predicate is determined not to be the valid data.

[0130] It should be noted that the status information is generated according to the predicate classification model and the valid predicate, wherein the status information includes correct and incorrect; and the correlation information of the valid predicate is obtained through the status information of the valid predicate.

[0131] It should be noted that the relevance information of the effective predicate is determined according to the predicate classification model, wherein the relevance information includes relevance and irrelevance; and whether the target data with the effective predicate is effective data is judged according to the relevance information.

[0132] As an example, P re All valid predicates p in it are added to P sel Only when the current P sel Only when P and p are related to REE result e, will the valid predicate p be added to P sel Go in.

[0133] In a specific implementation, when P sel ∧p→e forms a valid rule, M corr (P sel , p, e) means that the predicate classification model returns the generated status information, and when the status information is correct, it means that Psel and the current valid predicate p are related to the REE result e. corr (P sel , p, e) means that the predicate classification model returns the generated status information, and when the status information is wrong, it means P sel The currently valid predicate p is independent of the REE result e, that is, the currently valid predicate p will not be added to P sel middle.

[0134] The technical effect of the present invention is as follows: compared with the brute force enumeration method in the prior art which tries all permutations and combinations of predicates, by using the correlation classifier M corr (P sel ,p,e), filter out meaningless predicate combinations during data screening. M corr (P sel,p,e) In essence, the DQN approach is used to reduce the search space for data ,screening, and achieve the purpose of accelerating the discovery process; ,it can efficiently discover cross-table REE rules defined by multiple tuple variables.

[0135] The efficiency and accuracy of data screening algorithms that do not use the dynamic predicate expansion technology of the present invention are compared in multiple public data. The results show that after using dynamic predicate expansion, only 1% of the data screening accuracy is lost; compared with the data screening algorithm that does not use dynamic predicate expansion technology, the present invention speeds up by 2.52 times on average, and the highest speed-up effect even reaches 4.77 times; this fully proves that the dynamic predicate expansion technology of the present invention can effectively reduce the number of data screenings that need to be traversed and improve the execution efficiency of data screening.

[0136] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0137] Reference Figure 8 , showing a structural block diagram of a data screening device based on reinforcement learning provided by an embodiment of the present application;

[0138] The device is used to perform rule mining on target data and determine whether the rules mined are valid rules, specifically including:

[0139] An acquisition module 810, configured to acquire the target data and determine a data attribute of the target data, wherein the data attribute is a predicate in the target data;

[0140] A first determination module 820 is used to determine a candidate predicate set and a selected predicate set according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set;

[0141] A second determination module 830 is used to construct a predicate classification model according to the candidate predicates and the selected predicate set, and determine the valid predicates among the candidate predicates through the predicate classification model;

[0142] The determination module 840 is configured to add the valid predicate to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determine that the target data with the valid predicate is the valid data.

[0143] In one embodiment of the present invention, the acquisition module 810 includes:

[0144] A first acquisition submodule, used to acquire the target data;

[0145] A first determination submodule, used to determine a corresponding database according to the target data;

[0146] A first generating submodule, used for generating a data relationship table according to the database and the target data;

[0147] The second acquisition submodule is used to determine the data attribute corresponding to the target data in the data relationship table.

[0148] In one embodiment of the present invention, the second determining module 830 includes:

[0149] A first construction submodule is used to construct a predicate classification model through a feedforward neural network according to the candidate predicate and the selected predicate set;

[0150] The second determination submodule is used to determine the effective predicate according to the predicate classification model, the candidate predicate and the selected predicate set.

[0151] In one embodiment of the present invention, the first construction submodule includes:

[0152] A first generating unit, configured to perform deep reinforcement learning processing according to the candidate predicate and the selected predicate set to construct a reinforcement learning strategy;

[0153] A second generating unit, configured to generate training data according to the reinforcement learning strategy, the candidate predicates and the selected predicate set;

[0154] The first construction unit is used to construct the predicate classification model according to the training data and the reinforcement learning strategy.

[0155] In one embodiment of the present invention, the first generating unit includes:

[0156] A first construction subunit is used to construct the current network and the target network, and mark the candidate predicate as an execution action and mark the selected predicate set as a current state;

[0157] A first generating subunit, configured to generate a reward value according to the current network, the current state and the executed action;

[0158] The second construction subunit is used to construct the reinforcement learning strategy according to the reward value, wherein the reinforcement learning strategy includes determining that the candidate predicate and the selected predicate set are related when the reward value is positive; or determining that the candidate predicate and the selected predicate set are not related when the reward value is negative.

[0159] In one embodiment of the present invention, the second determining submodule includes:

[0160] a first determining unit, configured to determine an estimated value of the candidate predicate according to the predicate classification model and the candidate predicate;

[0161] The second determining unit is configured to determine that the candidate predicate is the valid predicate when the estimated value of the candidate predicate is greater than or equal to a preset threshold.

[0162] In one embodiment of the present invention, the determination module 840 includes:

[0163] A second generating submodule, configured to generate status information according to the predicate classification model and the valid predicate, wherein the status information includes correct and incorrect;

[0164] a third determination submodule, configured to determine correlation information of the effective predicate according to the predicate classification model, the state information and the effective predicate, wherein the correlation information includes correlation and irrelevance;

[0165] The first determination submodule is used to determine that the correlation information of the current effective predicate is the correlation when the state information returned by the predicate classification model according to the current effective predicate is the correct, and then determine that the target data with the effective predicate is the effective data.

[0166] Reference Fig. 9 , shows a computer device of a data screening method based on reinforcement learning of the present invention, which may specifically include the following:

[0167] The computer device 12 is in the form of a general-purpose computing device, and the components of the computer device 12 may include but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 connecting different system components (including the system memory 28 and the processing unit 16).

[0168] The bus 18 represents one or more of several types of bus 18 structures, including a memory bus 18 or memory controller, a peripheral bus 18, an accelerated graphics port, a processor or a local bus 18 using any of a variety of bus 18 architectures. These architectures include, by way of example, but are not limited to, an Industry Standard Architecture (ISA) bus 18, a Micro Channel Architecture (MAC) bus 18, an Enhanced ISA bus 18, an Audio Video Electronics Standards Association (VESA) local bus 18, and a Peripheral Component Interconnect (PCI) bus 18.

[0169] The computer device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the computer device 12, including volatile and non-volatile media, removable and non-removable media.

[0170] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. Computer device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write to non-removable, non-volatile magnetic media (commonly referred to as a "hard drive"). Although Fig. 9 Not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM or other optical medium) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 42, which are configured to perform the functions of various embodiments of the present invention.

[0171] A program / utility 40 having a set (at least one) of program modules 42 may be stored in, for example, a memory, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules 42, and program data, each of which or some combination may include an implementation of a network environment. The program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0172] The computer device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, cameras, etc.), one or more devices that enable an operator to interact with the computer device 12, and / or any device that enables the computer device 12 to communicate with one or more other computing devices (e.g., network cards, modems, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the computer device 12 may also communicate with one or more networks (e.g., local area networks (LANs)), wide area networks (WANs), and / or public networks (e.g., the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with the other modules of the computer device 12 via the bus 18. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with the computer device 12, including but not limited to: microcode, device drivers, redundant processing units 16, external disk drive arrays, RAID systems, tape drives, and data backup storage systems 34, etc.

[0173] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing a data screening method based on reinforcement learning provided in an embodiment of the present invention.

[0174] That is, when the processing unit 16 executes the program, the following steps are achieved: acquiring the target data and determining data attributes of the target data, wherein the data attributes are predicates in the target data; determining a candidate predicate set and a selected predicate set according to the data attributes, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set; constructing a predicate classification model according to the candidate predicates and the selected predicate set, and determining valid predicates in the candidate predicates through the predicate classification model; adding the valid predicates to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determining the target data with the valid predicates as the valid data.

[0175] In an embodiment of the present invention, the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a data screening method based on reinforcement learning as provided in all embodiments of the present application:

[0176] That is, when the program is executed by the processor, the following steps are implemented: acquiring the target data, and determining data attributes of the target data, wherein the data attributes are predicates in the target data; determining a candidate predicate set and a selected predicate set according to the data attributes, wherein the candidate predicate set includes a plurality of candidate predicates, and the selected predicate set is an empty set; constructing a predicate classification model according to the candidate predicates and the selected predicate set, and determining valid predicates in the candidate predicates through the predicate classification model; adding the valid predicates to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determining the target data with the valid predicates as the valid data.

[0177] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0178] Computer-readable signal media may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0179] The computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as an independent software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the operator's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet). The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other.

[0180] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0181] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0182] The above is a detailed introduction to a data screening method and device based on reinforcement learning provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A data screening method based on reinforcement learning, used for screening target data and determining whether the screened target data is valid data, characterized in that: include: Acquire the target data, and determine a data attribute of the target data, wherein the data attribute is a predicate type in the target data; Determining a candidate predicate set and a selected predicate set according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set; A predicate classification model is constructed based on the candidate predicates and the selected predicate set, and valid predicates among the candidate predicates are determined through the predicate classification model; a predicate classification model is constructed through a feedforward neural network based on the candidate predicates and the selected predicate set; a reinforcement learning strategy is constructed through deep reinforcement learning processing based on the candidate predicates and the selected predicate set; a current network and a target network are constructed, and the candidate predicates are marked as execution actions and the selected predicate set is marked as current states; a reward value is generated based on the current network, the current state and the execution action; and a reward value is generated based on the current network, the current state and the execution action. The reinforcement learning strategy is constructed according to the reward value, wherein the reinforcement learning strategy includes: when the reward value is positive, it is determined that the candidate predicate and the selected predicate set are related; or, when the reward value is negative, it is determined that the candidate predicate and the selected predicate set are not related; training data is generated according to the reinforcement learning strategy, the candidate predicate and the selected predicate set; the predicate classification model is constructed according to the training data and the reinforcement learning strategy; the valid predicate is determined according to the predicate classification model, the candidate predicate and the selected predicate set; The valid predicate is added to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, the target data with the valid predicate is determined to be the valid data.

2. The method according to claim 1, characterized in that The step of acquiring the target data and determining a data attribute of the target data, wherein the data attribute is a predicate type in the target data, comprises: Acquiring the target data; Determining a corresponding database according to the target data; generating a data relationship table according to the database and the target data; The data attribute corresponding to the target data is determined in the data relationship table.

3. The method according to claim 1, characterized in that The step of determining the effective predicate according to the predicate classification model, the candidate predicates and the selected predicate set comprises: determining an estimated value of the candidate predicate according to the predicate classification model and the candidate predicate; When the estimated value of the candidate predicate is greater than or equal to a preset threshold, the candidate predicate is determined to be the valid predicate.

4. The method according to claim 1, characterized in that: The step of determining the target data with the valid predicate as the valid data when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model comprises: generating state information according to the predicate classification model and the valid predicate, wherein the state information includes correct and incorrect; Determining the relevance information of the effective predicate according to the predicate classification model, the state information and the effective predicate, wherein the relevance information includes relevance and irrelevance; When the state information returned by the predicate classification model according to the current valid predicate is correct, the correlation information of the current valid predicate is determined to be relevant, and the target data with the valid predicate is determined to be the valid data.

5. A data screening device based on reinforcement learning, used for screening target data and determining whether the screened target data is valid data, characterized in that: include: an acquisition module, configured to acquire the target data and determine a data attribute of the target data, wherein the data attribute is a predicate type in the target data; A first determination module is used to determine a candidate predicate set and a selected predicate set according to the data attribute, wherein the candidate predicate set includes a plurality of candidate predicates and the selected predicate set is an empty set; The second determination module is used to construct a predicate classification model based on the candidate predicates and the selected predicate set, and determine the valid predicates in the candidate predicates through the predicate classification model; construct a predicate classification model through a feedforward neural network based on the candidate predicates and the selected predicate set; perform deep reinforcement learning processing based on the candidate predicates and the selected predicate set to construct a reinforcement learning strategy; construct a current network and a target network, and mark the candidate predicate as an execution action and mark the selected predicate set as a current state; generate a target network based on the current network, the current state and the execution action reward value; constructing the reinforcement learning strategy according to the reward value, wherein the reinforcement learning strategy includes determining that the candidate predicate and the selected predicate set are related when the reward value is positive; or determining that the candidate predicate and the selected predicate set are not related when the reward value is negative; generating training data according to the reinforcement learning strategy, the candidate predicate and the selected predicate set; constructing the predicate classification model according to the training data and the reinforcement learning strategy; determining the valid predicate according to the predicate classification model, the candidate predicate and the selected predicate set; A determination module is used to add the valid predicate to the selected predicate set, and when all the valid predicates in the selected predicate set and the selected predicate set are related to the predicate classification model, determine that the target data with the valid predicate is the valid data.

6. A computer device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 4 when executed by the processor.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images