Label sample generation system optimization method and label sample determination method and device

Abstract

The embodiment of the invention provides a label sample generation system optimization method and device and a label sample determination method and device, and relates to the field of artificial intelligence. According to the method, a corresponding sample subset is selected from a label-free sample set through a sample selection module, and then reward and punishment are performed on the sample selection module based on a machine learning classification module, so that optimization of the sample selection module is realized; and the optimized sample selection module is adopted to determine the corresponding label sample from the label-free sample set, so that the problems of unreal, unbalanced and insufficient sample labels in the prior art are solved, the label sample can be scientifically and efficiently determined, and the interpretability of the label sample is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence, and in particular to a label sample generation system optimization method, a label sample determination method and a device. Background Art

[0002] With the development of artificial intelligence technology, machine learning has been widely used in binary or multi-classification scenarios of massive samples, which can scientifically classify samples into corresponding categories. Binary or multi-classification machine learning models are highly dependent on the quality and quantity of classification labels. If the sample labels are untrue, unbalanced or insufficient, it may lead to an increase in the artificial intelligence misjudgment rate and even the failure of the sample classification task.

[0003] In the prior art, specific labels are usually supplemented through expert experience or traditional statistical methods. However, since the supplemented labels are non-objective or not all optimized by mathematical methods, the training effect of the classifier cannot meet expectations, resulting in a high error rate.

[0004] Therefore, how to supplement the sample labels so that the training effect of the classifier can reach the expected level is the problem to be solved at present. Summary of the invention

[0005] The present application provides a label sample generation system optimization method, a label sample determination method and a device, which are used to solve the technical problem that the training effect of the classifier existing in the existing technology cannot meet the expectations, resulting in a high error rate.

[0006] In a first aspect, the present application provides a label sample generation system optimization method, which is applied to a label sample generation system, wherein the label sample generation system includes: a sample selection module and a machine learning classification module, and the method includes:

[0007] The sample selection module determines a sample subset from the unlabeled sample set, and performs labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label;

[0008] The machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameter of the sample selection module and the sample parameter of each candidate sample;

[0009] Based on the model reward parameters and sample parameters, the sample selection module is adjusted to obtain an optimized sample selection module.

[0010] In a possible implementation, the configuration information includes: a second sample set, a test set, and a validation set, wherein the second sample set includes: a plurality of samples with a second label, the second label and the first label are mutually exclusive, and before the sample selection module determines the sample subset from the unlabeled sample set, the method further includes:

[0011] All samples in the unlabeled sample set are taken as samples with a first label, and classified by the machine learning classification module according to the second sample set, the test set and the validation set to obtain the initial model reward parameters of the sample selection module and the initial sample parameters of each sample in the unlabeled sample set.

[0012] In a possible implementation, the sample parameters include: a sample reward parameter and a sample state, and the sample selection module determines a sample subset from the unlabeled sample set, including:

[0013] The sample selection module selects the sample subset from the unlabeled sample set according to a preset ratio parameter, a sample state of each sample in the unlabeled sample set, and a sample reward parameter;

[0014] The number of samples in the sample subset is the product of the number of samples in the unlabeled sample set and the proportion parameter.

[0015] In a possible implementation, adjusting the parameters of the sample selection module based on the model reward parameter and the sample parameter to obtain an optimized sample selection module includes:

[0016] Determine a first balance parameter corresponding to the sample reward parameter and a second balance parameter corresponding to the model reward parameter;

[0017] For any one of the plurality of candidate samples, determining a “value-action” function of the sample selection module relative to the candidate sample according to the sample reward parameter, the first balance parameter, the model reward parameter and the second balance parameter of the candidate sample;

[0018] Determine a parameter update function of the sample selection model according to the “value-action” function of the plurality of candidate samples, the sample status and the sample selection parameters of the sample selection model;

[0019] Based on the parameter updating function, the sample selection parameters of the sample selection model are updated to obtain the optimized sample selection model.

[0020] In a possible implementation, the method further includes:

[0021] The machine learning classification module updates the sample status and sample reward parameters of the corresponding samples in the unlabeled sample set based on the sample parameters of multiple candidate samples, so that the optimized sample selection module repeatedly determines the sample subset based on the updated unlabeled sample set until the model reward parameters of the sample selection module meet the preset conditions, or the machine learning classification module performs classification processing for a preset number of times.

[0022] In a second aspect, the present application provides a label sample determination method, which is applied to a label sample generation system. The label sample generation system is optimized by using the label sample generation system optimization method shown in the first aspect and various possible implementations of the first aspect, and the method includes:

[0023] The sample selection module determines a sample subset from the unlabeled sample set, and performs labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label;

[0024] The machine learning classification module classifies the plurality of candidate samples having the first label according to the configuration information to obtain a sample reward parameter of each candidate sample;

[0025] Among multiple candidate samples, candidate samples whose sample reward parameters are greater than the first preset reward parameter are determined as first label samples corresponding to the first label, and candidate samples whose sample reward parameters are less than the second preset reward parameter are determined as second label samples corresponding to the second label.

[0026] In a possible implementation, the method further includes:

[0027] Determine a first number of first labeled samples and a second number of second labeled samples in the unlabeled sample set;

[0028] According to the first number and the second number, determining whether the number of labeled samples in the unlabeled sample set reaches a preset proportion;

[0029] When the number of labeled samples reaches a preset proportion in the unlabeled sample set, determining that the label sample determination procedure is completed;

[0030] When the number of labeled samples does not reach a preset proportion in the unlabeled sample set, the sample selection module re-determines a new sample subset from the unlabeled sample set based on the first number and the second number, and determines new labeled samples until the number of labeled samples reaches a preset proportion in the unlabeled sample set.

[0031] In a third aspect, the present application provides a label sample determination device, which is applied to a label sample generation system, and the label sample determination device includes: a sample selection module, a machine learning classification module, and a determination module;

[0032] The sample selection module is used to determine a sample subset from the unlabeled sample set, and perform labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label;

[0033] The machine learning classification module is used to classify the multiple candidate samples with the first label according to the configuration information to obtain a sample reward parameter for each candidate sample;

[0034] The determination module is used to determine, among the multiple candidate samples, the candidate samples whose sample reward parameters are greater than the first preset reward parameter as the first label samples corresponding to the first label, and to determine the candidate samples whose sample reward parameters are less than the second preset reward parameter as the second label samples corresponding to the second label.

[0035] In a fourth aspect, the present application provides a label sample generation system, the label sample generation system comprising: a sample selection module, a machine learning classification module, and an optimization module;

[0036] The sample selection module is used to determine a sample subset from the unlabeled sample set, and perform labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label;

[0037] The machine learning classification module is used to classify the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameter of the sample selection module and the sample parameter of each candidate sample;

[0038] The optimization module is used to adjust the parameters of the sample selection module based on the model reward parameters and the sample parameters to obtain an optimized sample selection module.

[0039] In a fifth aspect, the present application provides an electronic device, characterized in that it includes: a processor, and a memory communicatively connected to the processor;

[0040] The memory stores computer-executable instructions;

[0041] The processor executes the computer-executable instructions stored in the memory to implement the method described in the first aspect and various possible implementations of the first aspect, or the second aspect and various possible implementations of the second aspect.

[0042] In a sixth aspect, the present application provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the method described in the first aspect and various possible implementation methods of the first aspect, or the second aspect and various possible implementation methods of the second aspect.

[0043] In the seventh aspect, the present application provides a computer program product, characterized in that it includes a computer program, which, when executed by a processor, implements the method described in the first aspect and various possible implementations of the first aspect, or the second aspect and various possible implementations of the second aspect.

[0044] The label sample generation system optimization method, label sample determination method and device provided in the present application select a corresponding sample subset from an unlabeled sample set through a sample selection module, and then reward or punish the sample selection module based on a machine learning classification module to optimize the sample selection module; and then use the optimized sample selection module to determine the corresponding label samples from the unlabeled sample set, thereby solving the problems of untrue, unbalanced and insufficient sample labels existing in the prior art. This method can scientifically and efficiently determine label samples and improve the interpretability of label samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0046] Figure 1 A schematic diagram of a label sample generation system optimization method provided in an embodiment of the present application Figure 1 ;

[0047] Figure 2 A schematic diagram of a label sample generation system optimization method provided in an embodiment of the present application Figure 2 ;

[0048] Figure 3 A flowchart of a label sample determination method provided in an embodiment of the present application;

[0049] Figure 4 A schematic diagram of the structure of a label sample determination device provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the structure of a label sample generation system provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0052] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0053] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0054] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, products or devices.

[0056] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant countries and regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0057] In addition, this application involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and the use of artificial intelligence technology for automated decision-making, and a technical solution for making decisions that have a significant impact on personal rights and interests based on the results of automated decision-making. The application provides users with corresponding operation entrances for them to choose to agree or reject the results of automated decision-making; if the user chooses to reject, the expert decision-making process will be entered.

[0058] It should be noted that the label sample generation system optimization method, label sample determination method and device provided in the present application can be used in the field of financial technology, and can also be used in any field other than financial technology. The application field of the label sample generation system optimization method, label sample determination method and device in the present application is not limited.

[0059] With the development of artificial intelligence technology, machine learning has been widely used in binary or multi-classification scenarios of massive samples, which can scientifically classify samples into corresponding categories. Binary or multi-classification machine learning models are highly dependent on the quality and quantity of classification labels. If the sample labels are untrue, unbalanced or insufficient, it may lead to an increase in the artificial intelligence misjudgment rate and even the failure of the sample classification task.

[0060] In the prior art, specific labels are usually supplemented through expert experience or traditional statistical methods. However, since the supplemented labels are non-objective or not all optimized by mathematical methods, the training effect of the classifier cannot meet expectations, resulting in a high error rate.

[0061] Therefore, how to supplement the sample labels so that the training effect of the classifier can reach the expected level is the problem to be solved at present.

[0062] In response to the above problems, the present application provides a label sample generation system optimization method, a label sample determination method and device, which selects a corresponding sample subset from an unlabeled sample set through a sample selection module, and then rewards or punishes the sample selection module based on a machine learning classification module to optimize the sample selection module; and then uses the optimized sample selection module to determine the corresponding label samples from the unlabeled sample set, thereby solving the problems of inauthentic, unbalanced and insufficient sample labels existing in the prior art. This method can scientifically and efficiently determine label samples and improve the interpretability of label samples.

[0063] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] Figure 1 A schematic diagram of a label sample generation system optimization method provided in an embodiment of the present application Figure 1 This embodiment is applied to a label sample generation system, which includes: a sample selection module and a machine learning classification module.

[0065] Among them, the sample selection module is used to select a sample subset from the unlabeled sample set, and the machine learning classification module is used to score the sample selection module based on multiple candidate samples in the sample subset selected by the sample selection module.

[0066] It is understandable that the embodiments of the present application can be applied to reinforcement learning scenarios, for example. The sample selection module and the machine learning classification module can be in the same deep reinforcement learning neural network, for example.

[0067] Reinforcement learning scenarios mainly include four parts: environment, player, state, and action. In a game of reinforcement learning, the player can formulate an action strategy based on the current state and the previous rewards. The ultimate goal of the player is to obtain the maximum reward. The environment gives rewards, and the environment gives rewards or penalties based on the player's actions and current state, and updates the state at the same time.

[0068] In this embodiment, the state may, for example, refer to the sample states of multiple unlabeled samples in an unlabeled sample set; the environment may, for example, be played by a machine learning module, which is responsible for giving rewards and updating the state; the player may, for example, be played by a sample selection module, which trains its action strategies under different states and rewards, hoping that the selected labeled samples can obtain the highest reward; and the action may, for example, refer to the action strategy of the sample selection module.

[0069] like Figure 1 As shown, the label sample generation system optimization method provided in this embodiment includes:

[0070] S101: The sample selection module determines a sample subset from the unlabeled sample set.

[0071] The sample selection module may select at least one corresponding sample from the unlabeled sample set based on the expected label. The expected label may be, for example, a positive label, a negative label, and a type label, etc., which is not limited in this application. The following takes the expected label being a negative label as an example to explain the embodiment of this application.

[0072] For example, a negative sample selector (Ns) may be deployed in the sample selection module. The main purpose of the negative sample selector is to select negative samples that meet the negative label from multiple samples.

[0073] The unlabeled sample set includes multiple unlabeled samples. Before the sample selection module selects a sample subset from the unlabeled sample set, the multiple unlabeled samples may be configured with corresponding initial labels so that the sample selection module can make a selection based on the initial label of each sample.

[0074] It can be understood that the initial labels of the samples can be the same or different, and the indication is used to indicate that the current classification scenario is max(y i )+1 category.

[0075] In this scenario, supervised machine learning classifiers such as gradient boosting trees, neural networks, logistic regression or deep neural networks cannot work correctly. The goal of the negative sample selector Ns is to select the most suitable negative sample set from the unlabeled sample set, so as to achieve the optimal training effect in synchronization with the machine learning classification module.

[0076] The sample subset may include, for example, a preset number of samples, where the preset number may be, for example, pre-set or allocated by the system, and the present application does not impose any limitation on this.

[0077] S102: The sample selection module performs labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label.

[0078] The marking process may be, for example, marking the selection status of multiple samples in a sample subset, or marking multiple samples as candidate samples with desired labels, which is not limited in the present application.

[0079] The first label may be, for example, a negative label, a positive label, a type label, or a selection status label, which is not limited in the present application.

[0080] After determining the sample subset, the sample selection module will label each sample in the sample subset to obtain a plurality of candidate samples with the first label.

[0081] S103: The machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameters of the sample selection module and the sample parameters of each candidate sample.

[0082] The configuration information may include, for example, classification rules, scoring rules, and sample parameter determination rules for each candidate sample.

[0083] In this embodiment, the machine learning classification module may include, for example, a machine learning classifier (MLClassifier, Cl). Cl is required to perform a specific task in the objective world, such as finding customers who are expected to purchase a certain product from a set of customers to be classified.

[0084] Under normal circumstances, the premise for performing this task is to define customers who have actually purchased the product within a certain period of time as positive samples. However, in reality, it is impossible to identify negative samples, that is, customers who are not expected to purchase the product.

[0085] Therefore, the machine learning classifier Cl is required to train the sample selector Ns according to the configuration information so that the sample selector Ns can select negative sample customers who are not expected to purchase the product.

[0086] The model reward parameter may be, for example, the reward score of the current sample selector Ns, which is determined based on the multiple candidate samples with the first label selected by the sample selection module at that time. Since the purpose of the sample selection module is to select the negative sample set with the highest reward, the more accurate the negative sample set selected by the sample selection module is, the higher the model reward parameter is.

[0087] The sample parameter may be, for example, the sample reward given by the machine learning classifier Cl to each candidate sample. It is understandable that since the purpose of the sample selector Ns is to select the negative sample with the highest reward, the higher the sample parameter, the greater the probability that the sample selector Ns will select the sample corresponding to the sample parameter next time.

[0088] S104: Based on the model reward parameters and the sample parameters, the sample selection module is adjusted to obtain an optimized sample selection module.

[0089] After obtaining the model reward parameters and sample parameters, the sample selection model can be adjusted based on the above two parameters so that the sample selection module can select the sample subset with the highest reward in the next selection.

[0090] The embodiment of the present application provides a label sample generation system optimization method. First, the sample selection module determines a sample subset from an unlabeled sample set, and labels each sample in the sample subset to obtain multiple candidate samples with a first label; then the machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameters of the sample selection module and the sample parameters of each candidate sample; then, based on the model reward parameters and the sample parameters, the sample selection module is adjusted to obtain an optimized sample selection module; the method introduces the sample selection module, and optimizes the label sample generation system by placing the two intelligent agents, the sample selection module and the machine learning classification module, in the same deep reinforcement learning neural network, thereby outputting two optimized models.

[0091] Figure 2 A schematic diagram of a label sample generation system optimization method provided in an embodiment of the present application Figure 2 The sample parameters described in this embodiment may include, for example, sample reward parameters and sample status, and the configuration information may include, for example, a second sample set, a test set, and a validation set. The second sample set includes: a plurality of samples with a second label, and the second label is mutually exclusive with the first label. Figure 1 Based on the embodiment, a possible implementation method of the label sample generation system optimization method is explained. Figure 2 As shown, the label sample generation system optimization method provided in this embodiment includes:

[0092] S201: Take all samples in the unlabeled sample set as samples with a first label, and classify the all samples through the machine learning classification module according to the second sample set, the test set and the validation set to obtain the initial model reward parameters of the sample selection module and the initial sample parameters of each sample in the unlabeled sample set.

[0093] In this embodiment, the sample with the first label is a negative sample. In the case where the sample with the first label is a negative sample, since the second label and the first label are mutually exclusive, the sample with the second label is a positive sample.

[0094] The second sample set includes multiple positive samples. The test set and the validation set are pre-divided sets, which include multiple unlabeled samples and the true label corresponding to each unlabeled sample. The classification processing can be, for example: binary classification or multi-classification processing, which is not limited in this application and is subject to actual practice.

[0095] In this step, the machine learning classification module first classifies all positive samples in the unlabeled sample set to obtain the initial model reward parameters of the sample selection module and the initial sample parameters of each sample in the unlabeled sample set.

[0096] The sample state is used to indicate whether the corresponding sample is selected in the previous training. If it is selected, the sample state is the first sample state; if it is not selected, the sample state is the second sample state.

[0097] The sample reward parameter is used to indicate the reward that can be obtained by selecting the sample, which can be, for example, a reward score.

[0098] It can be understood that in this step, the machine learning classification module needs to classify negative samples and positive samples from multiple positive samples in the unlabeled sample set based on the second sample set, the test set and the verification set, and use its selection results as the corresponding initial sample state. At the same time, the corresponding initial sample reward parameters are set for each sample in the unlabeled sample set, and the initial model reward parameters are configured for the sample selection module. The initial model reward parameters can be set to preset values, or they can be freely set by the system.

[0099] S202: A sample selection module selects the sample subset from the unlabeled sample set according to a preset ratio parameter, the sample status of each sample in the unlabeled sample set, and a sample reward parameter, and labels each sample in the sample subset to obtain a plurality of candidate samples with a first label.

[0100] The preset ratio parameter may be, for example, a ratio parameter corresponding to positive and negative samples, and is used to determine the number of samples in the sample subset.

[0101] For example,

[0102] X={x1,...,x u ,...,x n},u,n∈N * ∧u<n,

[0103] Among them, x1-x u For u unlabeled samples in the unlabeled sample set U, any sample x in the unlabeled sample set U i There is a corresponding label y i ∈N*.

[0104] The preset positive and negative sample ratio parameter may be, for example, δ, to calculate |x u+1 ,...,x n | / δ is the number of samples in the sample subset, denoted as parameter μ, that is, |N|=μ, where N is the sample subset.

[0105] It can be understood that each sample in the unlabeled sample set corresponds to a sample state indicating whether it was selected last time, and a sample reward parameter that the sample selection module can obtain after selecting itself. During the selection process, the sample selection module can select a preset number of samples with the highest reward from the unlabeled sample set based on the sample states and sample reward parameters of multiple samples, and mark the preset number of samples to obtain a preset number of candidate samples with the first label. The preset number can be determined by a preset ratio parameter.

[0106] S203: The machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameters of the sample selection module and the sample parameters of each candidate sample.

[0107] Among them, during the classification process, the machine learning classification module will test multiple candidate samples based on the test set, and verify multiple candidate samples based on the verification set, so as to obtain the model reward parameters of the sample selection module and the sample parameters of multiple candidate samples.

[0108] For example, if this training is the jth training, the model reward parameter R j After the jth training, the machine learning classification model will generate a comprehensive evaluation index based on the validation set; at the same time, the machine learning classification model will generate a comprehensive evaluation index for each candidate sample x i Generate the corresponding sample reward parameter p(x i ), which is used to indicate the probability that the corresponding candidate sample is a sample with the first label.

[0109] The sum of the sample reward parameters of multiple candidate samples is the selection reward r of the sample selection module in the current training. j , select reward r j The higher it is, the more accurate the samples selected by the sample selection module are. It can be determined using the following formula:

[0110]

[0111] Among them, since the rewards of classical reinforcement learning have positive and negative values, the selection reward r in this embodiment j is always positive. Therefore, the sample reward r for candidate sample i in the jth round is required. ji The formula is converted to correspond to the reinforcement learning scenario. The specific conversion formula is as follows:

[0112]

[0113] Among them, T(x) is the conversion function, a ji For the sample selection model, in the jth round, according to the sample xi ∈U sample state s i Action taken.

[0114] Because the sample selection module takes corresponding actions a∈[0,1] for each sample in the unlabeled sample set according to its previous selection state s to decide whether to select the sample.

[0115] If a=1, it means that the sample was selected in the previous training process; if a=0, it means that the sample was not selected in the previous training process. Therefore, the action strategy of the sample selection module can be regarded as a Markov decision process:

[0116] π ω (s i ,a i )=P ω (a i |s i )=a i f ω (s i )+(1-a i )(1-f ω (s i ))

[0117] In one possible implementation, since the sample selection module needs to undergo a lot of training to ensure that the samples it selects can get the most rewards, it is necessary to model the action strategy of the sample selection module into a deep neural network. Without loss of generality, in this deep neural network, ReLu can be used as the activation function of the hidden layer, and sigmoid can be used as the activation function of the output layer.

[0118] S204: Determine a first balance parameter corresponding to the sample reward parameter and a second balance parameter corresponding to the model reward parameter.

[0119] S205: For any one of the multiple candidate samples, determine the “value-action” function of the sample selection module relative to the candidate sample according to the sample reward parameter, the first balance parameter, the model reward parameter and the second balance parameter of the candidate sample.

[0120] The first balance parameter and the second balance parameter may be preset, for example, and may be used for parameter adjustment.

[0121] Before determining the “value-action” function of the candidate samples, it is necessary to first determine the objective function of the sample selection module.

[0122] The objective function j(ω) can be expressed, for example, by the following formula:

[0123]

[0124] where ω is the parameter form of the sample selection module, Γ={(s1,a1,r1),...(s u ,a u ,r u )} is the trajectory of ω during this training process, s is the state of the sample in the unlabeled sample set, a is the action parameter of the sample selection module, which can be characterized as selection or non-selection, r is the sample reward parameter, and R is the model reward parameter. α is the first balance parameter corresponding to the sample reward parameter, and β is the second balance parameter corresponding to the model reward parameter.

[0125] The gradient of J(ω) with respect to ω is calculated as follows:

[0126]

[0127] in, It is the action strategy a that the sample selection module takes on the i-th sample in the unlabeled sample set during this training process. i The sum of the current and future trajectory rewards after . It can be defined as the “value-action” function V of the sample selection module i :

[0128]

[0129] S206: Determine a parameter update function of the sample selection model according to the “value-action” function of multiple candidate samples, the sample status, and the sample selection parameters of the sample selection model.

[0130] Among them, after obtaining the "value-action" function of multiple candidate samples, the "value-action" function can be brought into the above gradient formula, and the gradient of the objective function j(ω) can be expressed as:

[0131]

[0132] Based on the gradient of the objective function j(ω), the parameter update function of the sample selection model can be determined as:

[0133]

[0134] S207: Based on the parameter updating function, the sample selection parameters of the sample selection model are updated to obtain the optimized sample selection model.

[0135] In a possible implementation, the machine learning classification module may also update the sample states and sample reward parameters of the corresponding samples in the unlabeled sample set based on the sample parameters of the plurality of candidate samples.

[0136] It can be understood that for each classification process, each candidate sample will obtain a new sample parameter, which is used to indicate the latest sample state and sample reward parameter.

[0137] After generating new sample states and sample reward parameters, the machine learning classification model can update the sample parameters of the corresponding samples in the unlabeled sample set based on the new sample states and sample reward parameters, so that the optimized sample selection module can select the corresponding samples based on the latest sample states and sample selection parameters in the next round of training to determine the new sample subset used in the next round of training. The machine pre-learning classification module repeats the training process based on the new sample subset until the model reward parameters of the sample selection module meet the preset conditions, or the machine learning classification module performs the classification processing for a preset number of times.

[0138] The label sample generation system optimization method provided in this embodiment, before optimizing the model, first classifies all samples in the unlabeled sample set according to the second sample set, the test set and the validation set that are mutually exclusive with the first sample set, through the machine learning classification module, to obtain the initial model reward parameters of the sample selection module and the initial sample parameters of each sample in the unlabeled sample set, and then updates the sample selection parameters of the sample selection model based on the "value-action" function of each candidate sample. This method places two intelligent agents, the sample selection module and the machine learning classification module, in the same deep reinforcement learning neural network, and optimizes and updates based on the two models in the reinforcement learning scenario, thereby outputting two optimized models.

[0139] Figure 3 This is a flow chart of a label sample determination method provided in an embodiment of the present application. This embodiment is applied to a label sample generation system, which can be, for example, the above-mentioned Figure 1 or Figure 2 The label sample generation system optimized by the label sample generation system optimization method shown in the embodiment. Figure 3 As shown, the label sample determination method provided in this embodiment includes:

[0140] S301: The sample selection module determines a sample subset from the unlabeled sample set, and performs labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label.

[0141] Among them, step S301 is similar to the above-mentioned steps S101 and S102, and will not be repeated here.

[0142] S302: The machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain a sample reward parameter for each candidate sample.

[0143] Among them, step S302 is similar to the above step S103 and will not be repeated here.

[0144] S303: Determine, among the multiple candidate samples, the candidate samples whose sample reward parameters are greater than the first preset reward parameter as the first label samples corresponding to the first label, and determine the candidate samples whose sample reward parameters are less than the second preset reward parameter as the second label samples corresponding to the second label.

[0145] Among them, the first preset reward parameter is greater than the second preset reward parameter.

[0146] When the sample reward parameter of the candidate sample is greater than the first preset reward parameter, it indicates that the label corresponding to the candidate sample is the first label, that is, the sample selection module selects correctly. At this time, the candidate sample can be determined as the first label sample corresponding to the first label.

[0147] When the sample reward parameter of the candidate sample is less than the second preset reward parameter, it indicates that the label corresponding to the candidate sample is the second label, that is, the sample selection module makes an incorrect selection. At this time, the candidate sample can be determined as the second label sample corresponding to the second label.

[0148] It is understandable that the label sample generation system in this embodiment is applicable to binary classification scenarios and multi-classification scenarios. This embodiment is only explained in the binary classification scenario and does not limit the specific implementation scenario.

[0149] In the case where the implementation scenario is a multi-classification scenario, other preset reward parameters may exist in step S303. For example, in a three-classification scenario, there may be a third preset reward parameter; in a four-classification scenario, there may be a third reward parameter and a fourth reward parameter. However, its specific implementation is similar to the above step S303, and will not be repeated here.

[0150] In one possible implementation, after obtaining the first labeled sample and the second labeled sample, the first number of the first labeled samples and the second number of the second labeled samples in the unlabeled sample set can be determined, and based on the first number and the second number, it can be determined whether the number of labeled samples in the unlabeled sample set reaches a preset proportion.

[0151] The purpose of doing this is to determine whether the number of obtained label samples meets the requirements.

[0152] When the number of labeled samples reaches a preset proportion in the unlabeled sample set, determining that the label sample determination procedure is completed;

[0153] When the number of labeled samples does not reach a preset proportion in the unlabeled sample set, the sample selection module re-determines a new sample subset from the unlabeled sample set based on the first number and the second number, and determines new labeled samples until the number of labeled samples reaches a preset proportion in the unlabeled sample set.

[0154] The label sample determination method provided in the embodiment of the present application determines the label samples by combining two intelligent agents, namely the sample selection module and the machine learning classification module, thereby avoiding the defects of untrue, unbalanced and insufficient label samples in the prior art, achieving scientific and efficient determination of label samples, and improving the interpretability of label samples.

[0155] Figure 4 A schematic diagram of the structure of a label sample determination device provided in an embodiment of the present application. Applicable to a label sample generation system. The label sample determination device 400 includes: a sample selection module 401, a machine learning classification module 402 and a determination module 403;

[0156] The sample selection module 401 is used to determine a sample subset from the unlabeled sample set, and perform labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label;

[0157] The machine learning classification module 402 is used to classify the multiple candidate samples with the first label according to the configuration information to obtain a sample reward parameter for each candidate sample;

[0158] The determination module 403 is used to determine, among the multiple candidate samples, the candidate samples whose sample reward parameters are greater than the first preset reward parameter as the first label samples corresponding to the first label, and to determine the candidate samples whose sample reward parameters are less than the second preset reward parameter as the second label samples corresponding to the second label.

[0159] In a possible implementation, the device further includes: a determination module 404;

[0160] The determination module 403 is further configured to determine a first number of first labeled samples and a second number of second labeled samples in the unlabeled sample set;

[0161] The judging module 404 is configured to judge whether the number of labeled samples in the unlabeled sample set reaches a preset proportion according to the first number and the second number;

[0162] The determination module 403 is further configured to determine that the label sample determination procedure is completed when the number of label samples reaches a preset proportion in the unlabeled sample set;

[0163] When the number of labeled samples does not reach a preset proportion in the unlabeled sample set, the sample selection module re-determines a new sample subset from the unlabeled sample set based on the first number and the second number, and determines new labeled samples until the number of labeled samples reaches a preset proportion in the unlabeled sample set.

[0164] The present embodiment provides a label sample determination device, which can execute the label sample determination method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail in this embodiment.

[0165] Figure 5 This is a schematic diagram of the structure of a label sample generation system provided in an embodiment of the present application. Figure 5 As shown, the label sample generation system 500 includes: a sample selection module 501, a machine learning classification module 502 and an optimization module 503;

[0166] The sample selection module 501 is used to determine a sample subset from the unlabeled sample set, and perform labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label;

[0167] The machine learning classification module 502 is used to classify the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameter of the sample selection module and the sample parameter of each candidate sample;

[0168] The optimization module 503 is used to adjust the parameters of the sample selection module based on the model reward parameters and the sample parameters to obtain an optimized sample selection module.

[0169] In one possible implementation, the configuration information includes: a second sample set, a test set, and a validation set, wherein the second sample set includes: multiple samples with a second label, the second label is mutually exclusive with the first label, and the machine learning classification module 502 is also used to treat all samples in the unlabeled sample set as samples with the first label, and classify the full amount of samples according to the second sample set, the test set, and the validation set through the machine learning classification module to obtain the initial model reward parameters of the sample selection module and the initial sample parameters of each sample in the unlabeled sample set.

[0170] In a possible implementation, the sample selection module 501 is specifically used to select the sample subset from the unlabeled sample set according to a preset proportion parameter, the sample status of each sample in the unlabeled sample set, and a sample reward parameter; wherein the number of samples in the sample subset is the product of the number of samples in the unlabeled sample set and the proportion parameter.

[0171] In one possible implementation, the optimization module 503 is specifically used to determine a first balance parameter corresponding to a sample reward parameter and a second balance parameter corresponding to a model reward parameter; for any one of the multiple candidate samples, the "value-action" function of the sample selection module relative to the candidate sample is determined based on the sample reward parameter, the first balance parameter, the model reward parameter and the second balance parameter of the candidate sample; the parameter update function of the sample selection model is determined based on the "value-action" functions of the multiple candidate samples, the sample states and the sample selection parameters of the sample selection model; based on the parameter update function, the sample selection parameters of the sample selection model are updated to obtain the optimized sample selection model.

[0172] In one possible implementation, the machine learning classification module 502 is also used to update the sample status and sample reward parameters of the corresponding samples in the unlabeled sample set based on the sample parameters of multiple candidate samples, so that the optimized sample selection module repeatedly determines the sample subset based on the updated unlabeled sample set until the model reward parameters of the sample selection module meet the preset conditions, or the machine learning classification module performs the classification processing for a preset number of times.

[0173] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown, the electronic device 600 may include at least one processor 601, at least one storage medium 602, and a communication interface 603. Among them, the at least one processor 601 is used to implement the method provided by the above embodiment of the present application.

[0174] At least one memory 602 is used to store program instructions and / or data. The memory 602 is coupled to the processor 601. The coupling in the embodiment of the present application is an indirect coupling or communication connection between devices, units or modules, which can be electrical, mechanical or other forms, and is used for information exchange between devices, units or modules. The processor 601 may operate in conjunction with the memory 602. The processor 601 may execute program instructions stored in the memory 602. At least one of the at least one memory may be included in the processor.

[0175] The communication interface 603 is used to communicate with other devices through a transmission medium, so that the electronic device 600 can communicate with other devices. The communication interface 603 can be, for example, a transceiver, an interface, a bus, a circuit, or a device capable of implementing a transceiver function. The processor 601 can use the communication interface 603 to send and receive data and / or information, and is used to implement the method provided in the above embodiment of the present application. Please refer to the detailed description in the previous embodiment for details, which will not be repeated here.

[0176] The specific connection medium between the processor 601, the memory 602 and the communication interface 603 is not limited in the embodiment of the present application. Figure 6 In the embodiment, the processor 601, the memory 602 and the communication interface 603 are connected via a bus 606. The bus 606 is Figure 6 The connection between other components is only for illustration and is not intended to be limiting. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0177] A computer-readable storage medium is also provided in an embodiment of the present application, on which a computer program is stored. When the computer program is executed by a processor, the technical solution of the above-mentioned method embodiment is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0178] A computer program product is also provided in an embodiment of the present application, including a computer program. When the computer program is executed by a processor, the technical solution of the above-mentioned method embodiment is implemented. The implementation principle and technical effect are similar and will not be repeated here.

[0179] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0180] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0181] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0182] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0183] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0184] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0185] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0186] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0187] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A label sample generation system optimization method, characterized in that: Applied to a label sample generation system, the label sample generation system includes: a sample selection module and a machine learning classification module, the method includes: The sample selection module determines a sample subset from the unlabeled sample set, and performs labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label; The machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameter of the sample selection module and the sample parameter of each candidate sample; Based on the model reward parameters and the sample parameters, the sample selection module is adjusted to obtain an optimized sample selection module.

2. The method according to claim 1, characterized in that The configuration information includes: a second sample set, a test set, and a validation set, wherein the second sample set includes: a plurality of samples with a second label, the second label and the first label are mutually exclusive, and before the sample selection module determines a sample subset from the unlabeled sample set, the method further includes: All samples in the unlabeled sample set are taken as samples with a first label, and classified by the machine learning classification module according to the second sample set, the test set and the validation set to obtain the initial model reward parameters of the sample selection module and the initial sample parameters of each sample in the unlabeled sample set.

3. The method according to claim 2, characterized in that The sample parameters include: sample reward parameters and sample status. The sample selection module determines a sample subset from the unlabeled sample set, including: The sample selection module selects the sample subset from the unlabeled sample set according to a preset ratio parameter, a sample state of each sample in the unlabeled sample set, and a sample reward parameter; The number of samples in the sample subset is the product of the number of samples in the unlabeled sample set and the proportion parameter.

4. The method according to claim 3, characterized in that: The step of adjusting the parameters of the sample selection module based on the model reward parameter and the sample parameter to obtain an optimized sample selection module includes: Determine a first balance parameter corresponding to the sample reward parameter and a second balance parameter corresponding to the model reward parameter; For any one of the plurality of candidate samples, determining a “value-action” function of the sample selection module relative to the candidate sample according to the sample reward parameter, the first balance parameter, the model reward parameter and the second balance parameter of the candidate sample; Determine a parameter update function of the sample selection model according to the "value-action" function of multiple candidate samples, the sample status and the sample selection parameters of the sample selection model; Based on the parameter updating function, the sample selection parameters of the sample selection model are updated to obtain the optimized sample selection model.

5. The method according to claim 3, characterized in that: The method further comprises: The machine learning classification module updates the sample status and sample reward parameters of the corresponding samples in the unlabeled sample set based on the sample parameters of multiple candidate samples, so that the optimized sample selection module repeatedly determines the sample subset based on the updated unlabeled sample set until the model reward parameters of the sample selection module meet the preset conditions, or the machine learning classification module performs classification processing for a preset number of times.

6. A label sample determination method, characterized in that: Applied to a label sample generation system, the label sample generation system is optimized by using the label sample generation system optimization method according to any one of claims 1 to 5, the method comprising: The sample selection module determines a sample subset from the unlabeled sample set, and performs labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label; The machine learning classification module classifies the multiple candidate samples with the first label according to the configuration information to obtain a sample reward parameter for each candidate sample; Among the multiple candidate samples, the candidate samples whose sample reward parameters are greater than the first preset reward parameter are determined as the first label samples corresponding to the first label, and the candidate samples whose sample reward parameters are less than the second preset reward parameter are determined as the second label samples corresponding to the second label.

7. The method according to claim 6, characterized in that The method further comprises: Determine a first number of first labeled samples and a second number of second labeled samples in the unlabeled sample set; According to the first number and the second number, determining whether the number of labeled samples in the unlabeled sample set reaches a preset proportion; When the number of labeled samples reaches a preset proportion in the unlabeled sample set, determining that the label sample determination procedure is completed; When the number of labeled samples does not reach a preset proportion in the unlabeled sample set, the sample selection module re-determines a new sample subset from the unlabeled sample set based on the first number and the second number, and determines new labeled samples until the number of labeled samples reaches a preset proportion in the unlabeled sample set.

8. A label sample determination device, characterized in that: Applied to a label sample generation system, the device comprises: a sample selection module, a machine learning classification module and a determination module; The sample selection module is used to determine a sample subset from the unlabeled sample set, and perform labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label; The machine learning classification module is used to classify the multiple candidate samples with the first label according to the configuration information to obtain a sample reward parameter for each candidate sample; The determination module is used to determine, among the multiple candidate samples, the candidate samples whose sample reward parameters are greater than the first preset reward parameter as the first label samples corresponding to the first label, and to determine the candidate samples whose sample reward parameters are less than the second preset reward parameter as the second label samples corresponding to the second label.

9. A label sample generation system, characterized in that: The label sample generation system includes: a sample selection module, a machine learning classification module and an optimization module; The sample selection module is used to determine a sample subset from the unlabeled sample set, and perform labeling processing on each sample in the sample subset to obtain a plurality of candidate samples with a first label; The machine learning classification module is used to classify the multiple candidate samples with the first label according to the configuration information to obtain the model reward parameter of the sample selection module and the sample parameter of each candidate sample; The optimization module is used to adjust the parameters of the sample selection module based on the model reward parameters and the sample parameters to obtain an optimized sample selection module.

10. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.

Images