Data category imbalance processing method, device, system, and storage medium

Abstract

The present disclosure provides a method for processing data category imbalance, which is applied to the field of data processing technology. The method includes clustering minority class samples to obtain multiple clusters. Calculate the k nearest neighbor samples of the minority class samples in each cluster, obtain the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculate the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and process the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples. The present disclosure provides a data category imbalance processing device, a training method and device for a semi-supervised generative adversarial network, an abnormal transaction detection method and device, a system, and a storage medium.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and more specifically, to a method and device for processing data category imbalance, a training method and device for a semi-supervised generative adversarial network, an abnormal transaction detection method and device, a system, and a storage medium. Background Art

[0002] The number of samples in a certain category in the data set is quite different from the number of other samples. For example, in the data set collected to judge credit card fraud, the data of most users are normal, and only a small part is the data of defrauded users.

[0003] With the development of machine learning, some machine learning methods have gradually been used in the identification of abnormal consumer behavior and have achieved certain results. However, these methods still do not work well in minority class classification when dealing with samples with uneven distribution. Summary of the invention

[0004] In view of this, the present disclosure provides a method and device for processing data category imbalance, a training method and device for a semi-supervised generative adversarial network, an abnormal transaction detection method and device, a system, and a storage medium.

[0005] A first aspect of the present disclosure provides a method for processing data category imbalance, comprising:

[0006] Cluster the minority class samples to obtain multiple clusters;

[0007] Calculate the k nearest neighbor samples of the minority class samples in each cluster;

[0008] Obtain the number of majority class samples in the k nearest neighbor samples of the minority class samples in each of the clusters;

[0009] Calculate the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples;

[0010] The samples in the cluster are processed according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0011] In one embodiment of the present disclosure, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, processing the samples in the cluster according to the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples includes:

[0012] Obtain any minority class sample in the cluster;

[0013] Select any sample that is the k nearest neighbor of the minority class sample;

[0014] Interpolation is performed based on the minority class sample and any one of the k nearest neighbors of the minority class sample.

[0015] In one embodiment of the present disclosure, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, processing the samples in the cluster according to the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples includes:

[0016] Calculate the center point of the minority class samples in the cluster;

[0017] Select any sample that is k nearest to the center point;

[0018] Based on the central point and any one sample adjacent to the central point, interpolation is performed on the minority class samples in the cluster.

[0019] In one embodiment of the present disclosure, before clustering the minority class samples to obtain multiple clusters, the process includes:

[0020] A real data set is obtained, where the real data set includes majority class samples and minority class samples.

[0021] In one embodiment of the present disclosure, after clustering the minority class samples to obtain multiple clusters, the method further includes:

[0022] All samples in each cluster are normalized to a value range of (0, 1).

[0023] A second aspect of the present disclosure provides a training method for a semi-supervised generative adversarial network, comprising:

[0024] Obtain a training set, wherein the training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples;

[0025] Inputting the training samples in the training set into a semi-supervised generative adversarial network to obtain pseudo samples;

[0026] The semi-supervised generative adversarial network is trained based on the pseudo sample, the training set and the loss function to obtain a trained semi-supervised generative adversarial network.

[0027] In one embodiment of the present disclosure, the loss function of the generator in the semi-supervised generative adversarial network is:

[0028] max(E z～p(z) (log G(D(Z))));

[0029] Among them, Z represents a pseudo sample, E z～p(z) (log G(D(Z))) represents the expected probability that the pseudo sample Z generated by the generator G is discriminated as a minority class sample by the discriminator D, and max(*) represents the maximum value.

[0030] In one embodiment of the present disclosure, the training of the semi-supervised generative adversarial network based on the pseudo sample, the training set and the loss function to obtain the trained semi-supervised generative adversarial network includes:

[0031] Adding the generated pseudo samples to the training set to obtain a new training set;

[0032] The semi-supervised generative adversarial network is trained using the training set and the loss function using a gradient descent algorithm.

[0033] A third aspect of the present disclosure provides an abnormal transaction detection method, comprising:

[0034] Get the user's transaction data;

[0035] The transaction data of the user is input into a trained semi-supervised generative adversarial network to obtain a result of whether the transaction data is abnormal data.

[0036] The trained semi-supervised generative adversarial network is obtained by obtaining a training set, inputting the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples, and training the semi-supervised generative adversarial network based on the pseudo samples, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0037] A fourth aspect of the present disclosure provides a device for processing data category imbalance, comprising:

[0038] Clustering module, used to cluster minority samples to obtain multiple clusters;

[0039] The first calculation module is used to calculate the k nearest neighbor samples of the minority class samples in each cluster;

[0040] A first acquisition module is used to obtain the number of majority class samples in the k nearest neighbor samples of the minority class samples in each of the clusters;

[0041] A second calculation module is used to calculate the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples;

[0042] The processing module is used to process the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0043] In one embodiment of the present disclosure, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, the processing module includes:

[0044] An acquisition submodule, used to acquire any minority class sample in the cluster;

[0045] A first selection submodule is used to select any sample that is k nearest neighbors to the minority class sample;

[0046] The first interpolation module is used to perform interpolation based on the minority class sample and any one of the k nearest neighbors of the minority class sample.

[0047] In one embodiment of the present disclosure, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, the processing module includes:

[0048] A calculation submodule, used for calculating the center point of the minority class samples in the cluster;

[0049] The second selection submodule is used to select any sample that is k nearest to the center point;

[0050] The second interpolation module is used to interpolate the minority class samples in the cluster based on the central point and any one of the k nearest neighbors to the central point.

[0051] In one embodiment of the present disclosure, the device for processing data type imbalance further includes:

[0052] The data set acquisition module is used to acquire a real data set, where the real data set includes majority class samples and minority class samples.

[0053] In one embodiment of the present disclosure, the device for processing data type imbalance further includes:

[0054] The standardization processing module is used to perform standardization processing on all samples in each cluster with a value range of (0, 1).

[0055] A fifth aspect of the present disclosure provides a training device for a semi-supervised generative adversarial network, comprising:

[0056] A second acquisition module is used to acquire a training set, wherein the training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples;

[0057] A first input module, used to input the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples;

[0058] A training module is used to train the semi-supervised generative adversarial network based on the pseudo sample, the training set and the loss function to obtain a trained semi-supervised generative adversarial network.

[0059] In one embodiment of the present disclosure, the loss function of the generator in the semi-supervised generative adversarial network is:

[0060] max(E z～p(z) (log G(D(Z))));

[0061] Among them, Z represents a pseudo sample, E z～p(z) (log G(D(Z))) represents the expected probability that the pseudo sample Z generated by the generator G is discriminated as a minority class sample by the discriminator D, and max(*) represents the maximum value.

[0062] In one embodiment of the present disclosure, the training module includes:

[0063] Add a submodule to add the generated pseudo samples to the training set to obtain a new training set;

[0064] The training submodule is used to train the semi-supervised generative adversarial network using the training set and the loss function by adopting a gradient descent algorithm.

[0065] A sixth aspect of the present disclosure provides an abnormal transaction detection device, comprising:

[0066] The third acquisition module is used to obtain the user's transaction data;

[0067] The second input module is used to input the transaction data of the user into the trained semi-supervised generative adversarial network to obtain a result of whether the transaction data is abnormal data.

[0068] The trained semi-supervised generative adversarial network is obtained by obtaining a training set, inputting the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples, and training the semi-supervised generative adversarial network based on the pseudo samples, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0069] A seventh aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the method described above when executed.

[0070] An eighth aspect of the present disclosure provides a computer program, which includes computer executable instructions, and the instructions are used to implement the method as described above when executed.

[0071] According to the embodiment of the present disclosure, for minority class samples, the minority class samples are clustered to obtain multiple clusters, the k nearest neighbor samples of the minority class samples in each cluster are calculated, the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster is obtained, the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is calculated, and the samples in the cluster are processed according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples. According to the number of majority class samples in the cluster, different methods are used to process the samples in the cluster to reduce the error of the newly generated minority class samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0073] Figure 1 Schematically illustrates an exemplary system architecture according to an embodiment of the present disclosure;

[0074] Figure 2 A flowchart schematically shows a method for processing data category imbalance according to an embodiment of the present disclosure;

[0075] Figure 3 A schematic diagram schematically shows a flow chart of a training method for a semi-supervised generative adversarial network according to an embodiment of the present disclosure;

[0076] Figure 4 The following is a schematic diagram showing a flow chart of an abnormal transaction detection method according to an embodiment of the present disclosure;

[0077] Figure 5 A block diagram schematically shows a device for processing data category imbalance according to an embodiment of the present disclosure;

[0078] Figure 6 A block diagram schematically shows a training device for a semi-supervised generative adversarial network according to an embodiment of the present disclosure;

[0079] Figure 7 A block diagram schematically shows a training device for a semi-supervised generative adversarial network according to an embodiment of the present disclosure;

[0080] Figure 8 The block diagram of a computer system according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0081] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0082] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0083] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0084] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0085] An embodiment of the present disclosure provides a method for processing data category imbalance, including: clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0086] An embodiment of the present disclosure provides a training method for a semi-supervised generative adversarial network, including: obtaining a training set, wherein the training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples, inputting the training samples in the training set into a semi-supervised generative adversarial network to obtain pseudo samples, training the semi-supervised generative adversarial network based on the pseudo samples, the training set and a loss function to obtain a trained semi-supervised generative adversarial network.

[0087] The embodiment of the present disclosure provides an abnormal transaction detection method, including obtaining transaction data of a user, inputting the transaction data of the user into a trained semi-supervised generative adversarial network, and obtaining a result of whether the transaction data is abnormal data. The trained semi-supervised generative adversarial network obtains a training set, inputs the training samples in the training set into the semi-supervised generative adversarial network, obtains a pseudo sample, and trains the semi-supervised generative adversarial network based on the pseudo sample, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0088] Figure 1 The exemplary system architecture 100 according to the embodiment of the present disclosure is schematically shown. It should be noted that: Figure 1 What is shown is merely an example of a system architecture to which the embodiments of the present disclosure can be applied, in order to help those skilled in the art understand the technical content of the present disclosure, but it does not mean that the embodiments of the present disclosure cannot be used in other devices, systems, environments or scenarios.

[0089] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0090] Users can use terminal devices 101, 102, 103 to interact with server 105 through network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients and / or social platform software, etc. (only as examples).

[0091] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0092] The server 105 may be a server that provides various services, such as a background management server (only an example) that provides support for websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process the received data such as user requests, and feed back the processing results (such as whether the user transaction data is abnormal data) to the terminal device.

[0093] It should be noted that the method provided in the embodiment of the present disclosure can generally be executed by the server 105. Accordingly, the device corresponding to the method provided in the embodiment of the present disclosure can generally be set in the server 105. The method provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Accordingly, the device corresponding to the method provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal device 101, 102, 103 and / or the server 105. Alternatively, the method provided in the embodiment of the present disclosure can also be executed by the terminal device 101, 102, or 103, or it can also be executed by other terminal devices different from the terminal device 101, 102, or 103. Accordingly, the device corresponding to the method provided in the embodiment of the present disclosure can also be set in the terminal device 101, 102, or 103, or it can be set in other terminal devices different from the terminal device 101, 102, or 103.

[0094] For example, minority class samples, training sets, and user transaction data may be originally stored in any one of the terminal devices 101, 102, or 103 (for example, the terminal device 101, but not limited thereto), or stored on an external storage device and may be imported into the terminal device 101. Then, the terminal device 101 may locally execute the data category imbalance processing method, the semi-supervised generative adversarial network training method, and the abnormal transaction detection method provided in the embodiment of the present disclosure, or send the minority class samples, training sets, and user transaction data to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the minority class samples, training sets, and user transaction data may execute the method provided in the embodiment of the present disclosure.

[0095] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0096] Figure 2 The flowchart of the method for processing data category imbalance according to an embodiment of the present disclosure is schematically shown.

[0097] like Figure 2 As shown, the method includes operations S201 to S205.

[0098] In operation S201, minority class samples are clustered to obtain multiple clusters.

[0099] In operation S202 , k nearest neighbor samples of the minority class samples in each cluster are calculated.

[0100] In operation S203, the number of majority class samples in the k nearest neighbor samples of the minority class samples in each of the clusters is obtained.

[0101] In operation S204, the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is calculated.

[0102] In operation S205 , samples in the cluster are processed according to the ratio of the number of majority class samples to the number of k nearest neighbor samples in the cluster.

[0103] In the embodiments of the present disclosure, the number of minority class samples is much smaller than the number of majority class samples, and the minority class samples and the majority class samples are from a real data set, that is, the real data set includes the majority class samples and the minority class samples. The real data set can be data generated in various actual abnormal scenarios, such as credit card fraud in the financial field, irrational consumption of minors, etc. In addition, the present disclosure can also be applied to other fields, and the above examples shall not be understood as limiting the present disclosure.

[0104] In the embodiment of the present disclosure, after operation S201, all samples in each cluster may be normalized to a value range of (0, 1) so as to perform interpolation and expand the number of minority class samples.

[0105] In an embodiment of the present disclosure, in a real data set, the number of processed minority class samples and the number of majority class samples can be balanced, and the balanced real data set can be used to train a generative adversarial network, or other networks evolved from a generative adversarial network, such as a semi-supervised generative adversarial network, a cycle-consistent generative adversarial network, and the like.

[0106] According to the embodiment of the present disclosure, for minority class samples, the minority class samples are clustered to obtain multiple clusters, the k nearest neighbor samples of the minority class samples in each cluster are calculated, the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster is obtained, the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is calculated, and the samples in the cluster are processed according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples. According to the number of majority class samples in the cluster, different methods are used to process the samples in the cluster to reduce the error of the newly generated minority class samples.

[0107] In the embodiment of the present disclosure, the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples is divided into three cases. In the first case, the number of majority samples in the cluster accounts for the number of k nearest neighbor samples is not greater than the first preset threshold, in the second case, the number of majority samples in the cluster accounts for the number of k nearest neighbor samples is greater than the first preset threshold and less than the second preset threshold, and in the third case, the number of majority samples in the cluster accounts for the number of k nearest neighbor samples equal to the second preset threshold.

[0108] In one of the embodiments of the present disclosure, for the first case, operation S205, according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples, the samples in the cluster are processed specifically as follows: obtaining any minority class sample in the cluster, selecting any minority class sample adjacent to the minority class sample, and performing interpolation based on the minority class sample and the minority class samples adjacent to the minority class sample.

[0109] In one of the embodiments of the present disclosure, for the second case, operation S205, according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples, the samples in the cluster are processed specifically as follows: the center point of the minority class samples in the cluster is calculated, any sample adjacent to the center point is selected, and the minority class samples in the cluster are interpolated based on the center point and the samples adjacent to the center point.

[0110] Among them, for the third case, it means that all samples in this cluster are majority class samples, then this cluster is a noise cluster and will not be processed.

[0111] In the present disclosure, those skilled in the art can set the specific values ​​of the first preset threshold and the second preset threshold according to actual conditions, and the present disclosure does not impose any restrictions on this. The following is a schematic illustration using the ratio of the number of majority class samples in the cluster to the total number of samples as B, the first preset threshold as 1 / 2, and the second preset threshold as 1.

[0112] In one example, when B=1, it means that all samples in this cluster are majority class samples, and the cluster is a noise cluster and does not need to be expanded. When B is not equal to 1, it means that there are samples in this cluster that are minority class samples, and the cluster is a safe cluster or a boundary cluster, and minority class sample expansion is required.

[0113] Specifically, when B≤1 / 2, it means that the number of majority class samples in this cluster is not greater than the number of minority class samples, and the cluster is a safe cluster that is not at the boundary of positive and negative samples. If a new sample is generated, the probability that the new sample is near the boundary of the two categories is small. Select any sample of the k nearest neighbors (k is 11 in the experiment) around the minority class sample in the cluster for interpolation, which can alleviate the boundary fuzziness to a certain extent. When 1>B>1 / 2, it means that the number of majority class samples in this cluster is greater than the number of minority class samples, and the cluster is a boundary cluster. If a new sample is generated, the probability that the new sample is near the boundary of the two categories is large. First, the mean of all minority class samples is calculated to obtain the center of the minority class sample set, and then the samples of the k nearest neighbors around the center of the minority class sample in the cluster are interpolated, so that the newly generated minority class samples are kept similar to the existing minority class samples and are far away from the boundary between the majority class samples and the minority class samples.

[0114] Figure 3 The flowchart of the training method of the semi-supervised generative adversarial network according to the embodiment of the present disclosure is schematically shown.

[0115] like Figure 3 As shown, the method includes operations S301 to S303.

[0116] In operation S301, a training set is obtained, and the training samples in the training set are obtained by clustering the minority class samples, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0117] In operation S302, the training samples in the training set are input into a semi-supervised generative adversarial network to obtain pseudo samples.

[0118] In operation S303, the semi-supervised generative adversarial network is trained based on the pseudo sample, the training set, and the loss function to obtain a trained semi-supervised generative adversarial network.

[0119] In the disclosed embodiment, a generative adversarial network (GAN) is an algorithm model based on the idea of ​​zero-sum game, which includes a generator G (Generator) and a discriminator D (Discriminator). The generator randomly samples from the latent space as input and generates pseudo samples close to real samples as output. The input of the discriminator is a real sample or a pseudo sample output by the generator, and the discriminator needs to identify the pseudo sample output by the generator as much as possible. The generator interferes with the discriminator by generating pseudo samples close to real samples, and also improves the discriminator's ability to recognize real samples. The two eventually reach a Nash equilibrium through competition and learning.

[0120] In the disclosed embodiments, a semi-supervised generative adversarial network (SSGAN) can utilize samples generated by the generator in the GAN to improve and enhance sample classification tasks. While the discriminator can distinguish between real samples and pseudo samples generated by the generator, it can also perform the sample classification tasks required by the classifier, thereby improving the discriminative ability of the discriminator.

[0121] In the embodiment of the present disclosure, based on the ratio of the number of majority class samples to the number of k nearest neighbor samples in the cluster, processing the samples in the cluster may specifically be: when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster, selecting any sample that is the k nearest neighbor of the minority class sample, and performing interpolation based on the minority class sample and any sample that is the k nearest neighbor of the minority class sample; when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than the first preset threshold and less than the second preset threshold, calculating the center point of the minority class samples in the cluster, selecting any sample that is the k nearest neighbor of the center point, and interpolating the minority class samples in the cluster based on the center point and any sample that is the k nearest neighbor of the center point.

[0122] In an embodiment of the present disclosure, the training samples in the training set input to the generator are pseudo minority class samples after interpolation and expansion, that is, the set of original minority class samples and interpolated and expanded minority class samples. The pseudo samples generated by the generator during the training process are uncertain, and may generate minority class samples or multi-class samples. The purpose of training the generator is only to generate pseudo samples that make the judgment true. However, the interpolated and expanded minority class samples in the training set are closely related to the original minority class samples and have a high similarity, which can easily lead to overfitting when training the classifier, affecting the classification effect of the classifier. In one embodiment of the present disclosure, the loss function of the generator in the semi-supervised generative adversarial network is set: max(E z～p(z) (log G(D(Z)))), where Z represents a pseudo sample, Ez～p(z) (log G(D(Z))) represents the expected probability that the pseudo sample Z generated by the generator G is judged as a minority class sample by the discriminator D, and max(*) represents the maximum value. This can make the generated pseudo sample closer to the minority class sample.

[0123] In one embodiment of the present disclosure, operation S303 specifically adds the generated pseudo sample to the training set to obtain a new training set, and uses the training set and the loss function to train the semi-supervised generative adversarial network using a gradient descent algorithm. In one example, the judge is set to a fully connected deep neural network, which is divided into an input layer, five hidden layers and an output layer. The gradient descent algorithm is used for training, and the learning rate of the semi-supervised generative adversarial network is set to 0.001, and the learning rate of the judge is set to 0.01.

[0124] Figure 4 The flowchart of the abnormal transaction detection method according to the embodiment of the present disclosure is schematically shown.

[0125] like Figure 4 As shown, the method includes operations S401 to S403.

[0126] In operation S401, the transaction data of the user is obtained;

[0127] In operation S402, the transaction data of the user is input into a trained semi-supervised generative adversarial network to obtain a result of whether the transaction data is abnormal data.

[0128] Among them, the trained semi-supervised generative adversarial network obtains a training set, inputs the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples, and trains the semi-supervised generative adversarial network based on the pseudo samples, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0129] In the embodiment of the present disclosure, based on the ratio of the number of majority class samples to the number of k nearest neighbor samples in the cluster, processing the samples in the cluster may specifically be: when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster, selecting any sample that is the k nearest neighbor of the minority class sample, and interpolating based on the minority class sample and any sample that is the k nearest neighbor of the minority class sample; when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, calculating the center point of the minority class samples in the cluster, selecting any sample that is the k nearest neighbor of the center point, and interpolating the minority class samples in the cluster based on the center point and any sample that is the k nearest neighbor of the center point.

[0130] In the embodiment of the present disclosure, the loss function of the generator in the semi-supervised generative adversarial network is: max(E z～p(z) (log G(D(Z)))), where Z represents a pseudo sample, E z～p(z) (log G(D(Z))) represents the expected probability that the pseudo sample Z generated by the generator G is judged as a minority class sample by the discriminator D, and max(*) represents the maximum value. This can make the generated pseudo sample closer to the minority class sample.

[0131] In the embodiment of the present disclosure, the result of whether the transaction data output by the semi-supervised generative adversarial network is abnormal data can be a probability value or a definite result, and the present disclosure does not impose any restrictions on this.

[0132] Figure 5 The block diagram schematically shows a device for processing data category imbalance according to an embodiment of the present disclosure.

[0133] like Figure 5 As shown, the device 500 for processing data category imbalance includes a clustering module 510 , a first calculation module 520 , a first acquisition module 530 , a second calculation module 540 and a processing module 550 .

[0134] The clustering module 510 is used to cluster the minority class samples to obtain multiple clusters.

[0135] The first calculation module 520 is used to calculate the k nearest neighbor samples of the minority class samples in each cluster.

[0136] The first acquisition module 530 is used to obtain the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster.

[0137] The second calculation module 540 is used to calculate the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples.

[0138] The processing module 550 is used to process the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0139] In one embodiment of the present disclosure, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, the processing module 540 includes:

[0140] The acquisition submodule is used to obtain any minority class sample in the cluster;

[0141] The first selection submodule is used to select any sample that is k nearest neighbors to the minority class sample;

[0142] The first interpolation module is used to perform interpolation based on the minority class sample and any one of the k nearest neighbors of the minority class sample.

[0143] In one embodiment of the present disclosure, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, the processing module 540 includes:

[0144] The calculation submodule is used to calculate the center point of the minority class samples in the cluster;

[0145] The second selection submodule is used to select any sample that is k nearest to the center point;

[0146] The second interpolation module is used to interpolate the minority class samples in the cluster based on the center point and any one of the k nearest neighbors of the center point.

[0147] In one embodiment of the present disclosure, the device 500 for processing data type imbalance further includes:

[0148] The data set acquisition module is used to acquire a real data set, where the real data set includes majority class samples and minority class samples.

[0149] In one embodiment of the present disclosure, the device 500 for processing data type imbalance further includes:

[0150] The standardization processing module is used to perform standardization processing on all samples in each cluster with a value range of (0, 1).

[0151] Figure 6 A block diagram of a training device for a semi-supervised generative adversarial network according to an embodiment of the present disclosure is schematically shown.

[0152] like Figure 6 As shown, the training device 600 of the semi-supervised generative adversarial network includes a second acquisition module 601, a first input module 602, and a training module 603.

[0153] The second acquisition module 601 is used to acquire a training set, wherein the training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples;

[0154] A first input module 602 is used to input the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples;

[0155] The training module 603 is used to train the semi-supervised generative adversarial network based on the pseudo sample, the training set and the loss function to obtain a trained semi-supervised generative adversarial network.

[0156] Among them, according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples, processing the samples in the cluster can specifically be: when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster, selecting any sample that is the k nearest neighbor of the minority class sample, and interpolating based on the minority class sample and the minority class samples that are the k nearest neighbors of the minority class sample; when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than the first preset threshold and less than the second preset threshold, calculating the center point of the minority class samples in the cluster, selecting any sample that is the k nearest neighbor of the center point, and interpolating the minority class samples in the cluster based on the center point and any sample that is the k nearest neighbor of the center point.

[0157] In one embodiment of the present disclosure, the loss function of the generator in the semi-supervised generative adversarial network is:

[0158] max(E z～p(z) (log G(D(Z))));

[0159] Among them, Z represents a pseudo sample, E z～p(z) (log G(D(Z))) represents the expected probability that the pseudo sample Z generated by the generator G is discriminated as a minority class sample by the discriminator D, and max(*) represents the maximum value.

[0160] In one embodiment of the present disclosure, the training module 603 includes:

[0161] Add a submodule to add the generated pseudo samples to the training set to obtain a new training set;

[0162] The training submodule is used to train the semi-supervised generative adversarial network using the training set and the loss function by adopting a gradient descent algorithm.

[0163] Figure 7 A block diagram of a training device for a semi-supervised generative adversarial network according to an embodiment of the present disclosure is schematically shown.

[0164] like Figure 7 As shown, the abnormal transaction detection device includes a third acquisition module 701 and a second input module 702.

[0165] The third acquisition module 701 is used to acquire the transaction data of the user.

[0166] The second input module 702 is used to input the transaction data of the user into the trained semi-supervised generative adversarial network to obtain a result of whether the transaction data is abnormal data.

[0167] Among them, the trained semi-supervised generative adversarial network obtains a training set, inputs the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples, and trains the semi-supervised generative adversarial network based on the pseudo samples, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of each minority class sample in the cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples.

[0168] Among them, according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples, processing the samples in the cluster can specifically be: when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster, selecting any sample that is the k nearest neighbor of the minority class sample, and interpolating based on the minority class sample and any sample that is the k nearest neighbor of the minority class sample; when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than the first preset threshold and less than the second preset threshold, calculating the center point of the minority class samples in the cluster, selecting any sample that is the k nearest neighbor of the center point, and interpolating the minority class samples in the cluster based on the center point and any sample that is the k nearest neighbor of the center point.

[0169] Among them, the loss function of the generator in the semi-supervised generative adversarial network is: max(E z～p(z) (log G(D(Z)))), Z represents a pseudo sample, E z～p(z)(log G(D(Z))) represents the expected probability that the pseudo sample Z generated by the generator G is discriminated as a minority class sample by the discriminator D, and max(*) represents the maximum value.

[0170] According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, submodules, units, and subunits can be at least partially implemented as hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), systems on chips, systems on substrates, systems on packages, application specific integrated circuits (ASICs), or can be implemented by hardware or firmware in any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in any appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, submodules, units, and subunits can be at least partially implemented as computer program modules, and when the computer program modules are run, the corresponding functions can be performed.

[0171] For example, any of the clustering module 510, the first computing module 520, the first acquisition module 530, the second computing module 540, and the processing module 550 can be combined in one module / unit / subunit for implementation, or any of the modules / units / subunits can be split into multiple modules / units / subunits. Alternatively, at least part of the functions of one or more of these modules / units / subunits can be combined with at least part of the functions of other modules / units / subunits and implemented in one module / unit / subunit. According to an embodiment of the present disclosure, at least one of the clustering module 510, the first computing module 520, the first acquisition module 530, the second computing module 540, and the processing module 550 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation methods of software, hardware, and firmware, or by a suitable combination of any of them. Alternatively, at least one of the clustering module 510, the first calculation module 520, the first acquisition module 530, the second calculation module 540 and the processing module 550 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0173] Figure 8 A block diagram of a computer system suitable for implementing the method described above according to an embodiment of the present disclosure is schematically shown. Figure 8 The computer system shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0174] like Figure 8 As shown, the computer system 800 according to the embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 802 or the program loaded from the storage part 808 to the random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to the embodiment of the present disclosure.

[0175] In RAM 803, various programs and data required for the operation of system 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other through bus 804. Processor 801 performs various operations of the method flow according to the embodiment of the present disclosure by executing the program in ROM 802 and / or RAM 803. It should be noted that the program can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also perform various operations of the method flow according to the embodiment of the present disclosure by executing the program stored in the one or more memories.

[0176] According to an embodiment of the present disclosure, the system 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The system 800 may further include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, a mouse, etc.; an output section 807 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, a modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed, so that a computer program read therefrom is installed into the storage section 808 as needed.

[0177] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program contains a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0178] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present disclosure is implemented.

[0179] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, apparatus, or device.

[0180] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 802 and / or the RAM 803 described above and / or one or more memories other than the ROM 802 and the RAM 803 .

[0181] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0182] It will be appreciated by those skilled in the art that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations and / or combinations are not explicitly described in the present disclosure. In particular, the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways without departing from the spirit and teachings of the present disclosure. All of these combinations and / or combinations fall within the scope of the present disclosure.

[0183] The embodiments of the present disclosure are described above. However, these embodiments are only for illustrative purposes and are not intended to limit the scope of the present disclosure. Although the embodiments are described above separately, this does not mean that the measures in the various embodiments cannot be used in combination to advantage. The scope of the present disclosure is defined by the attached claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make a variety of substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A method for processing data category imbalance, comprising: Cluster the minority class samples to obtain multiple clusters; Calculate the k nearest neighbor samples of the minority class samples in each cluster; Obtain the number of majority class samples in the k nearest neighbor samples of the minority class samples in each of the clusters; Calculate the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples; Processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples; specifically comprising: When the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster; Selecting any one of the k nearest neighbors of the minority class sample; performing interpolation based on the minority class sample and any one of the k nearest neighbors of the minority class sample; When the ratio of the number of majority samples in the cluster to the total number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, the center point of the minority class samples in the cluster is calculated; any one of the samples that are k nearest neighbors to the center point is selected; and based on the center point and any one of the samples that are k nearest neighbors to the center point, the minority class samples in the cluster are interpolated.

2. The method according to claim 1, before clustering the minority class samples to obtain multiple clusters, comprising: A real data set is obtained, where the real data set includes majority class samples and minority class samples.

3. The method according to claim 1, after clustering the minority class samples to obtain a plurality of clusters, further comprises: All samples in each cluster are normalized to a value range of (0, 1).

4. A training method for a semi-supervised generative adversarial network, comprising: Obtain a training set, wherein the training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples; The processing of the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples comprises: When the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster; selecting any sample that is a k nearest neighbor of the minority class sample; and performing interpolation based on the minority class sample and any sample that is a k nearest neighbor of the minority class sample; When the ratio of the number of majority samples in the cluster to the total number of k-nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, calculate the center point of the minority class samples in the cluster; select any one of the k-nearest neighbors of the center point; interpolate the minority class samples in the cluster based on the center point and any one of the k-nearest neighbors of the center point; Inputting the training samples in the training set into a semi-supervised generative adversarial network to obtain pseudo samples; The semi-supervised generative adversarial network is trained based on the pseudo sample, the training set and the loss function to obtain a trained semi-supervised generative adversarial network.

5. According to the method of claim 4, the loss function of the generator in the semi-supervised generative adversarial network is: ； in, Z represents a pseudo sample, It represents the expected probability that the pseudo sample Z generated by the generator G is judged by the discriminator D as a minority class sample, Indicates taking the maximum value.

6. According to the method of claim 4 or 5, the training of the semi-supervised generative adversarial network based on the pseudo sample, the training set and the loss function to obtain a trained semi-supervised generative adversarial network comprises: Adding the generated pseudo samples to the training set to obtain a new training set; The semi-supervised generative adversarial network is trained using the training set and the loss function using a gradient descent algorithm.

7. A method for detecting abnormal transactions, comprising: Get the user's transaction data; Inputting the transaction data of the user into a trained semi-supervised generative adversarial network to obtain a result of whether the transaction data is abnormal data; The trained semi-supervised generative adversarial network is obtained by obtaining a training set, inputting the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples, and training the semi-supervised generative adversarial network based on the pseudo samples, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples. The processing of the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples comprises: When the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster; selecting any sample that is a k nearest neighbor of the minority class sample; and performing interpolation based on the minority class sample and any sample that is a k nearest neighbor of the minority class sample; When the ratio of the number of majority samples in the cluster to the total number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, the center point of the minority class samples in the cluster is calculated; any one of the samples that are k nearest neighbors to the center point is selected; and based on the center point and any one of the samples that are k nearest neighbors to the center point, the minority class samples in the cluster are interpolated.

8. A device for processing data type imbalance, comprising: Clustering module, used to cluster minority samples to obtain multiple clusters; The first calculation module is used to calculate the k nearest neighbor samples of the minority class samples in each cluster; A first acquisition module is used to obtain the number of majority class samples in the k nearest neighbor samples of the minority class samples in each of the clusters; A second calculation module is used to calculate the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples; A processing module, used to process the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples; specifically comprising: when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster; Select any one of the k-nearest samples of the minority class; interpolate based on the minority class sample and any one of the k-nearest samples of the minority class sample; when the ratio of the number of majority samples in the cluster to the total number of k-nearest samples is greater than a first preset threshold and less than a second preset threshold, calculate the center point of the minority class samples in the cluster; select any one of the k-nearest samples of the center point; interpolate the minority class samples in the cluster based on the center point and any one of the k-nearest samples of the center point.

9. The device according to claim 8, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, the processing module comprises: An acquisition submodule, used to acquire any minority class sample in the cluster; A first selection submodule is used to select any sample that is k nearest neighbors to the minority class sample; The first interpolation module is used to perform interpolation based on the minority class sample and any one of the k nearest neighbors of the minority class sample.

10. The method according to claim 1, when the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, the processing module comprises: A calculation submodule, used for calculating the center point of the minority class samples in the cluster; The second selection submodule is used to select any sample that is k nearest to the center point; The second interpolation module is used to interpolate the minority class samples in the cluster based on the central point and the samples of any one of the k nearest neighbors to the central point.

11. A training device for a semi-supervised generative adversarial network, comprising: A second acquisition module is used to acquire a training set, wherein the training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples; The processing of the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples comprises: When the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster; selecting any sample that is a k nearest neighbor of the minority class sample; and performing interpolation based on the minority class sample and any sample that is a k nearest neighbor of the minority class sample; When the ratio of the number of majority samples in the cluster to the total number of k-nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, calculate the center point of the minority class samples in the cluster; select any one of the k-nearest neighbors of the center point; interpolate the minority class samples in the cluster based on the center point and any one of the k-nearest neighbors of the center point; A first input module, used to input the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples; A training module is used to train the semi-supervised generative adversarial network based on the pseudo sample, the training set and the loss function to obtain a trained semi-supervised generative adversarial network.

12. The apparatus according to claim 11, wherein the loss function of the generator in the semi-supervised generative adversarial network is: ； in, Z represents a pseudo sample, It represents the expected probability that the pseudo sample Z generated by the generator G is judged by the discriminator D as a minority class sample, Indicates taking the maximum value.

13. An abnormal transaction detection device, comprising: The third acquisition module is used to obtain the user's transaction data; A second input module is used to input the transaction data of the user into the trained semi-supervised generative adversarial network to obtain a result of whether the transaction data is abnormal data; The trained semi-supervised generative adversarial network is obtained by obtaining a training set, inputting the training samples in the training set into the semi-supervised generative adversarial network to obtain pseudo samples, and training the semi-supervised generative adversarial network based on the pseudo samples, the training set and the loss function. The training samples in the training set are obtained by clustering minority class samples to obtain multiple clusters, calculating the k nearest neighbor samples of the minority class samples in each cluster, obtaining the number of majority class samples in the k nearest neighbor samples of the minority class samples in each cluster, calculating the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples, and processing the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples. The processing of the samples in the cluster according to the ratio of the number of majority class samples in the cluster to the number of k nearest neighbor samples comprises: When the ratio of the number of majority samples in the cluster to the number of k nearest neighbor samples is not greater than a first preset threshold, obtaining any minority class sample in the cluster; selecting any sample that is a k nearest neighbor of the minority class sample; and performing interpolation based on the minority class sample and any sample that is a k nearest neighbor of the minority class sample; When the ratio of the number of majority samples in the cluster to the total number of k nearest neighbor samples is greater than a first preset threshold and less than a second preset threshold, the center point of the minority class samples in the cluster is calculated; any one of the samples that are k nearest neighbors to the center point is selected; and based on the center point and any one of the samples that are k nearest neighbors to the center point, the minority class samples in the cluster are interpolated.

14. A computer system comprising: one or more processors; a memory for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any one of claims 1 to 3, or implement the method described in any one of claims 4 to 6, or implement the method described in claim 7.

15. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method described in any one of claims 1 to 3, or the method described in any one of claims 4 to 6, or the method described in claim 7.

Images