Data detection method and device, equipment and storage medium

Abstract

The invention provides a data detection method and device, equipment and a storage medium, the method can be applied to the technical fields of artificial intelligence, security detection and the like, and the method comprises the following steps: performing data detection on obtained to-be-detected target data through N data detection models to obtain N prediction probabilities that the target data belongs to target type data; determining detection thresholds corresponding to the N data detection models; and determining a detection result of the target data based on the N prediction probabilities and the detection threshold, the N data detection models being obtained by training based on a first training sample and a second training sample, and the second training sample being generated by the generative adversarial network based on the first training sample. According to the method, the N data detection models are used for data detection, so that the accuracy and stability of data detection can be improved. In addition, new training samples are generated by using the generative adversarial network, training data are increased, and the data detection accuracy and stability of the N data detection models are further improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data detection method, device, equipment and storage medium. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, AI technology has been used in many fields. For example, in some data detection scenarios, AI technology can be used to detect some fraudulent behaviors and protect the lives and property of users.

[0003] The current fraud detection method based on machine learning first trains a data detection model through training data, and then uses the data detection model to identify fraudulent behavior. However, the fraud detection method based on machine learning has low data detection accuracy and stability. Summary of the invention

[0004] The present application provides a data detection method, apparatus, device and storage medium, which can improve the accuracy and stability of data detection by performing data detection through N data detection models.

[0005] In a first aspect, the present application provides a data detection method, comprising:

[0006] Using N data detection models, respectively perform data detection on the acquired target data to be detected, and obtain N prediction probabilities that the target data belongs to the target type data, where N is a positive integer greater than 1;

[0007] Determine a detection threshold corresponding to the N data detection models, wherein the detection threshold is used to determine whether the target data is the target type data;

[0008] Determining a detection result of the target data based on the N prediction probabilities and the detection threshold;

[0009] The N data detection models are trained based on first training samples and second training samples, and the second training samples are generated by a generative adversarial network based on the first training samples.

[0010] In a second aspect, the present application provides a data detection device, comprising:

[0011] A prediction unit, configured to perform data detection on the acquired target data to be detected respectively through N data detection models, and obtain N prediction probabilities that the target data belongs to target type data, where N is a positive integer greater than 1;

[0012] A threshold determination unit, used to determine a detection threshold corresponding to the N data detection models, wherein the detection threshold is used to determine whether the target data is the target type data;

[0013] A detection unit, configured to determine a detection result of the target data based on the N prediction probabilities and the detection threshold;

[0014] The N data detection models are trained based on first training samples and second training samples, and the second training samples are generated by a generative adversarial network based on the first training samples.

[0015] In some embodiments, the detection unit is specifically used to determine the weights of the N data detection models; based on the weights of the N data detection models, the N prediction probabilities are weighted to obtain the weighted prediction probability of the target data; based on the weighted prediction probability and the detection threshold, the detection result is obtained.

[0016] In some embodiments, the training process of the N data detection models includes: processing the first training sample through the generative adversarial network to generate the second training sample; for any training sample between the first training sample and the second training sample, using the N data detection models to perform data detection on the training sample to obtain N prediction probabilities that the training sample belongs to the target type data; for each of the N data detection models, based on the prediction probability of the data detection model and the true label of the training sample, determining the loss of the data detection model, and updating the parameters in the data detection model based on the loss.

[0017] In some embodiments, the processing of the first training sample by the generative adversarial network to generate the second training sample is preceded by completing the training of the generative adversarial network, and the processing of the first training sample by the generative adversarial network to generate the second training sample includes: processing the first training sample by the trained generative adversarial network to generate the second training sample.

[0018] In some embodiments, the use of the N data detection models to perform data detection on the training samples to obtain N prediction probabilities that the training samples belong to the target type data includes: performing feature extraction on the training samples through a deep learning model to obtain feature information of the training samples; performing data detection on the feature information of the training samples through the N data detection models to obtain N prediction probabilities that the training samples belong to the target type data.

[0019] In some embodiments, the deep learning model is trained before extracting features from the training samples through the deep learning model to obtain feature information of the training samples. The training process of the deep learning model includes: extracting features from the third training sample through the feature extraction module in the deep learning model to obtain feature information of the third training sample; processing the feature information of the third training sample through the prediction module in the deep learning model to obtain the predicted probability of the third training sample; and training the deep learning model based on the predicted probability of the third training sample and the true label of the third training sample.

[0020] In some embodiments, extracting features from the training samples through a deep learning model to obtain feature information of the training samples includes: extracting features from the training samples through a feature extraction module in the trained deep learning model to obtain feature information of the training samples.

[0021] In some embodiments, the weights of the N data detection models are obtained by training together with the N data detection models, and the training process of the weights of the N data detection models includes: using the N data detection models obtained through the i-1th round of training, performing data detection on the i-th round training samples, and obtaining N prediction probabilities of the i-th round, where i is a positive integer; determining the loss of the N data detection models in the i-th round based on the N prediction probabilities of the i-th round, the weights of the N data detection models in the i-1th round, and the true labels of the i-th round training samples; based on the loss of the i-th round, updating the weights of the i-1th round to obtain the weights of the N data detection models in the i-th round.

[0022] In some embodiments, determining the loss of the N data detection models in the i-th round based on the N prediction probabilities of the i-th round, the weights of the N data detection models in the i-1 round, and the true labels of the i-th round training samples includes: determining the weighted prediction probabilities of the i-th round training samples in the i-th round based on the N prediction probabilities of the i-th round and the weights of the i-1 round; determining the loss of the N data detection models in the i-th round based on the weighted prediction probabilities of the i-th round training samples in the i-th round and the true labels of the i-th round training samples.

[0023] In some embodiments, the detection threshold is obtained by training together with the N data detection models, and the training process of the detection threshold includes: performing data detection on the i-th round training samples through the N data detection models obtained through the i-1th round training to obtain N prediction probabilities of the i-th round, where i is a positive integer; determining the weighted prediction probability of the i-th round training samples based on the N prediction probabilities of the i-th round and the weights of the N data detection models in the i-1th round; updating the detection threshold of the i-1th round based on the weighted prediction probability of the i-th round training samples and the true labels of the i-th round training samples to obtain the detection threshold of the i-th round.

[0024] In some embodiments, the detection threshold of the i-1th round is updated based on the weighted prediction probability of the i-th round training samples and the true label of the i-th round training samples to obtain the detection threshold of the i-th round, including: determining the evaluation indicators of the N data detection models in the i-th round based on the weighted prediction probability of the i-th round training samples, the true label of the i-th round training samples and the i-1th round detection threshold; determining the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round.

[0025] In some embodiments, the evaluation index includes at least one of precision, recall, F1 value, and ROC curve.

[0026] In some embodiments, determining the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round includes: determining the threshold setting indicators of the i-th round based on the evaluation indicators of the i-th round; determining the detection threshold of the i-th round based on the threshold setting indicators of the i-th round.

[0027] In some embodiments, determining the threshold setting indicator of the i-th round based on the evaluation indicator of the i-th round includes: using the evaluation indicator of the i-th round as the input of a preset threshold adjustment function, and using the output of the threshold adjustment function as the threshold setting indicator of the i-th round.

[0028] In some embodiments, obtaining the detection threshold of the i-th round based on the threshold setting indicator of the i-th round includes: adjusting the detection threshold of the i-1-th round to determine the threshold corresponding to the time when the threshold setting indicator of the i-th round is the maximum as the detection threshold of the i-th round.

[0029] In some embodiments, after obtaining the detection threshold of the i-th round, the detection threshold of the i-th round is verified, and the verification process of the detection threshold of the i-th round includes: performing data detection on the test sample through the N data detection models obtained by the i-th round of training to obtain N prediction probabilities of the test sample; determining the weighted prediction probability of the test sample based on the N prediction probabilities of the test sample and the weights of the N data detection models in the i-th round; determining the evaluation index of the N data detection models obtained by the i-th round of training based on the weighted prediction probability of the test sample, the detection threshold of the i-th round and the true label of the test sample; determining the verification result of the detection threshold of the i-th round based on the evaluation index of the N data detection models obtained by the i-th round of training.

[0030] In some embodiments, if the verification result of the detection threshold of the i-th round is qualified, the updating of the N data detection models and the detection threshold is stopped.

[0031] In a third aspect, a computing device is provided, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method in the first aspect or its implementations.

[0032] In a fourth aspect, a chip is provided for implementing the method in any aspect or its implementation in the first aspect. Specifically, the chip includes: a processor for calling and running a computer program from a memory, so that a device equipped with the chip executes the method in any aspect or its implementation in the first aspect.

[0033] In a fifth aspect, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the method in any aspect of the first aspect or its various implementations.

[0034] In a sixth aspect, a computer program product is provided, comprising computer program instructions, wherein the computer program instructions enable a computer to execute the method in any aspect of the first aspect or its various implementations.

[0035] In a seventh aspect, a computer program is provided, which, when executed on a computer, enables the computer to execute the method in any aspect of the first aspect or its implementation.

[0036] In summary, the embodiment of the present application performs data detection on the acquired target data to be detected through N data detection models, and obtains N predicted probabilities that the target data belongs to the target type data. Then, the detection threshold corresponding to the N data detection models is determined, and the detection threshold is used to determine whether the target data is the target type data, and then based on the N predicted probabilities and the detection threshold, the detection result of the target data is determined, wherein the N data detection models are obtained based on the first training sample and the second training sample training, and the second training sample is generated by the generative adversarial network based on the first training sample. In other words, the embodiment of the present application uses N data detection models for data detection, which can improve the accuracy and stability of data detection. In addition, when the embodiment of the present application trains the N data detection models, a new training sample (i.e., the second training sample) is generated using a generative adversarial network, which increases the training data, thereby achieving full training of the N data detection models, and further improving the data detection accuracy and stability of the N data detection models. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 A schematic diagram of an implementation environment involved in an embodiment of the present application;

[0039] Figure 2 A flowchart of a model training method provided in one embodiment of the present application;

[0040] Figure 3 A schematic diagram of a generative adversarial network;

[0041] Figure 4 Schematic diagram of generating a second training sample for a generative adversarial network;

[0042] Figure 5 Schematic diagram of data processing for N data detection models;

[0043] Figure 6 Schematic diagram of data processing for deep learning models and N data detection models;

[0044] Figure 7 A schematic diagram of a deep learning model;

[0045] Figure 8 A flowchart of a model training method provided in one embodiment of the present application;

[0046] Fig. 9 A schematic diagram of a data detection method flow chart provided in an embodiment of the present application;

[0047] Fig.10 is a schematic block diagram of a data detection device provided in an embodiment of the present application;

[0048] Fig.11 It is a schematic block diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here can be implemented in a sequence other than those illustrated or described here. In an embodiment of the present invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, but can also determine B according to A and / or other information. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units does not have to be limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. In the description of the present application, unless otherwise specified, "multiple" refers to two or more than two.

[0051] The technical solution proposed in this application can be applied to technical fields such as artificial intelligence and security detection to improve the accuracy and stability of data detection.

[0052] The following is an introduction to the relevant concepts involved in the embodiments of the present application.

[0053] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0054] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. The basic technologies of artificial intelligence generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0055] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

[0056] With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

[0057] In an embodiment of the present application, artificial intelligence technology is applied to data detection to identify some target type data, such as identifying fraudulent behavior data.

[0058] Ensemble Learning: A machine learning paradigm that improves the accuracy or stability of predictions by combining the predictions of multiple models.

[0059] Generative Adversarial Networks (GAN): A deep learning model whose goal is to generate new data that is indistinguishable from real data.

[0060] Fraud Detection: A method used to identify fraudulent activities, including but not limited to credit card fraud, e-commerce fraud, etc.

[0061] Current data detection methods (such as fraud detection methods) mainly include rule-based methods, statistical methods, and machine learning-based methods. Among them, the rule-based method mainly identifies fraudulent behavior through pre-set rules, but this method requires a lot of manual participation and has limited ability to identify complex fraudulent behavior. The statistical method mainly identifies abnormal behavior through statistical analysis, but this method is not very accurate and stable in identifying fraudulent behavior. The machine learning-based method identifies fraudulent behavior by training a model, but this method has high requirements on data quality, and the detection accuracy and stability are not high.

[0062] In order to solve the above technical problems, the embodiment of the present application performs data detection on the acquired target data to be detected by N data detection models, and obtains N predicted probabilities that the target data belongs to the target type data. Then, the detection threshold corresponding to the N data detection models is determined, and the detection threshold is used to determine whether the target data is the target type data, and then based on the N predicted probabilities and the detection threshold, the detection result of the target data is determined, wherein the N data detection models are obtained based on the first training sample and the second training sample training, and the second training sample is generated by the generative adversarial network based on the first training sample. In other words, the embodiment of the present application uses N data detection models for data detection, which can improve the accuracy and stability of data detection. In addition, when the embodiment of the present application trains the N data detection models, a new training sample (i.e., the second training sample) is generated by the generative adversarial network, which increases the training data, and then realizes the full training of the N data detection models, further improving the data detection accuracy and stability of the N data detection models.

[0063] The implementation environment of the embodiments of the present application is introduced below.

[0064] Figure 1 This is a schematic diagram of an implementation environment involved in an embodiment of the present application, including a terminal device 101 and a computing device 102.

[0065] like Figure 1As shown, the computing device 102 of the embodiment of the present application includes a generative adversarial network and N data detection models. Among them, the terminal device 101 obtains a training set, such as a first training sample, and sends the training set to the computing device 102. The computing device 102 uses the training set and the model training method provided in the embodiment of the present application to train the N data detection models. It should be noted that the above-mentioned generative adversarial network is a trained generative adversarial network model.

[0066] Exemplarily, the computing device 102 obtains a trained generative adversarial network and N untrained data detection models. Then, the computing device 102 obtains a first training sample based on the instruction of the terminal device 101. Then, the computing device 102 processes the first training sample through the trained generative adversarial network to generate a second training sample. Then, the computing device 102 uses the first training sample and the second training sample to train the N data detection models. For example, for any training sample in the first training sample and the second training sample, the training sample is subjected to data detection using N data detection models to obtain N predicted probabilities that the training sample belongs to the target type of data; for each of the N data detection models, the loss of the data detection model is determined based on the predicted probability of the data detection model and the true label of the training sample, and the parameters in the data detection model are updated based on the loss. Repeating the above steps, the trained N data detection models can be obtained.

[0067] In some embodiments, Figure 1 As shown, the application scenario also includes a database 103, which includes historical data, such as historical fraud data. In an embodiment of the present application, the terminal device 101 is connected to the database 103 in communication, and data can be written to the database 103. The computing device 102 is also connected to the database 102 in communication, and data can be read from the database 103. In one example, during the model training process of an embodiment of the present application, the computing device 102 obtains historical data from the database 103 as a first training sample when training the model. Then, the computing device 102 uses a generative adversarial network to process the first training sample and generate a second training sample. Then, the first training sample and the second training sample are used to train N data detection models to obtain N trained data detection models, and the trained N data detection models are stored in the computing device 102.

[0068] In the embodiment of the present application, the data detection method can be implemented by the terminal device 101 or by the computing device 102.

[0069] In some embodiments, if the data detection method of the embodiment of the present application is executed by the terminal device 101, the terminal device 101 obtains the trained N data detection models from the computing device 102. Then, the terminal device 101 performs data detection on the target data to be detected respectively through the obtained N data detection models, and obtains N prediction probabilities that the target data belongs to the target data type. Then, the terminal device 101 determines the detection result of the target data based on the N prediction probabilities.

[0070] In some embodiments, if the data detection method of the present application is executed by the computing device 102, the terminal device 101 sends the acquired target data to be detected to the computing device 102. The computing device 102 uses the N data detection models stored by itself to perform data detection on the target data respectively, and obtains N prediction probabilities that the target data belongs to the target type data. Finally, the computing device 102 determines the detection result of the target data based on the N prediction probabilities.

[0071] The embodiments of the present application do not limit the specific type of the terminal device 101. In some embodiments, the terminal device 101 may include but is not limited to: a mobile phone, a computer, an intelligent voice interaction device, an intelligent home appliance, a vehicle terminal, an aircraft, a wearable intelligent device, a medical device, etc. The device is often equipped with a display device, which may also be a display, a display screen, a touch screen, etc., and the touch screen may also be a touch screen, a touch panel, etc.

[0072] In some embodiments, the computing device 102 is a terminal device with data processing capabilities, such as a mobile phone, a computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, an aircraft, a wearable smart device, a medical device, and the like.

[0073] In some embodiments, computing device 102 is a server. The server may be one or more. When there are multiple servers, there are at least two servers for providing different services, and / or there are at least two servers for providing the same service, such as providing the same service in a load balancing manner, which is not limited in the embodiments of the present application. Among them, the above-mentioned server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. Server 102 can also become a node of the blockchain.

[0074] In the embodiment of the present application, the terminal device 101 and the computing device 102 may be directly or indirectly connected via wired communication or wireless communication, and the present application does not impose any limitation thereto.

[0075] It should be noted that the application scenarios of the embodiments of the present application include but are not limited to Figure 1 shown.

[0076] The technical solutions of the embodiments of the present application are described in detail below through some embodiments. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0077] First, the model training process of the embodiment of the present application is introduced.

[0078] Figure 2 Schematic diagram of a flow chart of a model training method provided in an embodiment of the present application. The execution subject of the embodiment of the present application is a device having a training model function, such as a model training device. In some embodiments, the model training device can be Figure 1 A computing device in Figure 1 The terminal device in Figure 1 For the convenience of description, the embodiment of the present application is described by taking the execution subject as a computing device as an example.

[0079] Combine the following Figure 2 , the training process of N data detection models is introduced.

[0080] like Figure 2 As shown, the N data detection model training process of the embodiment of the present application includes:

[0081] S101. Process a first training sample by generating an adversarial network to generate a second training sample.

[0082] In the embodiment of the present application, N data detection models are trained synchronously. When the computing device trains the N data detection models, it first collects training data, and the collected training data includes multiple training samples. For the convenience of description, the collected training samples are recorded as the first training samples.

[0083] It should be noted that the training data used in the model training process of the embodiment of the present application and the acquisition of the target data to be detected in the data detection process have all been legal and compliant in the past.

[0084] In some embodiments, the computing device divides the collected training data, ie, the first sample, into positive samples and negative samples according to whether the collected training data is target type data.

[0085] For example, for each first sample in the collected training data, if the first sample is target type data (such as fraud data), the first sample is classified as a positive sample, and the label can be set to 1. If the first sample is not target type data (such as not fraud data), the first sample is classified as a negative sample, and the label can be set to 0.

[0086] In the embodiment of the present application, the number of first training samples collected (e.g., fraudulent data collected) is very small, so more sample data needs to be constructed. In the embodiment of the present application, the computing device uses a generative adversarial network (GAN) to generate a second training sample as close to the real data (i.e., the first training sample) as possible based on the first training sample, so as to provide sufficient data for training.

[0087] In an embodiment of the present application, the computing device processes the first training sample through a generative adversarial network, and completes the training of the generative adversarial network before generating the second training sample.

[0088] In some embodiments, the generative adversarial network is trained by other computing devices, and the computing device of the embodiment of the present application obtains the trained generative adversarial network. For example, the computing device of the embodiment of the present application obtains the trained generative adversarial network from other computing devices. For another example, other computing devices save the trained generative adversarial network in the cloud, and the computing device of the embodiment of the present application obtains the trained generative adversarial network from the cloud.

[0089] In some embodiments, the computing device of the embodiments of the present application trains a generative adversarial network to obtain a trained generative adversarial network.

[0090] The following is an introduction to the training process of the generative adversarial network. For ease of description, the computing device of the embodiment of the present application is used as an example for the introduction. The training process of the generative adversarial network by other computing devices is consistent with the training process of the generative adversarial network by the embodiment of the present application, and can be referred to.

[0091] like Figure 3 As shown in the figure, the Generative Adversarial Network consists of two parts, one is the Generator and the other is the Discriminator. The task of the Generator is to receive a random noise input and then generate a data sample. The task of the Discriminator is to receive a data sample as input and then determine whether the data sample is real data.

[0092] During the training process, the generator and the discriminator will compete with each other. Specifically, the generator will try to generate more and more realistic data in order to fool the discriminator, while the discriminator will try to distinguish the generated data from the real data better and better.

[0093] In the embodiment of the present application, the training process of the generative adversarial network includes the following steps 11 to 14:

[0094] Step 11: Initialize the parameters of the generator and discriminator.

[0095] Step 12: Input a random noise into the generator, and the generator generates a data based on the noise.

[0096] Step 13: Input the generated data and some real data into the discriminator together. The discriminator outputs a prediction value based on the input data, indicating the probability that the data is real.

[0097] Step 14: Calculate the loss function based on the discriminator's predicted value and the true label, and use the gradient descent method to update the parameters of the generator and discriminator.

[0098] The embodiments of the present application do not limit the types of specific loss functions of the generator and the discriminator.

[0099] In some embodiments, the objective function of the generator is as shown in the following formula (1):

[0100]

[0101] Among them, G represents the generator, D represents the discriminator, z represents the input random noise, p(z) represents the probability distribution of random noise, G(z) represents the data generated by the generator based on random noise, and D(G(z)) represents the probability that the discriminator judges the generated data as real data.

[0102] In some embodiments, the objective function of the discriminator is as shown in the following formula (2):

[0103]

[0104] Among them, x represents the real data, pdata(x) represents the probability distribution of the real data, and D(x) represents the probability that the discriminator judges the real data as the real data.

[0105] During the training process, the above two objective functions will be optimized alternately, so that the data generated by the generator becomes closer and closer to the real data, and the discriminator's ability to judge the real data and generated data becomes stronger and stronger.

[0106] The specific training steps are as follows:

[0107] First, fix the generator G and update the discriminator D. This step minimizes the objective function of the discriminator. To achieve this, that is, to make the discriminator D better distinguish between generated data and real data. This optimization problem can be solved by the gradient descent method.

[0108] For example, the update of the discriminator is shown in formula (3):

[0109]

[0110] Among them, θ D is the parameter of the discriminator, α is the learning rate, is the loss function For parameter θ D gradient.

[0111] Next, fix the discriminator D and update the generator G. This step minimizes the objective function of the generator To achieve this, that is, to make the data generated by the generator G more and more like real data. This optimization problem can also be solved by the gradient descent method.

[0112] For example, the update of the discriminator is shown in formula (4):

[0113]

[0114] Among them, θ G are the parameters of the generator, α is the learning rate, is the loss function For parameter θ G gradient.

[0115] Repeat steps 12 to 14 until the stopping condition is met, for example, until the data generated by the generator and the real data cannot be distinguished by the discriminator, or until a preset number of iterations is reached.

[0116] In the embodiment of the present application, by means of this strategy of alternately optimizing the generator and the discriminator, the generative adversarial network can generate new data that is closer and closer to the real data. This provides a large amount of relatively high-quality training data for data detection tasks (such as fraud detection tasks), thereby improving the accuracy and stability of the model.

[0117] After the computing device obtains the trained generative adversarial network, it processes the first training sample through the trained generative adversarial network to generate a second training sample. Figure 4 As shown, the first training sample with a smaller amount of data is input into the trained generative adversarial network for processing to generate a second training sample with a larger amount of data, thereby increasing the training data of the model.

[0118] After the computing device generates a plurality of second training samples through the above steps, it executes the following step S102.

[0119] S102: For any training sample of the first training sample and the second training sample, use N data detection models to perform data detection on the training sample to obtain N prediction probabilities that the training sample belongs to the target type of data.

[0120] In an embodiment of the present application, an integrated learning method is used to detect data, that is, the data is detected by combining N data detection models to improve the accuracy and stability of data detection.

[0121] In some data detection scenarios, the available original training samples (i.e., the first training samples) are relatively few. In an embodiment of the present application, in order to improve the training accuracy of N data detection models, the computing device processes the first training samples through a generative adversarial network to generate second training samples, thereby increasing the training data of the model.

[0122] In the embodiment of the present application, the process of training N data detection models using each of the first training sample and the second training sample is consistent, and any one of the first training sample and the second training sample is used as an example for explanation.

[0123] Specifically, Figure 5 As shown, the computing device inputs the training sample into N data detection models for processing, and the N data detection models predict the prediction probability that the training sample belongs to the target type data, respectively, and then N prediction probabilities can be obtained. For example, the N data detection models include 3 data detection models, which are respectively recorded as the first data detection model, the second data detection model, and the third data detection model. The computing device inputs the training sample into the first data detection model, the second data detection model, and the third data detection model, respectively, so that the first data detection model predicts whether the training sample belongs to the target type data, obtains a prediction probability, and obtains the first prediction probability. Similarly, the second data detection model predicts whether the training sample belongs to the target type data, obtains a prediction probability, and obtains the second prediction probability. The third data detection model predicts whether the training sample belongs to the target type data, obtains a prediction probability, and obtains the third prediction probability.

[0124] In some embodiments, in order to further improve the detection performance of N data detection models for complex data, when the computing device trains the N data detection models, it first extracts useful features from the training data through a deep learning model. These features can help the N data detection models better understand the data and provide better input for subsequent model training. Based on this, in the above S102, N data detection models are used to perform data detection on the training samples, and the N predicted probabilities that the training samples belong to the target type data include the following S102-A and S102-B:

[0125] S102-A, extracting features from training samples through a deep learning model to obtain feature information of the training samples;

[0126] S102-B, performing data detection on the feature information of the training samples through N data detection models, and obtaining N prediction probabilities that the training samples belong to the target type data.

[0127] In this embodiment, if Figure 6 As shown, the computing device obtains a deep learning model, and uses the obtained deep learning model to extract feature information of training samples, and then inputs the extracted feature information of the training samples into N data detection models respectively to obtain N prediction probabilities that the training samples belong to the target type data.

[0128] The embodiments of the present application do not limit the specific type of the deep learning model, which can be any network model that can extract useful features from the original data.

[0129] Exemplarily, the above-mentioned deep learning model can be a deep neural network (DNN), a convolutional neural network (CNN) or a recurrent neural network (RNN).

[0130] In an embodiment of the present application, the deep learning model of the computing device performs feature extraction on the training samples, and completes the training of the deep learning model before obtaining the feature information of the training samples.

[0131] In some embodiments, the deep learning model is trained by other computing devices, and the computing device of the embodiment of the present application obtains the trained deep learning model. For example, the computing device of the embodiment of the present application obtains the trained deep learning model from other computing devices. For another example, other computing devices save the trained deep learning model in the cloud, and the computing device of the embodiment of the present application obtains the trained deep learning model from the cloud.

[0132] In some embodiments, the computing device of the embodiment of the present application trains the deep learning model to obtain a trained deep learning model.

[0133] The following is an introduction to the training process of the deep learning model. For ease of description, the computing device of the embodiment of the present application is used as an example for the execution subject. The training process of the deep learning model by other computing devices is consistent with the training process of the deep learning model by the embodiment of the present application, and can be referred to.

[0134] like Figure 7As shown, the deep learning model consists of at least two parts, one is a feature extraction module and the other is a prediction module. The feature extraction module is used to extract feature information of input data. The prediction module is used to make predictions based on the feature information extracted by the feature extraction model, such as predicting the probability that the input data belongs to the target type of data.

[0135] In the embodiment of the present application, the training process of the deep learning model includes the following steps 21 to 24:

[0136] Step 21: Initialize the parameters of the deep learning model.

[0137] Step 22: Perform feature extraction on the third training sample through the feature extraction module in the deep learning model to obtain feature information of the third training sample.

[0138] Step 23: Process the feature information of the third training sample through the prediction module in the deep learning model to obtain the prediction probability of the third training sample.

[0139] Step 25: Train the deep learning model based on the predicted probability of the third training sample and the true label of the third training sample.

[0140] In the embodiment of the present application, for the convenience of description, the training sample used to train the deep learning model is recorded as the third training sample. Optionally, the third training sample can be the first training sample mentioned above, or can be other training samples except the first training sample, which is not limited in the embodiment of the present application.

[0141] like Figure 7 As shown, the deep learning model of the embodiment of the present application includes a feature extraction module and a prediction module. When the deep learning model is trained, it includes training the feature extraction module and the prediction module.

[0142] Specifically, Figure 7 As shown, the computing device first initializes the parameters of the deep learning model, and then inputs the third training sample into the deep learning model. The feature extraction module in the deep learning model extracts the feature information of the third training sample. Then, the extracted feature information is input into the prediction module for prediction to obtain the predicted probability of the third training sample, for example, the prediction module predicts the predicted probability that the third training sample belongs to the target type data (e.g., fraud data).

[0143] Next, based on the predicted probability of the third training sample output by the prediction module and the true label of the third training sample, the loss of the deep learning model is calculated. It is determined whether the training end condition of the model is met, for example, whether the loss of the deep learning model is less than the preset loss, or whether the training times of the deep learning model reaches the preset times.

[0144] If the current deep learning model meets the training end conditions, the training of the deep learning model is stopped, and the deep learning model obtained by the current training is used as the trained deep learning model for the training of N data detection models.

[0145] If the current deep learning model does not meet the training end condition, the parameters of the deep learning model are updated using the gradient descent method based on the loss of the deep learning model determined above. For example, based on the loss of the deep learning model determined above, the update gradient of the parameters in the deep learning model is determined, and then based on the update gradient, the parameters of the deep learning model are updated. Next, the prediction module in the updated deep learning model is used to process the feature information of another third training sample to obtain the prediction probability of the third training sample, and the deep learning model is trained based on the prediction probability of the third training sample and the true label of the third training sample. Repeat steps 22 to 25 above until the difference between the predicted probability of the output of the deep learning model and the true label is small enough (that is, the loss of the deep learning model is less than the preset loss), or the number of training times of the deep learning model reaches the preset number of iterations.

[0146] The computing device implements the training of the deep learning model based on the above steps, or the computing device obtains the trained deep learning model from other storage devices. In this way, when the computing device trains N data detection models, it can extract features from the training samples (i.e., the training samples for training the N data detection models) through the feature extraction module in the trained deep learning model to obtain feature information of the training samples. Then, the feature information of the training samples is subjected to data detection through the N data detection models to obtain N predicted probabilities that the training samples belong to the target type of data.

[0147] In an embodiment of the present application, the computing device predicts N prediction probabilities that the first training sample or the second training sample belongs to the target type data through N data detection models based on the above steps, and then executes the following step S103.

[0148] S103. For each of the N data detection models, determine the loss of the data detection model based on the predicted probability of the data detection model and the true label of the training sample, and update the parameters in the data detection model based on the loss.

[0149] As can be seen from the above, when the computing device uses the training sample to train N data detection models, the N data detection models respectively predict the prediction probability that the training sample belongs to the target type data. For each data detection model, based on the prediction probability that the training sample belongs to the target type data predicted by the data detection model and the true label of the training sample, the loss of the data detection model is determined, and then based on the loss of the data detection model, the parameters of the data detection model are updated.

[0150] For example, assume that N data detection models include a first data detection model, a second data detection model, and a third data detection model. The first data detection model predicts the training sample to obtain a first prediction probability, the second data detection model predicts the training sample to obtain a second prediction probability, and the third data detection model predicts the training sample to obtain a third prediction probability. Then, the computing device determines a first loss based on the first prediction probability and the true label of the training sample, and updates the parameters in the first data detection model based on the first loss. The computing device determines a second loss based on the second prediction probability and the true label of the training sample, and updates the parameters in the second data detection model based on the second loss. The computing device determines a third loss based on the third prediction probability and the true label of the training sample, and updates the parameters in the third data detection model based on the third loss. Repeat the above steps until the training end condition of the model is met.

[0151] In an embodiment of the present application, when data detection is performed on target data based on N trained data detection models, the final detection result of the target data can be determined based on the N prediction probabilities of the N data detection models.

[0152] In an embodiment of the present application, based on the N prediction probabilities of the N data detection models, the method of determining the final detection result at least includes: weighting the N prediction probabilities to obtain weighted prediction probabilities, and then determining the final detection result of the target data based on the weighted prediction probabilities. When weighting the N prediction probabilities, it is necessary to determine the weights of the N data detection models, and then weight the N prediction probabilities based on the weights of each data detection model to obtain the weighted prediction probability.

[0153] In some embodiments, the weights of the N data detection models are obtained by training, that is, the weights of the N data detection models are trained together with the N data detection models. At this time, the training process of the weights of the N data detection models includes the following steps 31 to 33:

[0154] Step 31: Perform data detection on the i-th round training samples using the N data detection models obtained through the i-1th round training to obtain N prediction probabilities for the i-th round, where i is a positive integer;

[0155] Step 32: determine the loss of the N data detection models in the i-th round based on the N prediction probabilities in the i-th round, the weights of the N data detection models in the i-1-th round, and the true labels of the i-th round training samples;

[0156] Step 33: Based on the loss in the i-th round, the weights in the i-1th round are updated to obtain the weights of the N data detection models in the i-th round.

[0157] In an embodiment of the present application, multiple rounds of training are performed on N data detection models. In each round of training, not only the parameters in the N data detection models are updated, but also the weights of the N data detection models are updated.

[0158] Specifically, the computing device first initializes the parameters of N data detection models and the weights of N data detection models. Then, the computing device selects a batch of training samples, such as 10 training samples, from the training data set (i.e., the first training sample and the second training sample) as the first round of training samples. The first round of training samples are input into the N data detection models for data detection, and the N prediction probabilities that each training sample in the first round of training samples belongs to the target type data predicted by the N data detection models are obtained as the N prediction probabilities of the first round, that is, the N prediction probabilities of the first round include the N prediction probabilities of each training sample in the first round of training samples.

[0159] After the computing device obtains the N prediction probabilities of the first round, for each of the N data detection models, the computing device determines the loss of the data detection model based on the N prediction probabilities corresponding to the data detection model and the true labels of the first round of training samples, and then updates the parameters in the data detection model based on the loss, completing the first round of updates of the data detection model and obtaining the data detection model after the first round of training.

[0160] Furthermore, after obtaining the N prediction probabilities of the first round, the computing device can also determine the losses of the N data detection models in the first round based on the N prediction probabilities of the first round, the initial weights of the N data detection models, and the true labels of the first round training samples. Then, based on the losses of the N data detection models in the first round, the initial weights of the N data detection models are updated to obtain the weights of the N data detection models in the first round.

[0161] Next, the computing device selects another batch of training samples, for example, 10 training samples, from the training data set (i.e., the first training sample and the second training sample) as the second round of training samples. The second round of training samples are input into the N data detection models after the first round of training for data detection, and the N prediction probabilities that each training sample in the second round of training samples belongs to the target type data predicted by the N data detection models are obtained as the N prediction probabilities of the second round, that is, the N prediction probabilities of the second round include the N prediction probabilities of each training sample in the second round of training samples.

[0162] After the computing device obtains the N prediction probabilities of the second round, for each of the N data detection models, the computing device determines the loss of the data detection model based on the N prediction probabilities corresponding to the data detection model and the true labels of the second round of training samples. Then, based on the loss, the parameters in the data detection model after the first round of training are updated to complete the second round of updating of the data detection model and obtain the data detection model after the second round of training.

[0163] Furthermore, after obtaining the N prediction probabilities of the second round, the computing device can also determine the losses of the N data detection models in the second round based on the N prediction probabilities of the second round, the weights of the N data detection models in the first round, and the true labels of the second round training samples. Then, based on the losses of the N data detection models in the second round, the weights of the N data detection models in the first round are updated to obtain the weights of the N data detection models in the second round.

[0164] The computing device repeats the above steps to complete the update training of N data detection models and the weights of the N data detection models.

[0165] The embodiment of the present application does not limit the specific manner in which the computing device determines the loss of the N data detection models in the i-th round based on the N prediction probabilities in the i-th round, the weights of the N data detection models in the i-1 round, and the true labels of the i-th round training samples.

[0166] In some embodiments, the above step 32 includes the following steps 321 and 322:

[0167] Step 321: Determine the weighted prediction probability of the i-th round training sample in the i-th round based on the N prediction probabilities of the i-th round and the weight of the i-1-th round;

[0168] Step 322: Determine the losses of the N data detection models in the i-th round based on the weighted prediction probabilities of the i-th round training samples in the i-th round and the true labels of the i-th round training samples.

[0169] In this embodiment, the specific manner in which the computing device determines the losses of the N data detection models in each round is substantially the same. For ease of description, determining the losses of the N data detection models in the i-th round is used as an example for explanation.

[0170] In this embodiment, when the computing device determines the loss of the N data detection models in the i-th round based on the N prediction probabilities in the i-th round, the weights of the N data detection models in the i-1th round, and the true labels of the i-th round training samples, the computing device first determines the weighted prediction probability of the i-th round training samples in the i-th round based on the N prediction probabilities in the i-th round and the weights in the i-1th round. For example, for each training sample in the i-th round training samples, the N prediction probabilities of the training sample are weighted based on the weights of the N data detection models in the i-1th round to obtain the weighted prediction probability of the training sample in the i-th round. With reference to this step, the weighted prediction probability of each training sample in the i-th round training samples can be determined.

[0171] Next, based on the weighted prediction probability of each training sample in the i-th round of training samples and the true label of each training sample in the i-th round of samples, the losses of the N data detection models in the i-th round are determined.

[0172] Exemplarily, the computing device determines the loss of N data detection models in the i-th round based on the following formula (5):

[0173]

[0174] Where ω=(ω1,ω2,…,ω n ) is the weight of N data detection models, f j (x k ) is the jth data detection model for the kth sample x k The prediction result, y k is the true label of the kth sample, m is the number of samples, and n is the number of data detection models. is the weighted predicted probability of the kth training sample in the i-th round of training samples.

[0175] Based on the above method, after the computing device determines the loss of the N data detection models in the i-th round, based on the loss of the i-th round, the weight of the i-1th round is updated to obtain the weights of the N data detection models in the i-th round. For example, the computing device updates the weights of the N data detection models by using a gradient descent method.

[0176] Exemplarily, the computing device updates the weight of the i-1th round by the following formula (6) to obtain the weights of the N data detection models in the i-th round:

[0177]

[0178] Among them, α is the learning rate, is the loss function L with respect to weight ω j The partial derivative of .

[0179] From the above formula (6), we can see that if the prediction result of a data detection model is far away from the true label, the weight of the data detection model will be reduced. If the prediction result of a data detection model is close to the true label, the weight of the data detection model will be increased.

[0180] In an embodiment of the present application, during the training process of the data detection model, adjusting the weight of the model can allow the data detection model to adjust its influence on the final prediction result according to its performance during the training process, thereby further improving the accuracy and stability of the data detection model.

[0181] In some embodiments, the computing device uses N data detection models to detect the target data, obtains N prediction probabilities of the target data, and then determines the detection result of the target data based on these N prediction probabilities, determines the weighted prediction probability of the N prediction probabilities, and compares the weighted prediction probability with the detection threshold to determine the detection result of the target data.

[0182] In some embodiments, the above detection threshold is obtained by training, for example, the weights of N data detection models are trained together with the N data detection models. In this case, the training process of the detection threshold of the N data detection models includes the following steps 41 to 43:

[0183] Step 41: Perform data detection on the i-th round training samples using the N data detection models obtained by the i-1th round training to obtain N prediction probabilities for the i-th round, where i is a positive integer;

[0184] Step 42: Determine the weighted prediction probability of the training sample in the i-th round based on the N prediction probabilities in the i-th round and the weights of the N data detection models in the i-1th round;

[0185] Step 43: Based on the weighted prediction probability of the i-th round training samples and the true labels of the i-th round training samples, the detection threshold of the i-1th round is updated to obtain the detection threshold of the i-th round.

[0186] In an embodiment of the present application, the computing device also includes updating the detection threshold during the process of training N data detection models.

[0187] Specifically, the computing device first initializes the parameters of N data detection models and a detection threshold. Optionally, the initial detection threshold may be 0.5 or other values, depending on the judgment criteria for data detection (eg, fraudulent behavior).

[0188] Next, the computing device selects a batch of training samples, for example, 10 training samples, from the training data set (i.e., the first training sample and the second training sample) as the first round of training samples. The first round of training samples are input into the N data detection models for data detection, and the N prediction probabilities that each training sample in the first round of training samples belongs to the target type data predicted by the N data detection models are obtained as the N prediction probabilities of the first round, that is, the N prediction probabilities of the first round include the N prediction probabilities of each training sample in the first round of training samples.

[0189] After the computing device obtains the N prediction probabilities of the first round, for each of the N data detection models, the computing device determines the loss of the data detection model based on the N prediction probabilities corresponding to the data detection model and the true labels of the first round of training samples, and then updates the parameters in the data detection model based on the loss, completing the first round of updates of the data detection model and obtaining the data detection model after the first round of training.

[0190] Furthermore, after obtaining the N prediction probabilities of the first round, the computing device can also determine the weighted prediction probability of the first round of training samples based on the N prediction probabilities of the first round and the initial weights of the N data detection models. Then, based on the weighted prediction probability of the first round of training samples and the true labels of the first round of training samples, the initial detection threshold is updated to obtain the first round of detection threshold.

[0191] Next, the computing device selects another batch of training samples, for example, 10 training samples, from the training data set (i.e., the first training sample and the second training sample) as the second round of training samples. The second round of training samples are input into the N data detection models after the first round of training for data detection, and the N prediction probabilities that each training sample in the second round of training samples belongs to the target type data predicted by the N data detection models are obtained as the N prediction probabilities of the second round, that is, the N prediction probabilities of the second round include the N prediction probabilities of each training sample in the second round of training samples.

[0192] After the computing device obtains the N prediction probabilities of the second round, for each of the N data detection models, the computing device determines the loss of the data detection model based on the N prediction probabilities corresponding to the data detection model and the true labels of the second round of training samples. Then, based on the loss, the parameters in the data detection model after the first round of training are updated to complete the second round of updating of the data detection model and obtain the data detection model after the second round of training.

[0193] Furthermore, after obtaining the N prediction probabilities of the second round, the computing device can also determine the weighted prediction probabilities of the second round training samples based on the N prediction probabilities of the second round and the weights of the N data detection models in the first round. Then, based on the weighted prediction probabilities of the second round training samples and the true labels of the second round training samples, the detection threshold of the first round is updated to obtain the detection threshold of the second round.

[0194] The computing device repeats the above steps to complete the update training of N data detection models and detection thresholds.

[0195] In some embodiments, the computing device updates the weights and detection thresholds of the N data detection models during the update process of the N data detection models. Take the i-th round of training of the N data detection models as an example. The computing device selects a batch of training samples, such as 10 training samples, from the training data set as the i-th round of training samples. The i-th round of training samples are input into the N data detection models after the i-1th round of training for data detection, and the N predicted probabilities that each training sample in the i-th round of training samples predicted by the N data detection models belongs to the target type data are obtained as the N predicted probabilities of the i-th round. After the computing device obtains the N predicted probabilities of the i-th round, for each data detection model in the N data detection models, based on the N predicted probabilities corresponding to the data detection model and the true label of the i-th round of training samples, the loss of the data detection model is determined. Then, based on the loss, the parameters in the data detection model after the i-1th round of training are updated, the i-th round of update of the data detection model is completed, and the data detection model after the i-th round of training is obtained. Furthermore, after the computing device obtains the N prediction probabilities of the i-th round, it can also determine the loss of the N data detection models in the i-th round based on the N prediction probabilities of the i-th round, the weights of the N data detection models in the i-1 round, and the true labels of the i-th round training samples. Then, based on the loss of the N data detection models in the i-th round, the weights of the N data detection models in the i-1 round are updated to obtain the weights of the N data detection models in the i-th round. Further, after the computing device obtains the N prediction probabilities of the i-th round, it can also determine the weighted prediction probability of the i-th round training samples based on the N prediction probabilities of the i-th round and the weights of the N data detection models in the i-1 round. Then, based on the weighted prediction probability of the i-th round training samples and the true labels of the i-th round training samples, the detection threshold of the i-1 round is updated to obtain the detection threshold of the i-th round.

[0196] In the embodiment of the present application, the computing device updates the detection threshold of the i-1th round based on the weighted prediction probability of the i-th round training samples and the true label of the i-th round training samples, and the specific method of obtaining the detection threshold of the i-th round is not limited.

[0197] In some embodiments, the above step 43 includes the following steps 431 and 432:

[0198] Step 431: Determine the evaluation index of N data detection models in the i-th round based on the weighted prediction probability of the i-th round training samples, the true label of the i-th round training samples and the detection threshold of the i-1-th round;

[0199] Step 432: Determine the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round.

[0200] In this implementation, during the update process of the detection threshold in each round (for example, the i-th round), the computing device first determines the evaluation indicators of N data detection models in the i-th round by using the weighted prediction probability of the i-th round training samples, the true labels of the i-th round training samples, and the detection threshold of the i-1th round.

[0201] The embodiment of the present application does not limit the type of evaluation indicators.

[0202] Exemplarily, the evaluation index includes at least one of precision, recall, F1 value, and ROC (Receiver Operating Characteristic) curve. The specific selection of the evaluation index can be selected according to actual business needs.

[0203] In an embodiment of the present application, the computing device determines the specific method of evaluating the N data detection models in the i-th round based on the weighted prediction probability of the i-th round training samples, the true label of the i-th round training samples and the detection threshold of the i-1th round. The specific method can refer to the existing related technology and will not be repeated in the embodiment of the present application.

[0204] Next, the computing device determines the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round.

[0205] The embodiment of the present application does not limit the specific method in which the computing device determines the detection threshold of the i-th round based on the evaluation indicators of N data detection models in the i-th round.

[0206] In some embodiments, since the evaluation index of the N data detection models in the i-th round is determined based on the detection threshold of the i-1th round, that is, the evaluation index of the N data detection models in the i-th round is a function of the detection threshold of the i-1th round. Therefore, the computing device can adjust the detection threshold of the i-1th round so that the detection threshold of the N data detection models when the evaluation index of the i-th round is the maximum is determined as the detection threshold of the i-th round.

[0207] In some embodiments, the above step 432 includes the following steps 4321 and 4322:

[0208] Step 4321: Based on the evaluation index of the i-th round, determine the threshold setting index of the i-th round;

[0209] Step 4322: Determine the detection threshold of the i-th round based on the threshold setting index of the i-th round.

[0210] In an implementation, the computing device determines the threshold setting index for the i-th round based on the evaluation indexes of the N data detection models in the i-th round. For example, the evaluation indexes of the N data detection models in the i-th round are used to determine the threshold setting index for the i-th round. For another example, the evaluation indexes of the N data detection models in the i-th round are used as inputs of a preset threshold adjustment function, and the output of the threshold adjustment function is used as the threshold setting index for the i-th round.

[0211] Next, based on the threshold setting index of the i-th round, the detection threshold is adjusted to obtain the detection threshold of the i-th round.

[0212] For example, the detection threshold of the i-1th round is adjusted so that the threshold corresponding to the maximum threshold setting index of the i-th round is determined as the detection threshold of the i-th round.

[0213] Exemplarily, the computing device determines the detection threshold of the i-th round by the following formula (7):

[0214]

[0215] Among them, θ is the detection threshold of the i-1th round, F(.) is the threshold adjustment function, is the function of the evaluation index, F(θ) is the threshold setting index, θ * is the optimal threshold, that is, the detection threshold of the i-th round.

[0216] In some embodiments, after the detection threshold is obtained in each round of training, the detection threshold is verified. The verification process of the detection threshold of the i-th round includes the following steps 44 to 47:

[0217] Step 44: Perform data detection on the test sample using the N data detection models obtained through the i-th round of training to obtain N prediction probabilities of the test sample;

[0218] Step 45: Determine the weighted prediction probability of the test sample based on the N prediction probabilities of the test sample and the weights of the N data detection models in the i-th round;

[0219] Step 46: Determine the evaluation index of the N data detection models obtained by the i-th round of training based on the weighted prediction probability of the test sample, the detection threshold of the i-th round and the true label of the test sample;

[0220] Step 47: Based on the evaluation indicators of the N data detection models obtained from the i-th round of training, the verification result of the detection threshold of the i-th round is determined.

[0221] In an embodiment of the present application, after determining the detection threshold in each round based on the above steps, the computing device verifies on the verification set whether the evaluation indicators of N data detection models under the detection threshold are qualified.

[0222] Specifically, taking the detection threshold obtained in the i-th round of training as an example, after the computing device obtains the N data detection models obtained in the i-th round of training, the weights of the N data detection models in the i-th round, and the detection threshold of the i-th round based on the above steps, the computing device performs data detection on the test sample through the N data detection models obtained in the i-th round of training to obtain N prediction probabilities of the test sample. Next, based on the N prediction probabilities of the test sample and the weights of the N data detection models in the i-th round, the weighted prediction probability of the test sample is determined. Then, based on the weighted prediction probability of the test sample, the detection threshold of the i-th round and the true label of the test sample, the evaluation index of the N data detection models obtained in the i-th round of training is determined. Furthermore, based on the evaluation index of the N data detection models obtained in the i-th round of training, the verification result of the detection threshold of the i-th round is determined.

[0223] For example, if the evaluation indicators of the N data detection models obtained in the i-th round of training meet the preset requirements, the verification result of the detection threshold of the i-th round is determined to be qualified. If the evaluation indicators of the N data detection models obtained in the i-th round of training do not meet the preset requirements, the verification result of the detection threshold of the i-th round is determined to be unqualified.

[0224] In an embodiment of the present application, if it is determined that the verification result of the detection threshold of the i-th round is unqualified, the above-mentioned model training process is continued to train N data detection models and detection thresholds.

[0225] In some embodiments, if the verification result of the detection threshold of the i-th round is determined to be qualified, the computing device stops updating the N data detection models and the detection threshold. The computing device fixes the parameters of the N data detection models obtained by the i-th round of training, and fixes the detection threshold of the i-th round.

[0226] From the above, it can be seen that the embodiment of the present application provides an adaptive detection threshold adjustment method. Compared with simply setting a fixed threshold, such as 0.5, the method of the embodiment of the present application can adapt to different data distributions and model performance, and can further improve the accuracy of data detection.

[0227] The model training method provided in the embodiment of the present application processes the first training sample to generate a second training sample by generating an adversarial network; for any training sample in the first training sample and the second training sample, N data detection models are used to perform data detection on the training sample to obtain N predicted probabilities that the training sample belongs to the target type of data; for each of the N data detection models, the loss of the data detection model is determined based on the predicted probability of the data detection model and the true label of the training sample, and the parameters in the data detection model are updated based on the loss. That is, the embodiment of the present application generates the second training sample by generating an adversarial network, thereby increasing the amount of training data for the N data detection models, and can ensure the prediction accuracy and stability of the N data detection models.

[0228] The above introduces the overall model training process of the embodiment of the present application. The following is a specific example to further introduce the model training method provided in the embodiment of the present application.

[0229] Figure 8 A flow chart of a model training method provided in an embodiment of the present application is shown in FIG. Figure 8 As shown, including:

[0230] S201. Train a generative adversarial network.

[0231] Among them, the training process of the generative adversarial network can refer to the description of steps 11 to 14 above, which will not be repeated here.

[0232] S202: training a deep learning model.

[0233] In the embodiment of the present application, there is no restriction on the training order of the generative adversarial network and the deep learning model. That is, the above S202 can be executed before the above S201, or after the above S201, or synchronously with the above S201.

[0234] Among them, the training process of the deep learning model can refer to the description of steps 21 to 25 above, which will not be repeated here.

[0235] In this embodiment, it is assumed that the computing device completes the training of the generative adversarial network and the deep learning model.

[0236] In some embodiments, at least one of the above-mentioned generative adversarial network and deep learning model is not trained by the computing device of the embodiment of the present application, but is trained by other devices.

[0237] S203: Process the first training sample by generating an adversarial network to generate a second training sample.

[0238] The following steps S203 to S212 can be understood as the training process of N data detection models.

[0239] In the embodiment of the present application, since the data of the first training sample is relatively small, in order to improve the training effect of the N data detection models, in the embodiment of the present application, a large number of second training samples are generated based on the first training sample using a generative adversarial network. Then, the first training sample and the second training sample are used to train the N data detection models to improve the training accuracy and stability of the N data detection models.

[0240] S204: Obtain the i-th round training sample from the first training sample and the second training sample.

[0241] In an embodiment of the present application, the computing device performs multiple rounds of training on N data detection models, and the training process of each round is basically the same.

[0242] For the convenience of description, the i-th round of training is taken as an example.

[0243] During the i-th round of training, the computing device first selects a batch of training samples from a training set consisting of the first training samples and the second training samples as the i-th round of training samples.

[0244] The i-th round of training samples includes at least one training sample.

[0245] S205. Perform data detection on the i-th round training samples using the N data detection models obtained through the i-1th round training to obtain N prediction probabilities that the i-th round training samples belong to the target type data.

[0246] S206. For each of the N data detection models, the loss of the data detection model is determined based on the predicted probability of the data detection model and the true label of the i-th round of training samples, and the parameters in the data detection model obtained by the i-1th round of training are updated based on the loss to obtain the data detection model of the i-th round.

[0247] During the i-th round of training, the computing device inputs the i-th round training samples into the N data detection models obtained from the i-1-th round of training for data detection, and obtains N prediction probabilities that each of the i-th round training samples predicted by the N prediction data models belongs to the target type data.

[0248] Next, based on the N predicted probabilities that each training sample in the i-th round of training samples belongs to the target type of data and the true label of each training sample in the i-th round of training samples, the loss of each of the N data detection models in the i-th round is determined. For each of the N data detection models, based on the loss of the data detection model in the i-th round, the parameters of the data detection model obtained in the i-1th round of training are updated to obtain the data detection model of the i-th round.

[0249] The specific implementation process of the above S205 and S206 can refer to the relevant steps of the above S102 and S103.

[0250] After the computing device completes the i-th round of training of the N data detection models based on the above steps, step S207 is executed.

[0251] S207. Determine the loss of the N data detection models in the i-th round based on the N prediction probabilities in the i-th round, the weights of the N data detection models in the i-1-th round, and the true labels of the i-th round training samples.

[0252] Steps S207 and S208 can be understood as a process of updating the weights of N data detection models.

[0253] For example, based on the N prediction probabilities of the i-th round and the weights of the i-1th round, the weighted prediction probability of the i-th round training sample in the i-th round is determined; based on the weighted prediction probability of the i-th round training sample in the i-th round and the true label of the i-th round training sample, the loss of N data detection models in the i-th round is determined.

[0254] The implementation process of the above S207 can refer to the specific description of the above step 32, which will not be repeated here.

[0255] S208. Based on the loss in the i-th round, the weights in the i-1th round are updated to obtain the weights of the N data detection models in the i-th round.

[0256] The specific implementation process of the above S208 can refer to the specific description of the above step 33, which will not be repeated here.

[0257] S209: Determine the weighted prediction probability of the training samples in the i-th round based on the N prediction probabilities in the i-th round and the weights of the N data detection models in the i-1-th round.

[0258] Among them, steps S209 and S210 can be understood as a process of updating the detection threshold.

[0259] S210: Based on the weighted prediction probability of the i-th round training samples and the true labels of the i-th round training samples, the detection threshold of the i-1th round is updated to obtain the detection threshold of the i-th round.

[0260] For example, based on the weighted prediction probability of the i-th round training samples, the true label of the i-th round training samples and the detection threshold of the i-1th round, the evaluation indicators of the N data detection models in the i-th round are determined; based on the evaluation indicators of the N data detection models in the i-th round, the detection threshold of the i-th round is determined.

[0261] The specific implementation process of the above S209 and the above S210 can refer to the description of the above steps 42 and 43, which will not be repeated here.

[0262] S211, verifying the detection threshold of the i-th round.

[0263] After obtaining the detection threshold of the i-th round based on the above steps, the computing device verifies the detection threshold of the i-th round on the verification set.

[0264] The specific implementation process of the above S211 can refer to the description of the above steps 44 to 47, which will not be repeated here.

[0265] If the verification results of the detection threshold of the i-th round are synthesized, step S212 is executed.

[0266] If the verification result of the detection threshold of the i-th round is not synthesized, then i=i+1 is set and the process returns to execute step S204.

[0267] S212: Stop updating the N data detection models and detection thresholds.

[0268] In an embodiment of the present application, during the training of N data detection models, in order to increase the training effect of the model, a generative adversarial network is used to generate a second training sample based on the first training sample, and then the first training sample and the second training sample are used to train the N data detection models to improve the training accuracy and stability of the N data detection models. In addition, in an embodiment of the present application, during the training of N data detection models, the weights and detection thresholds of the N data detection models are dynamically adjusted and trained to improve the weights of the N data detection models and the accuracy of the detection thresholds. In this way, when N data detection models and detection thresholds are used for subsequent data detection, the accuracy of data detection can be further improved.

[0269] Combination of the above Figures 3 to 8 , describes in detail an embodiment of the model training method of the present application, and the data detection method provided in the embodiment of the present application is introduced below.

[0270] Fig. 91 is a flow chart of a data detection method provided in an embodiment of the present application. The execution subject of the embodiment of the present application is a device with a data detection function, for example, a data detection device. In some embodiments, the data detection device can be Figure 1 A computing device in Figure 1 The terminal device in Figure 1 For ease of description, the present application embodiment takes the execution subject as an example of a computing device, which can be the same device as the computing device for model training, or can be a different device, and the present application embodiment does not limit this.

[0271] like Fig. 9 As shown, the method of the embodiment of the present application includes the following steps:

[0272] S301. Perform data detection on the acquired target data to be detected by using N data detection models, and obtain N prediction probabilities that the target data belongs to target type data.

[0273] Among them, the N data detection models are trained based on the first training sample and the second training sample, the second training sample is generated by the generative adversarial network based on the first training sample, and N is a positive integer greater than 1.

[0274] The training process of the above-mentioned generative adversarial network and the training process of N data detection models based on the generative adversarial network can refer to the specific description of the above-mentioned embodiment, which will not be repeated here.

[0275] The embodiment of the present application does not limit the specific type of target type data.

[0276] For example, if the N data detection models are fraud detection models used to detect fraudulent behavior, the target type data is fraudulent data. In other words, the N data detection models in the embodiment of the present application are used to detect whether the target data belongs to the predicted type of fraudulent data.

[0277] In an embodiment of the present application, after the training of N data detection models is realized through the above steps, data detection is performed on the target data to be detected through the trained N data detection models to obtain N prediction probabilities that the target data belongs to the target data type.

[0278] S302: Determine detection thresholds corresponding to N data detection models.

[0279] The detection threshold is used to determine whether the target data is target type data.

[0280] The embodiment of the present application does not limit the specific method of determining the detection thresholds corresponding to the N data detection models mentioned above.

[0281] In some embodiments, the detection threshold is a preset value, such as 0.8, 0.7, etc.

[0282] In some embodiments, the above detection threshold is obtained by training, that is, during the training process of the N data detection models, the detection threshold is also dynamically trained and updated. For example, the evaluation index of the N data detection models under the detection threshold of the i-1th round is determined, and the detection threshold of the i-1th round is updated based on the evaluation index to obtain the detection threshold of the i-th round. Repeat the above steps until the update termination condition of the detection threshold is reached, such as when the number of updates reaches a preset number of times or the evaluation index reaches a preset requirement. Exemplarily, determining the evaluation index of the N data detection models under the detection threshold of the i-1th round may include: using the N data detection models obtained by the i-1th round of training, performing data detection on the i-th round training samples, and obtaining N prediction probabilities of the i-th round, where i is a positive integer; based on the N prediction probabilities of the i-th round and the weights of the N data detection models in the i-1th round, determining the weighted prediction probability of the i-th round training samples; based on the weighted prediction probability of the i-th round training samples, the true labels of the i-th round training samples and the i-1th round detection threshold, determining the evaluation index of the N data detection models in the i-th round. Exemplarily, determining the evaluation index of N data detection models under the detection threshold of the i-1th round may include: using the N data detection models obtained through the i-1th round of training, performing data detection on the i-th round of training samples, and obtaining N prediction probabilities of the i-th round, where i is a positive integer; determining the evaluation index of each of the N data detection models in the i-th round based on the N prediction probabilities of the i-th round, the true labels of the i-th round of training samples, and the i-1th round of detection threshold; determining the evaluation index of the N data detection models in the i-th round based on the evaluation index of each of the N data detection models in the i-th round, for example, taking the weighted sum of the evaluation index of each of the N data detection models in the i-th round to determine the evaluation index of the N data detection models in the i-th round. For another example, taking the average of the evaluation index of each of the N data detection models in the i-th round to determine the evaluation index of the N data detection models in the i-th round.

[0283] Specifically, the updating process of the detection threshold may refer to the description of steps 41 to 43 above.

[0284] S303: Determine the detection result of the target data based on the N prediction probabilities and the detection threshold.

[0285] In an embodiment of the present application, the computing device determines N prediction probabilities of the target data and the detection thresholds corresponding to the N data detection models based on the above steps, and then determines the detection results of the target data based on the N prediction probabilities and the detection thresholds.

[0286] The embodiment of the present application does not limit the specific manner in which the computing device determines the detection result of the target data based on N prediction probabilities and the detection threshold.

[0287] In some embodiments, the computing device determines an average predicted probability of N predicted probabilities of the target data and compares the average predicted probability to a detection threshold.

[0288] If the average predicted probability of the target data is greater than or equal to the detection threshold, it means that the target data belongs to the target type data. For example, if the average predicted probability of the target data is greater than the detection threshold, it means that the target data belongs to fraudulent data.

[0289] If the average predicted probability of the target data is less than the detection threshold, it means that the target data does not belong to the target type data. For example, if the average predicted probability of the target data is less than the detection threshold, it means that the target data does not belong to fraudulent data.

[0290] In some embodiments, the above S303 includes the following steps S303-A to S303-C:

[0291] S303-A, determining weights of N data detection models;

[0292] S303-B, weighting the N prediction probabilities based on the weights of the N data detection models to obtain a weighted prediction probability of the target data;

[0293] S303-C, obtaining a detection result based on the weighted prediction probability and the detection threshold.

[0294] In this embodiment, the computing device determines a weighted prediction probability of N prediction probabilities of the target data, and then determines a detection result of the target data based on the weighted prediction probability and a detection threshold.

[0295] Specifically, the computing device determines the weights of N data detection models.

[0296] The embodiment of the present application does not limit the specific method for determining the weights of the above-mentioned N data detection models.

[0297] In some embodiments, the weights of the above-mentioned N data detection models are preset values.

[0298] In some embodiments, the weights of the N data detection models are obtained by training. The training process of the weights of the N data detection models can refer to the specific description of steps 31 to 33 above.

[0299] The computing device weights the N prediction probabilities of the target data based on the weights of the N data detection models to obtain a weighted prediction probability of the target data. Then, based on the weighted prediction probability of the target data and a detection threshold, a detection result of the target data is determined.

[0300] Exemplarily, if the weighted prediction probability of the target data is greater than or equal to the detection threshold, it means that the target data belongs to the target type data. For example, if the weighted prediction probability of the target data is greater than the detection threshold, it means that the target data belongs to fraudulent data.

[0301] Exemplarily, if the weighted predicted probability of the target data is less than the detection threshold, it means that the target data does not belong to the target type data. For example, if the weighted predicted probability of the target data is less than the detection threshold, it means that the target data does not belong to fraudulent data.

[0302] The data detection method provided in the embodiment of the present application, through N data detection models, respectively performs data detection on the acquired target data to be detected, and obtains N predicted probabilities that the target data belongs to the target type data. Then, the detection threshold corresponding to the N data detection models is determined, and the detection threshold is used to determine whether the target data is the target type data, and then based on the N predicted probabilities and the detection threshold, the detection result of the target data is determined, wherein the N data detection models are obtained based on the first training sample and the second training sample training, and the second training sample is generated by the generative adversarial network based on the first training sample. In other words, the embodiment of the present application uses N data detection models for data detection, which can improve the accuracy and stability of data detection. In addition, when the embodiment of the present application trains the N data detection models, a new training sample (i.e., the second training sample) is generated using a generative adversarial network, which increases the training data, and then realizes the full training of the N data detection models, further improving the data detection accuracy and stability of the N data detection models.

[0303] Combination of the above Figures 2 to 9 , describes in detail the model training and image processing method embodiments of the present application, and the following is combined with Fig.10 , describe in detail the device embodiments of the present application.

[0304] Fig.10 1 is a schematic block diagram of a data detection device provided in an embodiment of the present application. The device 10 can be applied to a computing device.

[0305] like Fig.10 As shown, the data detection device 10 includes:

[0306] The prediction unit 11 is used to perform data detection on the acquired target data to be detected by using N data detection models, and obtain N prediction probabilities that the target data belongs to the target type data, where N is a positive integer greater than 1;

[0307] A threshold determination unit 12, used to determine a detection threshold corresponding to the N data detection models, wherein the detection threshold is used to determine whether the target data is the target type data;

[0308] A detection unit 13, configured to determine a detection result of the target data based on the N prediction probabilities and the detection threshold;

[0309] The N data detection models are trained based on first training samples and second training samples, and the second training samples are generated by a generative adversarial network based on the first training samples.

[0310] In some embodiments, the detection unit 13 is specifically used to determine the weights of the N data detection models; based on the weights of the N data detection models, weight the N prediction probabilities to obtain the weighted prediction probability of the target data; based on the weighted prediction probability and the detection threshold, obtain the detection result.

[0311] In some embodiments, the training process of the N data detection models includes: processing the first training sample through the generative adversarial network to generate the second training sample; for any training sample between the first training sample and the second training sample, using the N data detection models to perform data detection on the training sample to obtain N prediction probabilities that the training sample belongs to the target type data; for each of the N data detection models, based on the prediction probability of the data detection model and the true label of the training sample, determining the loss of the data detection model, and updating the parameters in the data detection model based on the loss.

[0312] In some embodiments, the processing of the first training sample by the generative adversarial network to generate the second training sample is preceded by completing the training of the generative adversarial network, and the processing of the first training sample by the generative adversarial network to generate the second training sample includes: processing the first training sample by the trained generative adversarial network to generate the second training sample.

[0313] In some embodiments, the use of the N data detection models to perform data detection on the training samples to obtain N prediction probabilities that the training samples belong to the target type data includes: performing feature extraction on the training samples through a deep learning model to obtain feature information of the training samples; performing data detection on the feature information of the training samples through the N data detection models to obtain N prediction probabilities that the training samples belong to the target type data.

[0314] In some embodiments, the deep learning model is trained before extracting features from the training samples through the deep learning model to obtain feature information of the training samples. The training process of the deep learning model includes: extracting features from the third training sample through the feature extraction module in the deep learning model to obtain feature information of the third training sample; processing the feature information of the third training sample through the prediction module in the deep learning model to obtain the predicted probability of the third training sample; and training the deep learning model based on the predicted probability of the third training sample and the true label of the third training sample.

[0315] In some embodiments, extracting features from the training samples through a deep learning model to obtain feature information of the training samples includes: extracting features from the training samples through a feature extraction module in the trained deep learning model to obtain feature information of the training samples.

[0316] In some embodiments, the weights of the N data detection models are obtained by training together with the N data detection models, and the training process of the weights of the N data detection models includes: using the N data detection models obtained through the i-1th round of training, performing data detection on the i-th round training samples, and obtaining N prediction probabilities of the i-th round, where i is a positive integer; determining the loss of the N data detection models in the i-th round based on the N prediction probabilities of the i-th round, the weights of the N data detection models in the i-1th round, and the true labels of the i-th round training samples; based on the loss of the i-th round, updating the weights of the i-1th round to obtain the weights of the N data detection models in the i-th round.

[0317] In some embodiments, determining the loss of the N data detection models in the i-th round based on the N prediction probabilities of the i-th round, the weights of the N data detection models in the i-1 round, and the true labels of the i-th round training samples includes: determining the weighted prediction probabilities of the i-th round training samples in the i-th round based on the N prediction probabilities of the i-th round and the weights of the i-1 round; determining the loss of the N data detection models in the i-th round based on the weighted prediction probabilities of the i-th round training samples in the i-th round and the true labels of the i-th round training samples.

[0318] In some embodiments, the detection threshold is obtained by training together with the N data detection models, and the training process of the detection threshold includes: performing data detection on the i-th round training samples through the N data detection models obtained through the i-1th round training to obtain N prediction probabilities of the i-th round, where i is a positive integer; determining the weighted prediction probability of the i-th round training samples based on the N prediction probabilities of the i-th round and the weights of the N data detection models in the i-1th round; updating the detection threshold of the i-1th round based on the weighted prediction probability of the i-th round training samples and the true labels of the i-th round training samples to obtain the detection threshold of the i-th round.

[0319] In some embodiments, the detection threshold of the i-1th round is updated based on the weighted prediction probability of the i-th round training samples and the true label of the i-th round training samples to obtain the detection threshold of the i-th round, including: determining the evaluation indicators of the N data detection models in the i-th round based on the weighted prediction probability of the i-th round training samples, the true label of the i-th round training samples and the i-1th round detection threshold; determining the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round.

[0320] In some embodiments, the evaluation index includes at least one of precision, recall, F1 value, and ROC curve.

[0321] In some embodiments, determining the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round includes: determining the threshold setting indicators of the i-th round based on the evaluation indicators of the i-th round; determining the detection threshold of the i-th round based on the threshold setting indicators of the i-th round.

[0322] In some embodiments, determining the threshold setting indicator of the i-th round based on the evaluation indicator of the i-th round includes: using the evaluation indicator of the i-th round as the input of a preset threshold adjustment function, and using the output of the threshold adjustment function as the threshold setting indicator of the i-th round.

[0323] In some embodiments, obtaining the detection threshold of the i-th round based on the threshold setting indicator of the i-th round includes: adjusting the detection threshold of the i-1-th round to determine the threshold corresponding to the time when the threshold setting indicator of the i-th round is the maximum as the detection threshold of the i-th round.

[0324] In some embodiments, after obtaining the detection threshold of the i-th round, the detection threshold of the i-th round is verified, and the verification process of the detection threshold of the i-th round includes: performing data detection on the test sample through the N data detection models obtained by the i-th round of training to obtain N prediction probabilities of the test sample; determining the weighted prediction probability of the test sample based on the N prediction probabilities of the test sample and the weights of the N data detection models in the i-th round; determining the evaluation index of the N data detection models obtained by the i-th round of training based on the weighted prediction probability of the test sample, the detection threshold of the i-th round and the true label of the test sample; determining the verification result of the detection threshold of the i-th round based on the evaluation index of the N data detection models obtained by the i-th round of training.

[0325] In some embodiments, if the verification result of the detection threshold of the i-th round is qualified, the updating of the N data detection models and the detection threshold is stopped.

[0326] It should be understood that the device embodiment and the method embodiment may correspond to each other, and similar descriptions may refer to the method embodiment. To avoid repetition, they will not be described here. Specifically, Fig.10 The device shown can execute the above-mentioned data detection method embodiment, and the aforementioned and other operations and / or functions of each module in the device are respectively for implementing the method embodiment corresponding to the computing device, which will not be repeated here for the sake of brevity.

[0327] The above describes the device of the embodiment of the present application from the perspective of the functional module in conjunction with the accompanying drawings. It should be understood that the functional module can be implemented in hardware form, can be implemented by instructions in software form, and can also be implemented by a combination of hardware and software modules. Specifically, the steps of the method embodiment in the embodiment of the present application can be completed by the hardware integrated logic circuit and / or software form instructions in the processor, and the steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or a combination of hardware and software modules in the decoding processor to perform. Optionally, the software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory, and completes the steps in the above method embodiment in conjunction with its hardware.

[0328] Fig.11is a schematic block diagram of a computing device provided in an embodiment of the present application, Fig.11 The computing device can be used to execute the above-mentioned model training method and / or data detection method.

[0329] like Fig.11 As shown, the computing device 30 may include:

[0330] The memory 31 and the processor 32, the memory 31 is used to store the computer program 33 and transmit the program code 33 to the processor 32. In other words, the processor 32 can call and run the computer program 33 from the memory 31 to implement the method in the embodiment of the present application.

[0331] For example, the processor 32 may be configured to execute the steps in the above method according to the instructions in the computer program 33 .

[0332] In some embodiments of the present application, the processor 32 may include but is not limited to:

[0333] General-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components, etc.

[0334] In some embodiments of the present application, the memory 31 includes but is not limited to:

[0335] Volatile memory and / or non-volatile memory. Among them, the non-volatile memory can be read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus random access memory (DR RAM).

[0336] In some embodiments of the present application, the computer program 33 may be divided into one or more modules, which are stored in the memory 31 and executed by the processor 32 to complete the method for recording pages provided by the present application. The one or more modules may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 33 in the computing device.

[0337] like Fig.11 As shown, the computing device 30 may also include:

[0338] The transceiver 34 may be connected to the processor 32 or the memory 31 .

[0339] The processor 32 may control the transceiver 34 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 34 may include a transmitter and a receiver. The transceiver 34 may further include an antenna, and the number of antennas may be one or more.

[0340] It should be understood that the various components in the computing device 30 are connected via a bus system, wherein the bus system includes not only a data bus but also a power bus, a control bus and a status signal bus.

[0341] According to one aspect of the present application, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a computer, the computer can perform the method of the above method embodiment. In other words, the present application embodiment also provides a computer program product containing instructions, and when the instructions are executed by a computer, the computer can perform the method of the above method embodiment.

[0342] According to another aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method of the above method embodiment.

[0343] In other words, when implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a server, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server, or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or a data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid state drive (solid state disk, SSD)), etc.

[0344] Those of ordinary skill in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0345] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the module is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0346] The modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. For example, each functional module in each embodiment of the present application may be integrated into a processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0347] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data detection method, characterized in that: include: Using N data detection models, respectively perform data detection on the acquired target data to be detected, and obtain N prediction probabilities that the target data belongs to the target type data, where N is a positive integer greater than 1; Determine a detection threshold corresponding to the N data detection models, wherein the detection threshold is used to determine whether the target data is the target type data; Determining a detection result of the target data based on the N prediction probabilities and the detection threshold; The N data detection models are trained based on first training samples and second training samples, and the second training samples are generated by a generative adversarial network based on the first training samples.

2. The method according to claim 1, characterized in that The determining the detection result of the target data based on the N prediction probabilities and the detection threshold comprises: Determining weights of the N data detection models; Based on the weights of the N data detection models, weighting the N prediction probabilities to obtain a weighted prediction probability of the target data; The detection result is obtained based on the weighted prediction probability and the detection threshold.

3. The method according to claim 2, characterized in that The training process of the N data detection models includes: Processing the first training sample by the generative adversarial network to generate the second training sample; For any training sample of the first training sample and the second training sample, use the N data detection models to perform data detection on the training sample to obtain N prediction probabilities that the training sample belongs to the target type data; For each of the N data detection models, the loss of the data detection model is determined based on the predicted probability of the data detection model and the true label of the training sample, and the parameters in the data detection model are updated based on the loss.

4. The method according to claim 3, characterized in that The first training sample is processed by the generative adversarial network to generate the second training sample, and before the training of the generative adversarial network is completed, the first training sample is processed by the generative adversarial network to generate the second training sample, including: The first training sample is processed by the trained generative adversarial network to generate the second training sample.

5. The method according to claim 3, characterized in that: The using the N data detection models to perform data detection on the training samples to obtain N prediction probabilities that the training samples belong to the target type data includes: Extracting features from the training samples using a deep learning model to obtain feature information of the training samples; Data detection is performed on the feature information of the training samples using the N data detection models to obtain N prediction probabilities that the training samples belong to the target type data.

6. The method according to claim 5, characterized in that Before extracting features from the training samples through the deep learning model to obtain feature information of the training samples, the deep learning model is trained. The training process of the deep learning model includes: Performing feature extraction on the third training sample by using the feature extraction module in the deep learning model to obtain feature information of the third training sample; Processing the feature information of the third training sample through the prediction module in the deep learning model to obtain the prediction probability of the third training sample; The deep learning model is trained based on the predicted probability of the third training sample and the true label of the third training sample.

7. The method according to claim 6, characterized in that The extracting features of the training samples by using a deep learning model to obtain feature information of the training samples includes: The feature extraction module in the trained deep learning model is used to extract features from the training samples to obtain feature information of the training samples.

8. The method according to claim 3, characterized in that The weights of the N data detection models are obtained by training together with the N data detection models, and the training process of the weights of the N data detection models includes: Using the N data detection models obtained through the i-1th round of training, data detection is performed on the i-th round of training samples to obtain the N prediction probabilities of the i-th round, where i is a positive integer; Determine the loss of the N data detection models in the i-th round based on the N prediction probabilities in the i-th round, the weights of the N data detection models in the i-1th round, and the true labels of the i-th round training samples; Based on the loss in the i-th round, the weights in the i-1-th round are updated to obtain the weights of the N data detection models in the i-th round.

9. The method according to claim 8, characterized in that The determining the loss of the N data detection models in the i-th round based on the N prediction probabilities in the i-th round, the weights of the N data detection models in the i-1th round, and the true labels of the i-th round training samples includes: Determine the weighted prediction probability of the i-th round training sample in the i-th round based on the N prediction probabilities of the i-th round and the weight of the i-1-th round; Based on the weighted prediction probability of the i-th round training sample in the i-th round and the true label of the i-th round training sample, the loss of the N data detection models in the i-th round is determined.

10. The method according to claim 3, characterized in that: The detection threshold is obtained by training together with the N data detection models, and the training process of the detection threshold includes: Using the N data detection models obtained through the i-1th round of training, data detection is performed on the i-th round of training samples to obtain the N prediction probabilities of the i-th round, where i is a positive integer; Determine the weighted prediction probability of the training sample in the i-th round based on the N prediction probabilities in the i-th round and the weights of the N data detection models in the i-1th round; Based on the weighted prediction probability of the i-th round training samples and the true labels of the i-th round training samples, the detection threshold of the i-1th round is updated to obtain the detection threshold of the i-th round.

11. The method according to claim 10, characterized in that The updating of the detection threshold of the i-1th round based on the weighted prediction probability of the i-th round training samples and the true label of the i-th round training samples to obtain the detection threshold of the i-th round includes: Determine the evaluation index of the N data detection models in the i-th round based on the weighted prediction probability of the i-th round training samples, the true label of the i-th round training samples and the detection threshold of the i-1-th round; Based on the evaluation indicators of the N data detection models in the i-th round, a detection threshold of the i-th round is determined.

12. The method according to claim 11, characterized in that The evaluation index includes at least one of precision, recall, F1 value, and subject characteristic ROC curve.

13. The method according to claim 11, characterized in that The determining the detection threshold of the i-th round based on the evaluation indicators of the N data detection models in the i-th round includes: Determining a threshold setting indicator for the i-th round based on the evaluation indicator for the i-th round; Based on the threshold setting index of the i-th round, the detection threshold of the i-th round is determined.

14. The method according to claim 13, characterized in that The step of determining the threshold setting index for the i-th round based on the evaluation index for the i-th round includes: The evaluation index of the i-th round is used as the input of a preset threshold adjustment function, and the output of the threshold adjustment function is used as the threshold setting index of the i-th round.

15. The method according to claim 14, characterized in that The step of obtaining the detection threshold of the i-th round based on the threshold setting indicator of the i-th round includes: The detection threshold of the i-1th round is adjusted to determine the threshold corresponding to the maximum threshold setting index of the i-th round as the detection threshold of the i-th round.

16. The method according to claim 10, characterized in that After obtaining the detection threshold of the i-th round, verifying the detection threshold of the i-th round, the verification process of the detection threshold of the i-th round includes: Using the N data detection models obtained through the i-th round of training, data detection is performed on the test sample to obtain N prediction probabilities of the test sample; Determine a weighted prediction probability of the test sample based on the N prediction probabilities of the test sample and the weights of the N data detection models in the i-th round; Determining evaluation indicators of the N data detection models obtained by the i-th round of training based on the weighted prediction probability of the test sample, the detection threshold of the i-th round and the true label of the test sample; Based on the evaluation indicators of the N data detection models obtained in the i-th round of training, the verification result of the detection threshold of the i-th round is determined.

17. The method according to claim 16, characterized in that If the verification result of the detection threshold of the i-th round is qualified, the updating of the N data detection models and the detection threshold is stopped.

18. A data detection device, characterized in that: include: A prediction unit, configured to perform data detection on the acquired target data to be detected respectively through N data detection models, and obtain N prediction probabilities that the target data belongs to target type data, where N is a positive integer greater than 1; a threshold determination unit, used to determine a detection threshold corresponding to the N data detection models, wherein the detection threshold is used to determine whether the target data is the target type data; A detection unit, configured to determine a detection result of the target data based on the N prediction probabilities and the detection threshold; The N data detection models are trained based on first training samples and second training samples, and the second training samples are generated by a generative adversarial network based on the first training samples.

19. A computer device comprising a processor and a memory; The memory is used to store computer programs; The processor is used to execute the computer program to implement the method according to any one of claims 1 to 17.

20. A computer-readable storage medium, characterized in that: For storing computer programs; The computer program enables a computer to execute the method according to any one of claims 1 to 17.

Images