A model training method and related apparatus

Abstract

Embodiments of the application disclose a model training method and related device in the field of artificial intelligence, wherein the method comprises: determining respective negative sample confidence degrees of each unlabeled sample in an unlabeled sample set based on a negative sample measurement mechanism; selecting a target negative sample from the unlabeled sample set according to the respective negative sample confidence degrees of each unlabeled sample; constructing a first training sample set based on the target negative sample and a target positive sample; selecting a reference positive sample and a reference negative sample according to the respective negative sample confidence degrees of each unlabeled sample in the unlabeled sample set except the target negative sample; constructing a second training sample set based on the target negative sample and the reference negative sample, and the target positive sample and the reference positive sample; and training a target classification model based on the first training sample set and the second training sample set. The method can make the trained model more comprehensively and accurately recognize various input data.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a model training method and related apparatus. Background Technology

[0002] In practical applications, there are many situations where there are only positive samples and a large number of unlabeled samples. This usually occurs because it is difficult to obtain negative samples or the negative samples are too diverse and dynamic.

[0003] For the above situation, PU learning (Positive and Unlabeled data learning) from semi-supervised learning is usually used to solve the problem. PU learning is usually based on a two-step method: the first step is to use a pre-defined classification method to select negative samples with high confidence from the unlabeled samples; the second step is to use the selected negative samples and the inherently correct positive samples to form a training sample set, and then use this training sample set to train the model.

[0004] However, the inventors of this application have discovered the following defects in the above-mentioned PU learning implementation method: In the second step, only the negative and positive samples selected in the first step are used for model training, while the remaining unlabeled samples that were not selected as negative samples are discarded. This will result in the trained model having limitations and making it difficult to comprehensively and accurately identify various input data. The reason is that the unlabeled samples that were not selected as negative samples often account for a large proportion and contain rich key information. If this key information is not learned during the model training process, the trained model will have difficulty accurately identifying input data that includes such key information. Summary of the Invention

[0005] This application provides a model training method and related apparatus, which enables the trained model to more comprehensively and accurately identify various input data.

[0006] In view of the above, the first aspect of this application provides a model training method, the method comprising:

[0007] Based on the negative sample measurement mechanism, the negative sample confidence level corresponding to each unlabeled sample in the unlabeled sample set is determined;

[0008] Based on the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, target negative samples are selected from the unlabeled sample set; and a first training sample set is constructed based on the target negative samples and target positive samples.

[0009] Based on the negative sample confidence scores of each unlabeled sample in the reference unlabeled sample set, reference positive samples and reference negative samples are selected from the reference unlabeled sample set, wherein the reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample; a second training sample set is constructed based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample.

[0010] A target classification model is trained based on the first training sample set and the second training sample set.

[0011] A second aspect of this application provides a model training apparatus, the apparatus comprising:

[0012] The confidence determination module is used to determine the negative sample confidence level of each unlabeled sample in the unlabeled sample set based on the negative sample measurement mechanism.

[0013] The first sample set construction module is used to select target negative samples from the unlabeled sample set according to the negative sample confidence scores corresponding to each unlabeled sample in the unlabeled sample set; and to construct a first training sample set based on the target negative samples and target positive samples.

[0014] The second sample set construction module is used to select reference positive samples and reference negative samples from the reference unlabeled sample set based on the negative sample confidence scores corresponding to each unlabeled sample in the reference unlabeled sample set. The reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample. Based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample, a second training sample set is constructed.

[0015] The model training module is used to train the target classification model based on the first training sample set and the second training sample set.

[0016] A third aspect of this application provides a computer device, the device comprising a processor and a memory:

[0017] The memory is used to store computer programs;

[0018] The processor is configured to perform the steps of the model training method as described in the first aspect above, according to the computer program.

[0019] A fourth aspect of this application provides a computer-readable storage medium for storing a computer program for performing the steps of the model training method described in the first aspect.

[0020] A fifth aspect of this application provides a computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the model training method described in the first aspect.

[0021] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0022] This application provides a model training method. When using PU learning to train a target classification model based on unlabeled samples and target positive samples, the method not only selects target negative samples (i.e., relatively reliable negative samples) from the unlabeled sample set according to the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, but also constructs a training sample set using the selected target negative samples and known target positive samples. Furthermore, it performs a second round of screening on the remaining unlabeled samples in the unlabeled sample set according to the negative sample confidence scores of the unlabeled samples other than the target negative samples, in order to select reference negative samples and reference positive samples that have reference value for model training. Then, it constructs another training sample set using the selected reference negative samples, the previously selected target negative samples, the selected reference positive samples, and the known target positive samples. Finally, the two constructed training sample sets are used to collaboratively train the target classification model. In this way, the unlabeled samples other than the target negative sample in the unlabeled sample set are fully and reasonably utilized, so that these unlabeled samples that were not selected as target negative samples can also participate in the training process of the target classification model. Correspondingly, during the model training process, the trained target classification model can learn the key information contained in such unlabeled samples, thereby improving the model performance of the trained target classification model and enabling the target classification model to more comprehensively and accurately identify various input data. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating an application scenario of the model training method provided in the embodiments of this application;

[0024] Figure 2 A schematic flowchart illustrating the model training method provided in this application embodiment;

[0025] Figure 3 A schematic diagram illustrating the implementation principle of the PU Bagging algorithm provided in this application embodiment;

[0026] Figure 4 A schematic diagram illustrating the implementation principle of the SPY algorithm provided in this application embodiment;

[0027] Figure 5 A schematic diagram illustrating the construction principle of the training sample set provided in the embodiments of this application;

[0028] Figure 6 A schematic diagram illustrating the implementation architecture of the training process of the target classification model provided in this application embodiment;

[0029] Figure 7 A schematic diagram illustrating the implementation architecture of the model training method provided in this application embodiment;

[0030] Figure 8 This is a schematic diagram of the structure of a model training device provided in an embodiment of this application;

[0031] Figure 9 This is a schematic diagram of another model training device provided in an embodiment of this application;

[0032] Figure 10 This is a schematic diagram of the structure of another model training device provided in the embodiments of this application;

[0033] Figure 11 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application;

[0034] Figure 12 This is a schematic diagram of the server structure provided in an embodiment of this application. Detailed Implementation

[0035] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0036] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0037] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0038] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0039] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0040] The solutions provided in this application relate to machine learning technology in artificial intelligence, and are specifically illustrated through the following embodiments:

[0041] In related technologies, when using PU learning to train a classification model based on unlabeled and positive samples, reliable negative samples are typically selected from the unlabeled sample set. These selected negative samples are then combined with positive samples to form a training sample set, which is used to train the classification model. Unlabeled samples that are not selected from the unlabeled sample set are usually discarded and not included in the model training process. However, these unlabeled samples often constitute a large proportion and contain rich key information. Discarding these unlabeled samples for model training will result in a classification model that struggles to comprehensively and accurately identify various input data.

[0042] To address the problems existing in the aforementioned related technologies, this application provides a model training method. This method can make reasonable and effective use of unlabeled samples other than reliable negative samples in the unlabeled sample set, so that these unlabeled samples can participate in the model training process, thereby improving the performance of the trained classification model and enabling it to more comprehensively and accurately identify various input data.

[0043] Specifically, in the model training method provided in this application embodiment, firstly, based on the negative sample evaluation mechanism, the confidence level of each unlabeled sample in the unlabeled sample set is determined. Then, based on the confidence level of each unlabeled sample in the unlabeled sample set, relatively reliable target negative samples are selected from the unlabeled sample set, and a first training sample set is constructed based on the selected target negative samples and existing target positive samples. Next, based on the confidence level of each unlabeled sample in the reference unlabeled sample set, reference positive samples and reference negative samples that are valuable for model training are selected from the reference unlabeled sample set. Here, the reference unlabeled sample set includes other unlabeled samples in the unlabeled sample set besides the target negative samples; and a second training sample set is constructed based on the target negative samples and reference negative samples, as well as the target positive samples and reference positive samples. Finally, a target classification model is trained based on the constructed first and second training sample sets.

[0044] The above-mentioned model training method, when using PU learning to train the target classification model based on unlabeled samples and target positive samples, not only selects target negative samples from the unlabeled sample set according to the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, and constructs a training sample set using the selected target negative samples and known target positive samples; it also performs a second round of screening on the remaining unlabeled samples in the unlabeled sample set according to the negative sample confidence scores of the unlabeled samples other than the target negative samples, in order to select reference negative samples and reference positive samples that have reference value for model training, and then constructs another training sample set using the selected reference negative samples, the previously selected target negative samples, the selected reference positive samples, and the known target positive samples; finally, the two constructed training sample sets are used to collaboratively train the target classification model. In this way, the unlabeled samples other than the target negative sample in the unlabeled sample set are fully and reasonably utilized, so that these unlabeled samples that were not selected as target negative samples can also participate in the training process of the target classification model. Correspondingly, during the model training process, the trained target classification model can learn the key information contained in such unlabeled samples, thereby improving the model performance of the trained target classification model and enabling the target classification model to more comprehensively and accurately identify various input data.

[0045] It should be understood that the model training method provided in this application embodiment can be executed by a computer device with data processing capabilities, which can be a terminal device or a server. Specifically, the terminal device can be a computer, smartphone, tablet computer, personal digital assistant (PDA), etc.; the server can be an application server or a web server. In actual deployment, it can be a standalone server, a cluster server composed of multiple physical servers, or a cloud server. The data involved in this application embodiment can be stored on a blockchain.

[0046] To facilitate understanding of the model training method provided in this application embodiment, the following example uses a server as the execution subject of the model training method to illustrate its application scenarios.

[0047] See Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of the model training method provided in the embodiments of this application. For example... Figure 1 As shown, this application scenario includes a server 110 and a database 120. The server 110 can retrieve data from the database 120 via a network, or the database 120 can be integrated into the server 110. The database 120 is used to store unlabeled samples and target positive samples in the target business scenario; the server 110 is used to execute the model training method provided in this application embodiment to train a target classification model suitable for the target business scenario based on the unlabeled samples and target positive samples stored in the database 120.

[0048] In practical applications, when server 110 needs to train a target classification model suitable for a target business scenario, server 110 needs to retrieve the unlabeled sample set and all target positive samples stored in database 120. It should be noted that the unlabeled sample set stores a large number of unlabeled samples, which refer to sample data in the target business scenario whose category is difficult to define; target positive samples refer to sample data in the target business scenario whose category can be determined to be the defined positive category.

[0049] After obtaining the unlabeled sample set and all target positive samples, server 110 can first use a negative sample measurement mechanism to determine the negative sample confidence level corresponding to each unlabeled sample in the unlabeled sample set. As an example, the server can use one or more of the following as the negative sample measurement mechanism: semi-supervised algorithm, unsupervised algorithm, and preset business rules. The semi-supervised algorithm here can be, for example, the SPY mechanism or PU Bagging (Positive and Unlabeled Data Bagging); the unsupervised algorithm here can be, for example, the K-means algorithm or the K-Nearest Neighbor (KNN) algorithm; and the preset business rules here can be data classification rules formulated based on universal data classification rules in the target business scenario.

[0050] After determining the confidence level of each negative sample in the unlabeled sample set, server 110 can perform a first round of screening based on the confidence level of each negative sample to select more reliable target negative samples from the unlabeled sample set. Then, using the selected target negative samples and the target positive samples retrieved from database 120, a first training sample set is constructed.

[0051] After server 110 completes the screening of target negative samples, it can use unlabeled samples from the unlabeled sample set other than the selected target negative samples to form a reference unlabeled sample set. Then, based on the negative sample confidence scores of each unlabeled sample in the reference unlabeled sample set, a second round of screening is performed to select reference positive samples and reference negative samples that can help with model training. Subsequently, the selected reference negative samples and the previously selected target negative samples are used to form a negative sample set, and the selected reference positive samples and the target positive samples retrieved from database 120 are used to form a positive sample set. The second training sample set is then constructed using the negative sample set and the positive sample set.

[0052] Finally, server 110 can use the constructed first training sample set and second training sample set to train the target classification model. For example, the target classification model to be trained may include sub-classification models corresponding to the first training sample set and the second training sample set, as well as a fusion structure. When training the target classification model, the sub-classification model corresponding to the first training sample set can be trained first using the first training sample set, and the sub-classification model corresponding to the second training sample set can be trained using the second training sample set. Then, based on the two trained sub-classification models, the fusion structure in the target classification model is trained using labeled training samples.

[0053] It should be understood that Figure 1The application scenarios shown are merely examples. In practical applications, the model training method provided in this application embodiment can also be applied to other scenarios. No limitations are made here on the applicable application scenarios of the model training method provided in this application embodiment.

[0054] The model training method provided in this application will be described in detail below through method embodiments.

[0055] See Figure 2 , Figure 2 This is a flowchart illustrating the model training method provided in this application. For ease of description, the following embodiments will still use a server as the execution subject of the model training method. Figure 2 As shown, the model training method includes the following steps:

[0056] Step 201: Based on the negative sample measurement mechanism, determine the negative sample confidence level corresponding to each unlabeled sample in the unlabeled sample set.

[0057] When the server uses PU learning to train the target classification model based on unlabeled samples and target positive samples, it needs to first use the negative sample measurement mechanism to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set.

[0058] It should be noted that the unlabeled sample set includes a large number of unlabeled samples, which refer to sample data whose category is difficult to define in the business scenario. Target positive samples refer to sample data in the business scenario whose category can be determined to be a preset positive category. The target classification model is a model used to identify whether the input data belongs to the preset positive category or the preset negative category.

[0059] Taking the method provided in this application embodiment for detecting abnormal behavior of a target account on a target network platform as an example. In this scenario, the target network platform can be a backend system based on the Internet that supports any network service activities, such as a game network platform, a social network platform, or a shopping network platform; the target account can be a specific type of registered account on the target network platform, such as a game account registered as an adult on a game network platform, or an account registered with real name on a social network platform or a shopping network platform; the abnormal behavior to be detected can be an unusual behavior that contradicts the normal behavior of the target account. For example, abnormal behavior can be the behavior of the target account being used by a target object that is not the target account.

[0060] In the above scenarios, specific reference verification methods are typically used to verify target accounts. For target accounts that fail reference verification and are identified as users who are not the target object (the target object is the legitimate user of the target account, meaning an object with the right to use the target account), the feature data of that target account (data reflecting the actual usage characteristics of the target account) can be considered positive target samples. For target accounts that pass reference verification and are identified as users who are the target object, since this identification result is likely generated through proxy verification (such as through black market proxy verification), the feature data of that target account will be considered unlabeled samples. Accordingly, the target classification model trained based on the above unlabeled samples and positive target samples is used to identify whether the actual user of the target account is the target object corresponding to the target account based on the feature data of the input target account.

[0061] It should be understood that the model training method provided in this application embodiment can be applied to any business scenario, and no limitation is made here on the business scenario to which the model training method provided in this application embodiment is applied. When the model training method provided in this application embodiment is applied to other business scenarios, the unlabeled samples and target positive samples will also be different, and the functions to be implemented by the trained target classification model will also be different. No limitation is made here on the unlabeled samples and target positive samples in this application embodiment, nor on the functions of the target classification model in this application embodiment.

[0062] It should be noted that the negative sample measurement mechanism is used to measure the probability that each unlabeled sample belongs to a negative sample; this negative sample measurement mechanism can be implemented, for example, based on a semi-supervised learning algorithm, an unsupervised learning algorithm, or a preset business rule. The negative sample confidence score corresponding to an unlabeled sample is a parameter used to characterize the probability that the unlabeled sample belongs to a negative sample. Taking the method provided in the embodiments of this application for detecting whether a target account has abnormal behavior as an example, in this scenario, the negative sample measurement mechanism can be used to measure the probability that the feature data of the target account verified by reference belongs to the feature data generated by the target object, and the negative sample confidence score corresponding to an unlabeled sample is used to characterize the probability that the feature data in the unlabeled sample is behavioral feature data generated by the target object.

[0063] In one possible implementation, the server can determine the corresponding negative sample confidence level for each unlabeled sample in the unlabeled sample set based on a negative sample measurement mechanism.

[0064] For example, the server can use the 1-DNF algorithm in PU learning to determine the negative sample confidence of each unlabeled sample. The basic idea of ​​the 1-DNF algorithm is to determine whether the frequency of each feature included in each unlabeled sample is higher than its frequency of occurrence in all target positive samples than in each unlabeled sample. If so, the feature is classified into the positive feature set. After completing the classification of each feature included in each unlabeled sample, the number of features belonging to the positive feature set included in each unlabeled sample can be determined as the negative sample confidence of that unlabeled sample. It should be understood that the fewer features belonging to the positive feature set included in an unlabeled sample, the greater the probability that the unlabeled sample is a negative sample; conversely, the more features belonging to the positive feature set included in an unlabeled sample, the less likely that the unlabeled sample is a negative sample.

[0065] It should be understood that in practical applications, the server may also use other negative sample measurement mechanisms (such as any one of semi-supervised learning algorithms, unsupervised learning algorithms, and preset business rules) to determine the corresponding negative sample confidence for each unlabeled sample in the unlabeled sample set; this application does not impose any restrictions on the negative sample measurement mechanism used when only one negative sample measurement mechanism is used to determine the corresponding negative sample confidence for each unlabeled sample.

[0066] In another possible implementation, the server can determine the negative sample confidence level of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism, based on multiple negative sample measurement mechanisms.

[0067] Specifically, the negative sample measurement mechanisms used in the first step of PU learning are usually relatively simple and have significant limitations. Using only one negative sample measurement mechanism to select target negative samples can easily lead to biased sampling, making the selected target negative samples unable to represent the complete negative sample population. To solve this problem, this application innovatively introduces multiple negative sample measurement mechanisms in the first step of PU learning. Through multiple negative sample measurement mechanisms, the negative sample confidence of each unlabeled sample under each negative sample measurement mechanism is determined. Then, based on the negative sample confidence of each unlabeled sample under each negative sample measurement mechanism, the target negative samples under each negative sample measurement mechanism are selected. Since the target negative samples selected through different negative sample measurement mechanisms can represent different types of negative sample features, multiple negative sample measurement mechanisms can select target negative samples with different distributions, thus making the selected target negative samples diverse.

[0068] As an example, the server can choose at least two from semi-supervised learning methods, unsupervised learning methods, and preset business rules as the negative sample measurement mechanism used in the first step of PU learning.

[0069] When the server uses a semi-supervised learning method as its negative sample measurement mechanism, the server can train a semi-supervised classification model based on unlabeled samples in the unlabeled sample set and the target positive sample. Then, through this semi-supervised classification model, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism can be determined.

[0070] Specifically, the server can select simulated positive and negative samples for semi-supervised learning from the unlabeled sample set and all target positive samples according to preset sample selection rules. Then, the selected simulated positive and negative samples are used to train a semi-supervised classification model. Furthermore, the semi-supervised classification model is used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism.

[0071] For example, the server can choose at least one of the more mature semi-supervised learning algorithms PUBagging and SPY in PU learning to determine the negative sample confidence of each unlabeled sample under the semi-supervised mechanism.

[0072] When the server selects PU Bagging to determine the negative sample confidence of each unlabeled sample under the semi-supervised mechanism, the server can divide the unlabeled sample set into m (m is an integer greater than 1) unlabeled sample subsets. Then, for each unlabeled sample subset, the unlabeled samples included in the unlabeled sample subset are used as the first simulated negative samples. Based on the target positive sample and the first simulated negative sample, the first semi-supervised classification model corresponding to the unlabeled sample subset is trained. Then, through the m first semi-supervised classification models, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the first semi-supervised mechanism is determined.

[0073] Figure 3 This is a schematic diagram illustrating the implementation principle of the PU Bagging algorithm provided in an embodiment of this application. Figure 3As shown, the server can divide the unlabeled sample set into m unlabeled sample subsets. For each unlabeled sample subset, all unlabeled sample data excluding those included in that subset can be considered as out-of-bag (OOB) data corresponding to that subset. For each unlabeled sample subset, the server can treat each unlabeled sample data included in that subset as a first simulated negative sample. Then, based on this first simulated negative sample and the target positive sample (directly treated as the first simulated positive sample in this semi-supervised learning process), a first semi-supervised classification model corresponding to that subset is trained. Furthermore, using this first semi-supervised classification model, each out-of-bag data corresponding to that subset is classified to determine the sub-negative sample confidence of each out-of-bag data corresponding to that subset. After training the first semi-supervised classification model for each of the m unlabeled sample subsets and obtaining the sub-negative sample confidence scores of the out-of-bag data for each of the m unlabeled sample subsets, the server can determine the negative sample confidence score of each unlabeled sample in the unlabeled sample set under the first semi-supervised mechanism based on the sub-negative sample confidence scores of the out-of-bag data for each of the m unlabeled sample subsets. Specifically, for each unlabeled sample data in the unlabeled sample set, it should belong to the out-of-bag data corresponding to the m-1 unlabeled sample subsets, that is, the unlabeled sample data should have m-1 sub-negative sample confidence scores. Based on this, the server can calculate the mean of these m-1 sub-negative sample confidence scores as the negative sample confidence score of the unlabeled sample data under the first semi-supervised mechanism. It should be understood that the higher the confidence level of the unlabeled sample data calculated in the above manner under the first semi-supervised mechanism, the more likely the unlabeled sample is to be a positive sample. Conversely, the lower the confidence level of the unlabeled sample data calculated in the above manner under the first semi-supervised mechanism, the more likely the unlabeled sample is to be a negative sample.

[0074] Thus, by using the PU Bagging algorithm to determine the negative sample confidence of each unlabeled sample under the first semi-supervised mechanism, the accuracy and reliability of the determined negative sample confidence can be guaranteed to a certain extent. That is, based on the negative sample confidence determined by the PU Bagging algorithm, it is beneficial to achieve accurate screening of target negative samples.

[0075] When the server selects SPY to determine the negative sample confidence of each unlabeled sample under the semi-supervised mechanism, the server can select a portion of the target positive samples from all the target positive samples, and use the selected target positive samples and each unlabeled sample in the unlabeled sample set as the second simulated negative samples; then, based on the unselected target positive samples and the second simulated negative samples, the second semi-supervised classification model is trained; then, through the second semi-supervised classification model, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the second semi-supervised mechanism is determined.

[0076] Figure 4 This is a schematic diagram illustrating the implementation principle of the SPY algorithm provided in the embodiments of this application. Figure 4 As shown, for all acquired target positive samples, the server can select a subset of target positive samples and treat this subset, along with each unlabeled sample in the unlabeled sample set, as second simulated negative samples. Then, based on these second simulated negative samples and the unselected target positive samples (considered second simulated positive samples in this semi-supervised learning process), a second semi-supervised classification model is trained. After training the second semi-supervised classification model, the server can use this model to classify each target positive sample and each unlabeled sample in the unlabeled sample set, determining the negative sample confidence of each target positive sample under the second semi-supervised mechanism, and the negative sample confidence of each unlabeled sample under the second semi-supervised mechanism. Since the training of the second semi-supervised classification model is based on the premise of using some target positive samples as second simulated negative samples, the second semi-supervised classification model obtained through training classifies these target positive samples. The confidence of these target positive samples under the second semi-supervised mechanism can essentially be used as a classification threshold. For unlabeled samples whose corresponding negative sample confidence is higher than the classification threshold, the server can consider that the unlabeled sample is likely to be a positive sample. For unlabeled samples whose corresponding negative sample confidence is lower than the classification threshold, the server can consider that the unlabeled sample is likely to be a negative sample.

[0077] Thus, by using the SPY algorithm to determine the negative sample confidence of each unlabeled sample under the second semi-supervised mechanism, the classification threshold for screening target negative samples can be determined directly based on the negative sample confidence of the target positive sample under the second semi-supervised mechanism. This avoids the problem of inaccurate threshold setting when manually setting the classification threshold, and ensures the reliability of the subsequently selected target negative samples.

[0078] It should be understood that the above-mentioned PU Bagging algorithm and SPY algorithm are only examples. In practical applications, the server can also use other types of semi-supervised learning algorithms to determine the negative sample confidence of each unlabeled sample under the semi-supervised mechanism. This application does not impose any restrictions on the semi-supervised learning algorithm used.

[0079] When the negative sample measurement mechanism adopted by the server includes unsupervised learning methods, the server can determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the unsupervised mechanism based on the distribution of each unlabeled sample and each target positive sample in the feature space.

[0080] Specifically, the server can map each unlabeled sample to a feature space based on its individual feature data, and similarly, map each target positive sample to a feature space based on its individual feature data. Then, based on the distribution of each unlabeled sample and each target positive sample in the feature space, the server determines the negative sample confidence of each unlabeled sample in the unsupervised sample set under unsupervised conditions. It should be understood that in the feature space, the greater the distance between an unlabeled sample and each target positive sample, the more likely the unlabeled sample is to be a negative sample; conversely, the closer the distance between an unlabeled sample and each target positive sample, the less likely the unlabeled sample is to be a negative sample.

[0081] For example, the server can choose at least one of the Kmeas algorithm and the KNN algorithm to determine the negative sample confidence of each unlabeled sample under unsupervised mechanism.

[0082] When the server selects the Kmeas algorithm to determine the negative sample confidence of each unlabeled sample under the unsupervised mechanism, the server can perform clustering on each unlabeled sample in the unlabeled sample set and each target positive sample to obtain multiple clusters; then, for each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the first unsupervised mechanism is determined according to the number of target positive samples included in its cluster.

[0083] Specifically, after mapping each unlabeled sample and each target positive sample to a feature space, the server selects K points (K is an integer greater than 1) from this feature space as initial cluster centers. Then, it calculates the distance (either Euclidean or cosine distance) between each sample point (i.e., the mapping point of the unlabeled sample and the target positive sample in the feature space) and the K cluster centers, and finds the cluster center closest to that sample point, assigning the sample point to the cluster corresponding to that cluster center. After assigning all sample points to their corresponding clusters, the cluster centers of each cluster are recalculated based on the positions of the sample points in each cluster. This process is repeated until a preset termination condition is met (e.g., the number of iterations reaches a preset number). After the termination condition is met, K clusters are obtained.

[0084] For each unlabeled sample in the unlabeled sample set, the server can determine the negative sample confidence of the unlabeled sample under the first unsupervised mechanism based on the number of target positive samples included in the cluster to which the unlabeled sample belongs. It should be understood that the fewer target positive samples included in the cluster to which the unlabeled sample belongs, the more likely the unlabeled sample is to be a negative sample. Conversely, the more target positive samples included in the cluster to which the unlabeled sample belongs, the less likely the unlabeled sample is to be a negative sample.

[0085] Thus, by using the K-means algorithm to determine the negative sample confidence of each unlabeled sample under the first unsupervised mechanism, we can measure whether an unlabeled sample is a negative sample based on the distribution of feature data included in the unlabeled sample and feature data included in the target positive sample. This can, to a certain extent, ensure the accuracy and reliability of the determined negative sample confidence.

[0086] When the server selects the KNN algorithm to determine the negative sample confidence of each unlabeled sample under the unsupervised mechanism, the server can determine multiple neighboring samples corresponding to each unlabeled sample in the feature space for each unlabeled sample in the unlabeled sample set; then, based on the number of target positive samples included in these multiple neighboring samples, the negative sample confidence of the unlabeled sample under the second unsupervised mechanism is determined.

[0087] Specifically, after mapping each unlabeled sample and each target positive sample to the feature space, the server can determine the distance between the mapping point corresponding to each unlabeled sample data and other mapping points in the feature space (i.e., mapping points corresponding to other unlabeled sample data and target positive sample data), and sort the distances between the mapping point and other mapping points. Based on this, the server selects the K (K is an integer greater than 1) mapping points that are closest to the unlabeled sample. The samples corresponding to these K mapping points are the neighboring samples of the unlabeled sample. Furthermore, the server can determine the negative sample confidence of the unlabeled sample under the second unsupervised mechanism based on the number of target positive samples included in these K neighboring samples. It should be understood that the fewer target positive samples included in the neighboring samples of the unlabeled sample, the more likely the unlabeled sample is to be a negative sample; conversely, the more target positive samples included in the neighboring samples of the unlabeled sample, the less likely the unlabeled sample is to be a negative sample.

[0088] Thus, by using the KNN algorithm to determine the negative sample confidence of each unlabeled sample under the second unsupervised mechanism, we can measure whether an unlabeled sample is a negative sample based on the distance between the unlabeled sample and the target positive sample in the feature space, which can, to a certain extent, ensure the accuracy and reliability of the determined negative sample confidence.

[0089] It should be understood that the K-means and KNN algorithms mentioned above are merely examples. In practical applications, the server may also use other types of unsupervised learning algorithms to determine the negative sample confidence of each unlabeled sample under the unsupervised mechanism. This application does not impose any restrictions on the unsupervised learning algorithms used.

[0090] When the negative sample measurement mechanism adopted by the server includes preset business rules, the server can obtain the reference features corresponding to each unlabeled sample in the unlabeled sample set; then, based on the business rules under the target business scenario and the reference features corresponding to each unlabeled sample in the unlabeled sample set, the server determines the negative sample confidence of each unlabeled sample in the unlabeled sample set under the strong rule mechanism; the target business scenario here refers to the business scenario that generates unlabeled samples.

[0091] Specifically, the server can determine the business rules applicable to classifying sample data into categories within the target business scenario that generates the unlabeled samples, as well as the reference features required for classifying sample data into categories based on these business rules. Based on this, the server needs to obtain the reference features corresponding to each unlabeled sample in the unlabeled sample set. Then, based on the business rules applicable to classifying sample data into categories within the target business scenario, and according to the reference features corresponding to each unlabeled sample, the server determines the negative sample confidence of each unlabeled sample under the strong rule mechanism.

[0092] Taking the method provided in this application embodiment for detecting whether a target account on a target network platform has abnormal behavior as an example, in this scenario, unlabeled samples are feature data of target accounts that have passed reference verification and have been identified as users as the target objects; when the server determines the negative sample confidence of each unlabeled sample under the strong rule mechanism based on business rules, it can obtain the historical reference verification count and the historical reference verification pass count of the target account to which each unlabeled sample belongs in the unlabeled sample set; then, for each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the strong rule mechanism is determined according to the historical reference verification count and the historical reference verification pass count of the target account to which the unlabeled sample belongs.

[0093] Specifically, in scenarios involving detecting abnormal behavior in a target account, the number of times the target account is triggered by reference verification (e.g., facial recognition) and the number of times the target account has successfully passed reference verification in the past can both reflect, to some extent, the likelihood that the actual user of the target account is not the target. For example, if a target account has been triggered by reference verification a large number of times in the past, it indicates that the actual user of the target account is very likely not the legitimate user (i.e., the target); similarly, if a target account has been triggered by reference verification in the past, and has failed reference verification in most of the times it has been triggered, it also indicates that the actual user of the target account is very likely not the legitimate user.

[0094] Based on the above theory, for each unlabeled sample in the unlabeled sample set, the server can obtain at least one of the following: the historical reference verification count (i.e., the number of times reference verification has been triggered historically) and the historical reference verification pass count (i.e., the number of times reference verification has passed historically) of the target account to which the unlabeled sample belongs, as a reference feature corresponding to that unlabeled sample. When the obtained reference feature is the historical reference verification count of the target account to which the unlabeled sample belongs, the server can determine the negative sample confidence of the unlabeled sample under the strong rule mechanism based on the historical reference verification count; it should be understood that the more historical reference verification counts the target account to which the unlabeled sample belongs, the more likely the unlabeled sample is to be a negative sample. When the obtained reference feature includes the historical reference verification count and the historical reference verification pass count of the target account to which the unlabeled sample belongs, the server can calculate the proportion of the historical reference verification pass count to the total historical reference verification count, and determine the negative sample confidence of the unlabeled sample under the strong rule mechanism accordingly; it should be understood that the larger the proportion of the historical reference verification pass count to the total historical reference verification count, the more likely the unlabeled sample is to be a negative sample.

[0095] It should be understood that when the method provided in this application is applied to other business scenarios, the server can obtain other types of data as reference features and use corresponding business rules to determine the negative sample confidence of unlabeled sample data under strong rules. This application does not limit the obtained reference features or the business rules used.

[0096] Since the reference features used to determine the confidence level of negative samples based on business rules are usually relatively simple, which is significantly different from the multiple features used in semi-supervised and unsupervised learning algorithms, determining the confidence level of negative samples based on business rules and selecting target negative samples under a strong rule mechanism based on such determined confidence level can ensure that the target negative samples selected under the strong rule mechanism are significantly different from the target negative samples selected through other mechanisms. This is beneficial for the trained model to learn richer information in the subsequent model training process.

[0097] Step 202: Select target negative samples from the unlabeled sample set according to the negative sample confidence scores of each unlabeled sample in the unlabeled sample set; construct a first training sample set based on the target negative samples and the target positive samples.

[0098] After the server determines the negative sample confidence level of each unlabeled sample in the unlabeled sample set, it can select unlabeled samples that are likely to be negative samples as target negative samples based on the negative sample confidence level of each unlabeled sample. Then, the first training sample set is constructed by using the selected target negative samples and the existing target positive samples.

[0099] The principle of constructing the first training sample set is as follows: Figure 5 As shown in (b), in Figure 5 In the diagram, (a) corresponds to the original unlabeled sample (white part) and the target positive sample (black part); the server selects the target negative sample from the unlabeled sample set, constructs the first training sample set using the target negative sample and the target positive sample, and discards the unlabeled samples in the unlabeled sample set except for the target negative sample, as shown below. Figure 5 As shown in (b).

[0100] In one possible implementation, if the server determines the negative sample confidence level of each unlabeled sample in the unlabeled sample set based solely on a negative sample measurement mechanism during step 201, then the server can select target negative samples from the unlabeled sample set based on the negative sample confidence levels determined in step 201. For example, assuming the server uses the 1-DNF algorithm in step 201 to determine the negative sample confidence level of an unlabeled sample as the number of features belonging to the positive feature set, then during step 202, the server can determine the unlabeled samples with zero features belonging to the positive feature set as target negative samples. Furthermore, the server can directly use the selected target negative samples and target positive samples to form the first training sample set.

[0101] In another possible implementation, if the server determines the negative sample confidence of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism based on multiple negative sample measurement mechanisms when performing step 201, then the server can construct the first training sample set corresponding to each negative sample measurement mechanism based on the target negative sample and each target positive sample under that negative sample measurement mechanism; here, the target negative sample under that negative sample measurement mechanism is selected according to the negative sample confidence of each unlabeled sample in the unlabeled sample set under that negative sample measurement mechanism.

[0102] Specifically, given that the server employs multiple negative sample measurement mechanisms and determines the negative sample confidence of each unlabeled sample under each mechanism, the server needs to construct a corresponding first training sample set for each mechanism. That is, for each negative sample measurement mechanism, the server needs to select unlabeled samples from the unlabeled sample set that are highly likely to be negative based on the negative sample confidence of each unlabeled sample under that mechanism, and use these as target negative samples under that mechanism. Then, using the target negative samples and existing target positive samples, the server constructs the first training sample set under that mechanism.

[0103] For example, when the server simultaneously employs semi-supervised learning methods, unsupervised learning methods, and preset business rules as multiple negative sample measurement mechanisms, the server can construct corresponding first training sample sets for each of the semi-supervised learning methods, unsupervised learning methods, and preset business rules.

[0104] For example, suppose the server uses semi-supervised learning methods including the PU Bagging algorithm and the SPY algorithm. After the server determines the negative sample confidence of each unlabeled sample under the first semi-supervised mechanism using the PU Bagging algorithm, it can select unlabeled samples whose negative sample confidence under the first semi-supervised mechanism is lower than the preset target negative sample classification threshold under the first semi-supervised mechanism as target negative samples under the first semi-supervised mechanism. Then, using the target negative samples under the first semi-supervised mechanism and the existing target positive samples, the first training sample set under the first semi-supervised mechanism is formed. After the server determines the negative sample confidence of each unlabeled sample and each target positive sample under the second semi-supervised mechanism using the SPY algorithm, it can determine the target negative sample classification threshold under the second semi-supervised mechanism based on the negative sample confidence of each target positive sample under the second semi-supervised mechanism. For example, the minimum value can be selected from the negative sample confidence of each target positive sample under the second semi-supervised mechanism as the target negative sample classification threshold under the second semi-supervised mechanism. Then, unlabeled samples whose negative sample confidence under the second semi-supervised mechanism is lower than the target negative sample classification threshold under the second semi-supervised mechanism are selected as target negative samples under the second semi-supervised mechanism. Then, the target negative samples under the second semi-supervised mechanism and the existing target positive samples are used to form the first training sample set under the second semi-supervised mechanism.

[0105] For example, suppose the server uses unsupervised learning methods including the K-means algorithm and the KNN algorithm. After determining the negative sample confidence of each unlabeled sample under the first unsupervised mechanism using the K-means algorithm, the server can select unlabeled samples whose negative sample confidence under the first unsupervised mechanism is lower than a preset target negative sample classification threshold under the first unsupervised mechanism as target negative samples under the first unsupervised mechanism. Then, using the target negative samples under the first unsupervised mechanism and the existing target positive samples, a first training sample set under the first unsupervised mechanism is formed. After determining the negative sample confidence of each unlabeled sample under the second unsupervised mechanism using the KNN algorithm, the server can select unlabeled samples whose negative sample confidence under the second unsupervised mechanism is lower than a preset target negative sample classification threshold under the second unsupervised mechanism as target negative samples under the second unsupervised mechanism. Then, using the target negative samples under the second unsupervised mechanism and the existing target positive samples, a first training sample set under the second unsupervised mechanism is formed.

[0106] For example, suppose the method provided in this application embodiment is used to detect whether a target account has abnormal behavior, and the reference features used are the historical reference verification count and the historical reference verification pass count of the target account corresponding to the unlabeled sample. For each unlabeled sample, the server calculates the ratio of the historical reference verification pass count to the historical reference verification count of the corresponding target account, as the negative sample confidence of the unlabeled sample under the strong rule mechanism. Then the server can preset a ratio threshold as the target negative sample classification threshold under the strong rule mechanism. For unlabeled samples whose negative sample confidence under the strong rule mechanism is higher than the target negative sample classification threshold under the strong rule mechanism, the server can regard them as target negative samples under the strong rule mechanism. Furthermore, the server can use each target negative sample under the strong rule mechanism and each existing target positive sample to form the first training sample set under the strong rule mechanism.

[0107] It should be understood that if the server uses other negative sample measurement mechanisms in step 201 to determine the negative sample confidence of each unlabeled sample under each negative sample measurement mechanism, then the server can correspondingly use other methods to select the target negative sample under each negative sample measurement mechanism based on the negative sample confidence of each unlabeled sample under each negative sample measurement mechanism, and construct the first training sample set corresponding to that negative sample measurement mechanism. This application does not impose any limitations on the method of selecting target negative samples under various negative sample measurement mechanisms.

[0108] In this way, by using multiple negative sample measurement mechanisms, the negative sample confidence of each unlabeled sample under each negative sample measurement mechanism can be determined, and the first training sample set corresponding to each negative sample measurement mechanism can be constructed accordingly. This makes the constructed first training sample set more diverse and carries richer information, which is beneficial for the model to learn richer knowledge based on the first training sample set corresponding to each of the multiple negative sample measurement mechanisms in the subsequent model training process.

[0109] Optionally, to further improve the reliability of the selected target negative samples, in this embodiment of the application, the server may also obtain the latent classification features corresponding to each unlabeled sample in the unlabeled sample set; the latent classification features can be determined according to the business scenario that generates the unlabeled samples; then, the server can select target negative samples from the unlabeled sample set based on the confidence level of each unlabeled sample and the latent classification features.

[0110] In some business scenarios, besides determining whether an unlabeled sample is positive or negative based on its own features, certain features related to the unlabeled sample can also be used to assist in determining whether it is positive or negative. These features used to assist in the determination are called latent classification features. When introducing latent classification features to filter target negative samples, the server can comprehensively consider the confidence level of each unlabeled sample and the latent classification features to select target negative samples from the set of unlabeled samples. For example, the server can determine unlabeled samples whose corresponding negative sample confidence level is within a specific confidence threshold range and / or whose corresponding latent classification features meet preset conditions as target negative samples.

[0111] Taking the method provided in this application embodiment for detecting whether a target account has abnormal behavior as an example, in this scenario, the unlabeled sample is the feature data of the target account that has passed reference verification (such as face verification) and has been identified as the user as the target object; in this case, for each unlabeled sample in the unlabeled sample set, the server can obtain the time spent by the target account to which the unlabeled sample belongs through reference verification, as the latent classification feature corresponding to the unlabeled sample; then, according to the negative sample confidence and latent classification feature corresponding to each unlabeled sample, the target negative sample is selected from the unlabeled sample set.

[0112] In practical business operations, it has been found that there are significant differences in the verification behavior during the reference verification process for target accounts where the actual user is not the legitimate user and for target accounts where the actual user is the legitimate user. Specifically, taking face verification as an example, when required to perform face verification, target accounts where the actual user is the legitimate user can usually pass the face verification quickly, while target accounts where the actual user is not the legitimate user often take a long time to pass the face verification because they need to use a proxy verification process. Based on this, in this embodiment, the server can obtain the time taken for the target account to pass face verification for each unlabeled sample, as the latent classification feature corresponding to that unlabeled sample; when selecting target negative samples from the set of unlabeled samples, the server can select unlabeled samples whose corresponding negative sample confidence is within a specific confidence threshold range and / or whose corresponding latent classification feature (i.e., the time taken to pass face verification) exceeds a preset time, as target negative samples.

[0113] It should be understood that when the method provided in this application is applied to other business scenarios, the server can determine other types of features as latent classification features according to the business scenario, and refer to the latent classification features corresponding to each unlabeled sample when selecting target negative samples. This application does not limit the latent classification features or the method of selecting target negative samples based on latent classification features.

[0114] Thus, introducing latent classification features in the process of selecting target negative samples can help select target negative samples more accurately from the set of unlabeled samples by using the negative sample confidence scores of each unlabeled sample determined in step 201, or make the selected target negative samples more abundant.

[0115] Step 203: Based on the negative sample confidence scores of each unlabeled sample in the reference unlabeled sample set, select reference positive samples and reference negative samples from the reference unlabeled sample set. The reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample. Based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample, construct a second training sample set.

[0116] After the server selects the target negative sample from the unlabeled sample set, it can form a reference unlabeled sample set from the remaining unlabeled samples in the unlabeled sample set (i.e., other unlabeled samples besides the target negative sample). Then, based on the negative sample confidence of each unlabeled sample in the reference unlabeled sample set, reference positive samples and reference negative samples that are valuable for model training are selected from the reference unlabeled sample set. Then, the selected reference negative samples and the target negative sample selected in step 202 are used to form a negative sample set, and the selected reference positive samples and the existing target positive samples are used to form a positive sample set. Combining the negative sample set and the positive sample set yields the second training sample set.

[0117] The principle of constructing the second training sample set is as follows: Figure 5 As shown in (c). Figure 5 (b) is a schematic diagram illustrating the implementation principle of constructing the first training sample set, as shown below. Figure 5 As shown in (b), the white portion corresponds to the target negative sample, the black portion corresponds to the target positive sample, and the gray portion corresponds to unlabeled samples that are not target negative samples (i.e., unlabeled samples in the reference unlabeled sample set). When constructing the first training sample set, the server can discard the unlabeled samples corresponding to the gray portion. When constructing the second training sample set, the server can effectively utilize the unlabeled samples corresponding to the gray portion, and determine each unlabeled sample as a reference negative sample or a reference positive sample based on the confidence level of each unlabeled sample; for example... Figure 5As shown in (c), for the unlabeled samples corresponding to the gray part in (b), they can be further identified as reference negative samples or reference positive samples. When constructing the second training sample set, the server can regard both the reference negative samples and the target negative samples as negative samples (i.e., the white part in (c)) and both the reference positive samples and the target positive samples as positive samples (i.e., the black part in (c)).

[0118] It should be noted that reference negative samples are unlabeled samples in the unlabeled sample set that may be negative. Compared to target negative samples, reference negative samples are less likely to be negative. However, since these unlabeled samples still have high reference value for model training, they need to be used together with target negative samples to construct the negative sample set in the second training sample set. Reference positive samples are unlabeled samples in the unlabeled sample set that may be positive. Compared to reference negative samples, reference positive samples are less likely to be negative; that is, the probability of a reference positive sample being positive is higher than its probability of being negative. Therefore, they can be used together with target positive samples to construct the positive sample set in the second training sample set.

[0119] In one possible implementation, if the server determines the negative sample confidence level of each unlabeled sample in the unlabeled sample set based solely on a negative sample measurement mechanism during step 201, then the server can select reference negative samples and reference positive samples from the reference unlabeled sample set based on the negative sample confidence levels of each unlabeled sample in the reference unlabeled sample set. For example, assuming the server uses the 1-DNF algorithm to determine the number of features belonging to the positive feature set in the unlabeled sample as the negative sample confidence level of that unlabeled sample during step 201, then during step 203, the server can determine, for each unlabeled sample in the reference unlabeled sample set, unlabeled samples whose number of features belonging to the positive feature set is less than a first preset threshold as reference negative samples, and unlabeled samples whose number of features belonging to the positive feature set is greater than a second preset threshold as reference positive samples.

[0120] In another possible implementation, if the server determines the negative sample confidence of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism based on multiple negative sample measurement mechanisms when performing step 201, then the server can construct a second training sample set corresponding to each negative sample measurement mechanism based on the target negative sample and reference negative sample, as well as the target positive sample and the reference positive sample under that negative sample measurement mechanism. Here, the reference negative sample and reference positive sample under that negative sample measurement mechanism are selected based on the negative sample confidence of each unlabeled sample in the reference unlabeled sample set corresponding to that negative sample measurement mechanism under that negative sample measurement mechanism. The reference unlabeled sample set corresponding to that negative sample measurement mechanism includes unlabeled samples in the unlabeled sample set other than the target negative sample under that negative sample measurement mechanism.

[0121] Specifically, given that the server employs multiple negative sample measurement mechanisms and determines the negative sample confidence of each unlabeled sample under each mechanism, the server needs to construct a corresponding second training sample set for each mechanism. That is, for each negative sample measurement mechanism, the server needs to select reference negative samples and reference positive samples from the reference unlabeled sample set based on the negative sample confidence of each unlabeled sample under that mechanism. Then, using the target negative samples and reference negative samples, as well as the target positive samples and reference positive samples under that mechanism, the server constructs the second training sample set corresponding to that mechanism.

[0122] For example, when the server simultaneously employs semi-supervised learning, unsupervised learning, and preset business rules as multiple negative sample measurement mechanisms, the server can construct corresponding second training sample sets for each of the semi-supervised learning, unsupervised learning, and preset business rules. It should be understood that the implementation of constructing the second training sample sets corresponding to each negative sample measurement mechanism is essentially similar to the implementation of constructing the first training sample sets corresponding to each negative sample measurement mechanism; the only difference is that the selection criteria (such as the reference negative sample classification threshold) used by the server to select the reference negative sample under each negative sample measurement mechanism are more lenient than the selection criteria (such as the target negative sample classification threshold) used to select the target negative sample under that negative sample measurement mechanism. Furthermore, the server also needs to determine the selection criteria for the reference positive sample under each negative sample measurement mechanism and select the reference negative sample accordingly. Alternatively, the server can directly consider all unlabeled negative samples in the reference unlabeled sample set, excluding the reference negative sample, as reference positive samples.

[0123] In this way, by using multiple negative sample measurement mechanisms, the negative sample confidence of each unlabeled sample under each negative sample measurement mechanism is determined, and a second training sample set corresponding to each negative sample measurement mechanism is constructed accordingly. On the one hand, this allows the unlabeled samples in the unlabeled sample set other than the target negative sample to be used reasonably and effectively. On the other hand, it also makes the constructed second training sample set more diverse and carries richer information, which is beneficial for the trained model to learn richer knowledge based on the second training sample set corresponding to each of the multiple negative sample measurement mechanisms during the subsequent model training process.

[0124] When the method provided in this application embodiment is used to detect whether a target account has abnormal behavior, in addition to constructing each first training sample set and each second training sample set through the above steps 202 and 203, the construction method of the training sample set can be further supplemented by combining historical information, so that the constructed training sample set is richer.

[0125] Specifically, when unlabeled samples are feature data of target accounts that have passed reference verification (such as face verification) and are identified as using the user as the target object, and target positive samples are feature data of target accounts that have failed reference verification and are identified as using the user who is not the target object, the server can obtain historical positive samples and historical negative samples. Here, historical positive samples are behavioral feature data of target accounts that have triggered reference verification n times (n is an integer greater than 1) and have failed in all n reference verifications, and historical negative samples are feature data of target accounts that have never triggered reference verification. Furthermore, the server can construct a third training sample set based on target negative samples, reference negative samples and historical negative samples, as well as target positive samples, reference positive samples and historical positive samples, and construct a fourth training sample set based on historical negative samples and historical positive samples.

[0126] In scenarios where abnormal behavior is detected in target accounts, the server employs a specific detection mechanism to monitor the usage behavior of each target account. If a target account is detected by the mechanism, it indicates that the actual user of that account may not be the target. In this case, further verification of the user is required through reference verification (such as facial recognition). Based on business experience, for target accounts that are repeatedly detected by the mechanism and required to undergo reference verification, if the actual user of that target account has never passed any reference verification (including failing reference verification and choosing to skip reference verification), it indicates that the actual user of that target account is highly likely not the target. Therefore, the feature data of such target accounts can be considered as historical positive samples. Conversely, for target accounts that are never detected by the mechanism, they have most likely never been used by non-target users. Therefore, the feature data of such target accounts can be considered as historical negative samples.

[0127] After acquiring historical positive and negative samples, the server can use the acquired historical negative samples to form a negative sample set along with the target and reference negative samples in the second training sample set, and use the acquired historical positive samples to form a positive sample set along with the target and reference positive samples in the second training sample set. Then, it can use this negative sample set and this positive sample set to form a third training sample set. It should be understood that when the server constructs second training sample sets corresponding to multiple negative sample measurement mechanisms, it can then combine historical positive and negative samples with the second training sample sets corresponding to each negative sample measurement mechanism to obtain third training sample sets corresponding to each of the multiple negative sample measurement mechanisms. Furthermore, the server can also construct a fourth training sample set independently using the acquired historical positive and negative samples.

[0128] Of course, in practical applications, the server can also use the acquired historical positive and negative samples, along with the first training sample sets corresponding to each negative sample measurement mechanism, to obtain the fifth training sample sets corresponding to each negative sample measurement mechanism. This application does not impose any limitations on the combination of historical positive and negative samples with the training sample sets constructed based on the negative sample measurement mechanisms.

[0129] Thus, in scenarios where abnormal behavior is detected in target accounts, introducing reliable historical information to assist in constructing a training sample set can make the constructed training sample set for model training more diverse, thereby enabling the model to learn richer knowledge during the model training process and improving the performance of the trained model.

[0130] Step 204: Train the target classification model based on the first training sample set and the second training sample set.

[0131] After constructing the first training sample set in step 202 and the second training sample set in step 203, the server can further train the target classification model based on the constructed first and second training sample sets. This target classification model is used to identify whether input data belongs to positive or negative sample data. Taking the method provided in this application embodiment as an example of detecting whether a target account exhibits abnormal behavior, the target classification model is used to identify whether the actual user of the target account is the target object based on the feature data of the input target account.

[0132] In one possible implementation, the server can train the target classification model in the following way: determine each training sample set used to train the target classification model as a basic training sample set; for each basic training sample set, train the corresponding sub-classification model based on the basic training sample set; after completing the training of the sub-classification models corresponding to each basic training sample set, the server can train the fusion structure in the target classification model based on the labeled training samples; the labeled training samples here include sample data and their corresponding labeled classification results; the target classification model includes the fusion structure and the corresponding sub-classification models of each basic training sample set, and the fusion structure is used to fuse the classification results of the corresponding sub-classification models of each basic training sample set.

[0133] Specifically, the server can consider all training sample sets used to train the target classification model as basic training sample sets. For example, if the server uses only one negative sample measurement mechanism to determine the negative sample confidence of each unlabeled sample, the server can construct a first training sample set and a second training sample set. In this case, the server can consider both the first and second training sample sets as basic training sample sets. As another example, if the server uses multiple negative sample measurement mechanisms to determine the negative sample confidence of each unlabeled sample under each mechanism, the server can construct first and second training sample sets corresponding to each of the multiple negative sample measurement mechanisms. In this case, the server can consider both the first and second training sample sets corresponding to each of the multiple negative sample measurement mechanisms as basic training sample sets. For example, when the method provided in this application is applied to a scenario of detecting whether a target account has abnormal behavior, the server can also combine historical information to construct a third training sample set and a fourth training sample set corresponding to each of the various negative sample measurement mechanisms. In this case, the server can regard the first, second, third, and fourth training sample sets corresponding to each of the various negative sample measurement mechanisms as basic training sample sets. In short, all training sample sets used to train the target classification model can be regarded as basic training sample sets.

[0134] The target classification model to be trained includes sub-classification models corresponding to each basic training sample set and a fusion structure. For example, assuming the server constructs p (p is an integer greater than 1) training sample sets for training the target classification model through steps 202 and 203, the target classification model should include p sub-classification models. The sub-classification model corresponding to each basic training sample set is used to determine its corresponding sub-classification result for the data input to the target classification model. For example, in this embodiment, each sub-classification model in the target classification model can be a Deep FM model. The fusion structure is used to fuse the sub-classification results output by each sub-classification model to obtain the classification result output by the target classification model. For example, in this embodiment, the fusion structure in the target classification model can be a self-attention fusion structure, that is, a model structure built based on a self-attention mechanism.

[0135] Figure 6 This is a schematic diagram illustrating the implementation architecture of the training process of the target classification model provided in this application embodiment. For example... Figure 6 As shown, when training a specific target classification model, the sub-classification models corresponding to each basic training sample set in the target classification model can be trained first. Specifically, for each sub-classification model, the server can iteratively train the sub-classification model using the basic training sample set corresponding to that sub-classification model until the sub-classification model meets the preset sub-classification model training termination condition. The sub-classification model training termination condition here can be, for example, the model performance of the sub-classification model meeting the preset performance requirements (such as the classification accuracy reaching the preset accuracy threshold), or it can be that the number of iterations for training the sub-classification model reaches the preset number. This application does not impose any limitations on this.

[0136] After training the sub-classification models corresponding to each basic training sample set in the target classification model, the server can fix the model parameters of each sub-classification model in the target classification model and further train the fusion structure in the target classification model using labeled training samples. Taking the self-attention fusion structure in the target classification model as an example, when training the fusion structure, the server can first classify the sample data in the labeled training samples through each sub-classification model in the target classification model to obtain the sub-classification results corresponding to the sample data; then, through the self-attention fusion structure in the target classification model, the sub-classification results corresponding to the sample data are fused to obtain the predicted classification result corresponding to the sample data; furthermore, based on the predicted classification result and the labeled classification result in the labeled training samples, a loss function is constructed; and based on the loss function, the model parameters of the self-attention fusion structure are adjusted.

[0137] It should be noted that the labeled training samples are pre-constructed training samples specifically designed for training the fusion structure in the target classification model. These labeled training samples include sample data and the corresponding labeled classification results, which are accurate. Since the fusion structure in the target classification model is usually trained after all sub-classification models have been trained, and the fusion structure has relatively few parameters, training it typically does not require a large number of labeled training samples. Taking the method provided in this application embodiment as an example of detecting abnormal behavior in a target account, the labeled training samples can include the feature data of the target account and the corresponding labeled results (i.e., whether the actual user of the target account is the target object corresponding to the target account). These labeled results can be obtained, for example, through a questionnaire survey (e.g., for some target accounts on a target network platform, a questionnaire survey can be used to determine whether the actual user of the target account has been used by other non-target objects).

[0138] like Figure 6 As shown, during the specific training of the fusion structure, the server can input sample data from the labeled training samples into the target classification model. Correspondingly, each sub-classification model in the target classification model can classify the sample data to obtain the corresponding sub-classification results. Then, the self-attention fusion structure in the target classification model can fuse the sub-classification results corresponding to the sample data based on the self-attention mechanism to obtain the predicted classification result corresponding to the sample data. Furthermore, the server can construct a loss function based on the difference between the predicted classification result and the labeled classification result in the labeled training samples, and adjust the model parameters of the self-attention fusion structure in the target classification model with the goal of minimizing the loss function. This process is repeated based on different labeled training samples until the target classification model meets a preset model training termination condition. This termination condition can be, for example, the model performance of the target classification model meeting a preset performance requirement (such as the classification accuracy of the target classification model reaching a preset accuracy threshold), or the number of iterations for training the target classification model reaching a preset number. This application does not impose any limitations on this.

[0139] The above-mentioned model training method, when using PU learning to train the target classification model based on unlabeled samples and target positive samples, not only selects target negative samples from the unlabeled sample set according to the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, and constructs a training sample set using the selected target negative samples and known target positive samples; it also performs a second round of screening on the remaining unlabeled samples in the unlabeled sample set according to the negative sample confidence scores of the unlabeled samples other than the target negative samples, in order to select reference negative samples and reference positive samples that have reference value for model training, and then constructs another training sample set using the selected reference negative samples, the previously selected target negative samples, the selected reference positive samples, and the known target positive samples; finally, the two constructed training sample sets are used to collaboratively train the target classification model. In this way, the unlabeled samples other than the target negative sample in the unlabeled sample set are fully and reasonably utilized, so that these unlabeled samples that were not selected as target negative samples can also participate in the training process of the target classification model. Correspondingly, during the model training process, the trained target classification model can learn the key information contained in such unlabeled samples, thereby improving the model performance of the trained target classification model and enabling the target classification model to more comprehensively and accurately identify various input data.

[0140] To facilitate understanding of the model training method provided in the embodiments of this application, the following example uses the application of this model training method in the scenario of detecting minors' gaming behavior. Figure 7 The schematic diagram shown illustrates the overall training method for this model.

[0141] Currently, some gaming platforms have implemented anti-addiction measures for underage game accounts (i.e., accounts registered under a minor's identity during real-name registration) to prevent minors from becoming addicted to games. These measures can restrict the login time and spending amount of underage game accounts based on factors such as the underage user's age and whether it is a holiday. However, some underage users, in order to avoid being restricted by the anti-addiction measures set by gaming platforms, borrow adult game accounts (i.e., accounts registered under an adult's identity during real-name registration) to play games.

[0142] To prevent underage users from circumventing anti-addiction measures in this way, gaming platforms employ specific detection mechanisms to examine adult game accounts and determine their risk level (reflecting whether the actual user is a minor). For high-risk adult game accounts, the server can trigger a facial verification mechanism, requiring the account holder to verify their identity as a registered user. For game accounts that fail facial verification, the server can confirm that the user is a minor and implement anti-addiction measures; the account's feature data (reflecting the user's behavioral characteristics) will be used as a positive sample. For game accounts that pass facial verification, since the verification result is likely generated by a minor having an adult verify on their behalf, the server cannot accurately determine whether the user is a minor; therefore, the account's feature data will be used as an unlabeled sample.

[0143] like Figure 7 As shown, after the server identifies unlabeled samples and target positive samples in the scenario for detecting minors' gaming behavior using the above method, it can introduce multiple negative sample measurement mechanisms to determine the negative sample confidence level of each unlabeled sample under each mechanism. For example, the server can use two mature semi-supervised algorithms in PU learning, PU Bagging and SPY, to determine the negative sample confidence level of each unlabeled sample under the first semi-supervised mechanism and the second semi-supervised mechanism. The server can also use unsupervised algorithms K-means and KNN to determine the negative sample confidence level of each unlabeled sample under the first unsupervised mechanism and the second unsupervised mechanism. Furthermore, the server can use business rules in the scenario to determine the negative sample confidence level of each unlabeled sample under a strong rule mechanism based on the corresponding reference features. Since the features used to determine the negative sample confidence level of unlabeled samples based on business rules are relatively singular, which is significantly different from the multi-feature approach used in machine learning, incorporating business rules to determine the negative sample confidence level of unlabeled samples can enrich the subsequently selected target negative samples.

[0144] After the server determines the negative sample confidence level of each unlabeled sample under each negative sample measurement mechanism, it can select target negative samples from the unlabeled samples based on the negative sample confidence level of each unlabeled sample under that mechanism. In this embodiment, latent classification features can be further used to assist in selecting target negative samples. Specifically, in actual business operations, it has been found that there are significant differences in the verification behavior during face verification between game accounts used by minors and game accounts used by adults. Specifically, when required to undergo face verification, game accounts used by adults can usually pass face verification quickly, while game accounts used by minors often take longer to pass because minors need to find other adults to verify on their behalf. Based on this, the server can also select target negative samples based on the time taken for the game account to which the unlabeled sample belongs to to pass face verification. The introduction of this latent classification feature helps to make the selected target negative samples more accurate.

[0145] Furthermore, for each negative sample measurement mechanism, the server can use the target negative samples selected based on the negative sample confidence of each unlabeled negative sample under that negative sample measurement mechanism, and each target positive sample, to form a first training sample set corresponding to that negative sample measurement mechanism. In addition, for each negative sample measurement mechanism, the server can also determine a reference unlabeled sample set corresponding to that negative sample measurement mechanism (including unlabeled samples in the unlabeled sample set other than the target negative samples under that negative sample measurement mechanism), and select reference negative samples and reference positive samples from the reference unlabeled sample set based on the negative sample confidence of each unlabeled sample in the reference unlabeled sample set under that negative sample measurement mechanism; then, using the currently selected reference negative samples, the previously selected target negative samples, and the currently selected reference negative samples and target positive samples, a second training sample set corresponding to that negative sample measurement mechanism is constructed.

[0146] Furthermore, the server can incorporate historical information to supplement the construction of the training sample set. For example, the server can use the feature data of game accounts that have triggered face verification multiple times in the past and failed in all of these verifications as historical positive samples, and the feature data of adult game accounts that have never triggered face verification in the past as historical negative samples. Then, for each negative sample measurement mechanism, the server can construct three corresponding training sample sets: the first is the first training sample set composed of target negative samples and target positive samples, as described above; the second is the second training sample set composed of target negative samples and reference negative samples, as well as target positive samples and reference positive samples, as described above; the third is to introduce historical positive samples and historical negative samples based on the second training sample set; in addition, the server can also construct a fourth training sample set using historical positive samples and historical negative samples separately.

[0147] Furthermore, the server can utilize the first, second, third, and fourth training sample sets corresponding to the various negative sample measurement mechanisms mentioned above to train the target classification model. Specifically, the server can treat each training sample set used to train the target classification model as a basic training sample set, and then use each basic training sample set to train its corresponding sub-classification model within the target classification model; here, the sub-classification model corresponding to each basic training sample set can specifically be a Deep FM model. After completing the training of the sub-classification models corresponding to each basic training sample set, the server can further train the self-attention fusion structure in the target classification model based on labeled training samples; here, labeled training samples can include the feature data of the game account and the labeling results corresponding to the behavioral feature data. The labeling results are used to characterize whether the actual user of the game account is an adult or a minor, and these labeling results can be obtained, for example, through a questionnaire survey; since the self-attention fusion structure has a small number of parameters, only a small number of labeled training samples are needed to train it.

[0148] The inventors of this application have found through experiments that applying the method provided in this application to detect gaming behavior of minors significantly improves the identification accuracy of adult game accounts where the actual user is a minor. Specifically, when each sub-classification model in the trained target classification model uses Deep FM, the identification accuracy is improved by 70.6% compared to using a single Logistic Regression classifier. Furthermore, with the same model structure, the target classification model trained using the method provided in this application improves the identification accuracy by 17.37% compared to a classifier trained using traditional PU learning.

[0149] It should be noted that the model training method provided in this application embodiment can be applied not only to the above-mentioned application scenarios for detecting minors' gaming behavior, but also to other application scenarios. For example, this application embodiment can also be used to detect whether the actual user of a target account on a social networking platform or an online shopping platform is the target object corresponding to the target account, where the target object refers to the real-name registered legitimate user corresponding to the target account.

[0150] When the embodiments of this application are applied to an application scenario that detects whether the actual user of a target account is the target object corresponding to the target account, the target positive sample can be the feature data of the target account that has not passed the reference verification and has been identified as the actual user who is not the corresponding target object (used to reflect the usage behavior characteristics of the target account), and the unlabeled sample can be the feature data of the target account that has passed the reference verification and has been identified as the actual user who is the corresponding target object.

[0151] For the set of unlabeled samples that includes all unlabeled samples, the server can use the above method. Figure 2 The illustrated embodiments describe various negative sample measurement mechanisms, determining the negative sample confidence level for each unlabeled sample under each mechanism. Furthermore, for each negative sample measurement mechanism, target negative samples are selected from the unlabeled samples based on their respective negative sample confidence levels. In addition, in this application scenario, the server can further incorporate the time spent verifying the target account corresponding to the unlabeled sample as a latent classification feature, combining this latent feature to select target negative samples.

[0152] In this application scenario, the server constructs the first and second training sample sets corresponding to each of the various negative sample measurement mechanisms in the same way as described above. In this scenario, the server can also introduce historical information to supplement the construction of the training sample sets. Specifically, the server can use the feature data of target accounts that have triggered reference verification multiple times in the past and failed in all of these verifications as historical positive samples, and the feature data of target accounts that have never triggered reference verification in the past as historical negative samples. Similarly, for each negative sample measurement mechanism, the server can also use the training sample set construction method in the previous application scenario to construct the corresponding three training sample sets; and, separately, construct a fourth training sample set using historical positive and historical negative samples. The training method for the target classification model in this application scenario is similar to that in the previous application scenario, and will not be repeated here.

[0153] It should be understood that, in addition to the two application scenarios mentioned above, the embodiments of this application can also be applied to other application scenarios. No limitation is made here on the application scenarios applicable to the model training method provided in the embodiments of this application.

[0154] In response to the model training method described above, this application also provides a corresponding model training apparatus so that the above model training method can be applied and implemented in practice.

[0155] See Figure 8 , Figure 8 This is consistent with the above text Figure 2 The diagram shows the structure of a model training device 800 corresponding to the model training method shown. Figure 8 As shown, the model training device 800 includes:

[0156] The confidence determination module 801 is used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set based on the negative sample measurement mechanism.

[0157] The first sample set construction module 802 is used to select target negative samples from the unlabeled sample set according to the negative sample confidence scores corresponding to each unlabeled sample in the unlabeled sample set; and to construct a first training sample set based on the target negative samples and target positive samples.

[0158] The second sample set construction module 803 is used to select reference positive samples and reference negative samples from the reference unlabeled sample set according to the negative sample confidence scores corresponding to each unlabeled sample in the reference unlabeled sample set. The reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample. Based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample, a second training sample set is constructed.

[0159] The model training module 804 is used to train the target classification model based on the first training sample set and the second training sample set.

[0160] Optional, in Figure 8 Based on the model training device shown, the confidence determination module 801 is specifically used to: determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism based on multiple negative sample measurement mechanisms;

[0161] The first sample set construction module 802 is specifically used to: construct the first training sample set based on the target negative sample and the target positive sample under each negative sample measurement mechanism; the target negative sample under the negative sample measurement mechanism is selected according to the negative sample confidence of each unlabeled sample in the unlabeled sample set under the negative sample measurement mechanism.

[0162] The second sample set construction module 803 is specifically used to: construct the second training sample set for each negative sample measurement mechanism based on the target negative sample, reference negative sample, reference positive sample, and the target positive sample under the negative sample measurement mechanism; the reference negative sample and reference positive sample under the negative sample measurement mechanism are selected according to the negative sample confidence of each unlabeled sample in the reference unlabeled sample set corresponding to the negative sample measurement mechanism; the reference unlabeled sample set corresponding to the negative sample measurement mechanism includes unlabeled samples in the unlabeled sample set other than the target negative sample under the negative sample measurement mechanism.

[0163] Optional, in Figure 8 Based on the model training apparatus shown, the confidence determination module 801 is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under each of the negative sample measurement mechanisms through at least two of the following methods:

[0164] Based on the unlabeled samples in the unlabeled sample set and the target positive sample, a semi-supervised classification model is trained; the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism is determined through the semi-supervised classification model.

[0165] Based on the distribution of each unlabeled sample in the unlabeled sample set and each target positive sample in the feature space, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the unsupervised mechanism is determined.

[0166] Obtain the reference features corresponding to each unlabeled sample in the unlabeled sample set; determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the strong rule mechanism based on the business rules under the target business scenario and the reference features corresponding to each unlabeled sample in the unlabeled sample set; the target business scenario is the business scenario that generates the unlabeled samples.

[0167] Optionally, the confidence determination module 801 is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism through at least one of the following methods:

[0168] The unlabeled sample set is divided into m unlabeled sample subsets, where m is an integer greater than 1. For each unlabeled sample subset, the unlabeled samples included in the unlabeled sample subset are used as first simulated negative samples. Based on the target positive sample and the first simulated negative samples, a first semi-supervised classification model corresponding to the unlabeled sample subset is trained. Through the m first semi-supervised classification models, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the first semi-supervised mechanism is determined.

[0169] Select a portion of the target positive samples from each of the target positive samples, and use the selected target positive samples and each unlabeled sample in the unlabeled sample set as the second simulated negative samples; train a second semi-supervised classification model based on the unselected target positive samples and the second simulated negative samples; determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the second semi-supervised mechanism through the second semi-supervised classification model.

[0170] Optionally, the confidence determination module 801 is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under unsupervised mechanism through at least one of the following methods:

[0171] Clustering is performed on each unlabeled sample and each target positive sample in the unlabeled sample set to obtain multiple clusters; for each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the first unsupervised mechanism is determined according to the number of target positive samples included in its cluster.

[0172] For each unlabeled sample in the unlabeled sample set, multiple neighboring samples corresponding to the unlabeled sample are determined in the feature space; based on the number of target positive samples included in the multiple neighboring samples, the negative sample confidence of the unlabeled sample under the second unsupervised mechanism is determined.

[0173] Optionally, when the unlabeled sample is feature data of a target account that has passed reference verification and is identified as targeting a user, the confidence determination module 801 is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the strong rule mechanism in the following ways:

[0174] Obtain at least one of the following: the number of historical reference verifications and the number of historical reference verifications passed for each unlabeled sample in the unlabeled sample set to its respective target account.

[0175] For each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the strong rule mechanism is determined based on at least one of the historical reference verification count and the historical reference verification pass count of the target account to which the unlabeled sample belongs.

[0176] Optional, in Figure 8 Based on the model training device shown, see Figure 9 , Figure 9 This is a schematic diagram of another model training device 900 provided in an embodiment of this application. Figure 9 As shown, the device also includes:

[0177] The latent feature acquisition module 901 is used to acquire the latent classification features corresponding to each unlabeled sample in the unlabeled sample set; the latent classification features are determined according to the business scenario that generated the unlabeled samples;

[0178] The first sample set construction module 802 is specifically used for:

[0179] Based on the negative sample confidence and latent classification features corresponding to each unlabeled sample in the unlabeled sample set, the target negative sample is selected from the unlabeled sample set.

[0180] Optional, in Figure 9 Based on the model training device shown, when the unlabeled sample is feature data of a target account that has passed reference verification and is identified as the target user, the latent feature acquisition module 901 is specifically used for:

[0181] For each unlabeled sample in the unlabeled sample set, the time taken for the target account to which the unlabeled sample belongs to pass the reference verification is obtained, and this time is used as the latent classification feature corresponding to the unlabeled sample.

[0182] Optional, in Figure 8 Based on the model training apparatus shown, the unlabeled samples are feature data of target accounts that have passed reference verification and are identified as using users as the target object, while the target positive samples are feature data of target accounts that have not passed the reference verification and are identified as using users who are not the target object; see also Figure 10 , Figure 10 This is a schematic diagram of another model training device 1000 provided in an embodiment of this application. (See attached diagram.) Figure 10 As shown, the device also includes:

[0183] The historical sample acquisition module 1001 is used to acquire historical positive samples and historical negative samples; the historical positive samples are the behavioral feature data of target accounts that have triggered the reference verification n times and failed in all n reference verifications, where n is an integer greater than 1; the historical negative samples are the feature data of target accounts that have never triggered the reference verification.

[0184] The third sample set construction module 1002 is used to construct a third training sample set based on the target negative sample, the reference negative sample and the historical negative sample, as well as the target positive sample, the reference positive sample and the historical positive sample;

[0185] The fourth sample set construction module 1003 is used to construct a fourth training sample set based on the historical negative samples and the historical positive samples;

[0186] The model training module 804 is specifically used to train the target classification model based on the first training sample set, the second training sample set, the third training sample set, and the fourth training sample set.

[0187] Optional, in Figure 8 Based on the model training device shown, the model training module 804 is specifically used for:

[0188] Determine the training sample sets used to train the target classification model, and use them as the basic training sample sets;

[0189] For each of the basic training sample sets, a sub-classification model corresponding to the basic training sample set is trained based on the basic training sample set;

[0190] After training the sub-classification models corresponding to each of the basic training sample sets, the fusion structure in the target classification model is trained based on the labeled training samples; the labeled training samples include sample data and their corresponding labeled classification results; the target classification model includes the fusion structure and the sub-classification models corresponding to each of the basic training sample sets, and the fusion structure is used to fuse the classification results of the sub-classification models corresponding to each of the basic training sample sets.

[0191] Optionally, the model training module 804 is specifically used to train the fusion structure in the following manner:

[0192] The sample data in the labeled training samples are classified by each sub-classification model in the target classification model to obtain the sub-classification results corresponding to the sample data.

[0193] By using the self-attention fusion structure in the target classification model, the sub-classification results corresponding to the sample data are fused to obtain the predicted classification result corresponding to the sample data;

[0194] Based on the predicted classification results and the labeled classification results in the labeled training samples, a loss function is constructed; based on the loss function, the model parameters of the self-attention fusion structure are adjusted.

[0195] The aforementioned model training device makes full and reasonable use of unlabeled samples other than the target negative sample in the unlabeled sample set, so that these unlabeled samples that were not selected as target negative samples can also participate in the training process of the target classification model. Accordingly, during the model training process, the trained target classification model can learn the key information contained in such unlabeled samples, thereby improving the model performance of the trained target classification model and enabling the target classification model to more comprehensively and accurately identify various input data.

[0196] This application also provides a computer device for model training. Specifically, the computer device may be a terminal device or a server. The terminal device and server provided in this application will be described below from the perspective of hardware implementation.

[0197] See Figure 11 , Figure 11 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 11 As shown, for ease of explanation, only the parts related to the embodiments of this application are shown. For specific technical details not disclosed, please refer to the method section of the embodiments of this application. The terminal can be any terminal device including mobile phones, tablets, personal digital assistants, point-of-sale (POS) terminals, in-vehicle computers, etc. Taking a computer as an example:

[0198] Figure 11 This is a block diagram illustrating a portion of the structure of a computer associated with the terminal provided in an embodiment of this application. (Reference) Figure 11 The computer includes: a radio frequency (RF) circuit 1110, a memory 1120, an input unit 1130 (including a touch panel 1131 and other input devices 1132), a display unit 1140 (including a display panel 1141), a sensor 1150, an audio circuit 1160 (which can connect to a speaker 1161 and a microphone 1162), a wireless fidelity (WiFi) module 1170, a processor 1180, and a power supply 1190, etc. Those skilled in the art will understand that... Figure 11The computer architecture shown does not constitute a limitation on the computer and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0199] The memory 1120 can be used to store software programs and modules. The processor 1180 executes various computer functions and data processing by running the software programs and modules stored in the memory 1120. The memory 1120 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer (such as audio data, telephone directory, etc.). In addition, the memory 1120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0200] The processor 1180 is the control center of the computer, connecting various parts of the computer through various interfaces and lines. It performs various computer functions and processes data by running or executing software programs and / or modules stored in the memory 1120, and by calling data stored in the memory 1120. Optionally, the processor 1180 may include one or more processing units; preferably, the processor 1180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1180.

[0201] In this embodiment of the application, the processor 1180 included in the terminal also has the following functions:

[0202] Based on the negative sample measurement mechanism, the negative sample confidence level corresponding to each unlabeled sample in the unlabeled sample set is determined;

[0203] Based on the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, target negative samples are selected from the unlabeled sample set; and a first training sample set is constructed based on the target negative samples and target positive samples.

[0204] Based on the negative sample confidence scores of each unlabeled sample in the reference unlabeled sample set, reference positive samples and reference negative samples are selected from the reference unlabeled sample set, wherein the reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample; a second training sample set is constructed based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample.

[0205] A target classification model is trained based on the first training sample set and the second training sample set.

[0206] Optionally, the processor 1180 is further configured to execute steps of any implementation of the model training method provided in the embodiments of this application.

[0207] See Figure 12 , Figure 12 This is a schematic diagram of the structure of a server 1200 provided in an embodiment of this application. The server 1200 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1222 (e.g., one or more processors) and memory 1232, and one or more storage media 1230 (e.g., one or more mass storage devices) for storing application programs 1242 or data 1244. The memory 1232 and storage media 1230 can be temporary or persistent storage. The program stored in the storage media 1230 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the server. Furthermore, the CPU 1222 may be configured to communicate with the storage media 1230 and execute the series of instruction operations in the storage media 1230 on the server 1200.

[0208] Server 1200 may also include one or more power supplies 1226, one or more wired or wireless network interfaces 1250, one or more input / output interfaces 1258, and / or one or more operating systems, such as Windows Server. TM Mac OS X TM Unix TM Linux TM FreeBSD TM etc.

[0209] The steps performed by the server in the above embodiments can be based on this Figure 12 The server structure shown.

[0210] CPU 1222 is used to perform the following steps:

[0211] Based on the negative sample measurement mechanism, the negative sample confidence level corresponding to each unlabeled sample in the unlabeled sample set is determined;

[0212] Based on the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, target negative samples are selected from the unlabeled sample set; and a first training sample set is constructed based on the target negative samples and target positive samples.

[0213] Based on the negative sample confidence scores of each unlabeled sample in the reference unlabeled sample set, reference positive samples and reference negative samples are selected from the reference unlabeled sample set, wherein the reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample; a second training sample set is constructed based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample.

[0214] A target classification model is trained based on the first training sample set and the second training sample set.

[0215] Optionally, the CPU 1222 can also be used to execute any implementation of the model training method provided in the embodiments of this application.

[0216] This application also provides a computer-readable storage medium for storing a computer program that executes any one of the implementation methods of the model training method described in the foregoing embodiments.

[0217] This application also provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the implementation methods of the model training method described in the foregoing embodiments.

[0218] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0219] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0220] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0221] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0222] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing computer programs.

[0223] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0224] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A model training method, characterized in that, The method includes: Based on the negative sample measurement mechanism, the negative sample confidence level of each unlabeled sample in the unlabeled sample set is determined, including: based on multiple negative sample measurement mechanisms, the negative sample confidence level of each unlabeled sample in the unlabeled sample set is determined under each negative sample measurement mechanism, wherein the unlabeled sample is feature data of a target account that has been verified by reference and identified as the user as the target object; Based on the negative sample confidence scores of each unlabeled sample in the unlabeled sample set, target negative samples are selected from the unlabeled sample set; based on the target negative samples and target positive samples, a first training sample set is constructed, wherein the target positive samples are feature data of target accounts that fail the reference verification and are identified as being used by users who are not the target objects; Based on the negative sample confidence scores of each unlabeled sample in the reference unlabeled sample set, reference positive samples and reference negative samples are selected from the reference unlabeled sample set, wherein the reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample; a second training sample set is constructed based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample. Based on the first training sample set and the second training sample set, a target classification model is trained; The determination of the negative sample confidence level of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism, based on multiple negative sample measurement mechanisms, includes at least two of the following: Based on the unlabeled samples in the unlabeled sample set and the target positive sample, a semi-supervised classification model is trained; the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism is determined through the semi-supervised classification model. Based on the distribution of each unlabeled sample in the unlabeled sample set and each target positive sample in the feature space, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the unsupervised mechanism is determined. Obtain the reference features corresponding to each unlabeled sample in the unlabeled sample set; determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the strong rule mechanism based on the business rules under the target business scenario and the reference features corresponding to each unlabeled sample in the unlabeled sample set; the target business scenario is the business scenario that generates the unlabeled samples.

2. The method according to claim 1, characterized in that, The construction of the first training sample set based on the target negative sample and the target positive sample includes: For each of the negative sample measurement mechanisms, a first training sample set is constructed based on the target negative samples and the target positive samples under the negative sample measurement mechanism; the target negative samples under the negative sample measurement mechanism are selected according to the negative sample confidence of each unlabeled sample in the unlabeled sample set under the negative sample measurement mechanism. The construction of a second training sample set based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample, includes: For each of the negative sample measurement mechanisms, a second training sample set is constructed based on the target negative sample, reference negative sample, reference positive sample, and the target positive sample under the negative sample measurement mechanism; the reference negative sample and reference positive sample under the negative sample measurement mechanism are selected according to the negative sample confidence of each unlabeled sample in the reference unlabeled sample set corresponding to the negative sample measurement mechanism; the reference unlabeled sample set corresponding to the negative sample measurement mechanism includes unlabeled samples in the unlabeled sample set other than the target negative sample under the negative sample measurement mechanism.

3. The method according to claim 1, characterized in that, The semi-supervised classification model is trained based on the unlabeled samples in the unlabeled sample set and the target positive sample; the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism is determined by the semi-supervised classification model, including at least one of the following: The unlabeled sample set is divided into m unlabeled sample subsets, where m is an integer greater than 1. For each unlabeled sample subset, the unlabeled samples included in the unlabeled sample subset are used as first simulated negative samples. Based on the target positive sample and the first simulated negative samples, a first semi-supervised classification model corresponding to the unlabeled sample subset is trained. Through the m first semi-supervised classification models, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the first semi-supervised mechanism is determined. Select a portion of the target positive samples from each of the target positive samples, and use the selected target positive samples and each unlabeled sample in the unlabeled sample set as the second simulated negative samples; train a second semi-supervised classification model based on the unselected target positive samples and the second simulated negative samples; determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the second semi-supervised mechanism through the second semi-supervised classification model.

4. The method according to claim 1, characterized in that, The step of determining the negative sample confidence of each unlabeled sample in the unlabeled sample set under unsupervised mechanism based on the distribution of each unlabeled sample and each target positive sample in the feature space includes at least one of the following: Clustering is performed on each unlabeled sample and each target positive sample in the unlabeled sample set to obtain multiple clusters; for each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the first unsupervised mechanism is determined according to the number of target positive samples included in its cluster. For each unlabeled sample in the unlabeled sample set, multiple neighboring samples corresponding to the unlabeled sample are determined in the feature space; based on the number of target positive samples included in the multiple neighboring samples, the negative sample confidence of the unlabeled sample under the second unsupervised mechanism is determined.

5. The method according to claim 1, characterized in that, The process involves obtaining the reference features corresponding to each unlabeled sample in the unlabeled sample set; determining the negative sample confidence of each unlabeled sample in the unlabeled sample set under a strong rule mechanism based on the business rules under the target business scenario and the reference features corresponding to each unlabeled sample in the unlabeled sample set; the target business scenario is the business scenario that generates the unlabeled samples, including: Obtain at least one of the following: the number of historical reference verifications and the number of historical reference verifications passed for each unlabeled sample in the unlabeled sample set to its respective target account. For each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the strong rule mechanism is determined based on at least one of the historical reference verification count and the historical reference verification pass count of the target account to which the unlabeled sample belongs.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Obtain the latent classification features corresponding to each unlabeled sample in the unlabeled sample set; the latent classification features are determined based on the business scenario that generated the unlabeled samples; The step of selecting target negative samples from the unlabeled sample set based on the negative sample confidence scores corresponding to each unlabeled sample in the unlabeled sample set includes: Based on the negative sample confidence and latent classification features corresponding to each unlabeled sample in the unlabeled sample set, the target negative sample is selected from the unlabeled sample set.

7. The method according to claim 6, characterized in that, The step of obtaining the latent classification features corresponding to each unlabeled sample in the unlabeled sample set includes: For each unlabeled sample in the unlabeled sample set, the time taken for the target account to which the unlabeled sample belongs to pass the reference verification is obtained, and this time is used as the latent classification feature corresponding to the unlabeled sample.

8. The method according to claim 1, characterized in that, The method further includes: Obtain historical positive samples and historical negative samples; the historical positive samples are the feature data of target accounts that have triggered the reference verification n times and failed in all n reference verifications, where n is an integer greater than 1; the historical negative samples are the feature data of target accounts that have never triggered the reference verification. A third training sample set is constructed based on the target negative sample, the reference negative sample, and the historical negative sample, as well as the target positive sample, the reference positive sample, and the historical positive sample; a fourth training sample set is constructed based on the historical negative sample and the historical positive sample. The step of training a target classification model based on the first training sample set and the second training sample set includes: The target classification model is trained based on the first training sample set, the second training sample set, the third training sample set, and the fourth training sample set.

9. The method according to claim 1 or 8, characterized in that, The step of training a target classification model based on the first training sample set and the second training sample set includes: Determine the training sample sets used to train the target classification model, and use them as the basic training sample sets; For each of the basic training sample sets, a sub-classification model corresponding to the basic training sample set is trained based on the basic training sample set; After training the sub-classification models corresponding to each of the basic training sample sets, the fusion structure in the target classification model is trained based on the labeled training samples; the labeled training samples include sample data and their corresponding labeled classification results; the target classification model includes the fusion structure and the sub-classification models corresponding to each of the basic training sample sets, and the fusion structure is used to fuse the classification results of the sub-classification models corresponding to each of the basic training sample sets.

10. The method according to claim 9, characterized in that, The process of training the fusion structure in the target classification model based on labeled training samples includes: The sample data in the labeled training samples are classified by each sub-classification model in the target classification model to obtain the sub-classification results corresponding to the sample data. By using the self-attention fusion structure in the target classification model, the sub-classification results corresponding to the sample data are fused to obtain the predicted classification result corresponding to the sample data; Based on the predicted classification results and the labeled classification results in the labeled training samples, a loss function is constructed; based on the loss function, the model parameters of the self-attention fusion structure are adjusted.

11. A model training device, characterized in that, The device includes: The confidence determination module is used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set based on the negative sample measurement mechanism. This includes: determining the negative sample confidence of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism based on multiple negative sample measurement mechanisms. The unlabeled sample is feature data of a target account that has been verified by reference and identified as a user as the target object. The first sample set construction module is used to select target negative samples from the unlabeled sample set according to the negative sample confidence scores corresponding to each unlabeled sample in the unlabeled sample set; and to construct a first training sample set based on the target negative samples and target positive samples, wherein the target positive samples are feature data of target accounts that fail the reference verification and are identified as being used by users who are not the target objects. The second sample set construction module is used to select reference positive samples and reference negative samples from the reference unlabeled sample set based on the negative sample confidence scores corresponding to each unlabeled sample in the reference unlabeled sample set. The reference unlabeled sample set includes unlabeled samples in the unlabeled sample set other than the target negative sample. Based on the target negative sample and the reference negative sample, as well as the target positive sample and the reference positive sample, a second training sample set is constructed. The model training module is used to train the target classification model based on the first training sample set and the second training sample set; The confidence determination module is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under each negative sample measurement mechanism through at least two of the following methods: Based on the unlabeled samples in the unlabeled sample set and the target positive sample, a semi-supervised classification model is trained; the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism is determined through the semi-supervised classification model. Based on the distribution of each unlabeled sample in the unlabeled sample set and each target positive sample in the feature space, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the unsupervised mechanism is determined. Obtain the reference features corresponding to each unlabeled sample in the unlabeled sample set; determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the strong rule mechanism based on the business rules under the target business scenario and the reference features corresponding to each unlabeled sample in the unlabeled sample set; the target business scenario is the business scenario that generates the unlabeled samples.

12. The apparatus according to claim 11, characterized in that, The first sample set construction module is specifically used to: construct the first training sample set based on the target negative sample and the target positive sample under each negative sample measurement mechanism; the target negative sample under the negative sample measurement mechanism is selected according to the negative sample confidence of each unlabeled sample in the unlabeled sample set under the negative sample measurement mechanism. The second sample set construction module is specifically used to: construct the second training sample set for each negative sample measurement mechanism, based on the target negative sample, reference negative sample, reference positive sample, and the target positive sample under the negative sample measurement mechanism; the reference negative sample and reference positive sample under the negative sample measurement mechanism are selected according to the negative sample confidence of each unlabeled sample in the reference unlabeled sample set corresponding to the negative sample measurement mechanism; the reference unlabeled sample set corresponding to the negative sample measurement mechanism includes unlabeled samples in the unlabeled sample set other than the target negative sample under the negative sample measurement mechanism.

13. The apparatus according to claim 11, characterized in that, The confidence determination module is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the semi-supervised mechanism through at least one of the following methods: The unlabeled sample set is divided into m unlabeled sample subsets, where m is an integer greater than 1. For each unlabeled sample subset, the unlabeled samples included in the unlabeled sample subset are used as first simulated negative samples. Based on the target positive sample and the first simulated negative samples, a first semi-supervised classification model corresponding to the unlabeled sample subset is trained. Through the m first semi-supervised classification models, the negative sample confidence of each unlabeled sample in the unlabeled sample set under the first semi-supervised mechanism is determined. Select a portion of the target positive samples from each of the target positive samples, and use the selected target positive samples and each unlabeled sample in the unlabeled sample set as the second simulated negative samples; train a second semi-supervised classification model based on the unselected target positive samples and the second simulated negative samples; determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the second semi-supervised mechanism through the second semi-supervised classification model.

14. The apparatus according to claim 11, characterized in that, The confidence determination module is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under unsupervised mechanism through at least one of the following methods: Clustering is performed on each unlabeled sample and each target positive sample in the unlabeled sample set to obtain multiple clusters; for each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the first unsupervised mechanism is determined according to the number of target positive samples included in its cluster. For each unlabeled sample in the unlabeled sample set, multiple neighboring samples corresponding to the unlabeled sample are determined in the feature space; based on the number of target positive samples included in the multiple neighboring samples, the negative sample confidence of the unlabeled sample under the second unsupervised mechanism is determined.

15. The apparatus according to claim 11, characterized in that, The confidence determination module is specifically used to determine the negative sample confidence of each unlabeled sample in the unlabeled sample set under the strong rule mechanism in the following ways: Obtain at least one of the following: the number of historical reference verifications and the number of historical reference verifications passed for each unlabeled sample in the unlabeled sample set to its respective target account. For each unlabeled sample in the unlabeled sample set, the negative sample confidence of the unlabeled sample under the strong rule mechanism is determined based on at least one of the historical reference verification count and the historical reference verification pass count of the target account to which the unlabeled sample belongs.

16. The apparatus according to any one of claims 11 to 15, characterized in that, The device further includes: The latent feature acquisition module is used to acquire the latent classification features corresponding to each unlabeled sample in the unlabeled sample set; the latent classification features are determined based on the business scenario that generated the unlabeled samples. The first sample set construction module is specifically used for: Based on the negative sample confidence and latent classification features corresponding to each unlabeled sample in the unlabeled sample set, the target negative sample is selected from the unlabeled sample set.

17. The apparatus according to claim 16, characterized in that, The latent feature acquisition module is specifically used for: For each unlabeled sample in the unlabeled sample set, the time taken for the target account to which the unlabeled sample belongs to pass the reference verification is obtained, and this time is used as the latent classification feature corresponding to the unlabeled sample.

18. The apparatus according to claim 11, characterized in that, The device further includes: The historical sample acquisition module is used to acquire historical positive samples and historical negative samples; the historical positive samples are the behavioral feature data of target accounts that have triggered the reference verification n times and failed in all n reference verifications, where n is an integer greater than 1; the historical negative samples are the feature data of target accounts that have never triggered the reference verification. The third sample set construction module is used to construct a third training sample set based on the target negative sample, the reference negative sample and the historical negative sample, as well as the target positive sample, the reference positive sample and the historical positive sample; The fourth sample set construction module is used to construct a fourth training sample set based on the historical negative samples and the historical positive samples; The model training module is specifically used to train the target classification model based on the first training sample set, the second training sample set, the third training sample set, and the fourth training sample set.

19. The apparatus according to claim 11 or 18, characterized in that, The model training module is specifically used for: Determine the training sample sets used to train the target classification model, and use them as the basic training sample sets; For each of the basic training sample sets, a sub-classification model corresponding to the basic training sample set is trained based on the basic training sample set; After training the sub-classification models corresponding to each of the basic training sample sets, the fusion structure in the target classification model is trained based on the labeled training samples; the labeled training samples include sample data and their corresponding labeled classification results. The target classification model includes the fusion structure and the sub-classification models corresponding to each of the basic training sample sets. The fusion structure is used to fuse the classification results of the sub-classification models corresponding to each of the basic training sample sets.

20. The apparatus according to claim 19, characterized in that, The model training module is specifically used to train the fusion structure in the following ways: The sample data in the labeled training samples are classified by each sub-classification model in the target classification model to obtain the sub-classification results corresponding to the sample data. By using the self-attention fusion structure in the target classification model, the sub-classification results corresponding to the sample data are fused to obtain the predicted classification result corresponding to the sample data; Based on the predicted classification results and the labeled classification results in the labeled training samples, a loss function is constructed; Based on the loss function, the model parameters of the self-attention fusion structure are adjusted.

21. A computer device, characterized in that, The device includes a processor and a memory; The memory is used to store computer programs; The processor is configured to execute the model training method according to any one of claims 1 to 10 according to the computer program.

22. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program for performing the model training method according to any one of claims 1 to 10.

23. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or the instructions are executed by the processor, the model training method according to any one of claims 1 to 10 is implemented.

Images