Label classification model training and object screening method, device, and storage medium

Abstract

Embodiments of the present application provide a label classification model training and object screening method and device, and a storage medium, relating to the technical field of artificial intelligence. In the method, a label classification model to be trained is iteratively trained based on a sample data set, and a target label classification model is output. In one iteration process, multiple preset level labels are divided into positive level labels and negative level labels based on the real label values of a sample object under the multiple preset level labels, rather than binding the sample object to one level label. Therefore, when obtaining a target loss function for parameter adjustment based on a first predicted label value of the sample object under the positive level labels and a second predicted label value of the sample object under the negative level labels, the partial order relationship of the sample object under multiple level labels is considered, the label classification model is more reasonable in practical significance, and the effect of screening user account targeted multimedia content based on the label classification model is improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a label classification model training and object screening method, device and storage medium. BACKGROUND

[0002] With the development of Internet technology, various applications are emerging, and people can obtain more and more multimedia content. In order to meet the needs of different target objects for multimedia content and save the time of target objects searching for multimedia content, it is necessary to push multimedia content to each target object, so as to achieve good revenue effect.

[0003] The related technology trains the model by using the softmax function and the cross-entropy loss function to predict the grade label of the target object, and then pushes the related content based on the obtained grade label. However, the above method only learns to improve the score of the actual grade label of the target object, and at the same time suppresses the scores of the grade labels of other grades, ignoring the partial order relationship between each grade label, thereby resulting in low prediction accuracy of the model, and further affecting the effect of pushing multimedia content. SUMMARY

[0004] Embodiments of the present application provide a label classification model training and object screening method, device, equipment and storage medium, which are used to improve the prediction accuracy of the label classification model and the effect of pushing multimedia content.

[0005] In one aspect, the present application provides a label classification model training method, which comprises:

[0006] obtaining a sample data set, wherein each sample data at least contains real label values of a sample object under a plurality of preset grade labels;

[0007] based on the sample data set, iteratively training a label classification model to be trained, and outputting a trained target label classification model, wherein in one iteration process, based on first prediction label values of a sample object in each sample data under corresponding each positive grade label and second prediction label values of the sample object under corresponding each negative grade label, a target loss function for parameter adjustment is obtained, and the each positive grade label and the each negative grade label are obtained based on real label values of the corresponding sample object under the plurality of preset grade labels.

[0008] In one aspect, the present application provides an object screening method, which comprises:

[0009] obtaining feature data of each candidate object;

[0010] input the feature data of each candidate object into a trained target label classification model respectively to obtain a target label value of each candidate object under each preset level label, wherein the trained target label classification model is obtained by using the label classification model training method described above;

[0011] determine a corresponding depth intention score of each candidate object based on the target label value of each candidate object under each preset level label;

[0012] select at least one target object from the candidate objects based on the corresponding depth intention score of each candidate object.

[0013] In one aspect, an embodiment of the present application provides a label classification model training device, which comprises:

[0014] a first obtaining module configured to obtain a sample data set, wherein each sample data comprises at least real label values of a sample object under a plurality of preset level labels;

[0015] a training module configured to perform iterative training on a label classification model to be trained based on the sample data set, and output a trained target label classification model, wherein in one iteration process, a target loss function for parameter adjustment is obtained based on first prediction label values of a sample object in each sample data under corresponding positive level labels and second prediction label values of the sample object under corresponding negative level labels, wherein the positive level labels and the negative level labels are obtained by dividing the plurality of preset level labels based on real label values of the corresponding sample object under the plurality of preset level labels.

[0016] Optionally, the training module further comprises a parameter adjustment module.

[0017] The parameter adjustment module is specifically configured to:

[0018] for each sample data, the following steps are performed respectively:

[0019] determine a first loss value based on the first prediction label values of the sample object in the sample data under the corresponding positive level labels;

[0020] determine a second loss value based on the second prediction label values of the sample object in the sample data under the corresponding negative level labels;

[0021] determine a target loss value corresponding to the sample data based on the first loss value and the second loss value;

[0022] Obtain a target loss function for parameter tuning based on the obtained target loss values respectively corresponding to the sample data.

[0023] Optionally, each sample data further comprises feature data of the sample object;

[0024] The training module further comprises a prediction module.

[0025] The prediction module is specifically configured to:

[0026] Before obtaining the target loss function for parameter tuning based on the first prediction label values of the sample object in each sample data under the corresponding respective positive level labels and the second prediction label values of the sample object under the corresponding respective negative level labels, input the feature data of the sample object contained in the respective sample data into the label classification model to be trained to obtain the first prediction label values of the corresponding respective sample objects under the corresponding respective positive level labels and the second prediction label values of the corresponding respective sample objects under the corresponding respective negative level labels.

[0027] Optionally, the parameter tuning module is further configured to:

[0028] From the respective real label values corresponding to one sample data, determine the first type of real label values greater than or equal to a preset threshold and the second type of real label values less than the preset threshold;

[0029] Take the respective preset level labels corresponding to the first type of real label values as the positive level labels corresponding to the sample object in the one sample data;

[0030] Take the respective preset level labels corresponding to the second type of real label values as the negative level labels corresponding to the sample object in the one sample data.

[0031] Optionally, the training module further comprises a setting module.

[0032] The setting module is specifically configured to:

[0033] Set respective active levels for the plurality of preset level labels; and,

[0034] In the respective positive level labels and the respective negative level labels corresponding to each sample data, set the maximum active level in the respective positive level labels to be less than the minimum active level in the respective negative level labels.

[0035] Optionally, the setting module is further configured to:

[0036] According to a retention time length of the sample object in the target application and an active number of the sample object in the target application, a real label value of the sample object in each of the plurality of preset level labels is determined.

[0037] In one aspect, an embodiment of the present application provides an object screening device, which comprises:

[0038] A second acquisition module is configured to acquire feature data of each candidate object.

[0039] A prediction module is configured to input the feature data of each candidate object into a trained target label classification model to obtain a target label value of each candidate object in each of a plurality of preset level labels, wherein the trained target label classification model is obtained by using the label classification model training device.

[0040] An evaluation module is configured to determine a corresponding deep intention score of each candidate object based on the target label value of each candidate object in each of the plurality of preset level labels.

[0041] A screening module is configured to screen at least one target object from the candidate objects based on the corresponding deep intention score of each candidate object.

[0042] Optionally, the evaluation module is specifically configured to:

[0043] For each candidate object, the following steps are performed:

[0044] The target label value of one candidate object in each of the plurality of preset level labels is normalized to obtain a candidate probability of one candidate object in each of the plurality of preset level labels.

[0045] Based on the obtained candidate probability of each candidate object and the weight corresponding to each of the plurality of preset level labels, a deep intention score of one candidate object is determined.

[0046] Optionally, the screening module is specifically configured to:

[0047] The corresponding deep intention score of each candidate object is sorted in descending order of the deep intention score to obtain a target sorting result.

[0048] The candidate object corresponding to the deep intention score in the first M positions in the target sorting result is taken as a target object, wherein M is greater than or equal to 1.

[0049] Optionally, the candidate object is a candidate object for a target application.

[0050] The screening module is further configured to:

[0051] After the at least one target object is selected from the candidate objects based on the depth intention score corresponding to each of the candidate objects, the related content of the target application is recommended to the at least one target object.

[0052] In an aspect, an embodiment of the present application provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the label classification model training method and / or the object selection method when executing the program.

[0053] In an aspect, an embodiment of the present application provides a computer readable storage medium storing a computer program executable by a computer device, wherein the program, when executed on the computer device, causes the computer device to perform the steps of the label classification model training method and / or the object selection method.

[0054] In an embodiment of the present application, based on the real label values of the sample objects under the multiple preset level labels, the multiple preset level labels are divided into positive level labels and negative level labels corresponding to the sample objects, instead of being limited to binding the sample objects to a certain level label. Therefore, in the training process, when obtaining a target loss function for adjusting model parameters based on the first prediction label values of the sample objects under the corresponding positive level labels and the second prediction label values of the sample objects under the corresponding negative level labels in each sample data, the partial order relationship of the sample objects under multiple level labels is comprehensively considered, so that the label classification model is more reasonable in practical significance, and the prediction effect of the label classification model is improved. In the scenario of targeted delivery of multimedia content, using the above trained target label classification model to predict the target label values of candidate objects can effectively improve the accuracy of label value prediction, so that the target objects are selected from the candidate objects based on the target label values of the candidate objects, and the corresponding multimedia content is pushed to each target object selected, which can effectively improve the effect of targeted delivery of multimedia content. BRIEF DESCRIPTION OF DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0056] Figure 1 A system architecture diagram is provided for an embodiment of the present application.

[0057] Figure 2A flowchart of a label classification model training method provided by an embodiment of the present application is shown in FIG. 1.

[0058] Figure 3 An interface diagram of a novel application provided by an embodiment of the present application is shown in FIG. 2.

[0059] Figure 4 An interface diagram of a shopping application provided by an embodiment of the present application is shown in FIG. 3.

[0060] Figure 5 A diagram of dividing positive and negative grade labels provided by an embodiment of the present application is shown in FIG. 4.

[0061] Figure 6 A flowchart of a label classification model training method provided by an embodiment of the present application is shown in FIG. 5.

[0062] Figure 7 A flowchart of an object screening method provided by an embodiment of the present application is shown in FIG. 6.

[0063] Figure 8 A flowchart of a method for predicting a target label value provided by an embodiment of the present application is shown in FIG. 7.

[0064] Figure 9 A diagram of an advertisement delivery interface provided by an embodiment of the present application is shown in FIG. 8.

[0065] Figure 10 A flowchart of a label classification model training and object screening method provided by an embodiment of the present application is shown in FIG. 9.

[0066] Figure 11 A diagram of an advertisement delivery interface provided by an embodiment of the present application is shown in FIG. 10.

[0067] Figure 12 A structural diagram of a label classification model training device provided by an embodiment of the present application is shown in FIG. 11.

[0068] Figure 13 A structural diagram of an object screening device provided by an embodiment of the present application is shown in FIG. 12.

[0069] Figure 14 A structural diagram of a computer device provided by an embodiment of the present application is shown in FIG. 13. DETAILED DESCRIPTION

[0070] In order to make the objectives, technical solutions and beneficial effects of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely used to explain the present application and should not be used to limit the present application.

[0071] In order to facilitate understanding, the terms involved in the embodiments of the present application are explained below.

[0072] Natural language processing (NLP) is an important direction in the field of computer science and the field of artificial intelligence. It studies various theories and methods that can realize effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, the research in this field will involve natural language, i.e., the language used in daily life, so it is closely related to the study of linguistics. Natural language processing technology usually includes text processing, semantic understanding, machine translation, and artificial intelligence (AI) is the use of digital computers or digital computer-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which attempts to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, enabling machines to have perception, reasoning, and decision-making functions.

[0073] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction system, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0074] Machine learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and other disciplines. It is a discipline that studies how computers simulate or implement human learning behavior to acquire new knowledge or skills, and reorganize existing knowledge structure to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental approach to enabling computers to have intelligence, and its applications are widespread in various fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and inductive learning, etc. For example, in the embodiments of the present application, a label classification model is trained using machine learning technology. After obtaining the label classification model, the label classification model is used to predict the target label value of each candidate object under each preset level label. Then, based on the obtained target label value, at least one target object is selected from the plurality of candidate objects, and the multimedia content is targeted to the selected target object, wherein the multimedia content can be an advertisement, a video, an article, etc.

[0075] Single-label classification task: For a set of feature inputs, only one class label corresponds, and the model learns the corresponding relationship.

[0076] Multi-label classification task: For a set of feature inputs, several class labels correspond, and the model learns the corresponding relationship.

[0077] Softmax function: A function mapping relationship that normalizes a set of data using an exponential method.

[0078] Circle loss function: A new loss function that fully enumerates the prediction score partial order relationship between multiple categories from a unified perspective of cross-entropy loss and hinge loss. The decision boundary is proved to be circular, so it is named circle loss function.

[0079] DeepFM model: Combining the advantages of breadth and depth models, jointly training factorization machine (FM) model and deep neural network (DNN) model, learning low-order feature combination and high-order feature combination.

[0080] The design idea of the embodiments of the present application is introduced below.

[0081] In the scenario of targeted pushing of multimedia content, the model is usually trained using the softmax function and the cross-entropy loss function to predict the level label of the target object. Then, based on the obtained level label, the related content is targeted. However, this method only learns to improve the score of the actual level label of the target object, while suppressing the scores of the level labels of other levels.

[0082] For example, four active level labels are set in advance, and the order from low to high according to the active level is: active level label 0, active level label 1, active level label 2, and active level label 3. Set the actual level label of user A to be active level label 1. When training the model using the softmax function and the cross-entropy loss function, the model only learns to improve the score of user A in the active level label 1, while suppressing the scores of user A in the active level label 0, the active level label 2 and the active level label 3.

[0083] When using the trained model to predict the active level label of the candidate user, the candidate user will get a high score in one active level label and a low score in the other three active level labels, and then the active level label with the high score will be taken as the predicted level label of the candidate user.

[0084] However, in the practical sense, there is a partial order relationship between the various level labels of the user, that is, when predicting the level label, the actual level label of the user account and other level labels lower than the actual level label should be predicted to output a high score. For example, assuming that the actual level label of the user account A is the active level label 1, in the practical sense, since the active level of the active level label 1 is higher than that of the active level label 0, the user account A naturally meets the active level label condition of the high active level, that is, the user account A should obtain a high score in the active level label 1 and the active level label 0, and should not only obtain a high score in the active level label 1.

[0085] In the model training process, if the partial order relationship between the various level labels described above is ignored, the rationality of the model prediction will be affected, thereby leading to low prediction accuracy of the model, and further affecting the effect of targeted delivery of multimedia content.

[0086] Therefore, the embodiment of the present application provides a label classification model training method, in which a sample data set is obtained, wherein each sample data at least contains real label values of a sample object under a plurality of preset level labels. Then, based on the sample data set, a label classification model to be trained is iteratively trained, and a trained target label classification model is output, wherein in one iteration process, based on first prediction label values of the sample object under corresponding each positive level label and second prediction label values under corresponding each negative level label in each sample data, a target loss function for parameter adjustment is obtained, each positive level label and each negative level label are obtained by dividing a plurality of preset level labels based on real label values of the corresponding sample object under the plurality of preset level labels.

[0087] In a possible implementation, after obtaining the label classification model, the label classification model can be used to screen target objects for targeted delivery of multimedia content.

[0088] Specifically, the feature data of each candidate object is obtained, and then the feature data of each candidate object is input into the trained target label classification model to obtain target label values of each candidate object under a plurality of preset level labels. Then, based on the target label values of each candidate object under the plurality of preset level labels, the corresponding deep intention score of each candidate object is determined. Then, based on the corresponding deep intention score of each candidate object, at least one target object is screened from each candidate object. Then, the corresponding multimedia content is pushed to each target object screened.

[0089] In the embodiments of the present application, based on the real label values of the sample objects under the plurality of preset level labels, the plurality of preset level labels are divided into positive level labels and negative level labels corresponding to the sample objects, instead of being limited to binding the sample objects to a certain level label. Therefore, in the training process, based on the first prediction label values of the sample objects in each sample data under the corresponding each positive level label and the second prediction label values of the sample objects under the corresponding each negative level label, when the target loss function is obtained to adjust the model parameters, the partial order relationship of the sample objects under the plurality of level labels is comprehensively considered, so that the label classification model is more reasonable in practical significance, and the prediction effect of the label classification model is improved. In the scenario of targeted delivery of multimedia content, using the above trained target label classification model to predict the target label value of the candidate object can effectively improve the accuracy of label value prediction, so that the target object is selected from each candidate object based on the target label value of the candidate object, and the corresponding multimedia content is pushed to each target object selected, which can effectively improve the effect of targeted delivery of multimedia content.

[0090] Reference Figure 1 FIG. 1 is a system architecture diagram to which a label classification model training method and an object screening method provided by the embodiments of the present application are applicable, and the architecture at least includes a terminal device 101 and a server 102.

[0091] The terminal device 101 can install a target application with a label classification model training function and / or an object screening function. The target application can be a client application, a web application, a program application, etc. The terminal device 101 can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.

[0092] The server 102 can be a background server of the target application, and provides corresponding services for the target application. The server 102 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers. The server 102 can also be a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content distribution networks (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal device 101 and the server 102 can be directly or indirectly connected through wired or wireless communication, which is not limited in the present application.

[0093] The label classification model training method and the object screening method in the embodiments of the present application can be executed by the terminal device 101, or can be executed by the server 102, or the label classification model training method can be executed by the terminal device 101 and the object screening method can be executed by the server 102, or the label classification model training method can be executed by the server 102 and the object screening method can be executed by the terminal device 101. Two of the two implementation modes are described below.

[0094] In the first implementation mode, the label classification model training method and the object screening method are executed by the terminal device 101.

[0095] In the label classification model training phase, the user submits a sample data set on the terminal device 101, wherein each sample data at least contains real label values of a sample object under a plurality of preset level labels. The terminal device 101 iteratively trains the label classification model to be trained based on the sample data set, and outputs a trained target label classification model, wherein in one iteration process, based on the first predicted label values of the sample object under the corresponding each positive level label and the second predicted label values of the sample object under the corresponding each negative level label in each sample data, a target loss function for parameter adjustment is obtained. Each positive level label and each negative level label are obtained by dividing the plurality of preset level labels based on the real label values of the corresponding sample object under the plurality of preset level labels. After obtaining the target label classification model, the target label classification model is saved in the terminal device 101.

[0096] In the object screening phase, the user submits feature data of each candidate object on the terminal device 101, and the terminal device 101 respectively inputs the feature data of each candidate object into the trained target label classification model to obtain target label values of each candidate object under a plurality of preset level labels. Then, based on the target label values of each candidate object under the plurality of preset level labels, the corresponding depth intention score of each candidate object is determined. Then, based on the corresponding depth intention score of each candidate object, at least one target object is screened from each candidate object. Then, the multimedia content is recommended to the at least one target object.

[0097] In the second implementation mode, the label classification model training method and the object screening method are executed by the server 102.

[0098] In the label classification model training phase: the user submits a sample data set on the terminal device 101, wherein each sample data at least contains real label values of a sample object under a plurality of preset level labels respectively. The terminal device 101 sends the sample data set to the server 102. The server 102 iteratively trains the label classification model to be trained based on the sample data set, and outputs a trained target label classification model, wherein in one iteration process, based on the first prediction label values of the sample object under the corresponding each positive level label and the second prediction label values under the corresponding each negative level label in each sample data, a target loss function for parameter adjustment is obtained, each positive level label and each negative level label are obtained based on the real label values of the corresponding sample object under the plurality of preset level labels, and the plurality of preset level labels are divided. After obtaining the target label classification model, the target label classification model is saved in the server 102.

[0099] In the object screening phase: the user submits the feature data of each candidate object on the terminal device 101, and the terminal device 101 sends the feature data of each candidate object to the server 102. The server 102 respectively inputs the feature data of each candidate object into the trained target label classification model to obtain the target label values of each candidate object under the plurality of preset level labels. Then, based on the target label values of each candidate object under the plurality of preset level labels, the corresponding depth intention score of each candidate object is determined respectively. Then, based on the corresponding depth intention score of each candidate object, at least one target object is screened from each candidate object. The server 102 further recommends multimedia content to the at least one target object.

[0100] Based on Figure 1 The system architecture diagram shown in the embodiment of the application provides a label classification model training method, as shown in the flowchart of the method, Figure 2 The flowchart of the method can be executed by the terminal device 101 or the server 102 as shown in Figure 1 The flowchart of the method can be executed by the terminal device 101 or the server 102 as shown in

[0101] Step S201, obtaining a sample data set.

[0102] Specifically, the sample data set includes a plurality of sample data, and each sample data at least contains real label values of a sample object under a plurality of preset level labels respectively, wherein the sample object can be a user account, a team account, a device identifier, etc.

[0103] The preset level labels are respectively set with corresponding levels, wherein the level corresponding to the preset level label can be an active level, an important level, an efficiency level, etc. The real label value of the sample object under the preset level label represents the matching degree of the sample object and the preset level label. The higher the matching degree of the sample object and the preset level label, the greater the real label value of the sample object under the preset level label; the lower the matching degree of the sample object and the preset level label, the smaller the real label value of the sample object under the preset level label.

[0104] In a possible implementation, the preset level labels are active level labels for the target application, and each active level label corresponds to an active level. According to the retention time length of the sample object in the target application and the active times of the sample object in the target application, the real label values of the sample object under the preset level labels are determined respectively.

[0105] Specifically, the target application can be an instant messaging application, a novel application, a video application, a live broadcast application, a shopping application, etc. The retention time length of the sample object in the target application refers to the time length of using the target application after activating the target application, such as one day, one week, one month, one year, etc. The active times of the sample object in the target application include average active times per unit time, total active times, etc., and the unit time can be 1 day, 3 days, 7 days, etc. Once active can be starting the target application, performing an operation in the target application, etc. The longer the retention time length of the sample object in the target application and the greater the active times of the sample object in the target application, the higher the active level of the actual level label corresponding to the sample object.

[0106] For example, it is assumed that the target application is a novel application Y, when the user starts the novel application Y, the novel application Y displays a main interface, as shown in FIG. 1. Figure 3 The user starting the novel application Y or the user clicking a science fiction novel W in the main interface indicates one active time of the user account in the novel application Y. If the user account downloads the novel application Y one week ago, it can be determined that the retention time length of the user account in the novel application Y is one week.

[0107] For example, it is assumed that the target application is a shopping application T, when the user starts the shopping application T, the shopping application T displays a main interface, as shown in FIG. 2. Figure 4 The user starting the shopping application T or the user clicking a short-sleeve purchase link in the main interface indicates one active time of the user account in the shopping application T. If the user account downloads the shopping application T one month ago, it can be determined that the retention time length of the user account in the shopping application T is one month.

[0108] In step S202, based on the sample data set, the label classification model to be trained is iteratively trained, and a trained target label classification model is output.

[0109] Specifically, in each iteration, a subset of sample data is selected from the sample dataset for iterative training. The sample data selected in each iteration can be completely different, or it can contain some identical sample data. Alternatively, in each iteration, all sample data in the sample dataset can be used for training.

[0110] In one iteration, the target loss function for parameter tuning is obtained based on the first predicted label value of the sample object under each positive level label and the second predicted label value under each negative level label of each sample object in each sample data. The positive level label and the negative level label are obtained by dividing multiple preset level labels based on the real label values ​​of the corresponding sample object under multiple preset level labels.

[0111] Specifically, for each sample object in the sample data, based on the actual label values ​​of the sample object under multiple preset level labels, the multiple preset level labels are divided into positive level labels and negative level labels. Among them, the maximum activity level in each positive level label is less than the minimum activity level in each negative level label.

[0112] For example, such as Figure 5 As shown, multiple preset level labels are set, including: activity level label 0 (activity level 0), activity level label 1 (activity level 1), activity level label 2 (activity level 2), activity level label 3 (activity level 3), and activity level label 4 (activity level 4).

[0113] Based on the actual label values ​​of the sample objects under multiple preset level labels, the multiple preset level labels are divided into positive level labels and negative level labels. The positive level labels include activity level label 0, activity level label 1 and activity level label 2, and the negative level labels include activity level label 3 and activity level label 4. The maximum activity level in each positive level label is activity level 2, and the maximum activity level in each negative level label is activity level 3.

[0114] Because when dividing multiple preset level labels into positive and negative level labels, the maximum active level in each positive level label is set to be less than the minimum active level in each negative level label, when training the label classification model, the sample object can obtain high scores in the actual level label and other level labels with lower levels than the actual level label, thereby improving the rationality and accuracy of the model prediction.

[0115] The label classification model to be trained is used to predict first predicted label values of the sample objects in each sample data under corresponding respective positive level labels and second predicted label values of the sample objects under corresponding respective negative level labels, to obtain a target loss function for parameter adjustment. Then the target loss function is used to adjust parameters of the label classification model to be trained. The end condition of training the label classification model can be that the number of iterations reaches a preset number, or the target loss function for parameter adjustment meets a preset condition.

[0116] In the embodiments of the present application, based on the real label values of the sample objects under the plurality of preset level labels, the plurality of preset level labels are divided into positive level labels and negative level labels corresponding to the sample objects, instead of being limited to binding the sample objects to a certain level label. Therefore, when obtaining the target loss function for parameter adjustment based on the first predicted label values of the sample objects in each sample data under corresponding respective positive level labels and the second predicted label values of the sample objects under corresponding respective negative level labels in the training process, the partial order relationship of the sample objects under the plurality of level labels is comprehensively considered, so that the label classification model is more reasonable in practical significance, and the prediction effect of the label classification model is improved.

[0117] Optionally, in the step S201, when constructing the sample data set, the sample data obtained from the target application side is often sparse. If the label classification model is trained based only on these sample data, it is difficult to achieve good prediction effect. Therefore, in the embodiments of the present application, the sample data obtained from the target application side is used as positive sample data, and then sample data is collected from the recommendation log system as negative sample data, and the sample data set is constructed based on the obtained positive sample data and negative sample data.

[0118] Specifically, the sample data obtained from the target application side is the data of sample objects with longer retention duration in the target application and more active times in the target application, and this part of sample objects are deep conversion user accounts of the target application. The negative sample data is the data of non-deep conversion user accounts exposed / clicked / activated sampled from the recommendation log system. Sample data can be obtained from different target application sides, and target label classification models corresponding to different target applications are trained respectively.

[0119] When constructing the real label values of the sample objects under the plurality of preset level labels, N+1 preset level labels are first set, which are level label 0 to level label N, each preset level label corresponds to an active level, which is active level 0 to active level N, and N is an integer greater than 1.

[0120] The preset level label with the active level of 0 is taken as the actual level label corresponding to the negative sample data, and the actual level label corresponding to each positive sample data is determined from each preset level label corresponding to other active levels.

[0121] For the sample object in each positive sample data, the real label value of the actual level label corresponding to the sample object is set to 1, and the real label values of other preset level labels lower than the active level of the actual level label are also set to 1. The real label values of other preset level labels higher than the active level of the actual level label are set to 0.

[0122] For the sample object in each negative sample data, the real label value of the actual level label (the preset level label with the active level of 0) corresponding to the sample object is set to 1, and the real label values of the preset level labels corresponding to the active levels 1 to N are set to 0.

[0123] In the embodiment, the sample data obtained from the target application side is taken as the positive sample data, and the sample data collected from the recommendation log system is taken as the negative sample data, so that the sample data is expanded. Meanwhile, when constructing the real label values of the sample object under the multiple preset level labels, the real label values of the actual level label corresponding to the object and other preset level labels lower than the active level of the actual level label are all filled with high scores, so that the partial order relationship between the preset level labels can be learned during model training, thereby effectively improving the accuracy and rationality of the label classification model.

[0124] Optionally, in the step S202, each positive level label and each negative level label corresponding to the sample object in each sample data is obtained in the following manner:

[0125] From the real label values corresponding to each sample data, the first type of real label values greater than or equal to a preset threshold value and the second type of real label values less than the preset threshold value are determined. Then, each preset level label corresponding to the first type of real label values is taken as the positive level label corresponding to the sample object in a sample data. Each preset level label corresponding to the second type of real label values is taken as the negative level label corresponding to the sample object in the sample data.

[0126] Specifically, the preset threshold value is an anchor point created when the label classification model is trained, which is used to prevent excessive deviation during learning. The real label values of each positive level label corresponding to the sample object are all greater than or equal to the preset threshold value, and the real label values of each negative level label corresponding to the sample object are all less than the preset threshold value.

[0127] For example, when constructing the level label, a plurality of preset level labels are set, including an active level label 0 (active level 0), an active level label 1 (active level 1), an active level label 2 (active level 2), an active level label 3 (active level 3), and an active level label 4 (active level 4).

[0128] When the actual level label corresponding to the sample object is the active level label 2, the true label value of the active level label 2 is set to 1, and the true label values corresponding to the active level label 0 and the active level label 1 are both set to 1. The true label values corresponding to the active level label 3 and the active level label 4 are both set to 0.

[0129] When the model is trained, the preset threshold is set to 1, and since the true label values corresponding to the active level label 0, the active level label 1, and the active level label 2 are all greater than or equal to 1, the active level label 0, the active level label 1, and the active level label 2 are used as the positive level labels of the sample object, and the active level label 3 and the active level label 4 are used as the negative level labels of the sample object.

[0130] In the embodiment of the present application, when constructing the level label of the sample data, the plurality of preset level labels corresponding to the sample object are divided into positive level labels with true label values greater than or equal to a preset threshold and negative level labels with true label values less than the preset threshold, so that the label classification model learns the partial order relationship between the preset level labels in the training process, prevents excessive deviation in the learning process, and improves the prediction effect of the label classification model.

[0131] Optionally, in the above step S202, each sample data further includes feature data of the sample object, wherein the feature data of the sample object includes age, gender, city, occupation, education level, historical behavior data, and the like.

[0132] The feature data of the sample object included in each sample data is input into the label classification model to be trained, to obtain first prediction label values of the corresponding sample object under each corresponding positive level label and second prediction label values of the corresponding sample object under each corresponding negative level label.

[0133] Specifically, the label classification model to be trained can be any deep learning model. The label classification model to be trained extracts features of the feature data of the sample object to obtain a feature vector of the sample object. Then, based on the feature vector of the sample object, the first prediction label values of the sample object under each positive level label and the second prediction label values of the sample object under each negative level label are predicted.

[0134] For example, as shown in FIG. 2, the feature data of the sample object includes age, gender, city, occupation, education level, and historical behavior data. Figure 6As shown, the sample data includes feature data of a sample user account and real label values of the sample user account under 5 preset level labels respectively, wherein the feature data includes age, gender, city, education level, and historical behavior data. The 5 preset level labels are active level label 0 (active level 0), active level label 1 (active level 1), active level label 2 (active level 2), active level label 3 (active level 3), and active level label 4 (active level 4). The real label values of the sample user account under the active level label 0, the active level label 1, and the active level label 2 are all 1. The real label values of the sample user account under the active level label 3 and the active level label 4 are both 0.

[0135] The feature data of the sample user account is input into the label classification model (DeepFM model) to be trained, and the DeepFM model performs feature extraction on the feature data of the sample user account to obtain a feature vector of the sample user account. Then, based on the feature vector of the sample user account, the predicted label values of the sample user account under each preset level label are predicted.

[0136] Since the real label values of the sample user account under the active level label 0, the active level label 1, and the active level label 2 are all greater than or equal to the preset threshold 1, the active level label 0, the active level label 1, and the active level label 2 are determined as the positive level labels corresponding to the sample user account. The predicted label values of the sample user account under the active level label 0, the active level label 1, and the active level label 2 are taken as the first predicted label values of the sample user account under each positive level label.

[0137] Since the real label values of the sample user account under the active level label 3 and the active level label 4 are both less than the preset threshold 1, the active level label 3 and the active level label 4 are determined as the negative level labels corresponding to the sample user account. The predicted label values of the sample user account under the active level label 3 and the active level label 4 are taken as the second predicted label values of the sample user account under each positive level label.

[0138] Based on the first predicted label values of the sample user account under the active level label 0, the active level label 1, and the active level label 2, and the second predicted label values of the sample user account under the active level label 3 and the active level label 4, a target loss function is determined. Then, the target loss function is used to adjust the parameters of the DeepFM model, and the next iteration training is performed until the target loss function meets the preset condition, and the trained target label classification model is output.

[0139] In a possible implementation, a target loss function used when training the label classification model is a circle loss function. Specifically, in each iteration of the training process, the target loss function used for parameter tuning is determined in the following manner:

[0140] For each sample data, the following steps are performed:

[0141] Based on the first predicted label value of the sample object in one sample data under the corresponding each positive level label, a first loss value is determined. Then, based on the second predicted label value of the sample object in one sample data under the corresponding each negative level label, a second loss value is determined. According to the first loss value and the second loss value, a target loss value corresponding to one sample data is determined. Based on the obtained target loss value corresponding to each sample data respectively, a target loss function used for parameter tuning is obtained.

[0142] In a specific implementation, according to the first loss value and the second loss value, the target loss value corresponding to one sample data satisfies the following formula (1):

[0143]

[0144] Wherein, L k denotes the target loss value corresponding to the sample data k, Ω pos denotes the positive level label set, Ω neg denotes the negative level label set, s j denotes the first loss value of the sample object S under the negative level label j, s i denotes the second loss value of the sample object S under the negative level label i.

[0145] The sum of the target loss values corresponding to each sample data respectively is obtained to obtain the target loss function used for parameter tuning, and then the target loss function and an optimizer (Optimizer) are used to perform parameter optimization on the label classification model to be trained.

[0146] In the embodiments of the present application, the circle loss function is used to adjust the parameters of the label classification model, and the partial order relationship between each level label is fully considered, so that more label realistic meanings are used when the model is learned, thereby making the label classification model more reasonable and accurate.

[0147] Based on Figure 1 the system architecture diagram shown, the embodiments of the present application provide a flow of an object screening method, as Figure 7 shown, the flow of the method can be executed by Figure 1 the terminal device 101 or the server 102 shown, including the following steps:

[0148] In step S701, the feature data of each candidate object is obtained.

[0149] Specifically, each candidate object can be a candidate object for a target application. For different target applications, the process of the above-mentioned label classification model training method can be adopted to train different target label classification models, and then the corresponding target label classification model is adopted to screen target objects for the target application.

[0150] The feature data of the candidate object includes age, gender, city, occupation, education level, historical behavior data, etc.

[0151] Step S702, respectively input the feature data of each candidate object into the trained target label classification model to obtain the target label value of each candidate object under each of the plurality of preset level labels.

[0152] Specifically, the training process of the trained target label classification model has been described in the foregoing, which will not be repeated here. For each candidate object, the feature data of the candidate object is input into the trained target label classification model. The target label classification model extracts features from the feature data of the candidate object to obtain a feature vector of the candidate object, and then predicts the target label value of the candidate object under the plurality of preset level labels based on the feature vector of the candidate object.

[0153] For example, as shown in FIG. 6, the plurality of preset level labels are set to include: an active level label 0 (active level 0), an active level label 1 (active level 1), an active level label 2 (active level 2), an active level label 3 (active level 3), and an active level label 4 (active level 4). Figure 8 The feature data of the candidate user includes 25 years old, female, Shanghai, undergraduate, and historical behavior data. The feature data of the candidate user account is input into the trained target label classification model (DeepFM model), and the DeepFM model extracts features from the feature data of the candidate user account to obtain a feature vector of the candidate user account. Then, based on the feature vector of the candidate user account, the target label value of the candidate user account under each of the above-mentioned five preset level labels is predicted, which are respectively: active level label 0 (target label value = 1), active level label 1 (target label value = 1), active level label 2 (target label value = 1), active level label 3 (target label value = -1), and active level label 4 (target label value = -1).

[0154] Step S703, based on the target label value of each candidate object under each of the plurality of preset level labels, respectively determine the corresponding deep intention score of each candidate object.

[0155]

[0156] ​Specifically, the deep intention score represents a matching degree of the candidate object and subsequent targeted multimedia content, where the targeted multimedia content can be a promotional advertisement of a target application, a promotional advertisement of a commodity, or a video, an article, an audio, etc. The higher the deep intention score, the higher the matching degree of the candidate object and the subsequent targeted multimedia content. Screening the candidate object with a high deep intention score for targeted multimedia content can effectively improve the delivery effect.

[0157] In a possible implementation, the following steps are performed for each candidate object respectively:

[0158] The target label values of a candidate object under the plurality of preset level labels are normalized to obtain candidate probabilities of the candidate object under the plurality of preset level labels, and then a deep intention score of the candidate object is determined based on the obtained candidate probabilities and weights corresponding to the plurality of preset level labels respectively.

[0159] Specifically, the target label values of a candidate object under the plurality of preset level labels can be normalized by using a Sigmoid function to obtain candidate probabilities of the candidate object under the plurality of preset level labels. The weights corresponding to the plurality of preset level labels respectively can be preset, and each weight represents an influence degree of the preset level label on the deep intention score. The weights corresponding to the plurality of preset level labels respectively can be adjusted according to actual conditions. The calculation formula of the deep intention score is specifically shown in the following formula (2):

[0160]

[0161] wherein, X represents the deep intention score, s i represents the target label value of the candidate object S under the preset level label i, a i represents the weight corresponding to the preset level label i.

[0162] For example, it is assumed that the target label values of a candidate user account output by the target label classification model under five preset level labels are: active level label 0 (target label value = 1), active level label 1 (target label value = 1), active level label 2 (target label value = 1), active level label 3 (target label value = -1), and active level label 4 (target label value = -1).

[0163] After the normalization of the above target label values, the candidate probabilities of the candidate object under the plurality of preset level labels are: active level label 0 (candidate probability = 0.76), active level label 1 (candidate probability = 0.76), active level label 2 (candidate probability = 0.76), active level label 3 (0.27), and active level label 4 (0.27).

[0164] It is set that the active level label 0 corresponds to the weight 0, the active level label 1 corresponds to the weight 1, the active level label 2 corresponds to the weight 2, the active level label 3 corresponds to the weight 3, and the active level label 4 corresponds to the weight 4. The depth intention score of the candidate user account is calculated by using the above formula (2) = 4.17 points.

[0165] It should be noted that the implementation manner of determining the depth intention score of the candidate object in the embodiments of the present application is not limited to the above-mentioned one, and the depth intention score of the candidate object can also be determined directly according to the target label value of the candidate object under each of the plurality of preset level labels and the weight corresponding to each of the plurality of preset level labels. The present application does not make specific limitations.

[0166] In step S704, at least one target object is selected from the candidate objects based on the depth intention score corresponding to each of the candidate objects.

[0167] Specifically, when selecting the candidate objects, the embodiments of the present application at least provide the following two implementation manners:

[0168] In one possible implementation manner, the depth intention scores corresponding to each of the candidate objects are sorted in descending order of the depth intention scores, and a target sorting result is obtained. The candidate objects corresponding to the depth intention scores in the top M positions in the target sorting result are taken as the target objects, where M is greater than or equal to 1.

[0169] For example, it is set that M = 3, the depth intention score of the candidate user account A is 4.5 points, the depth intention score of the candidate user account B is 4 points, the depth intention score of the candidate user account C is 3 points, the depth intention score of the candidate user account D is 3.7 points, and the depth intention score of the candidate user account F is 6 points.

[0170] The depth intention scores corresponding to each of the candidate objects are sorted in descending order of the depth intention scores, and the target sorting result obtained is: the candidate user account F, the candidate user account A, the candidate user account B, the candidate user account D, and the candidate user account C. The candidate user account F and the candidate user account A are taken as the target user accounts.

[0171] In one possible implementation manner, the candidate objects with the depth intention score greater than or equal to a preset score in each of the candidate objects are taken as the target objects, where M is greater than or equal to 1.

[0172] For example, it is assumed that the preset score is set to 4, the deep intention score of the candidate user account A is 4.5, the deep intention score of the candidate user account B is 4, the deep intention score of the candidate user account C is set to 3, the deep intention score of the candidate user account D is 3.7, and the deep intention score of the candidate user account F is 6.

[0173] Since the deep intention scores of the candidate user account A, the candidate user account B, the candidate user account C, and the candidate user account F are all greater than or equal to the preset score, the candidate user account A, the candidate user account B, the candidate user account C, and the candidate user account F are regarded as target user accounts.

[0174] Optionally, after the at least one target object is selected from the candidate objects, the corresponding multimedia content is pushed to the at least one target object. When the candidate objects are candidate objects of the target application, after the at least one target object is selected from the candidate objects, the related content of the target application is recommended to the at least one target object.

[0175] For example, it is assumed that the target application is a novel application Y, and the candidate user accounts are user accounts registered in an instant messaging application. The candidate user account A is selected as a target user account from the candidate user accounts, and a promotion advertisement of the novel application Y is targetedly pushed to the candidate user account A. After the candidate user account A logs in the instant messaging application, the instant messaging application loads and displays the promotion advertisement of the novel application Y, as shown in Figure 9 The promotion advertisement of the novel application Y is displayed in a circle of friends interface of the instant messaging application. The user can click a “learn more” button to download or start the novel application Y.

[0176] In the embodiment of the present application, in the process of training the label classification model, the plurality of preset level labels corresponding to the sample object are divided into positive level labels and negative level labels, and then the target loss function for parameter tuning is determined based on the predicted label values under the positive level labels and the negative level labels, so that the model learns the partial order relationship between the level labels in the training process, thereby improving the rationality and accuracy of the model prediction, and further improving the accuracy of user screening and the effect of targetedly pushing multimedia content.

[0177] In order to better explain the embodiments of the present application, a label classification model training method and an object screening method provided by the embodiments of the present application are introduced below by taking the scenario of targetedly pushing advertisements as an example, which is executed by a server, as shown in Figure 10 The method includes the following steps:

[0178] Step one, constructing a sample data set.

[0179] Obtaining positive sample data from the side of the advertisement master corresponding to the novel application Y, wherein the positive sample data is the data of the sample user account with a longer retention time in the novel application Y and more active times in the novel application Y, and the sample user account is a deep conversion user account of the novel application Y. The negative sample data is the data of the non-deep conversion user account exposed, clicked or activated from the recommendation log system. Based on the obtained positive sample data and negative sample data, a sample data set is constructed.

[0180] Step two, constructing sample level labels.

[0181] Five preset level labels are set, which are active level label 0, active level label 1, active level label 2, active level label 3 and active level label 4. Each preset level label corresponds to an active level, which is active level 0, active level 1, active level 2, active level 3 and active level 4. The active level label 0 is used as the actual level label corresponding to the negative sample data, and the actual level label corresponding to each positive sample data is determined from each preset level label corresponding to the other active levels.

[0182] For the sample user account in each positive sample data, the true label value of the actual level label corresponding to the sample user account is set to 1, and the true label values of the other preset level labels lower than the active level of the actual level label are also set to 1. The true label values of the other preset level labels higher than the active level of the actual level label are set to 0.

[0183] For the sample user account in each negative sample data, the true label value of the actual level label (preset level label 0) corresponding to the sample user account is set to 1, and the true label values of the other preset level labels (active level label 1, active level label 2, active level label 3 and active level label 4) are set to 0.

[0184] Each sample data further includes feature data of the sample user account, wherein the feature data of the sample object includes age, gender, city, education level and historical behavior data.

[0185] Step three, model learning.

[0186] The sample data set obtained above is used to iteratively train the label classification model to be trained until the target loss function for parameter tuning meets the preset condition, and the trained target label classification model is output.

[0187] In one iteration process, the following steps are included:

[0188] Randomly sample k sample data from the sample data set, k is greater than or equal to 1. For each sample data, input the feature data of the sample user account into the label classification model to be trained, and obtain the predicted label value of the sample user account under each preset level label.

[0189] If the real label values of the sample user account under the active level label 0, the active level label 1 and the active level label 2 are all greater than or equal to the preset threshold 1, the active level label 0, the active level label 1 and the active level label 2 are determined as the positive level labels corresponding to the sample user account. The predicted label values of the sample user account under the active level label 0, the active level label 1 and the active level label 2 are taken as the first predicted label values of the sample user account under each positive level label.

[0190] If the real label values of the sample user account under the active level label 3 and the active level label 4 are all less than the preset threshold 1, the active level label 3 and the active level label 4 are determined as the negative level labels corresponding to the sample user account. The predicted label values of the sample user account under the active level label 3 and the active level label 4 are taken as the second predicted label values of the sample user account under each positive level label.

[0191] The first predicted label values of the sample user account under the active level label 0, the active level label 1 and the active level label 2, and the second predicted label values of the sample user account under the active level label 3 and the active level label 4 are substituted into the above formula (1) to obtain the target loss value corresponding to the sample data.

[0192] The target loss values corresponding to the k sample data are summed to obtain the target loss function for parameter adjustment. The target loss function is used to adjust the parameters of the label classification model to be trained.

[0193] Step four, prediction phase.

[0194] For each candidate user account in the plurality of candidate user accounts, input the feature data of the candidate user account into the trained target label classification model to obtain the target label value of the candidate user account under the five preset level labels. Substitute the target label values of the candidate user account under the five preset level labels into the above formula (2) to obtain the deep intention score of the candidate user account.

[0195] According to the order of the deep intention scores from large to small, sort the deep intention scores corresponding to each candidate user account to obtain a target ranking result. The candidate user accounts corresponding to the top M deep intention scores in the target ranking result are taken as the target user accounts, wherein M is greater than or equal to 1.

[0196] Each target user account is a deep intention account of the target application, and each target user account can be used as a delivery target to deliver the promotion advertisement of the target application to each target user account. If the target user account is set as a video application account, after a user logs in to the video application using the target user account, the video application can display the promotion advertisement of the novel application Y on the main interface of the video application. As shown in FIG. 11, the main interface of the video application displays recommended video information 1101, and simultaneously displays the promotion advertisement 1102 of the novel application Y. Figure 11

[0197] In the embodiment of the present application, based on the real label values of the sample objects under the plurality of preset level labels, the plurality of preset level labels are divided into positive level labels and negative level labels corresponding to the sample objects, instead of being limited to binding the sample objects to a certain level label. Therefore, in the training process, when obtaining the target loss function for parameter tuning based on the first prediction label values of the sample objects under the corresponding respective positive level labels and the second prediction label values of the sample objects under the corresponding respective negative level labels in each sample data, the partial order relationship of the sample objects under the plurality of level labels is comprehensively considered, so that the label classification model is more reasonable in practical significance, thereby improving the prediction effect of the label classification model. In the scenario of targeted advertising, using the trained target label classification model to predict the target label value of the candidate object can effectively improve the accuracy of label value prediction, so that the deep intention score of the candidate object is determined based on the target label value of the candidate object, and at least one target object is selected from the plurality of candidate objects based on the deep intention score, and the corresponding advertisement is pushed to each target object selected, which can effectively improve the effect of targeted advertising.

[0198] In order to verify the effect of the label classification model training method and the object screening method provided in the embodiments of the present application in the scenario of targeted advertising, the present inventors tested the effect of the novel application Y, and the test results are shown in Table 1 as follows:

[0199] Table 1.

[0200]

[0201] Among them, the next stay indicates that the retention duration is 1 day, the 2 stay indicates that the retention duration is 2 days, the 3 stay indicates that the retention duration is 3 days, and the 7 stay indicates that the retention duration is 7 days. The next stay ratio represents the ratio of the number of target user accounts with a retention duration of 1 day in the novel application Y to the number of target user accounts that activated the novel application Y in the test period. The 2 stay ratio represents the ratio of the number of target user accounts with a retention duration of 2 days in the novel application Y to the number of target user accounts that activated the novel application Y in the test period. The 3 stay ratio and the 7 stay ratio represent the same meaning as the next stay ratio and the 2 stay ratio, which will not be described here. ​

[0202] The test result after the advertisement of the novel application Y is targeted to each target user account after the target user accounts are screened using the prior art scheme before the test.

[0203] It can be seen by comparison that after the target user accounts are screened and the advertisement is targeted using the technical scheme in the embodiments of the present application, the 2 stay ratio, 3 stay ratio, and 7 stay ratio of each target user account in the novel application Y are increased, and at the same time, the 3-day average active times and 7-day average active times of each target user account in the novel application Y are also increased. It can be seen that the technical scheme provided in the embodiments of the present application can effectively screen the deep intention user accounts of the novel application Y, thereby improving the advertisement delivery effect.

[0204] Based on the same technical concept, the embodiments of the present application provide a structural schematic diagram of a label classification model training device, as shown in Figure 12 The device 1200 includes:

[0205] A first acquisition module 1201 is configured to obtain a sample data set, wherein each sample data at least contains real label values of a sample object under a plurality of preset level labels respectively;

[0206] A training module 1202 is configured to perform iterative training on a label classification model to be trained based on the sample data set, and output a trained target label classification model, wherein in one iteration process, based on first prediction label values of a sample object under corresponding respective positive level labels and second prediction label values under corresponding respective negative level labels in each sample data, a target loss function for parameter adjustment is obtained, and the respective positive level labels and the respective negative level labels are obtained based on real label values of the corresponding sample object under a plurality of preset level labels.

[0207] Optionally, the training module 1202 further includes a parameter adjustment module 1203.

[0208] The parameter adjustment module 1203 is specifically configured to:

[0209] For each sample data, the following steps are performed respectively:

[0210] Based on the first prediction label values of the sample object under the corresponding respective positive level labels in one sample data, a first loss value is determined;

[0211] determine a second loss value based on the second predicted label values of the sample object in the one sample data under the respective negative level labels;

[0212] determine a target loss value corresponding to the one sample data based on the first loss value and the second loss value;

[0213] obtain a target loss function for parameter tuning based on the obtained target loss values respectively corresponding to the sample data.

[0214] Optionally, each sample data further comprises feature data of the sample object;

[0215] The training module 1202 further comprises a prediction module 1204;

[0216] The prediction module 1204 is specifically configured to:

[0217] Before obtaining the target loss function for parameter tuning based on the first predicted label values of the sample object in each sample data under the respective positive level labels and the second predicted label values of the sample object in each sample data under the respective negative level labels, input the feature data of the sample object contained in the respective sample data into the label classification model to be trained to obtain the first predicted label values of the corresponding sample object under the respective positive level labels and the second predicted label values of the corresponding sample object under the respective negative level labels.

[0218] Optionally, the parameter tuning module 1203 is further configured to:

[0219] determine, from the respective real label values corresponding to the one sample data, first real label values greater than or equal to a preset threshold and second real label values less than the preset threshold;

[0220] use the respective preset level labels corresponding to the first real label values as the positive level labels corresponding to the sample object in the one sample data;

[0221] use the respective preset level labels corresponding to the second real label values as the negative level labels corresponding to the sample object in the one sample data.

[0222] Optionally, the training module 1202 further comprises a setting module 1205;

[0223] The setting module 1205 is specifically configured to:

[0224] set respective active levels for the plurality of preset level labels; and,

[0225] In each of the sample data corresponding to each positive level label and each negative level label, the maximum active level in the each positive level label is set to be less than the minimum active level in the each negative level label.

[0226] Optionally, the setting module 1205 is further configured to:

[0227] According to the retention duration of the sample object in the target application and the active times of the sample object in the target application, the real label value of the sample object under each of the plurality of preset level labels is determined.

[0228] Based on the same technical concept, an embodiment of the present application provides a structural schematic diagram of an object screening device, as shown in Figure 13 The device 1300 includes:

[0229] The second acquisition module 1301 is configured to acquire feature data of each candidate object.

[0230] The prediction module 1302 is configured to input the feature data of each candidate object into a trained target label classification model respectively, to obtain target label values of each candidate object under a plurality of preset level labels, wherein the trained target label classification model is obtained by using the label classification model training device.

[0231] The evaluation module 1303 is configured to determine a corresponding deep intention score of each candidate object based on the target label values of each candidate object under the plurality of preset level labels.

[0232] The screening module 1304 is configured to screen at least one target object from the candidate objects based on the corresponding deep intention scores of the candidate objects.

[0233] Optionally, the evaluation module 1303 is specifically configured to:

[0234] For each candidate object, the following steps are performed respectively:

[0235] The target label values of a candidate object under the plurality of preset level labels are normalized to obtain candidate probabilities of the candidate object under the plurality of preset level labels.

[0236] Based on the obtained candidate probabilities and weights corresponding to the plurality of preset level labels respectively, a deep intention score of the candidate object is determined.

[0237] Optionally, the screening module 1304 is specifically configured to:

[0238] According to an order from large to small of the depth intention scores, the depth intention scores corresponding to the respective candidate objects are sorted to obtain a target sorting result.

[0239] The candidate objects corresponding to the top M depth intention scores in the target sorting result are taken as target objects, where M is greater than or equal to 1.

[0240] Optionally, the respective candidate objects are candidate objects for a target application.

[0241] The screening module 1304 is further configured to:

[0242] After screening at least one target object from the respective candidate objects based on the respective depth intention scores corresponding to the respective candidate objects, the related content of the target application is recommended to the at least one target object.

[0243] In the embodiments of the present application, the plurality of preset level labels are divided into positive level labels and negative level labels corresponding to the sample objects based on the real label values of the sample objects under the plurality of preset level labels, instead of being limited to binding the sample objects to a certain level label. Therefore, in the training process, when obtaining the target loss function for parameter tuning based on the first prediction label values of the sample objects under the respective positive level labels and the second prediction label values of the sample objects under the respective negative level labels in the respective sample data, the partial order relationship of the sample objects under the plurality of level labels is comprehensively considered, so that the label classification model is more reasonable in practical significance, and the prediction effect of the label classification model is improved. In the scenario of targeted delivery of multimedia content, using the above trained target label classification model to predict the target label values of the candidate objects can effectively improve the accuracy of label value prediction, so that the target objects are screened from the respective candidate objects based on the target label values of the candidate objects, and the respective target objects are pushed corresponding multimedia content, which can effectively improve the effect of targeted delivery of multimedia content.

[0244] Based on the same technical concept, the embodiments of the present application provide a computer device, as shown in Figure 14 The computer device includes at least one processor 1401 and a memory 1402 connected to the at least one processor. In the embodiments of the present application, the specific connection medium between the processor 1401 and the memory 1402 is not limited, Figure 14 For example, the processor 1401 and the memory 1402 are connected through a bus. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0245] In the embodiments of the present application, the memory 1402 stores instructions executable by the at least one processor 1401, and the at least one processor 1401 can execute the steps of the label classification model training method and / or the object screening method by executing the instructions stored in the memory 1402.

[0246] The processor 1401 is the control center of the computer device, can connect various parts of the computer device through various interfaces and lines, and train the label classification model and / or perform object screening by running or executing the instructions stored in the memory 1402 and calling the data stored in the memory 1402. Optionally, the processor 1401 can include one or more processing units, and the processor 1401 can integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface and application program, and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1401. In some embodiments, the processor 1401 and the memory 1402 can be implemented on the same chip, and in some embodiments, they can also be implemented on independent chips respectively.

[0247] The processor 1401 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as hardware processor execution or executed by a combination of hardware and software modules in the processor.

[0248] The memory 1402, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 1402 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 1402 is any other medium capable of carrying or storing desired program codes in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 1402 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0249] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium storing a computer program executable by a computer device, which, when the program is executed on the computer device, causes the computer device to perform the steps of the label classification model training method and / or the object screening method.

[0250] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, etc.) containing computer-usable program code.

[0251] The present application is described with reference to flowcharts and / or block diagrams of the method, device (system) and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a device that implements the flowcharts and / or block diagrams. Figure 1one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0252] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the flowchart Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0253] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart Figure 1 one or more processes and / or blocks Figure 1 an apparatus for performing the functions specified in the flowchart

[0254] While the preferred embodiments of the application have been described, additional variations and modifications can be made to the embodiments by those skilled in the art once they learn of the basic inventive concepts. Therefore, the appended claims are intended to encompass within their scope all such variations and modifications as are included within the scope of the invention.

[0255] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for training a label classification model, the method comprising: The method comprises the following steps: determining real label values of a sample object in a plurality of preset level labels according to a retention time length of the sample object in a target application and an active number of the sample object in the target application; obtaining a sample data set, wherein each sample data at least comprises real label values of a sample object in a plurality of preset level labels; setting corresponding active levels for the plurality of preset level labels; and setting a maximum active level in each positive level label and a minimum active level in each negative level label in each sample data corresponding to each positive level label and each negative level label; when constructing the real label values of the sample object in the plurality of preset level labels, filling a high score value into the real label values of the actual level label corresponding to the sample object and other preset level labels with lower active levels than the actual level label; iteratively training a label classification model to be trained based on the sample data set, and outputting a trained target label classification model, wherein in one iteration process, a target loss function for parameter adjustment is obtained based on first prediction label values of a sample object in each sample data in corresponding each positive level label and second prediction label values of the sample object in corresponding each negative level label, and the each positive level label and the each negative level label are obtained by dividing the plurality of preset level labels based on real label values of the corresponding sample object in the plurality of preset level labels.

2. The method of claim 1, wherein, The target loss function for parameter adjustment is obtained based on the first prediction label values of the sample object in each sample data in corresponding each positive level label and the second prediction label values of the sample object in corresponding each negative level label, and comprises: the following steps are performed for each sample data: determining a first loss value based on the first prediction label values of the sample object in corresponding each positive level label in one sample data; determining a second loss value based on the second prediction label values of the sample object in corresponding each negative level label in the one sample data; determining a target loss value corresponding to the one sample data based on the first loss value and the second loss value; obtaining the target loss function for parameter adjustment based on the target loss values corresponding to the each sample data.

3. The method of claim 1, wherein, Each sample data further comprises feature data of the sample object. Before the target loss function for parameter adjustment is obtained based on the first prediction label values of the sample object in each sample data in corresponding each positive level label and the second prediction label values of the sample object in corresponding each negative level label, the following steps are further included: inputting the feature data of the sample object contained in the each sample data into the label classification model to be trained, to obtain the first prediction label values of the corresponding sample object in corresponding each positive level label and the second prediction label values of the corresponding sample object in corresponding each negative level label.

4. The method of claim 1, wherein, The positive grade label and the negative grade label corresponding to the sample object in each sample data are obtained in the following manner: From the real label values corresponding to each sample data, a first type of real label value greater than or equal to a preset threshold value and a second type of real label value less than the preset threshold value are determined; The preset grade label corresponding to the first type of real label value is taken as the positive grade label corresponding to the sample object in the sample data; The preset grade label corresponding to the second type of real label value is taken as the negative grade label corresponding to the sample object in the sample data.

5. A method of object screening, characterized by, The method comprises: obtaining feature data of each candidate object; inputting the feature data of each candidate object into a trained target label classification model to obtain target label values of each candidate object under a plurality of preset grade labels, wherein the trained target label classification model is obtained by using the method of any one of claims 1 to 4; based on the target label values of each candidate object under a plurality of preset grade labels, determining the depth intention score corresponding to each candidate object respectively; based on the depth intention score corresponding to each candidate object, screening at least one target object from the candidate objects.

6. The method of claim 5, wherein, The method of determining the depth intention score corresponding to each candidate object respectively based on the target label values of each candidate object under a plurality of preset grade labels comprises: for each candidate object, the following steps are performed respectively: normalizing the target label values of a candidate object under a plurality of preset grade labels to obtain candidate probabilities of a candidate object under the plurality of preset grade labels; based on the obtained candidate probabilities and the weights corresponding to the plurality of preset grade labels, determining the depth intention score of a candidate object.

7. The method of claim 5, wherein, The method of screening at least one target object from the candidate objects based on the depth intention score corresponding to each candidate object comprises: sorting the depth intention scores corresponding to each candidate object in descending order of the depth intention scores to obtain a target sorting result; the candidate objects corresponding to the top M depth intention scores in the target sorting result are taken as target objects, wherein M is greater than or equal to 1.

8. The method of any one of claims 5 to 7, wherein, The candidate objects are candidate objects for a target application. After the method of screening at least one target object from the candidate objects based on the depth intention score corresponding to each candidate object, the method further comprises: recommending related content of the target application to the at least one target object. 9.A label classification model training apparatus, characterized by comprising: The method comprises: a first obtaining module configured to determine real label values of a sample object under a plurality of preset grade labels according to a retention duration of the sample object in a target application and an active number of the sample object in the target application; obtaining a sample data set, wherein each sample data contains at least a real label value of a sample object under a plurality of preset level labels respectively; setting a corresponding active level for each of the plurality of preset level labels; and in each sample data corresponding to each positive level label and each negative level label, setting the maximum active level in the each positive level label, which is less than the minimum active level in the each negative level label; when constructing the real label value of the sample object under the plurality of preset level labels, filling the actual level label corresponding to the sample object and the real label value of other preset level labels lower than the active level of the actual level label with a high score value; a training module configured to perform iterative training on a label classification model to be trained based on the sample data set, and output a trained target label classification model, wherein in one iteration process, based on a first predicted label value of a sample object under each corresponding positive level label in each sample data and a second predicted label value under each corresponding negative level label, a target loss function for parameter adjustment is obtained, and the each positive level label and the each negative level label are obtained by dividing the plurality of preset level labels based on the real label value of the corresponding sample object under the plurality of preset level labels.

10. An object screening apparatus, characterized by, comprising: a second obtaining module configured to obtain feature data of each candidate object; a prediction module configured to input the feature data of each candidate object into the trained target label classification model respectively, and obtain target label values of the each candidate object under a plurality of preset level labels respectively, wherein the trained target label classification model is obtained by using the device of claim 9; an evaluation module configured to determine a depth intention score corresponding to each of the each candidate object based on the target label values of the each candidate object under the plurality of preset level labels respectively; a screening module configured to screen at least one target object from the each candidate object based on the depth intention score corresponding to each of the each candidate object.

11. A computer device comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, The processor executes the program to realize the steps of the method of any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer program is stored in the computer device and executed by the computer device, and when the program runs on the computer device, the computer device executes the steps of the method of any one of claims 1-8.

Images