User group determination method and device, storage medium and computing equipment

Abstract

The embodiment of the invention provides a user group determination method and device, a storage medium and computing equipment. Comprising the steps of obtaining content features related to content to be delivered and user features related to a user to be pushed; inputting the content features and the user features into a comparative learning model, and calculating feature similarity between the content features and the user features; inputting the target user features of which the feature similarity is greater than a threshold value and the content features into a sorting model, and calculating user scores of target users corresponding to the target user features; wherein the user score represents the degree of interest of the target user in the to-be-delivered content; and based on the user score of each target user, determining a user group interested in the to-be-delivered content.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more specifically, embodiments of the present disclosure relate to a method, apparatus, storage medium, and computing device for determining a user group. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure. No description herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] As more and more users access the Internet, the traditional indiscriminate content delivery method is becoming less and less effective.

[0004] In this regard, the Lookalike targeting technology is proposed in related technologies. This technology is based on the seed user portrait and relationship chain to find similar users, that is, to automatically expand according to the common attributes of the seed population to expand the audience coverage, thereby improving the content delivery effect. For example, social factors such as family, social identity, status, related groups, cultural factors such as culture and subculture, and psychological factors such as behavior, motivation, and interest can form the screening criteria for similar population expansion Lookalike. Based on the relationship chain, user groups with similar relationships can be determined, and then content can be delivered to users in these user groups with similar relationships.

[0005] However, although the Lookalike targeting technology can improve the delivery effect to a certain extent compared with indiscriminate delivery by identifying user groups with similar relationships, the content that users in the user groups determined based on relationships are interested in is still diverse, and it is impossible to ensure that all users in the user groups are interested in the delivered content. Therefore, the effect of content delivery based on the Lookalike targeting technology is still not ideal. Summary of the invention

[0006] In a first aspect of the embodiments of the present disclosure, a method for determining a user group is provided. The method comprises:

[0007] Obtaining content features related to the content to be delivered and user features related to the user to be pushed;

[0008] Inputting the content feature and the user feature into a contrastive learning model, and calculating the feature similarity between the content feature and the user feature;

[0009] Inputting the target user features and the content features whose feature similarity is greater than a threshold into the sorting model, and calculating the user score of the target user corresponding to the target user features; wherein the user score represents the interest degree of the target user in the content to be delivered;

[0010] Based on the user score of each target user, a user group that is interested in the content to be delivered is determined.

[0011] Optionally, the contrastive learning model includes a user tower network and a content tower network; wherein the user tower network is used to process user features, and the content tower network is used to process content features;

[0012] The step of inputting the content feature and the user feature into a contrastive learning model and calculating the feature similarity between the content feature and the user feature includes:

[0013] Inputting the user characteristics into the user tower network for calculation, and inputting the content characteristics into the content tower network for calculation;

[0014] The output of the user tower network and the output of the content tower network are further input into a fully connected layer for calculation to obtain the feature similarity between the content feature and the user feature.

[0015] Optionally, the contrastive learning model further includes an embedding layer for receiving model input, and the output of the embedding layer is connected to the user tower network and the content tower network respectively;

[0016] Before inputting the user features into the user tower network for calculation and inputting the content features into the content tower network for calculation, the method further includes:

[0017] Inputting the content features and the user features into the embedding layer of the contrastive learning model, and calculating a content embedding vector corresponding to the content features and a user embedding vector corresponding to the user features;

[0018] The step of inputting the user features into the user tower network for calculation, and inputting the content features into the content tower network for calculation, comprises:

[0019] The user embedding vector is input into the user tower network for calculation, and the content embedding vector is input into the content tower network for calculation.

[0020] Optionally, the sorting model includes a multi-task sorting model; wherein the multi-task includes a click task of predicting the content clicked by the user and a play task of predicting the content played by the user;

[0021] The step of inputting the target user feature whose feature similarity is greater than a threshold and the content feature into the sorting model, and calculating the user score of the target user corresponding to the target user feature, comprises:

[0022] The target user features and the content features whose feature similarity is greater than a threshold are input into the sorting model, and the probability of the target user clicking and playing the content to be delivered is predicted through the click task and the play task respectively, and the target user is further scored according to the probability to obtain the user score of the target user.

[0023] Optionally, the method further includes:

[0024] Acquire the business scenario to which the content to be delivered belongs and the scenario characteristics of the business scenario;

[0025] The target user feature with feature similarity greater than a threshold and the content feature are input into the sorting model, the probability of the target user clicking and playing the content to be delivered is predicted through the click task and the play task respectively, and the target user is further scored according to the probability to obtain the user score of the target user, including:

[0026] The target user features, the content features and the scenario features whose feature similarity is greater than a threshold are input into the sorting model, and the probability of the target user clicking and playing the content to be delivered in the business scenario is predicted respectively through the click task and the play task, and the target user is further scored according to the probability to obtain the user score of the target user.

[0027] Optionally, determining a user group interested in the to-be-delivered content based on the user score of each target user includes:

[0028] The user scores of the target users are sorted, and the users with the target scores of the first preset number of the sorted results are selected to form a user group interested in the content to be delivered.

[0029] Optionally, determining a user group interested in the to-be-delivered content based on the user score of each target user includes:

[0030] The user score of each target user is compared with a score threshold, and users with a score greater than the score threshold are selected to form a user group that is interested in the content to be delivered.

[0031] Optionally, the score threshold is calculated in the following way:

[0032] Sorting the user scores of the target users, and taking the target scores of the first preset number of the sorted results;

[0033] The total score and the average score of the preset number of target scores are calculated, and a linear weighted calculation is performed based on the total score and the average score to obtain a score threshold.

[0034] Optionally, the method further includes:

[0035] The content to be delivered is pushed to the user group.

[0036] In a second aspect of the embodiments of the present disclosure, a device for determining a user group is provided, the device comprising:

[0037] An acquisition unit, which acquires content features related to the content to be delivered and user features related to the user to be pushed;

[0038] A first calculation unit, inputting the content feature and the user feature into a contrastive learning model, and calculating a feature similarity between the content feature and the user feature;

[0039] A second calculation unit inputs the target user feature and the content feature whose feature similarity is greater than a threshold into a ranking model, and calculates a user score of the target user corresponding to the target user feature; wherein the user score represents the interest degree of the target user in the content to be delivered;

[0040] The determination unit determines, based on the user score of each target user, a user group that is interested in the content to be delivered.

[0041] Optionally, the contrastive learning model includes a user tower network and a content tower network; wherein the user tower network is used to process user features, and the content tower network is used to process content features;

[0042] The first computing unit comprises:

[0043] A first calculation subunit, inputting the user characteristics into the user tower network for calculation, and inputting the content characteristics into the content tower network for calculation;

[0044] The second calculation subunit further inputs the output of the user tower network and the output of the content tower network into a fully connected layer for calculation to obtain a feature similarity between the content feature and the user feature.

[0045] Optionally, the contrastive learning model further includes an embedding layer for receiving model input, and the output of the embedding layer is connected to the user tower network and the content tower network respectively;

[0046] Before the first computing subunit, it also includes:

[0047] A feature conversion subunit inputs the content feature and the user feature into an embedding layer of the contrastive learning model, and calculates a content embedding vector corresponding to the content feature and a user embedding vector corresponding to the user feature;

[0048] The first calculation subunit is further used to input the user embedding vector into the user tower network for calculation, and input the content embedding vector into the content tower network for calculation.

[0049] Optionally, the sorting model includes a multi-task sorting model; wherein the multi-task includes a click task of predicting the content clicked by the user and a play task of predicting the content played by the user;

[0050] The second computing unit comprises:

[0051] The third calculation subunit inputs the target user feature whose feature similarity is greater than the threshold and the content feature into the sorting model, and predicts the probability of the target user clicking and playing the content to be delivered respectively through the click task and the play task;

[0052] The scoring subunit scores the target user according to the probability to obtain a user score of the target user.

[0053] Optionally, the device further comprises:

[0054] An acquisition subunit is used to acquire the business scenario to which the content to be delivered belongs and the scenario characteristics of the business scenario;

[0055] The third calculation subunit is further used to input the target user features whose feature similarity is greater than a threshold, the content features and the scenario features into the sorting model, and predict the probability of the target user clicking and playing the content to be delivered in the business scenario through the click task and the play task respectively.

[0056] Optionally, the determining unit includes:

[0057] The sorting subunit sorts the user scores of the target users, and selects the users with the first preset number of target scores from the sorting results to form a user group interested in the content to be delivered.

[0058] Optionally, the determining unit includes:

[0059] The comparison subunit compares the user score of each target user with a score threshold, and selects users with scores greater than the score threshold to form a user group that is interested in the content to be delivered.

[0060] Optionally, the threshold is calculated by a threshold calculation subunit:

[0061] The threshold calculation subunit sorts the user scores of the target users, takes the first preset number of target scores of the sorting results; calculates the total score and the average score of the preset number of target scores, and performs linear weighted calculation based on the total score and the average score to obtain a score threshold.

[0062] Optionally, the device further comprises:

[0063] A delivery unit pushes the content to be delivered to the user group.

[0064] In a third aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, comprising:

[0065] When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for determining a user group as described in any of the preceding items.

[0066] In a fourth aspect of the embodiments of the present disclosure, there is provided a computing device, comprising:

[0067] processor;

[0068] a memory for storing instructions executable by the processor;

[0069] The processor is configured to execute the executable instructions to implement the method for determining a user group as described in any of the preceding items.

[0070] According to the user group determination scheme provided by the embodiment of the present disclosure, a comparative learning model is used to perform a rough screening of all users to which the content to be delivered is targeted, and the ranking model is used to predict the interest level of the remaining users in the content to be delivered and score them, and a fine screening is performed based on the scored user scores, thereby selecting a user group with a high interest in the content to be delivered. Since the user group is determined based on the content to be delivered, it can be ensured that the users in the user group are all interested in the content to be delivered, and delivering content to such a user group can improve the delivery effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0072] Figure 1 The following schematically shows a schematic diagram of a content delivery system provided by the present disclosure;

[0073] Figure 2 The following schematically shows a method for determining a user group provided by the present disclosure;

[0074] Figure 3 The schematic diagram of the contrastive learning model provided by the present disclosure is schematically shown;

[0075] Figure 4 The schematic diagram of the sorting model provided by the present disclosure is schematically shown;

[0076] Figure 5 The medium schematic diagram provided by the present disclosure is schematically shown;

[0077] Figure 6 The following schematically shows a schematic diagram of a device for determining a user group provided by the present disclosure;

[0078] Figure 7 The schematic diagram of the computing device provided by the present disclosure is schematically shown.

[0079] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0080] The principles and spirit of the present disclosure will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present disclosure, and are not intended to limit the scope of the present disclosure in any way. On the contrary, these embodiments are provided to make the present disclosure more thorough and complete, and to fully convey the scope of the present disclosure to those skilled in the art.

[0081] Those skilled in the art will appreciate that the embodiments of the present disclosure may be implemented as a system, device, apparatus, method or computer program product. Therefore, the present disclosure may be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0082] According to an embodiment of the present disclosure, a method for determining a user group, a computer-readable storage medium, an apparatus, and a computing device are proposed.

[0083] It should be understood herein that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction rather than having any limiting meaning.

[0084] The principle and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

[0085] The data involved in the present disclosure may be data authorized by the user or fully authorized by all parties. The collection, dissemination, and use of the data shall comply with the requirements of relevant national laws and regulations. The implementation methods / examples of the present disclosure may be combined with each other. SUMMARY OF THE INVENTION

[0087] The present disclosure aims to provide a user group determination scheme, which uses a comparative learning model to perform a rough screening of all users to which the content to be delivered is targeted, and then the ranking model is used to predict the interest of the remaining users in the content to be delivered and score them, and a fine screening is performed based on the scored user scores, so as to select a user group with high interest in the content to be delivered. Since the user group is determined based on the content to be delivered, it can be ensured that the users in the user group are all interested in the content to be delivered, and delivering content to such a user group can improve the delivery effect.

[0088] After introducing the basic principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

[0089] Application Scenario Overview

[0090] First reference Figure 1 A system architecture diagram of a content delivery system is shown. In the system architecture diagram, various network nodes can communicate information with the help of the network, and then complete interaction and data processing. The system architecture diagram may include a server 12 that communicates data with one or more clients 11 via a network 13, and a database 14 that can be integrated with the server 12 or independent of the server 12.

[0091] The server 12 is used to determine the user group and deliver content to the determined user group. In some embodiments, the server 12 can be subdivided into a first server for determining the user group and a second server for delivering content to the determined user group.

[0092] The server 12 may store a pre-trained comparative learning model and a ranking model, and determine the user group interested in the content to be delivered through the comparative learning model and the ranking model; and then deliver the content to be delivered to the client 11 where the users in the user group are located. When the server 12 is divided into a first server and a second server, the first server may store a pre-trained comparative learning model and a ranking model, and determine the user group interested in the content to be delivered through the comparative learning model and the ranking model, and transmit the user group to the second server, and then the second server delivers the content to be delivered to the client 11 where the users in the user group are located.

[0093] In the system architecture diagram, each network 13 may include a wired or wireless telecommunication device, through which the network devices on which the client 11 and the server 12 are based can exchange data. For example, each network 13 may include a local area network ("LAN"), a wide area network ("WAN"), an intranet, the Internet, a mobile phone network, a virtual private network (VPN), a cellular or other mobile communication network, Bluetooth, NFC, or any combination thereof. In the discussion of the exemplary embodiments, it should be understood that the terms "data" and "information" may be used interchangeably herein to refer to text, images, audio, video, or any other form of information that may exist in a computer-based environment.

[0094] The client 11 may include a server, a desktop computer, a laptop computer, a tablet computer, a smart phone, a handheld computer, a personal digital assistant ("PDA"), or any other wired or wireless processor-driven device.

[0095] The server 12 may include a server, a server cluster, or a cloud platform built based on a server cluster.

[0096] The data / relationships that need to be read or the processing that needs to be performed by the server 12 in the process of determining the user group may need to be obtained from the connected database 14 , and the data / relationships that need to be written or the processing results involved in the process may need to be written into the connected database 14 .

[0097] Figure 1 In the figure, the computing device 15 can be in an integrated relationship or a separate relationship with the server 12. In the latter case, it can generally be connected through an internal network or a dedicated network, or it can also be connected through an encrypted public network. In particular, when it is an integrated relationship, a more efficient and faster transmission speed internal bus connection may be used. The computing device 15, when it is an integrated relationship or a separate relationship, can access the database 14 directly (not shown in the figure) or through the server 12.

[0098] The computing device 15 can be properly programmed to control the implementation of the method in this specification through such instructions. In particular, when it is an integrated relationship, the transaction processed by the computing device 15 can be regarded as the processing of the server 12 without special distinction.

[0099] Exemplary Methods

[0100] Combine the following Figure 1 The application scenario shown is shown in Figure 2To describe the method for determining a user group according to an exemplary embodiment of the present disclosure. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this regard. On the contrary, the embodiments of the present disclosure can be applied to any applicable scenario.

[0101] like Figure 2 As shown, the method for determining the user group can be applied to the aforementioned server and may include the following steps:

[0102] Step 210: Obtain content features related to the content to be delivered and user features related to the user to be pushed;

[0103] Step 220: inputting the content feature and the user feature into a contrastive learning model, and calculating the feature similarity between the content feature and the user feature;

[0104] Step 230: Input the target user features and the content features whose feature similarity is greater than a threshold into the ranking model, and calculate the user score of the target user corresponding to the target user features; wherein the user score represents the interest degree of the target user in the content to be delivered;

[0105] Step 240: Based on the user score of each target user, determine the user group that is interested in the content to be delivered.

[0106] The content to be delivered in this specification may include but is not limited to videos (such as short videos, long videos), audio (such as radio stations, music songs, user-recorded audio, audio e-books, etc.), text images (such as e-books, comic books, illustrated news, blogs, Weibo, etc.), etc.

[0107] In order to improve the delivery effect of the content to be delivered, it is necessary to make the users in the determined user group as interested as possible in the content to be delivered. Only when the users are interested in the content to be delivered will they be motivated to click on the content to be delivered, thereby increasing the user conversion rate. The higher the user conversion rate, the better the delivery effect and the more accurate the determined user group.

[0108] In view of this, the embodiments of this specification match the content to be delivered with the user group that is most interested in the content to be delivered by comprehensively considering the content features of the content to be delivered and the user features of the users to be pushed. In implementation, a comparative learning model is used to perform a rough screening of all users for the content to be delivered, and the remaining users are then predicted and scored by a sorting model, and a fine screening is performed based on the scored user scores, thereby selecting a user group with a high interest in the content to be delivered. Since the user group is determined based on the content to be delivered, it can be ensured that the users in the user group are all interested in the content to be delivered, and content delivery to such a user group can improve the delivery effect.

[0109] The contrastive learning model and ranking model used in this manual are introduced below.

[0110] [Comparative Learning Model]

[0111] In an exemplary embodiment, the contrastive learning model may include a user tower network and a content tower network; wherein the user tower network is used to process user features, and the content tower network is used to process content features;

[0112] The step 220 of inputting the content feature and the user feature into a contrastive learning model to calculate the feature similarity between the content feature and the user feature may further include:

[0113] Inputting the user characteristics into the user tower network for calculation, and inputting the content characteristics into the content tower network for calculation;

[0114] The output of the user tower network and the output of the content tower network are further input into a fully connected layer for calculation to obtain the feature similarity between the content feature and the user feature.

[0115] In this embodiment, contrastive learning is generally a self-supervised learning method that aims to learn effective representations of data by maximizing the similarity between similar samples and minimizing the similarity between different samples. The core idea is to construct "positive sample pairs" (similar sample pairs) and "negative sample pairs" (dissimilar sample pairs) to allow the model to learn the intrinsic structure of the data without labels. Since contrastive learning can be trained without relying on labeled data, it is suitable for large-scale unlabeled data sets.

[0116] Contrastive learning models may include but are not limited to SimCLR (Simple Contrastive Learning of Representations), MoCo (Momentum Contrast), BYOL (Bootstrap Your Own Latent), DSSM (Deep Structured Semantic Model), etc.

[0117] Below Figure 3 The DSSM model trained by contrastive learning is shown as an example. The difference between this model and the traditional DSSM model is that:

[0118] The dual tower network of the traditional DSSM model is improved, and one of the tower networks is defined as the user tower network 31, and the other is still the content tower network 32. In addition, an auxiliary task 33 is designed, and the user tower network 31 and the auxiliary task 33 share user features. The user tower network 31, the content tower network 32 and the auxiliary task 33 can be composed of several activation functions ReLu.

[0119] Among them, the user tower network 31 is used to process user characteristics (such as user portraits, user behaviors, etc.), and the content tower is used to process content characteristics.

[0120] Since it involves the content to be delivered for different services, for example, when multiple different services in the same application APP need to deliver content, the content tower network 32 needs to be subdivided according to the content of different services. Taking the music application APP as an example, the application may include audio book services, podcast services, and song services, so the content tower network 32 can be divided into an audio book tower network, a podcast tower network, and a song tower network. The user characteristics and content characteristics of different services only update the network parameters of their respective content tower networks, and the network parameters of other content tower networks are not updated by gradient backhaul.

[0121] When training the comparative learning model, the data set for model training can be the real user behavior data. The behavior data of each user u clicking on content i under different services constitutes the positive sample set (u, i + )∈P, the behavior data of user u not clicking on content i constitutes the negative sample set (u, i - )∈N.

[0122] The user tower network is the main task, and each content tower network is a secondary task, plus auxiliary task 33. Negative sampling is performed on other clicked content in the same task (batch), where K is the number of negative samples, and different sample data updates the corresponding content tower network; auxiliary contrast learning obtains half of the user features through masks as one side input of auxiliary task 33, and the remaining half of the user features are used as the other side input of auxiliary task 33, and the user features are learned through self-supervision. Among them, the relevant features of the user tower network are shared in the entire model.

[0123] The model training adopts self-supervised training, taking the positive sample set P and the negative sample set N as the model input for calculation, and using the loss function as Figure 3 The self-supervised loss function for auxiliary tasks and the loss functions (supervised loss) of the user tower network and the content tower network maximize the similarity between similar samples and minimize the similarity between different samples, thereby learning the model parameters in the model.

[0124] In an exemplary embodiment, Figure 3 As shown, the contrastive learning model may further include an embedding layer for receiving model input, and the output of the embedding layer is respectively connected to the user tower network and the content tower network;

[0125] Accordingly, before inputting the user features into the user tower network for calculation and inputting the content features into the content tower network for calculation, the above-mentioned process may further include:

[0126] Inputting the content features and user features into the embedding layer of the contrastive learning model, and calculating a content embedding vector corresponding to the content features and a user embedding vector corresponding to the user features;

[0127] The step of inputting the user features into the user tower network for calculation, and inputting the content features into the content tower network for calculation, may include:

[0128] The user embedding vector is input into the user tower network for calculation, and the content embedding vector is input into the content tower network for calculation.

[0129] In this embodiment, user features and content features are converted into vector representations through ID mapping, namely embedding vectors. Through embedding, features that cannot be understood by computers can be numerically converted into features in a unified vector space, and similarity can be measured based on vector data. In addition, after being converted into embedding vectors,

[0130] This manual adopts contrastive learning and constructs user tower network auxiliary tasks, which is different from the traditional content tower auxiliary tasks. It can enable the user tower network to learn high-quality embeddings for users with few user behaviors and insufficient features, thereby improving the accuracy of determining user groups.

[0131] [Sorting Model]

[0132] In an exemplary embodiment, the ranking model may include a multi-task ranking model; wherein the multi-task includes a click task of predicting a user's click content and a play task of predicting a user's play content;

[0133] The step 230 inputs the target user feature whose feature similarity is greater than a threshold and the content feature into the ranking model, and calculates the user score of the target user corresponding to the target user feature, and may further include:

[0134] The target user features and the content features whose feature similarity is greater than a threshold are input into the sorting model, and the probability of the target user clicking and playing the content to be delivered is predicted through the click task and the play task respectively, and the target user is further scored according to the probability to obtain the user score of the target user.

[0135] In this embodiment, the ranking model may include but is not limited to Caser (Convolutional Sequence Embedding Recommendation), DIEN (Deep Interest Evolution Network), DIN (Deep Interest Network) and other refined ranking models.

[0136] Please refer to the following Figure 4 Schematic diagram of the sorting model shown. Figure 4As shown in the figure, there is a click tower network (Ctr Towaer) corresponding to the task of predicting the user's click on the content, and a conversion tower network (Cvr Towaer) corresponding to the task of predicting the user's play of the content. The overall sorting model is a PLE (Pre-Layered Embedding) structure, which improves the model effect by introducing the DIN refined sorting model and the factor machine (FM) module; the model sample is the behavioral data of online users' exposure, click and play; the model input is the user characteristics of the target user and the content characteristics of the content to be delivered, and the model output is the user score of each target user for the content to be delivered.

[0137] In an exemplary embodiment, before step 230, the following steps may also be included:

[0138] Acquire the business scenario to which the content to be delivered belongs and the scenario characteristics of the business scenario;

[0139] Accordingly, step 230 may further include:

[0140] The target user features, the content features and the scenario features whose feature similarity is greater than a threshold are input into the sorting model, and the probability of the target user clicking and playing the content to be delivered in the business scenario is predicted respectively through the click task and the play task, and the target user is further scored according to the probability to obtain the user score of the target user.

[0141] In actual applications, users of different businesses are quite different. For example, the content they are interested in is usually different. Therefore, when determining user groups for different business scenarios, it is necessary to focus on fine-tuning model training based on the training set of this business scenario. Figure 4 As shown, the scene features can be directly concatenated with other processed user features and content features to score the target user.

[0142] After the comparative learning model and the ranking model are introduced, after the user scores are obtained by scoring the initially screened target users according to the ranking model, step 240 is further performed to determine the user group that is interested in the content to be delivered.

[0143] Since the user score represents the target user's interest in the content to be delivered, in one embodiment, the user scores of the target users may be sorted, and the users with the first preset number of target scores in the sorting results constitute a user group interested in the content to be delivered.

[0144] In another embodiment, the user scores of the target users may be compared with a score threshold, and users with scores greater than the score threshold may constitute a user group that is interested in the content to be delivered.

[0145] The score threshold may be an empirical value set manually, or may be a dynamic threshold calculated in real time according to the scores of each user determined each time.

[0146] Taking the dynamic threshold as an example, the score threshold can be calculated in the following way:

[0147] Sorting the user scores of the target users, and taking the target scores of the first preset number of the sorted results;

[0148] The total score and the average score of the preset number of target scores are calculated, and a linear weighted calculation is performed based on the total score and the average score to obtain a score threshold.

[0149] In this embodiment, since the target users initially screened by the comparative learning model for each content to be delivered and the scoring results of the target users by the sorting model are different, in order to more accurately screen out user groups, a more accurate score threshold can be given by the above-mentioned calculation of the dynamic threshold, thereby avoiding inaccurate results caused by the inflexibility of the fixed score threshold.

[0150] After determining the user group, the embodiments of this specification can further push the content to be delivered to the user group.

[0151] Since the user group is determined based on the content to be delivered, it can be ensured that the users in the user group are all interested in the content to be delivered. Delivering content to such a user group can improve the delivery effect.

[0152] Exemplary Media

[0153] After introducing the method of the exemplary embodiment of the present disclosure, next, refer to Figure 5 A medium according to an exemplary embodiment of the present disclosure is described.

[0154] In this exemplary embodiment, the above method can be implemented by a program product, such as a portable compact disk read-only memory (CD-ROM) and including program code, and can be run on a device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus, or a device.

[0155] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0156] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0157] The program code contained on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RE, etc., or any suitable combination of the foregoing.

[0158] Program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0159] In summary, the present disclosure may provide a computer-readable storage medium. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device may be enabled to execute the aforementioned embodiment of the method for determining a user group.

[0160] Exemplary Devices

[0161] After introducing the medium of the exemplary embodiment of the present disclosure, next, reference is made to Figure 6 A device according to an exemplary embodiment of the present disclosure is described.

[0162] Figure 6 A block diagram schematically shows a device for determining a user group according to an embodiment of the present disclosure, corresponding to the aforementioned Figure 4 The method embodiment shown. The user group determination device may include:

[0163] The acquisition unit 610 acquires content features related to the content to be delivered and user features related to the user to be pushed;

[0164] A first calculation unit 620 inputs the content feature and the user feature into a contrastive learning model to calculate a feature similarity between the content feature and the user feature;

[0165] The second calculation unit 630 inputs the target user feature and the content feature whose feature similarity is greater than the threshold into the sorting model, and calculates the user score of the target user corresponding to the target user feature; wherein the user score represents the interest degree of the target user in the content to be delivered;

[0166] The determination unit 640 determines a user group that is interested in the content to be delivered based on the user score of each target user.

[0167] Optionally, the contrastive learning model includes a user tower network and a content tower network; wherein the user tower network is used to process user features, and the content tower network is used to process content features;

[0168] The first calculation unit 620 includes:

[0169] A first calculation subunit 624 inputs the user characteristics into the user tower network for calculation, and inputs the content characteristics into the content tower network for calculation;

[0170] The second calculation subunit 626 further inputs the output of the user tower network and the output of the content tower network into a fully connected layer for calculation to obtain the feature similarity between the content feature and the user feature.

[0171] Optionally, the contrastive learning model further includes an embedding layer for receiving model input, and the output of the embedding layer is connected to the user tower network and the content tower network respectively;

[0172] Before the first calculation subunit 624, it also includes:

[0173] The feature conversion subunit 622 inputs the content feature and the user feature into the embedding layer of the contrastive learning model, and calculates the content embedding vector corresponding to the content feature and the user embedding vector corresponding to the user feature;

[0174] The first calculation subunit 624 is further configured to input the user embedding vector into the user tower network for calculation, and input the content embedding vector into the content tower network for calculation.

[0175] Optionally, the sorting model includes a multi-task sorting model; wherein the multi-task includes a click task of predicting the content clicked by the user and a play task of predicting the content played by the user;

[0176] The second calculation unit 630 includes:

[0177] The third calculation subunit 634 inputs the target user feature whose feature similarity is greater than the threshold and the content feature into the sorting model, and predicts the probability of the target user clicking and playing the content to be delivered respectively through the click task and the play task;

[0178] The scoring subunit 636 scores the target user according to the probability to obtain a user score of the target user.

[0179] Optionally, the device further comprises:

[0180] An acquisition subunit 632 is configured to acquire the business scenario to which the content to be delivered belongs and the scenario characteristics of the business scenario;

[0181] The third calculation subunit 634 is further used to input the target user features whose feature similarity is greater than a threshold, the content features and the scenario features into the sorting model, and predict the probability of the target user clicking and playing the content to be delivered in the business scenario through the click task and the play task respectively.

[0182] Optionally, the determining unit 640 includes:

[0183] The sorting subunit 642 sorts the user scores of the target users, and selects the users with the first preset number of target scores from the sorting results to form a user group that is interested in the content to be delivered.

[0184] Optionally, the determining unit 640 includes:

[0185] The comparison subunit 646 compares the user score of each target user with a score threshold, and selects users with scores greater than the score threshold to form a user group that is interested in the content to be delivered.

[0186] Optionally, the threshold is calculated by the threshold calculation subunit 644:

[0187] The threshold calculation subunit 644 sorts the user scores of the target users, takes the first preset number of target scores of the sorting results; calculates the total score and the average score of the preset number of target scores, performs linear weighted calculation based on the total score and the average score, and obtains the score threshold.

[0188] Optionally, the device further comprises:

[0189] The delivery unit 650 pushes the content to be delivered to the user group.

[0190] Exemplary Computing Devices

[0191] After introducing the method, medium and apparatus of the exemplary embodiments of the present disclosure, next, reference is made to Figure 7 A computing device according to an exemplary embodiment of the present disclosure is described.

[0192] Figure 7 The computing device 1500 shown is merely an example and should not bring any limitation to the functionality and scope of use of the embodiments of the present disclosure.

[0193] like Figure 7 As shown, computing device 1500 is in the form of a general computing device. Components of computing device 1500 may include, but are not limited to: at least one processing unit 1501, at least one storage unit 1502, and a bus 1503 connecting different system components (including processing unit 1501 and storage unit 1502).

[0194] The bus 1503 includes a data bus, a control bus, and an address bus.

[0195] The storage unit 1502 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 15021 and / or a cache memory 15022 , and may further include a readable medium in the form of a non-volatile memory, such as a read-only memory (ROM) 15023 .

[0196] The storage unit 1502 may also include a program / utility 15025 having a set (at least one) of program modules 15024, such program modules 15024 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0197] The computing device 1500 may also communicate with one or more external devices 1504 (eg, a keyboard, pointing device, etc.).

[0198] Such communication may be performed via input / output (I / O) interface 1505. In addition, computing device 1500 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via network adapter 1506. Figure 7 As shown, network adapter 1506 communicates with other modules of computing device 1500 via bus 1503. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with computing device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0199] Through Figure 7 The computing device 1500 shown can implement the aforementioned method for determining a user group. More specifically, the storage unit 1502 stores instructions executable by the processing unit 1501. When the processing unit 1501 executes the instructions, the aforementioned method for determining a user group is implemented.

[0200] It should be noted that, although several units / modules or sub-units / modules of the user group determination device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules for embodiment.

[0201] In addition, although the operations of the disclosed method are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0202] Although the spirit and principle of the present disclosure have been described with reference to several specific embodiments, it should be understood that the present disclosure is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present disclosure is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A method for determining a user group, comprising: Obtaining content features related to the content to be delivered and user features related to the user to be pushed; Inputting the content feature and the user feature into a contrastive learning model, and calculating the feature similarity between the content feature and the user feature; Inputting the target user features and the content features whose feature similarity is greater than a threshold into the sorting model, and calculating the user score of the target user corresponding to the target user features; wherein the user score represents the interest degree of the target user in the content to be delivered; Based on the user score of each target user, a user group that is interested in the content to be delivered is determined.

2. The method according to claim 1, wherein the contrastive learning model includes a user tower network and a content tower network; wherein, The user tower network is used to process user features, and the content tower network is used to process content features; The step of inputting the content feature and the user feature into a contrastive learning model and calculating the feature similarity between the content feature and the user feature includes: Inputting the user characteristics into the user tower network for calculation, and inputting the content characteristics into the content tower network for calculation; The output of the user tower network and the output of the content tower network are further input into a fully connected layer for calculation to obtain the feature similarity between the content feature and the user feature.

3. According to the method of claim 2, the contrastive learning model further comprises an embedding layer for receiving model input, and the output of the embedding layer is connected to the user tower network and the content tower network respectively; Before inputting the user features into the user tower network for calculation and inputting the content features into the content tower network for calculation, the method further includes: Inputting the content features and the user features into the embedding layer of the contrastive learning model, and calculating a content embedding vector corresponding to the content features and a user embedding vector corresponding to the user features; The step of inputting the user features into the user tower network for calculation, and inputting the content features into the content tower network for calculation, comprises: The user embedding vector is input into the user tower network for calculation, and the content embedding vector is input into the content tower network for calculation.

4. The method according to claim 1, wherein the sorting model comprises a multi-task sorting model; wherein: The multiple tasks include a click task of predicting the content that the user will click and a play task of predicting the content that the user will play; The step of inputting the target user feature whose feature similarity is greater than a threshold and the content feature into the sorting model, and calculating the user score of the target user corresponding to the target user feature, comprises: The target user features and the content features whose feature similarity is greater than a threshold are input into the sorting model, and the probability of the target user clicking and playing the content to be delivered is predicted through the click task and the play task respectively, and the target user is further scored according to the probability to obtain the user score of the target user.

5. The method according to claim 4, further comprising: Acquire the business scenario to which the content to be delivered belongs and the scenario characteristics of the business scenario; The target user feature with feature similarity greater than a threshold and the content feature are input into the sorting model, the probability of the target user clicking and playing the content to be delivered is predicted through the click task and the play task respectively, and the target user is further scored according to the probability to obtain the user score of the target user, including: The target user features, the content features and the scenario features whose feature similarity is greater than a threshold are input into the sorting model, and the probability of the target user clicking and playing the content to be delivered in the business scenario is predicted respectively through the click task and the play task, and the target user is further scored according to the probability to obtain the user score of the target user.

6. The method according to claim 1, wherein determining the user group interested in the content to be delivered based on the user score of each target user comprises: The user scores of the target users are sorted, and the users with the target scores of the first preset number of the sorted results are selected to form a user group interested in the content to be delivered.

7. The method according to claim 1, wherein determining the user group interested in the content to be delivered based on the user score of each target user comprises: The user score of each target user is compared with a score threshold, and users with scores greater than the score threshold are selected to form a user group that is interested in the content to be delivered.

8. A device for determining a user group, the device comprising: An acquisition unit, which acquires content features related to the content to be delivered and user features related to the user to be pushed; A first calculation unit, inputting the content feature and the user feature into a contrastive learning model, and calculating a feature similarity between the content feature and the user feature; A second calculation unit inputs the target user feature and the content feature whose feature similarity is greater than a threshold into a ranking model, and calculates a user score of the target user corresponding to the target user feature; wherein the user score represents the interest degree of the target user in the content to be delivered; The determination unit determines, based on the user score of each target user, a user group that is interested in the content to be delivered.

9. A computer-readable storage medium, comprising: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the method for determining a user group as described in any one of claims 1 to 7.

10. A computing device comprising: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the executable instructions to implement the method for determining a user group according to any one of claims 1 to 7.

Images