A training method and device for user identification model

Abstract

The present application provides a method and device for training a user identification connection model, wherein the method includes: for an advertising monitoring scenario, using a first deep learning network to extract word vectors from advertising click data within a preset time range, and inputting the word vectors into a first natural language processing model for pre-training; for an application scenario, using a second deep learning network to extract word vectors from account behavior data within a preset time range, and inputting the word vectors into a second natural language processing model for pre-training; based on supervised samples in which user identification has been connected, fine-tuning the parameters of the pre-trained first natural language processing model and second natural language processing model. The embodiments of the present application can improve the accuracy of user identification connection.

Description

Technical Field

[0001] The present application relates to the technical field of big data analysis, and in particular to a method and device for training a user identification model. Background Art

[0002] In the era of big data, the amount of log data collected in the advertising monitoring field is growing by billions every day. These logs contain a large amount of user behavior data, and the same user may leave data information in multiple scenarios. In the later big data analysis, how to accurately capture and integrate user information of the same identity is the key to big data analysis and is also related to the accuracy of the conclusion. Simply put, the traffic data, accommodation data, travel data, etc. of the same user will be reflected in various ways in different demand scenarios. How to accurately track users and locate multiple user data to the same person will determine the accuracy of subsequent big data analysis and will generally affect the results of user portrait analysis of a certain group of people.

[0003] When building accurate user portraits, we face the following problem: log collection cannot successfully collect all user IDs, and each business line has its own defined UID to identify users, resulting in fragmented user IDs. Therefore, in order to integrate user tags, connecting user IDs (also known as ID-Mapping) has become an urgent need.

[0004] In the existing technology, most of the solutions are based on rules. For the device ID and the account ID in other specific scenarios (such as a social APP), the region, model, timestamp and other information of both sides are obtained respectively, and then the corresponding rules are formulated for matching. Common rules include: 1) the model is the same and the region is the same; 2) the number of times the behavior roughly overlaps exceeds a certain threshold. If the above two rules are met, it can be determined that the behavior data in these two scenarios belongs to the same user, but for a device ID, there will be many user IDs that meet the above conditions, so the corresponding threshold needs to be adjusted. The setting of the threshold is generally determined based on the acceptable consistency level.

[0005] During the research process, the applicant found that in the prior art, 1) the rule-based solution can only perform approximate matching, that is, a batch of IDs are obtained and further precise screening cannot be performed; 2) the similarities of user behaviors in different scenarios are not fully utilized, that is, the behavioral data is not mined, and the accuracy cannot meet the needs of the production environment. Summary of the invention

[0006] In view of this, the purpose of the present application is to provide a method and device for training a user identification access model to improve the accuracy of user identification access.

[0007] In a first aspect, an embodiment of the present application provides a method for training a user identification access model, including:

[0008] For the advertising monitoring scenario, a first deep learning network is used to extract word vectors from the advertising click data within a preset time range, and the word vectors are input into a first natural language processing model for pre-training;

[0009] For application scenarios, a second deep learning network is used to extract word vectors from account behavior data within a preset time range, and the word vectors are input into a second natural language processing model for pre-training;

[0010] Based on the supervised samples whose user identities have been connected, fine-tuning is performed on the pre-trained first natural language processing model and the second natural language processing model.

[0011] In a possible implementation, the first deep learning network includes a first word vector model;

[0012] The first deep learning network is used to extract word vectors from the ad click data within a preset time range, including:

[0013] Processing the advertisement click data within a preset time range according to the order of timestamps to obtain an advertisement click time series;

[0014] Each element in the advertisement click time series is taken as a word, and the advertisement click time series is taken as a sentence, and is input into a first word vector model for training to obtain a word vector of the advertisement click time series.

[0015] In a possible implementation, the second deep learning network includes a second word vector model;

[0016] The second deep learning network is used to extract word vectors from the account behavior data within a preset time range, including:

[0017] The account behavior data within the preset time range is processed in the order of timestamps to obtain the account behavior time series;

[0018] Each element in the account behavior time series is taken as a word, and the account behavior time series is taken as a sentence, and is input into the second word vector model for training to obtain the word vector of the account behavior time series.

[0019] In a possible implementation, the word vector is input into a first natural language processing model for pre-training, including:

[0020] Randomly mask a preset proportion of advertising features from multiple advertising features corresponding to each advertisement in the word vector;

[0021] The masked word vector is input into the first natural language processing model for pre-training.

[0022] In a possible implementation, the word vector is input into a second natural language processing model for pre-training, including:

[0023] Randomly mask a preset proportion of account behavior features from multiple account behavior features corresponding to each account in the word vector;

[0024] The masked word vector is input into the second natural language processing model for pre-training.

[0025] In a possible implementation, fine tuning the parameters of the pre-trained first natural language processing model and the second natural language processing model based on the supervised sample whose user identification has been obtained includes:

[0026] Based on the supervised samples to which the user identification has been connected, first behavior sequence features in the advertisement monitoring scenario and second behavior sequence features in the application scenario are respectively obtained;

[0027] Inputting the first behavior sequence feature and the second behavior sequence feature into the pre-trained first natural language processing model and the second natural language processing model respectively, to obtain a first target vector and a second target vector respectively;

[0028] Concatenate the first target vector and the second target vector, and perform a pooling operation on the concatenated target vector;

[0029] Perform batch normalization on the target vector obtained after the pooling operation;

[0030] Performing classification processing on the target vector obtained after batch normalization processing to obtain the target label of the supervised sample;

[0031] Based on the target labels and original labels of the supervised samples, fine-tune the parameters of the pre-trained first natural language processing model and the second natural language processing model.

[0032] In one possible implementation, the advertising click data includes any multiple of the following advertising characteristics: device identification, advertising identification, advertising industry identification, advertising media identification, advertising product identification, advertising material identification, timestamp, and number of advertising clicks per unit time; the account behavior data includes any multiple of the following account behavior characteristics: likes, comments, reposts, blogs, regions, models, and timestamps.

[0033] In a second aspect, an embodiment of the present application further provides a training device for a user identification access model, including:

[0034] A first pre-training module is used to extract word vectors from advertisement click data within a preset time range using a first deep learning network for an advertisement monitoring scenario, and input the word vectors into a first natural language processing model for pre-training;

[0035] A second pre-training module is used to extract word vectors from account behavior data within a preset time range using a second deep learning network for an application scenario, and input the word vectors into a second natural language processing model for pre-training;

[0036] A fine-tuning module is used to fine-tune the pre-trained first natural language processing model and the second natural language processing model based on supervised samples whose user identifiers have been opened.

[0037] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0038] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are executed.

[0039] The embodiment of the present application provides a training method for a user identification connection model. First, for the advertising monitoring scenario, a first deep learning network is used to extract word vectors from the advertising click data within a preset time range, and the word vectors are input into the first natural language processing model for pre-training; then for the application scenario, a second deep learning network is used to extract word vectors from the account behavior data within a preset time range, and the word vectors are input into the second natural language processing model for pre-training; finally, based on the supervised samples in which the user identification has been connected, the pre-trained first natural language processing model and the second natural language processing model are finely tuned. Since the amount of advertising click data and account behavior data is large and sparse, and contains a lot of noise, the accuracy of the word vectors can be improved by extracting word vectors from these data through deep learning. By learning user behavior data in two different scenarios through a natural language processing model, the accuracy of user identification connection can be improved.

[0040] Furthermore, for the advertising monitoring scenario, the advertising click data within the preset time range is processed in the order of timestamps to obtain the advertising click time series; each element in the advertising click time series is used as a word and the advertising click time series is used as a sentence, and is input into the first word vector model for training to obtain the word vector of the advertising click time series. For the application scenario, the account behavior data within the preset time range is processed in the order of timestamps to obtain the account behavior time series; each element in the account behavior time series is used as a word and the account behavior time series is used as a sentence, and is input into the second word vector model for training to obtain the word vector of the account behavior time series. In-depth data mining from the perspective of time series can more accurately portray user portraits.

[0041] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 A flow chart of a method for training a user identification access model provided in an embodiment of the present application is shown;

[0044] Figure 2 A schematic diagram of the structure of a user identification access model provided in an embodiment of the present application is shown;

[0045] Figure 3 A schematic diagram of the structure of a training device for a user identification access model provided in an embodiment of the present application is shown;

[0046] Figure 4 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0047] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0048] Considering that most of the existing technologies are based on rule-based solutions, for the device ID and the account ID in other specific scenarios (such as a social APP), the region, model, timestamp and other information on both sides are obtained respectively, and then the corresponding rules are formulated for matching. Common rules include: 1) the model is the same and the region is the same; 2) the number of times the behavior roughly overlaps exceeds a certain threshold. If the above two rules are met, it can be determined that the behavior data in these two scenarios belongs to the same user, but for a device ID, there will be many user IDs that meet the above conditions, so the corresponding threshold needs to be adjusted. The setting of the threshold is generally determined based on the acceptable consistency level.

[0049] During the research process, the applicant found that in the prior art, 1) the rule-based solution can only perform approximate matching, that is, a batch of IDs are obtained and further precise screening cannot be performed; 2) the similarities of user behaviors in different scenarios are not fully utilized, that is, the behavioral data is not mined, and the accuracy cannot meet the needs of the production environment.

[0050] Based on this, an embodiment of the present application provides a method and device for training a user identification access model, which is described below through an embodiment.

[0051] To facilitate understanding of this embodiment, a training method for a user identification model disclosed in an embodiment of the present application is first introduced in detail.

[0052] Please refer to Figure 1 , Figure 1 A flowchart of a method for training a user identification model provided in an embodiment of the present application. Figure 1 As shown, the following steps may be included:

[0053] S101. For an advertisement monitoring scenario, extract word vectors from advertisement click data within a preset time range using a first deep learning network, and input the word vectors into a first natural language processing model for pre-training;

[0054] S102. For an application scenario, using a second deep learning network to extract word vectors from account behavior data within a preset time range, and inputting the word vectors into a second natural language processing model for pre-training;

[0055] S103: Based on the supervised samples whose user identifiers have been connected, fine-tune the parameters of the pre-trained first natural language processing model and the second natural language processing model.

[0056] In step S101, the ad click data includes any number of the following ad features: device identification, ad identification, ad attributes (advertising industry identification, advertising media identification, advertising product identification, advertising material identification), timestamp, and number of ad clicks per unit time. Among them, the ad click log contains ad features such as device identification, ad identification, advertising industry identification, advertising media identification, advertising product identification, advertising material identification, and timestamp (accurate to milliseconds). In order to reduce the magnitude of data and minimize the loss of data information, the timestamp is processed into days, and the user behavior is aggregated to generate a new feature field: the number of times the device clicks on the ad on that day. Regarding the number of ad clicks per unit time, the unit time can be one day, and the number of ad clicks in one day refers to the number of times the device clicks on the ad on that day. In a specific implementation, the ad click data within a preset time range can be the ad click data within three months. The ad click data of the user within three months can be obtained by collecting the ad click logs of the user for the past three months, and performing feature extraction and processing on these ad click logs.

[0057] The first deep learning network includes a first word vector model, such as a word2vec model. The Word2vec model is a group of related models used to generate word vectors. These models are shallow and double-layer neural networks used for training to reconstruct linguistic word texts. This embodiment is not limited to this, and the first word vector model can also be other word vector models.

[0058] The first natural language processing model can be a BERT model, which aims to pre-train deep bidirectional representations by jointly conditioning the context in all layers. Therefore, the pre-trained BERT representation can be fine-tuned through an additional output layer.

[0059] In this step, since the amount of ad click data is large and sparse and contains a lot of noise, word vectors are extracted from these data through deep learning to improve the accuracy of word vectors.

[0060] In a possible implementation, Figure 2As shown, the first deep learning network may include a first word vector model, which sorts the advertisement click data within a preset time range in the order of timestamps to obtain an advertisement click time series; each element in the advertisement click time series is used as a word, and the advertisement click time series is used as a sentence, and is input into the first word vector model for training to obtain a word vector of the advertisement click time series. In this embodiment, a high-dimensional and sparse advertisement click time series can be converted into a low-dimensional and dense word vector. In-depth mining of data from the perspective of time series can more accurately portray user portraits.

[0061] In one possible implementation, a preset proportion of advertising features are randomly masked from the multiple advertising features corresponding to each advertisement in the word vector; the masked word vector is input into the first natural language processing model for pre-training. Specifically, this embodiment improves the MLM (Masked Language Modeling). The original MLM randomly masks a preset number of complete words in the word vector. The MLM of this embodiment randomly masks a part of each word in the word vector (advertising identifier or advertising attribute), that is, a preset proportion of advertising features in the multiple advertising features corresponding to each advertisement in the word vector, for example, randomly masking 20% ​​of the tokens, and replacing the masked position with a 0 vector, or replacing it with an attribute identifier. When predicting these advertisements, this embodiment can not only use the most recently clicked advertisements, but also use its attributes to predict, so that the embeddings of different attributes can be aligned in the semantic space.

[0062] In step S102, assuming that the application scenario is a Weibo scenario, the account behavior data includes any number of the following account behavior features: likes, comments, forwarding, blogging, region, model, and timestamp. Among them, the account behavior log contains data such as likes, comments, forwarding, blogging, region, model, and timestamp (accurate to milliseconds). In a specific implementation, the account behavior data within the preset time range can be account behavior data within three months. The user's account behavior data within three months can be obtained by collecting the user's account behavior logs for the past three months, and extracting and processing features from these account behavior logs. This embodiment is not limited to this, and the application scenario can also be other social APP scenarios.

[0063] The second deep learning network includes a second word vector model, such as a word2vec model. The Word2vec model is a group of related models used to generate word vectors. These models are shallow and double-layer neural networks used for training to reconstruct linguistic word texts. This embodiment is not limited to this, and the second word vector model can also be other word vector models.

[0064] The second natural language processing model can be a BERT model, which is designed to pre-train deep bidirectional representations by jointly conditioning the context in all layers. Therefore, the pre-trained BERT representation can be fine-tuned through an additional output layer.

[0065] In this step, since the account behavior data is large and sparse and contains a lot of noise, extracting word vectors from these data through deep learning can improve the accuracy of word vectors.

[0066] In a possible implementation, Figure 2 As shown, the second deep learning network may include a second word vector model, which sorts the account behavior data within a preset time range in the order of timestamps to obtain an account behavior time series; each element in the account behavior time series is used as a word, and the account behavior time series is used as a sentence, and is input into the second word vector model for training to obtain a word vector of the account behavior time series. In this embodiment, a high-dimensional and sparse account behavior time series can be converted into a low-dimensional and dense word vector. In-depth mining of data from the perspective of time series can more accurately portray user portraits.

[0067] In one possible implementation, a preset proportion of account behavior features are randomly masked from the multiple account behavior features corresponding to each account in the word vector; the masked word vector is input into the second natural language processing model for pre-training. Specifically, this embodiment improves the MLM (Masked Language Modeling). The original MLM randomly masks a preset number of complete words in the word vector. The MLM of this embodiment randomly masks a part of each word in the word vector (likes or models), that is, a preset proportion of account behavior features in the multiple account behavior features corresponding to each account in the word vector, such as randomly masking 20% ​​of the tokens, and replacing the masked position with a 0 vector, or replacing it with a model. When predicting these account behaviors, this embodiment can not only use the most recent account behavior, but also use its model to predict, so that the embeddings of different models can be aligned in the semantic space.

[0068] In step S103, the supervised samples whose user identifications have been connected refer to some IDpair pairs whose user identifications have been connected.

[0069] In this step, based on the supervised samples in which the user identification has been connected, the pre-trained first natural language processing model and the second natural language processing model are fine-tuned. By learning the user behavior data in two different scenarios through the natural language processing model, the accuracy of user identification connection can be improved.

[0070] In a possible implementation, step S103 may include the following sub-steps:

[0071] S1031, based on the supervised samples to which the user identification has been connected, respectively obtain the first behavior sequence features in the advertisement monitoring scenario and the second behavior sequence features in the application scenario;

[0072] S1032, inputting the first behavior sequence feature and the second behavior sequence feature into the pre-trained first natural language processing model and the second natural language processing model respectively, to obtain a first target vector and a second target vector respectively;

[0073] S1033, concatenating the first target vector and the second target vector, and performing a pooling operation on the concatenated target vector;

[0074] S1034, performing batch normalization processing on the target vector obtained after the pooling operation;

[0075] S1035, performing classification processing on the target vector obtained after batch normalization processing to obtain the target label of the supervised sample;

[0076] S1036. Based on the target labels and original labels of the supervised samples, fine-tune the parameters of the pre-trained first natural language processing model and the second natural language processing model.

[0077] In step S1031, in a specific implementation, it is also necessary to process the first behavior sequence features in the advertisement monitoring scenario and the second behavior sequence features in the application scenario into an input format of a natural language processing model.

[0078] In step S1033, the first target vector and the second target vector are concatenated. For example, the first target vector is (16, 128), and the second target vector is also (16, 128). After the two are concatenated, (16, 256) is obtained.

[0079] Through the above steps S101-S103, the training is completed to obtain the final user identification connection model. When the user identification connection model of this embodiment is used to connect user identifications, the ad click data in the ad monitoring scenario and the account behavior data in the application scenario that need to be connected are first input into the user identification connection model for pre-training, and then based on the supervised samples of the user identification that have been connected, the pre-trained first natural language processing model and the second natural language processing model are finely tuned, and finally the user identifications of two different application scenarios are connected, which can improve the accuracy of user identification connection and is conducive to subsequent precision marketing and intelligent recommendation work.

[0080] The embodiment of the present application provides a training method for a user identification connection model. First, for the advertising monitoring scenario, a first deep learning network is used to extract word vectors from the advertising click data within a preset time range, and the word vectors are input into the first natural language processing model for pre-training; then for the application scenario, a second deep learning network is used to extract word vectors from the account behavior data within a preset time range, and the word vectors are input into the second natural language processing model for pre-training; finally, based on the supervised samples in which the user identification has been connected, the pre-trained first natural language processing model and the second natural language processing model are finely tuned. Since the amount of advertising click data and account behavior data is large and sparse, and contains a lot of noise, the accuracy of the word vectors can be improved by extracting word vectors from these data through deep learning. By learning user behavior data in two different scenarios through a natural language processing model, the accuracy of user identification connection can be improved.

[0081] Furthermore, for the advertising monitoring scenario, the advertising click data within the preset time range is processed in the order of timestamps to obtain the advertising click time series; each element in the advertising click time series is used as a word and the advertising click time series is used as a sentence, and is input into the first word vector model for training to obtain the word vector of the advertising click time series. For the application scenario, the account behavior data within the preset time range is processed in the order of timestamps to obtain the account behavior time series; each element in the account behavior time series is used as a word and the account behavior time series is used as a sentence, and is input into the second word vector model for training to obtain the word vector of the account behavior time series. In-depth data mining from the perspective of time series can more accurately portray user portraits.

[0082] Based on the same technical concept, the embodiments of the present application also provide a training device, electronic device, and computer storage medium for a user identification model. For details, please refer to the following embodiments.

[0083] Please refer to Figure 3 , Figure 3 A schematic diagram of a training device for a user identification model provided in an embodiment of the present application. Figure 3 As shown, the device may include:

[0084] A first pre-training module 10 is used to extract word vectors from advertisement click data within a preset time range using a first deep learning network for an advertisement monitoring scenario, and input the word vectors into a first natural language processing model for pre-training;

[0085] A second pre-training module 20 is used to extract word vectors from account behavior data within a preset time range using a second deep learning network for an application scenario, and input the word vectors into a second natural language processing model for pre-training;

[0086] The fine-tuning module 30 is used to fine-tune the pre-trained first natural language processing model and the second natural language processing model based on the supervised samples whose user identifiers have been opened.

[0087] In a possible implementation, the first deep learning network may include a first word vector model; and the first pre-training module 10 may include:

[0088] A first sequence generating unit is used to process the advertisement click data within a preset time range according to the sequence of timestamps to obtain an advertisement click time sequence;

[0089] The first word vector generating unit is used to input each element in the advertisement click time series as a word and the advertisement click time series as a sentence into the first word vector model for training to obtain the word vector of the advertisement click time series.

[0090] In a possible implementation, the second deep learning network may include a second word vector model; and the second pre-training module 20 may include:

[0091] A second sequence generating unit is used to process the account behavior data within a preset time range according to the order of timestamps to obtain an account behavior time series;

[0092] The second word vector generating unit is used to input each element in the account behavior time series as a word and the account behavior time series as a sentence into the second word vector model for training to obtain the word vector of the account behavior time series.

[0093] In a possible implementation, the first pre-training module 10 may further include:

[0094] A first masking unit is used to randomly mask a preset proportion of advertisement features from a plurality of advertisement features corresponding to each advertisement in the word vector;

[0095] The first pre-training unit is used to input the masked word vector into the first natural language processing model for pre-training.

[0096] In a possible implementation, the second pre-training module 20 may further include:

[0097] A second masking unit is used to randomly mask a preset proportion of account behavior features from a plurality of account behavior features corresponding to each account in the word vector;

[0098] The second pre-training unit is used to input the masked word vector into the second natural language processing model for pre-training.

[0099] In a possible implementation, the fine parameter adjustment module 30 may include:

[0100] An acquisition unit, configured to acquire, based on the supervised samples to which the user identification has been connected, the first behavior sequence features in the advertisement monitoring scenario and the second behavior sequence features in the application scenario;

[0101] An input unit, used to input the first behavior sequence feature and the second behavior sequence feature into the pre-trained first natural language processing model and the second natural language processing model, respectively, to obtain a first target vector and a second target vector, respectively;

[0102] A concatenation and pooling unit, used for concatenating the first target vector and the second target vector, and performing a pooling operation on the concatenated target vector;

[0103] Batch normalization unit, used to perform batch normalization on the target vector obtained after the pooling operation;

[0104] A classification unit, used to classify the target vector obtained after batch normalization to obtain the target label of the supervised sample;

[0105] A fine-tuning unit is used to fine-tune the parameters of the pre-trained first natural language processing model and the second natural language processing model based on the target label and the original label of the supervised sample.

[0106] In one possible implementation, the advertising click data may include any multiple of the following advertising characteristics: device identification, advertising identification, advertising industry identification, advertising media identification, advertising product identification, advertising material identification, timestamp, and number of advertising clicks per unit time; the account behavior data may include any multiple of the following account behavior characteristics: likes, comments, reposts, blogs, regions, and models.

[0107] The present application embodiment discloses an electronic device, such as Figure 4As shown, it includes: a processor 401, a memory 402 and a bus 403. The memory 402 stores machine-readable instructions executable by the processor 401. When the electronic device is running, the processor 401 communicates with the memory 402 through the bus 403. When the machine-readable instructions are executed by the processor 401, the method described in the previous method embodiment is performed. The specific implementation can be referred to the method embodiment, which will not be repeated here.

[0108] A computer program product of a user identification method provided in an embodiment of the present application includes a computer-readable storage medium storing a non-volatile program code executable by a processor. The instructions included in the program code can be used to execute the method described in the previous method embodiment. The specific implementation can be found in the method embodiment and will not be repeated here.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0110] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0111] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0112] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0113] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0114] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for training a user identification model, characterized in that: include: For the advertising monitoring scenario, a first deep learning network is used to extract word vectors from the advertising click data within a preset time range, and the word vectors are input into a first natural language processing model for pre-training; For the application scenario, a second deep learning network is used to extract word vectors from the account behavior data within a preset time range, and the word vectors are input into a second natural language processing model for pre-training; wherein the advertisement click data and the account behavior data are data within the same time range; Based on the supervised samples whose user identities have been connected, fine-tuning is performed on the pre-trained first natural language processing model and the second natural language processing model.

2. The method according to claim 1, characterized in that The first deep learning network includes a first word vector model; The first deep learning network is used to extract word vectors from the ad click data within a preset time range, including: The advertisement click data within a preset time range is processed according to the order of timestamps to obtain an advertisement click time series; Each element in the advertisement click time series is taken as a word, and the advertisement click time series is taken as a sentence, and is input into a first word vector model for training to obtain a word vector of the advertisement click time series.

3. The method according to claim 1, characterized in that The second deep learning network includes a second word vector model; The second deep learning network is used to extract word vectors from the account behavior data within a preset time range, including: The account behavior data within the preset time range is processed in the order of timestamps to obtain the account behavior time series; Each element in the account behavior time series is taken as a word, and the account behavior time series is taken as a sentence, and is input into the second word vector model for training to obtain the word vector of the account behavior time series.

4. The method according to claim 1, characterized in that: The word vector is input into the first natural language processing model for pre-training, including: Randomly mask a preset proportion of advertising features from multiple advertising features corresponding to each advertisement in the word vector; The masked word vector is input into the first natural language processing model for pre-training.

5. The method according to claim 1, characterized in that The word vector is input into the second natural language processing model for pre-training, including: Randomly mask a preset proportion of account behavior features from multiple account behavior features corresponding to each account in the word vector; The masked word vector is input into the second natural language processing model for pre-training.

6. The method according to claim 1, characterized in that Based on the supervised samples whose user identifiers have been connected, fine tuning the parameters of the pre-trained first natural language processing model and the second natural language processing model includes: Based on the supervised samples to which the user identification has been connected, first behavior sequence features in the advertisement monitoring scenario and second behavior sequence features in the application scenario are respectively obtained; Inputting the first behavior sequence feature and the second behavior sequence feature into the pre-trained first natural language processing model and the second natural language processing model respectively, to obtain a first target vector and a second target vector respectively; Concatenate the first target vector and the second target vector, and perform a pooling operation on the concatenated target vector; Perform batch normalization on the target vector obtained after the pooling operation; Performing classification processing on the target vector obtained after batch normalization processing to obtain the target label of the supervised sample; Based on the target labels and original labels of the supervised samples, fine-tune the parameters of the pre-trained first natural language processing model and the second natural language processing model.

7. The method according to claim 1, characterized in that The advertising click data includes any multiple of the following advertising features: device identification, advertising identification, advertising industry identification, advertising media identification, advertising product identification, advertising material identification, timestamp, and number of advertising clicks per unit time; the account behavior data includes any multiple of the following account behavior features: likes, comments, reposts, blogs, regions, models, and timestamps.

8. A training device for a user identification model, characterized in that: include: A first pre-training module is used to extract word vectors from advertisement click data within a preset time range using a first deep learning network for an advertisement monitoring scenario, and input the word vectors into a first natural language processing model for pre-training; A second pre-training module is used to extract word vectors from account behavior data within a preset time range using a second deep learning network for an application scenario, and input the word vectors into a second natural language processing model for pre-training; wherein the advertisement click data and the account behavior data are data within the same time range; A fine-tuning module is used to fine-tune the pre-trained first natural language processing model and the second natural language processing model based on supervised samples whose user identifiers have been opened.

9. An electronic device, characterized in that: include: A processor, a storage medium and a bus, wherein the storage medium stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the storage medium via the bus, and the processor executes the machine-readable instructions to perform the steps of any method as claimed in claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are executed.

Images