User recommendation model training method and device

Abstract

The invention discloses a user recommendation model training method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the steps of determining search word sorting data of each user category based on obtained full-amount search word data, further sorting search words of each sample by combining the user category corresponding to each sample to obtain preprocessed data, training a retrieval enhancement language model based on the preprocessed data, and obtaining a retrieval enhancement language model; and obtaining a target user recommendation model. According to the embodiment, the user data can be preprocessed based on optimization of the search term, the reasoning speed of the model can be increased, and the determination efficiency of the recommended user is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a training method and device for a user recommendation model. Background Art

[0002] In the field of Internet technology, as new energy technology matures, the primary task of new energy automobile companies is to occupy a larger user market to ensure the development of their own brands. For different automobile brands, an important operation to occupy the user market is to attract new users. The main way to attract new users is to find people or objects to find people.

[0003] The commonly used existing method of attracting new users is to screen new users through various models, such as similarity models, classification models, clustering models, etc. However, when attracting new users based on existing models, the length of user features may be very long. For example, the features of the search terms when users search for new energy on the website will result in a long input feature sequence for the model, which will cause the model's calculation amount to increase geometrically and take a long time to train. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method and device for training a user recommendation model, which can reduce the amount of model calculation and improve the model calculation efficiency.

[0005] To achieve the above object, according to one aspect of an embodiment of the present invention, a method for training a user recommendation model is provided, comprising:

[0006] Obtain a target number of sample data and full search term data, where each sample has a user category;

[0007] Determine the ranking data of the search terms for each user category based on the full amount of search term data;

[0008] Based on the ranking data of the search terms of each user category and the user category corresponding to each sample, the search terms contained in each sample in the target number of sample data are ranked to obtain preprocessed data of the sample data;

[0009] The retrieval enhanced language model is trained based on the preprocessed data of the sample data to obtain the target user recommendation model.

[0010] Optionally, the ranking data of the search terms of each user category is determined based on the target quantity and the total search term data, including:

[0011] Determine the frequency of each search term based on the full amount of search term data;

[0012] Calculate the probability distribution data of the search terms based on the frequency of each search term;

[0013] Ranking data of the search terms for each user category is determined based on the probability distribution data.

[0014] Optionally, determining the ranking data of the search terms of each user category based on the probability distribution data includes:

[0015] Determining the frequency of search terms that meet a preset probability distribution condition based on the probability distribution data;

[0016] The search terms of each user category are classified and sorted in descending order of the frequency of the search terms that meet the preset probability distribution conditions to obtain sorting data of the search terms of each user category.

[0017] Optionally, based on the sorting data of the search terms of each user category and the user category corresponding to each sample, the search terms contained in each sample in the target number of sample data are sorted to obtain preprocessed data of the sample data, including:

[0018] Determine the first search term in the search terms included in each sample within the ranking data range of the search terms of the user category corresponding to each sample;

[0019] Determine a second search term other than the first search term in the search terms included in each sample;

[0020] The search terms of each sample are sorted in order from the first search term to the second search term to obtain preprocessed data of the sample data.

[0021] Optionally, a retrieval enhanced language model is trained based on the preprocessed data of the sample data to obtain a target user recommendation model, including:

[0022] According to the first preset characteristic sequence length, each piece of data in the preprocessed data of the sample data is intercepted to obtain target sample data;

[0023] Determine the data of the target user category in the target sample data as positive sample data, and determine the data of categories other than the target category as negative sample data;

[0024] The retrieval enhancement model is trained based on the positive sample data and the negative sample data to obtain the target user recommendation model.

[0025] Optionally, the method further comprises:

[0026] Obtaining the user data to be recommended, the user data to be recommended including the search terms and corresponding frequencies of the user to be recommended;

[0027] The search terms of the users to be recommended are sorted based on the frequencies corresponding to the search terms of the users to be recommended, to obtain sorting data;

[0028] The sorted data is intercepted according to a preset second preset feature sequence length to obtain pre-processed data of the user to be recommended;

[0029] The preprocessed data of the user to be recommended is input into the target user recommendation model to obtain the target recommended user data.

[0030] Optionally, the retrieval enhanced language model adopts a dual-tower structure and includes a global information processing unit and a local information processing unit, and the global information processing unit and the local information processing unit are simplified to modify the exponential form of calculation into a linear form of calculation.

[0031] Optionally, the implementation process of changing the exponential calculation method to a linear calculation method is as follows:

[0032] Deleting the activation function unit and the initialization parameter variable in the global information processing unit and the local information processing unit to obtain a first global information processing simplification unit and a first local information processing simplification unit, wherein the first global information processing simplification unit and the first local information processing simplification unit include calculation items in the form of e exponents;

[0033] Transform the calculation item in the form of e exponential into a general calculation item to obtain a simplified second global information processing simplified unit and a second local information processing simplified unit;

[0034] Taylor expansion is performed on the second global information processing simplification unit and the second local information processing simplification unit, and variables in the expansion results are normalized to convert the exponential form calculation method into a linear form calculation method.

[0035] According to another aspect of an embodiment of the present invention, a training device for a user recommendation model is provided, comprising:

[0036] An acquisition module is used to acquire data of a target number of samples and full search term data, wherein each sample has a user category;

[0037] A determination module, used to determine the ranking data of the search terms of each user category based on the full amount of search term data;

[0038] A processing module, configured to sort the search terms contained in each sample in the target number of sample data based on the sorting data of the search terms of each user category and the user category corresponding to each sample, so as to obtain pre-processed data of the sample data;

[0039] The training module is used to train the retrieval enhanced language model based on the preprocessed data of the sample data to obtain the target user recommendation model.

[0040] Optionally, the determination module is further used to: determine the frequency of each search term based on the full search term data; calculate the probability distribution data of the search term based on the frequency of each search term; and determine the ranking data of the search term of each user category based on the probability distribution data.

[0041] Optionally, the determination module is also used to: determine the frequency of search terms that meet preset probability distribution conditions based on the probability distribution data; classify and sort the search terms of each user category in order from high to low according to the frequency of search terms that meet the preset probability distribution conditions, and obtain sorting data of the search terms of each user category.

[0042] Optionally, the processing module is also used to: determine the first search term among the search terms contained in each sample within the sorting data range of the search terms of the user category corresponding to each sample; determine the second search term other than the first search term among the search terms contained in each sample; and sort the search terms of each sample in order from the first search term to the second search term to obtain preprocessing data of the sample data.

[0043] Optionally, the training module is also used to: intercept each data in the preprocessed data of the sample data according to the first preset feature sequence length to obtain target sample data; determine the data of the target user category in the target sample data as positive sample data, and determine the data of categories other than the target category as negative sample data; train the retrieval enhancement model based on the positive sample data and the negative sample data to obtain a target user recommendation model.

[0044] Optionally, the device also includes: an acquisition module, which is also used to acquire user data to be recommended, and the user data to be recommended includes the search terms of the user to be recommended and the corresponding frequencies; a sorting module, which is used to sort the search terms of the user to be recommended based on the frequencies corresponding to the search terms of the user to be recommended, and obtain sorting data; an interception module, which is used to intercept the sorting data according to a preset second preset feature sequence length, and obtain pre-processed data of the user to be recommended; an input module, which is used to input the pre-processed data of the user to be recommended into the target user recommendation model, and obtain target recommended user data.

[0045] Optionally, the retrieval enhanced language model adopts a dual-tower structure and includes a global information processing unit and a local information processing unit. The global information processing unit and the local information processing unit are simplified to modify the exponential calculation method into a linear calculation method.

[0046] Optionally, the implementation process of changing the exponential calculation method to a linear calculation method is as follows:

[0047] Deleting the activation function unit and the initialization parameter variable in the global information processing unit and the local information processing unit to obtain a first global information processing simplification unit and a first local information processing simplification unit, wherein the first global information processing simplification unit and the first local information processing simplification unit include calculation items in the form of e exponents;

[0048] Transform the calculation item in the form of e exponential into a general calculation item to obtain a simplified second global information processing simplified unit and a second local information processing simplified unit;

[0049] Taylor expansion is performed on the second global information processing simplification unit and the second local information processing simplification unit, and variables in the expansion results are normalized to convert the exponential form calculation method into a linear form calculation method.

[0050] According to another aspect of an embodiment of the present invention, there is provided an electronic device, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for training a user recommendation model provided in an embodiment of the present invention.

[0051] According to another aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for training a user recommendation model provided by an embodiment of the present invention is implemented.

[0052] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method for training a user recommendation model provided by an embodiment of the present invention is implemented.

[0053] An embodiment of the above invention has the following advantages or beneficial effects: it can sort search terms according to the user's search term data, and the sorting data of different user categories can reflect the differences between different user categories. Unimportant search terms can be filtered out during sorting, and only the remaining important search terms are used as features for training, which can reduce the feature length of model training and improve training efficiency.

[0054] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0056] Figure 1 is a schematic diagram of main steps of a method for training a user recommendation model according to an embodiment of the present invention;

[0057] Figure 2 is a structural diagram of a person-finding module provided in an embodiment of the present invention;

[0058] Figure 3 is a schematic diagram of main modules of a device for training a user recommendation model according to an embodiment of the present invention;

[0059] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0060] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0061] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0062] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution disclosed in the present invention are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0063] It should be noted that in the technical solution of the present invention, the collection, use, storage, sharing and transfer of user personal information involved are in compliance with the provisions of relevant laws and regulations, and it is necessary to inform the user and obtain the user's consent or authorization. When applicable, the user's personal information is de-identified and / or anonymized and / or encrypted.

[0064] Figure 1 FIG. 1 is a schematic diagram of the main process of the method for determining recommended users according to an embodiment of the present invention. Figure 1 As shown, including:

[0065] Step S101, obtaining a target amount of sample data and full search term data, wherein each sample has a user category.

[0066] The data source stores the full amount of user data, including user data and user search term data. Sample data and full amount of search term data are obtained from the data source according to the preset target quantity. The data of each sample is the data of each user, including the user category identifier, the user's search term data and user information. The user types include new energy users, non-new energy users, and random users. The user category identifier can be 0, 1, and 2, corresponding to the three user categories respectively. The target quantity should not be set too small. The target number of sample data should include data of each category, or at least two categories of data. It needs to include categories that the trained model can recognize. For example, if the model ultimately needs to recognize new energy users, the target number of samples needs to include sample data of new energy users.

[0067] Optionally, the data in the data source may be data obtained through a specific search interface, for example, search term data historically input by a user obtained through a designated browsing interface including a search box.

[0068] Step S102: determining the ranking data of the search terms of each user category based on the full amount of search term data.

[0069] The frequency of occurrence of each search term in the full amount of search terms is counted, and the distribution of the frequency of occurrence of the search terms is calculated in combination with the target number. The frequency of occurrence of the search terms is screened based on the distribution, and classified according to the category of the user corresponding to the search term to obtain the frequency data of the search terms in each category. The search terms in each category are sorted according to the frequency to obtain the frequency ranking data of the search terms in each category. Specifically, the TOP N search term ranking data of different categories can be obtained. The value of N can be set according to the actual situation and is not limited to this. Among them, the frequency of occurrence of the search term can be set to the number of times the user has searched for the search term.

[0070] Optionally, when filtering the occurrence frequencies of search terms based on the distribution, search terms with lower occurrence frequencies may be filtered out to reduce the number of search terms, thereby reducing the length of features composed of subsequent search terms.

[0071] Optionally, when sorting the search terms of each user category, statistics can be collected on the browsing time of the pop-up interface after the user searches based on the search terms to obtain the total browsing time of each search term, and sort them according to the time length to obtain the sorting data of each search term. In order to reduce the number of search terms, search terms with shorter browsing time can be filtered.

[0072] Step S103 , based on the sorting data of the search terms of each user category and the user category corresponding to each sample, sorting processing is performed on the search terms contained in each sample in the target number of sample data to obtain preprocessing data of the sample data.

[0073] According to the sorting data of the frequency of search terms for each user category, the search terms in each sample are sorted to obtain preprocessed data. For example, the search terms in each sample are sorted in the order of the sorting data. An example is given to illustrate the specific flow of the sorting operation: for example, the user type of the sample is a new energy user, and the search terms included are "a", "b", "c", "d" and "e", and the corresponding TOP N search terms for new energy users are ranked 1.b, 2.e, 3.a, 4.f, 5.g, 6.c, 7.d. The search term ranking of the sample determined based on the above data is beacd.

[0074] Step S104: training a retrieval enhanced language model based on the preprocessed data of the sample data to obtain a target user recommendation model.

[0075] The preprocessed data of the sample data is used as the training sample data of the model and input into the retrieval enhanced language model. The model parameters are adjusted according to the output results of the model until the end condition of the model training is met, that is, the error rate is less than the preset threshold, and the target user recommendation model is obtained.

[0076] According to the training method of the user recommendation model provided by the embodiment of the present invention, the acquired search terms can be classified and sorted according to user categories, and each sample data can be sorted according to the user category to obtain preprocessed data. Further, a target user recommendation model is obtained by training a retrieval enhanced language model based on the preprocessed data, thereby improving the accuracy of the model output results. By searching for data with lower frequency of search terms, the amount of input data during model training can be reduced, the feature length can be reduced, the speed of model training and reasoning can be greatly improved, and the efficiency of model training can be improved.

[0077] Optionally, the ranking data of search terms for each user category is determined based on the full amount of search term data, including: determining the frequency of each search term based on the full amount of search term data; calculating the probability distribution data of the search term based on the frequency of each search term; and determining the ranking data of the search terms for each user category based on the probability distribution data. According to the acquired full amount of search term data, the occurrence frequency of each search term is counted to obtain the search term-frequency data. The probability distribution of the frequency is calculated based on the data of the search term frequency. Specifically, a probability distribution function such as normal distribution, chi-square distribution, t distribution, F distribution, etc. can be selected, and the search term frequency / (target number / user type number) is used as the independent variable for calculation. All search terms within a variance of the probability distribution function are sorted to obtain the TOP N search terms for each category. Among them, the setting of the N value is the same in each user category, that is, the TOP N search term data of the same length are intercepted for each category of users.

[0078] Optionally, the ranking data of the frequency of search terms of each user category can also be obtained by counting the occurrence frequency of search terms of each user category and sorting them in descending order to obtain the top N search terms. The search terms can be screened according to a preset frequency threshold, and the screened search terms are used for ranking processing.

[0079] Optionally, determining the ranking data of the search terms of each user category based on the probability distribution data includes: determining the frequency of the search terms that meet the preset probability distribution conditions based on the probability distribution data; classifying and sorting the search terms of each user category in order of the frequency of the search terms that meet the preset probability distribution conditions from high to low, and obtaining the ranking data of the search terms of each user category. Obtain all search terms within a variance of the probability distribution function, determine the frequency of occurrence of each search term, sort all search terms according to the frequency, and obtain the ranking data of the search terms corresponding to each user category. For example, the search term ranking corresponding to the user with user category 0 is: beacd.

[0080] Optionally, when performing sorting, the search terms may be sorted in descending order according to the probability value of each search term to obtain sorting data of the search terms in each category.

[0081] Optionally, based on the ranking data of search terms of each user category and the user category corresponding to each sample, the search terms contained in each sample in the target number of sample data are sorted to obtain preprocessed data of the sample data, including: determining the first search term in the search terms contained in each sample within the ranking data range of the search terms of the user category corresponding to each sample; determining the second search term in the search terms contained in each sample except the first search term; sorting the search terms of each sample in the order from the first search term to the second search term to obtain preprocessed data of the sample data. When arranging the search terms of each sample, first obtain the ranking data of the TOP N search terms of the user category corresponding to the sample, select the search terms in the TOP N range in each sample as the first search term, and determine the search terms in the search terms of the sample except the first search term as the second search term. Sorting is performed in the order from the first search term to the second search term to obtain preprocessed data of each sample. Among them, the sorting of the search terms in the first search term can be performed according to the sorting in the TOP N; the sorting of the search terms in the second search term can be sorted according to the frequency, or it can be sorted randomly.

[0082] According to the training method of the user recommendation model provided by the embodiment of the present invention, search terms can be sorted according to the user's search term data. The sorting data of different user categories can reflect the differences between different user categories. Unimportant search terms can be filtered out during sorting, and only the remaining important search terms are used as features for training, which can reduce the feature length of model training and improve training efficiency.

[0083] Optionally, training a retrieval enhancement language model based on the preprocessed data of the sample data to obtain a target user recommendation model includes: intercepting each piece of data in the preprocessed data of the sample data according to a first preset feature sequence length to obtain target sample data; determining the data of the target user category in the target sample data as positive sample data, and determining the data of categories other than the target category as negative sample data; training a retrieval enhancement model based on the positive sample data and the negative sample data to obtain a target user recommendation model. The preprocessed data of the sample data obtained after sorting also needs to perform a unified feature interception operation, that is, intercepting the data of each sample according to a preset length. Specifically, the search terms of each piece of data in the preprocessed data are digitally encoded to obtain each piece of data, that is, a feature sequence of the search terms of each user, and intercepting according to the first preset feature sequence length to obtain a search term feature sequence of each user with the same length. The intercepted data is classified according to the user category of each user and the preset target user category, and the data belonging to the target user category is determined as the positive sample data required for model training, and the data not belonging to the target user category is determined as the negative sample data required for model training. For example, user categories are 0, 1, and 2, and the target user category is determined to be 1. Then the user category 1 is the positive sample data, and the user categories 0 and 2 are the negative sample data. The retrieval enhancement model is trained based on the determined positive sample data and negative sample data to obtain the target user recommendation model.

[0084] Optionally, when truncating, the search term may be cut into character dimensions, and the corresponding digital code may be obtained by marking, and a string of codes corresponding to the search term may be merged with other search term features to obtain a feature sequence of the sample.

[0085] Optionally, when performing feature sequence interception, the sorted search terms are first encoded to obtain the user's search term feature sequence, which is intercepted according to the first preset feature sequence length. If only part of the code of a search term is intercepted during interception, the part of the code can be retained, or the part of the code can be uniformly set to 0.

[0086] According to the training method of the user recommendation model provided by the embodiment of the present invention, it is possible to use more important search term features for processing in the data preprocessing stage, including sorting the search terms by frequency. This stage processing method can retain the important features of the user while greatly reducing the user's feature sequence, which helps to speed up the model training and ensure the accuracy of the model's output results.

[0087] Optionally, the method further includes: obtaining user data to be recommended, the user data to be recommended including search terms and corresponding frequencies of the user to be recommended; sorting the search terms of the user to be recommended based on the frequencies corresponding to the search terms of the user to be recommended to obtain sorting data; intercepting the sorting data according to a preset second preset feature sequence length to obtain preprocessed data of the user to be recommended; inputting the preprocessed data of the user to be recommended into the target user recommendation model to obtain target recommended user data. Obtaining user data to be recommended, including data of search terms of the user to be recommended, including search terms and corresponding frequencies. According to the frequency of the search terms, sorting the search terms of each user from high to low according to the frequency to obtain sorting data. According to the preset second preset feature sequence length, intercepting the feature sequence after digital encoding of the sorting data to obtain preprocessing data of the user to be recommended. Inputting the preprocessing data into the target user recommendation model to obtain output target recommended user data. Specifically, the preprocessing data input into the target user recommendation model includes data of multiple users, and the target recommended user data among the multiple users output by the model is obtained. The target recommended user is a user of the determined target user category; in order to ensure the accuracy of the model output results, the preset second preset feature sequence length should be the same as the preset first preset feature sequence length. The data of the user to be recommended includes the search term data and user information of the user to be recommended, and the range of users to be recommended can be set according to the actual situation. The corresponding conditions can be set for screening, for example, all non-new energy users.

[0088] Optionally, the retrieval enhancement language model adopts a dual-tower structure and includes a global information processing unit and a local information processing unit. The global information processing unit and the local information processing unit are simplified to modify the exponential calculation method into a linear calculation method.

[0089] Optionally, the implementation process of changing the exponential calculation method to a linear calculation method is as follows:

[0090] Deleting the activation function unit and the initialization parameter variable in the global information processing unit and the local information processing unit to obtain a first global information processing simplification unit and a first local information processing simplification unit, wherein the first global information processing simplification unit and the first local information processing simplification unit include calculation items in the form of e exponents;

[0091] Transform the calculation item in the form of e exponential into a general calculation item to obtain a simplified second global information processing simplified unit and a second local information processing simplified unit;

[0092] Taylor expansion is performed on the second global information processing simplification unit and the second local information processing simplification unit, and variables in the expansion results are normalized to convert the exponential form calculation method into a linear form calculation method.

[0093] Optionally, after the sample data is pre-processed, it is mainly calculated by the person-finding module of the model. The module structure is as follows: Figure 2 As shown. The module adopts a dual-tower structure. The left input is the embedding (a way to convert discrete variables into continuous vector representation) vector of the seed user, and the right input is the embedding vector of the target user. Both embeddings need to pass through a layer of FC (fully connected layer) to complete the mapping to the low-dimensional space. Since the embedding vector of the target user on the right is obtained through user representation learning, in order to prevent overfitting, the first layer FC of the dual tower is shared. Among them, the seed user is the new energy user, and the target user is the user to be recommended.

[0094] After the data passes through the FC layer, the tower on the left can get the embeddings corresponding to each cluster. The embeddings of each cluster and the embeddings of the target user are input to the global attention unit (vector unit of global importance weight) and the local attention unit (vector unit of local importance weight) to get global-Attention (vector of global importance weight) and local-Attention (vector of local importance weight). Global-Attention (i.e. global information processing unit) and local-Attention (i.e. local information processing unit) are two variants of Attention, which can better capture global and local information.

[0095] The vector of the seed user passes through the global-Attention module:

[0096]

[0097] Among them, E s is the seed user embedding vector, W u and W g They are the initialization parameters of the model and the variables that change during the model training process.

[0098] The seed user vector and target user vector enter the local-Attention module:

[0099]

[0100] Among them, E s is the seed user embedding vector, E u is the target user embedding vector, Wl is the initialization parameter of the model. After the embedding vector data of the seed user and the target user pass through the global-Attention and local-Attention modules, they become the corresponding global and local vectors. In essence, it is still a vector transformation, but it can better capture global and local information. Because it takes a lot of time for the embedding vector data to pass through the global-Attention and local-Attention modules, it is necessary to improve the two modules.

[0101] Based on the above steps, the global-Attention and local-Attention modules are explained and simplified. The tanh module included is as follows:

[0102]

[0103] Tanh module is a kind of activation function. After the data passes through the activation function, its nonlinearity will increase, which can better fit the model. Since there are multiple activation functions in the entire model framework, they can be omitted in these two Attention modules. In order to show the optimization method more intuitively. u , W g , W l As the initialization parameters of the model, that is, variables, we can ignore them here. The global-Attention and local-Attention modules become as follows:

[0104]

[0105]

[0106] The global-Attention and local-Attention modules temporarily ignore the tanh module and some variables (i.e., the deleted activation function units and initialization parameter variables), which makes it easier to observe the flow of data.

[0107] The global-Attention and local-Attention modules have been simplified. Based on the simplified modules, the softmax method is as follows:

[0108]

[0109] The data x is operated in the form of e exponential operation through the softmax method. Compared with linear operation, this calculation method will increase the amount of calculation. For the simplified Attention module (i.e., the first global information processing simplified unit and the first local information processing simplified unit, and is a calculation term in the form of e exponential form), and That is its data x:

[0110]

[0111]

[0112] The i-th data in the global-Attention module is the q vector of the i-th data of the seed user and the k and v vectors of the j-th data multiplied, added and divided by the q and k vectors of all seed user data. Similarly, the i-th data in the local-Attention module is the q and v vectors of the i-th data of the seed user and the k vector of the j-th data of the target user multiplied, added and divided by the q and k vectors of all seed user and target user data. Among them, the q vector is the query vector, the k vector is the key vector, and the v vector is the value vector.

[0113] Further analysis of the global-Attention and local-Attention modules shows that and Both modules essentially perform weighted averaging on the v vector of the jth data of the seed user. Therefore, the global-Attention and local-Attention modules are further simplified to obtain the second global information processing simplified unit and the second local information processing simplified unit as follows:

[0114]

[0115]

[0116] This processing method does not need to be limited to the exponential form of e, but is more concerned with data transformation. The processing of data changes from the exponential form of e to a more general method.

[0117] Will and The two modules become common modules and In order to retain the distribution characteristics similar to Attention, it is required here and It is always true that it is greater than or equal to zero. After becoming a universal module, the present invention proposes a Taylor expansion-based form to optimize and accelerate the original softmax method.

[0118] The e-exponential form of the multiplication of the q vector of the i-th data of the seed user and the k vector of the j-th data in the module becomes the multiplication of the q vector of the i-th data of the seed user and the k vector of the j-th data and then adding 1. Similarly, In this method, the exponential form of the multiplication of the q vector of the seed user's i-th data and the k vector of the target user's j-th data becomes the multiplication of the q vector of the seed user's i-th data and the k vector of the target user's j-th data and then adding 1:

[0119]

[0120]

[0121] To ensure and The two methods are non-negative. The q vector of the seed user's i-th data, the k vector of the seed user's j-th data, and the k vector of the target user's j-th data are normalized respectively. Thus, the global-Attention and local-Attention modules are simplified to the following forms:

[0122]

[0123]

[0124] Finally, the exponential form of the nonlinear e in the global-Attention and local-Attention modules is approximated to a linear form, so the complexity of the global-Attention and local-Attention modules of the model is reduced from O(n 2 ) becomes O(n), thus speeding up the training and reasoning of the model.

[0125] Optionally, the similarity calculation is based on the seed user features and the target user features. After the similarity is output, the target user and the seed user are sorted according to their similarity, and the number of user recommendations determined according to the preset needs is selected and output in the sorting order. Among them, the Sigmoid function is a nonlinear function, the purpose of which is to increase the nonlinear ability of the entire model, thereby increasing the model's ability to fit the data.

[0126] Optionally, the target user recommendation model can also include an object-to-person module, which encodes the features of the items that the seed user has browsed in a certain historical period, finds the relevant target users through similarity calculation, sorts the target users, and takes out the top data as the target user recommendation data.

[0127] According to the training method of the user recommendation model provided by the embodiment of the present invention, the data for model training can be preprocessed, including sorting the search terms of each sample according to the sorting data of the search terms corresponding to the user category, and intercepting according to the preset length to obtain a feature sequence of a certain length, which can retain important features while greatly reducing the feature length, reducing the time consumed during model training, and improving the accuracy of the model. In addition, by improving the activation function of the model, the exponential function is improved to a linear function, which further accelerates the training and reasoning speed of the model.

[0128] Figure 3 3 is a schematic diagram of the main modules of a training device 300 for a user recommendation model provided by an embodiment of the present invention, including:

[0129] An acquisition module 301 is used to acquire data of a target number of samples and full search term data, wherein each sample has a user category;

[0130] A determination module 302, configured to determine ranking data of search terms of each user category based on the full amount of search term data;

[0131] A processing module 303 is used to sort the search terms contained in each sample in the target number of sample data based on the sorting data of the search terms of each user category and the user category corresponding to each sample, so as to obtain pre-processed data of the sample data;

[0132] The training module 304 is used to train the retrieval enhanced language model based on the pre-processed data of the sample data to obtain a target user recommendation model.

[0133] According to the training device of the user recommendation model provided by the embodiment of the present invention, it is possible to classify and sort the acquired search terms according to user categories, and sort each sample data according to the user category to obtain preprocessed data. Further, a target user recommendation model is obtained by training the retrieval enhanced language model based on the preprocessed data, thereby improving the accuracy of the model output results. By using data with lower search term frequency, the amount of input data during model training can be reduced, the feature length can be reduced, the speed of model training and reasoning can be greatly improved, and the model training efficiency can be improved.

[0134] Optionally, the determination module 302 is further used to: determine the frequency of each search term based on the full search term data; calculate the probability distribution data of the search term based on the frequency of each search term; and determine the ranking data of the search term of each user category based on the probability distribution data.

[0135] Optionally, the determination module 302 is also used to: determine the frequency of search terms that meet preset probability distribution conditions based on the probability distribution data; classify and sort the search terms of each user category in order from high to low according to the frequency of search terms that meet the preset probability distribution conditions, and obtain sorting data of the search terms of each user category.

[0136] Optionally, the processing module 303 is also used to: determine the first search term among the search terms contained in each sample within the sorting data range of the search terms of the user category corresponding to each sample; determine the second search term other than the first search term among the search terms contained in each sample; and sort the search terms of each sample in order from the first search term to the second search term to obtain preprocessing data of the sample data.

[0137] Optionally, the training module 304 is also used to: intercept each data in the preprocessed data of the sample data according to the first preset feature sequence length to obtain target sample data; determine the data of the target user category in the target sample data as positive sample data, and determine the data of categories other than the target category as negative sample data; train the retrieval enhancement model based on the positive sample data and the negative sample data to obtain a target user recommendation model.

[0138] Optionally, the device also includes: an acquisition module 301, which is also used to acquire the user data to be recommended, and the user data to be recommended includes the search terms and corresponding frequencies of the users to be recommended; a sorting module 305, which is used to sort the search terms of the users to be recommended based on the frequencies corresponding to the search terms of the users to be recommended, and obtain sorting data; a interception module 306, which is used to intercept the sorting data according to a preset second preset feature sequence length, and obtain pre-processed data of the users to be recommended; an input module 307, which is used to input the pre-processed data of the users to be recommended into the target user recommendation model, and obtain target recommended user data.

[0139] Optionally, the retrieval enhanced language model adopts a dual-tower structure and includes a global information processing unit and a local information processing unit. The global information processing unit and the local information processing unit are simplified to modify the exponential calculation method into a linear calculation method.

[0140] Optionally, the implementation process of changing the exponential calculation method to a linear calculation method is as follows:

[0141] Deleting the activation function unit and the initialization parameter variable in the global information processing unit and the local information processing unit to obtain a first global information processing simplification unit and a first local information processing simplification unit, wherein the first global information processing simplification unit and the first local information processing simplification unit include calculation items in the form of e exponents;

[0142] Transform the calculation item in the form of e exponential into a general calculation item to obtain a simplified second global information processing simplified unit and a second local information processing simplified unit;

[0143] Taylor expansion is performed on the second global information processing simplification unit and the second local information processing simplification unit, and variables in the expansion results are normalized to convert the exponential form calculation method into a linear form calculation method.

[0144] According to the training device of the user recommendation model provided by the embodiment of the present invention, the data for model training can be preprocessed, including sorting the search terms of each sample according to the sorting data of the search terms corresponding to the user category, and intercepting according to the preset length to obtain a feature sequence of a certain length, which can retain important features while greatly reducing the feature length, reducing the time consumed during model training, and improving the accuracy of the model. In addition, by improving the activation function of the model, the exponential function is improved to a linear function, which further accelerates the training and reasoning speed of the model.

[0145] Figure 4 An exemplary system architecture 400 is shown to which the method for training a user recommendation model or the apparatus for training a user recommendation model according to an embodiment of the present invention can be applied.

[0146] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0147] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0148] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0149] Server 405 may be a server that provides various services, such as a backend management server (only an example) that provides support for shopping websites browsed by users using terminal devices 401, 402, and 403. The backend management server may analyze and process the received training request and other data, and feed back the processing results (such as a user recommendation model - only an example) to the terminal device.

[0150] It should be noted that the training method of the user recommendation model provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the training device of the user recommendation model is generally set in the server 405 .

[0151] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0152] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device or a server suitable for implementing an embodiment of the present invention. Figure 5 The terminal device or server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0153] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0154] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0155] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0156] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0157] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0158] The units or modules involved in the embodiments of the present invention may be implemented in software or hardware. The units or modules described may also be provided in a processor. For example, they may be described as follows: a processor includes an acquisition module, a determination module, a processing module, and a training module. The names of these modules do not, in some cases, constitute limitations on the modules themselves. For example, the acquisition module may also be described as a "module for acquiring data of a target number of samples and full search term data."

[0159] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0160] Obtain a target number of sample data and full search term data, where each sample has a user category;

[0161] Determine the ranking data of the search terms for each user category based on the full amount of search term data;

[0162] Based on the ranking data of the search terms of each user category and the user category corresponding to each sample, the search terms contained in each sample in the target number of sample data are ranked to obtain preprocessed data of the sample data;

[0163] The retrieval enhanced language model is trained based on the preprocessed data of the sample data to obtain the target user recommendation model.

[0164] According to the technical solution of the embodiment of the present invention, the acquired search terms can be classified and sorted according to user categories, and each sample data can be sorted according to the user category to obtain preprocessed data. Furthermore, a target user recommendation model is obtained by training a retrieval enhanced language model based on the preprocessed data, thereby improving the accuracy of the model output results. By searching for data with lower frequency of search terms, the amount of input data during model training can be reduced, the feature length can be reduced, the speed of model training and reasoning can be greatly improved, and the efficiency of model training can be improved.

[0165] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the above-mentioned training method of the user recommendation model.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for training a user recommendation model, characterized in that: include: Obtain a target number of sample data and full search term data, where each sample has a user category; Determine the ranking data of the search terms of each user category based on the full amount of search term data; Based on the ranking data of the search terms of each user category and the user category corresponding to each sample, the search terms contained in each sample in the target number of sample data are ranked to obtain pre-processed data of the sample data; The retrieval enhanced language model is trained based on the pre-processed data of the sample data to obtain a target user recommendation model.

2. The method according to claim 1, characterized in that The determining the ranking data of the search terms of each user category based on the full amount of search term data includes: Determining the frequency of each search term based on the full amount of search term data; Calculating probability distribution data of the search terms based on the frequency of each search term; Ranking data of the search terms for each user category is determined based on the probability distribution data.

3. The method according to claim 2, characterized in that The determining the ranking data of the search terms of each user category based on the probability distribution data includes: Determining the frequency of search terms that meet a preset probability distribution condition based on the probability distribution data; The search terms of each user category are classified and sorted in order from high to low frequency of the search terms that meet the preset probability distribution condition, so as to obtain sorting data of the search terms of each user category.

4. The method according to claim 2 or 3, characterized in that: The step of sorting the search terms contained in each sample in the target number of sample data based on the sorted data of the search terms of each user category and the user category corresponding to each sample to obtain preprocessed data of the sample data includes: Determine a first search term in the search terms included in each sample within a ranking data range of search terms of the user category corresponding to each sample; Determining a second search term other than the first search term among the search terms included in each sample; The search terms of each sample are sorted in order from the first search term to the second search term to obtain preprocessed data of the sample data.

5. The method according to claim 1, characterized in that The pre-processed data based on the sample data is used to train the retrieval enhanced language model to obtain a target user recommendation model, including: According to the first preset characteristic sequence length, each piece of data in the preprocessed data of the sample data is intercepted to obtain target sample data; Determine the data of the target user category in the target sample data as positive sample data, and determine the data of categories other than the target category as negative sample data; A retrieval enhancement model is trained based on the positive sample data and the negative sample data to obtain a target user recommendation model.

6. The method according to claim 1, characterized in that The method further comprises: Acquire user data to be recommended, wherein the user data to be recommended includes search terms and corresponding frequencies of the user to be recommended; sorting the search terms of the to-be-recommended users based on the frequencies corresponding to the search terms of the to-be-recommended users to obtain sorting data; The sorted data is intercepted according to a preset second preset feature sequence length to obtain pre-processed data of the user to be recommended; The pre-processed data of the user to be recommended is input into the target user recommendation model to obtain target recommended user data.

7. The method according to claim 1, characterized in that The retrieval enhanced language model adopts a double-tower structure and includes a global information processing unit and a local information processing unit. The global information processing unit and the local information processing unit are simplified to modify the exponential calculation method into a linear calculation method.

8. The method according to claim 7, characterized in that The implementation process of modifying the exponential calculation method to a linear calculation method is as follows: Deleting the activation function unit and the initialization parameter variable in the global information processing unit and the local information processing unit to obtain a first global information processing simplification unit and a first local information processing simplification unit, wherein the first global information processing simplification unit and the first local information processing simplification unit include calculation items in the form of e exponents; Transforming the calculation item in the form of e exponential into a general calculation item to obtain a simplified second global information processing simplified unit and a second local information processing simplified unit; Taylor expansion is performed on the second global information processing simplification unit and the second local information processing simplification unit, and variables in the expansion results are normalized to convert the exponential form calculation method into a linear form calculation method.

9. A training device for a user recommendation model, characterized in that: include: An acquisition module is used to acquire data of a target number of samples and full search term data, wherein each sample has a user category; A determination module, configured to determine ranking data of search terms of each user category based on the full amount of search term data; A processing module, configured to perform sorting processing on the search terms contained in each sample of the target number of sample data based on the sorting data of the search terms of each user category and the user category corresponding to each sample, so as to obtain preprocessing data of the sample data; The training module is used to train the retrieval enhanced language model based on the pre-processed data of the sample data to obtain a target user recommendation model.

10. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 8.

11. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images