Item recommendation method, device, equipment and storage medium

Abstract

The embodiment of the present application discloses an item recommendation method, device, equipment and storage medium, which relates to the field of data processing technology, and includes: obtaining user features of the current user; inputting the user features into a first sub-neural network model to obtain a first feature vector of the current user, the first sub-neural network model is obtained through knowledge distillation training; according to the first feature vector and the second feature vectors of multiple candidate items, the item recommendation result of the current user is determined from multiple candidate items, each candidate item corresponds to a pre-saved second feature vector, the second feature vector is obtained by processing the item features of the candidate item by the second sub-neural network model, the second sub-neural network model is obtained through knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model. The above method can solve the technical problem that a simple neural network model in the related art cannot achieve high-performance item recommendation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to an item recommendation method, device, equipment and storage medium. Background Art

[0002] At present, with the development of Internet products, providing personalized services to users to improve their product satisfaction has become one of the key tasks of various Internet products. Among them, recommendation services, as a means of personalized services, are widely used in Internet products. For item recommendation scenarios, recommendation services can be implemented manually, that is, the backend service personnel select the items recommended to the user, or through intelligent methods such as machine learning, for example, the backend server predicts the items recommended to the user using machine learning. In some related technologies, neural networks, as a means of implementing machine learning, are often used in the recommendation services of Internet products. Generally speaking, the greater the depth of the neural network model and the more complex the structure, the better the performance of the neural network model, and the closer the recommended items finally predicted are to the needs of the user. However, when deploying complex neural network models in the backend server, there are high requirements for deployment resources, and the complex neural network model runs slowly, which is not conducive to the rapid prediction of recommended items. In contrast, when using a neural network model with low depth and simple structure to predict recommended items, although it is conducive to the deployment of the backend server and has a faster running speed, it will reduce the performance of the neural network model, so that the recommended items finally predicted do not meet the needs of the user, reducing the user's experience.

[0003] Therefore, in recommendation services, how to use simple neural network models to achieve high-performance item recommendations has become a technical problem that needs to be solved urgently. Summary of the invention

[0004] The embodiments of the present application provide an item recommendation method, apparatus, device and storage medium to solve the technical problem in the related art that a simple neural network model cannot achieve high-performance item recommendation.

[0005] In a first aspect, an embodiment of the present application provides an item recommendation method, comprising:

[0006] Get the user characteristics of the current user;

[0007] Inputting the user feature into a first sub-neural network model to obtain a first feature vector of the current user, wherein the first sub-neural network model is obtained through knowledge distillation training;

[0008] According to the first feature vector and the second feature vectors of multiple candidate items, the item recommendation result of the current user is determined from the multiple candidate items, each of the candidate items corresponds to a pre-saved second feature vector, the second feature vector is obtained by processing the item features of the candidate items by a second sub-neural network model, the second sub-neural network model is obtained by knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model.

[0009] In a second aspect, the present application also provides an item recommendation device, including:

[0010] A feature acquisition module is used to acquire user features of the current user;

[0011] A vector determination module, used for inputting the user feature into a first sub-neural network model to obtain a first feature vector of the current user, wherein the first sub-neural network model is obtained through knowledge distillation training;

[0012] A result determination module is used to determine the item recommendation result of the current user from multiple candidate items based on the first feature vector and the second feature vectors of the multiple candidate items, each of the candidate items corresponds to a pre-saved second feature vector, the second feature vector is obtained by processing the item features of the candidate items by a second sub-neural network model, the second sub-neural network model is obtained by knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model.

[0013] In a third aspect, the present application also provides an item recommendation device, including:

[0014] one or more processors;

[0015] A memory for storing one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the item recommendation method as described in the first aspect.

[0017] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the item recommendation method as described in the first aspect.

[0018] The above-mentioned method, device, equipment and storage medium for recommending items determine the first feature vector of the current user through the first sub-neural network model, and then determine the item recommendation result according to the first feature vector and the second feature vector of each candidate item, and the second feature vector is obtained by pre-processing the second sub-neural network model, and the second sub-neural network model and the first sub-neural network model are obtained by knowledge distillation training and correspond to the same teacher model. The technical means solves the technical problem that a simple neural network model in the related art cannot achieve high-performance item recommendation. By using the technical means of knowledge distillation, a complex neural network model can be used as a teacher model to learn a large number of features, and then the "knowledge" of the teacher model can be transferred to a simple student model (i.e., the first sub-neural network model and the second sub-neural network model), which ensures that the simple student model has high performance, and by using the substantially unchanged characteristics of the second feature vector, the second feature vector is directly saved and reused after being obtained, without the need to deploy the second sub-neural network model online, and only the first sub-neural network model needs to be deployed online, which further simplifies the complexity of the online deployment of the neural network model, and still ensures the accuracy of item recommendation, thereby improving the monetization ability of item recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of an item recommendation method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a recommendation system provided in an embodiment of the present application;

[0021] Figure 3 An example framework diagram of a recommendation system provided in an embodiment of the present application;

[0022] Figure 4 A flowchart of another item recommendation method provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a model distillation provided in an embodiment of the present application;

[0024] Figure 6 A characteristic distillation schematic diagram provided for an embodiment of the present application;

[0025] Figure 7 A schematic diagram of the combination of model distillation and feature distillation provided in the embodiments of the present application;

[0026] Figure 8 A schematic diagram of a neural network model in the training phase provided in an embodiment of the present application;

[0027] Fig. 9 A schematic diagram of a neural network model in the application phase provided in an embodiment of the present application;

[0028] Fig.10 A schematic diagram of the structure of an item recommendation device provided in an embodiment of the present application;

[0029] Fig.11 A schematic diagram of the structure of an item recommendation device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0031] For the neural network model, it includes a training process and an application process. The training process is specifically to enable the neural network model to learn the pre-collected features so that the neural network model can output the expected results, that is, the training process is the process of enabling the neural network model to have learning capabilities. The application process refers to deploying the trained neural network model online, that is, deploying it in the background server, so that the neural network model provides online services (the online service in the embodiment is a recommendation service).

[0032] It is understandable that features play a vital role in the prediction task of the neural network model. The more features the neural network model learns, the better the performance of the neural network model, and the closer the recommended items predicted by it are to the needs of users. However, for most Internet products, there are often problems with sparse features, such as a small number of users who have access to items and a small number of items displayed to users. At this time, sparse features will seriously affect the performance of the neural network model, so that the recommended items finally obtained do not meet the needs of users, reducing the user experience. In some related technologies, in order to solve the problem that the performance of the neural network model cannot be guaranteed in the scenario of sparse features, the data of multiple columns are usually fused during the training process to train a unified neural network model, and then the items displayed in each column are predicted by the neural network model. Among them, the column refers to the area where the items are displayed, and each column displays an item. The data of the column includes the features corresponding to the items displayed in the column and the features corresponding to the user, wherein the features corresponding to the item may include the name and category of the item, etc., and the features corresponding to the user may include the basic information of the user (such as user ID, etc.). However, since different columns correspond to different features, after fusing the column features, the neural network model cannot learn the true distribution of the features corresponding to each column, which still reduces the performance of the neural network model.

[0033] In addition, in order to ensure the consistency of the neural network model in the training process and the application process or to meet the efficiency requirements of the application process (even if the neural network model has a faster running speed), some features obtained in the training process are usually ignored during online services. At this time, the ignored features can be called dominant features, and correspondingly, the features obtained in both the training process and the application process can be called common features. That is, the dominant features are applied in the training process, and the common features are applied in the training process and the application process. Further, after ignoring the dominant features, the neural network model in the application process only obtains the common features, which will lead to the performance degradation of the online service. Then, in order to ensure the performance of the neural network model in the application process, a multi-task learning method can be adopted, that is, a dominant feature is obtained separately through a neural network model to distinguish it from the neural network model in the training process. However, in this case, each task may not be able to meet the harmless guarantee, that is, the neural network model that obtains the dominant feature may damage the neural network model in the training process. In addition, the more dominant features are used in a training process, the more workload is required to adjust the weights of each task according to the dominant features.

[0034] Therefore, an embodiment of the present application provides an item recommendation method to implement high-performance item recommendation using a simple neural network model, and can ensure the performance of the neural network model when features are sparse, and ensure the performance of the neural network model during the application process when the neural network model is trained using advantageous features.

[0035] In the item recommendation method provided in the embodiment of the present application, a neural network model is used to determine the items recommended to the user. Among them, the neural network model used in the application process is obtained by knowledge distillation training. It can be understood that knowledge distillation is a method of model compression (reducing the number of model parameters while ensuring model performance). Under knowledge distillation, a complex neural network model and a simple neural network model are pre-constructed. After that, the complex neural network model is first trained in the training process so that the complex neural network model can extract the required content from a large number of highly redundant feature sets. After that, the "knowledge" learned by the complex neural network model is transferred to the simple neural network model, that is, the simple neural network model learns the "knowledge" in the complex neural network model so that the simple neural network model also has the performance of the complex neural network model. It can be understood that "knowledge" can be understood as the learning ability of the neural network model. After that, only the simple neural network model needs to be deployed in the application process. Among them, the complex neural network model is the teacher model in knowledge distillation, and the simple neural network model is the student model in knowledge distillation. The teacher model uses a large number of features during the training process, and in order to make up for the lack of sparse features, the feature content of the teacher model is enriched during the training process. In one embodiment, the teacher model training process uses both common features and dominant features, wherein common features refer to user features and item features, and dominant features refer to interactive features between users and items. Interactive features can be understood as interactive behaviors between users and items, such as the behavior of users clicking on items in a column, etc. It can be understood that the interactive features between users and items are constantly changing, and with the increase in usage time, the number of users and items, the interactive features will gradually increase. Increasing the interactive features can improve the accuracy of the teacher model. However, as the interactive features gradually increase, the delay of the student model in the application process will increase. Therefore, the interactive features are determined as dominant features. For the training process of the student model, only common features are used, and interactive features are not used to avoid increasing the delay of the student model. Since the teacher model uses advantageous features, when the "knowledge" of the teacher model is transferred to the student model, although the student model has a simple structure, it still has the ability to learn advantageous features, so it has better performance. In addition, since knowledge distillation does not use multi-task learning, the teacher model will not violate the harmlessness guarantee when learning advantageous features.

[0036] Exemplarily, the item recommendation method provided in the embodiment of the present application can be executed by an item recommendation device, which can be implemented by software and / or hardware. The item recommendation device can be composed of two or more physical entities or one physical entity.

[0037] In the embodiment, the item recommendation device is described as a background server as an example. The background server is used to provide background services for users of the client, wherein the client refers to an application for local use by the user, and the type of the application can be set according to actual conditions. If the application is an educational application, the user can learn through the educational application. At this time, the services that the background server can provide include but are not limited to course search services, course recommendation services, course learning services, etc. It should be noted that the embodiment of the physical entity type where the client is located is not limited. For example, the physical entity where the client is located is a mobile phone, a tablet computer, or a smart interactive tablet.

[0038] For example, Figure 1 A flowchart of an item recommendation method provided in an embodiment of the present application. Figure 1 , the item recommendation method specifically includes:

[0039] Step 110: Obtain user characteristics of the current user.

[0040] Among them, the current user refers to the user who currently needs to recommend items. User characteristics include basic information of the user, wherein the content of the basic information can be set according to the actual situation. For example, if the client is an educational application, then the basic information of the user may include user ID, grade and region of the user, etc. Optionally, basic information is information that the user authorizes the backend server to obtain. The backend server cannot obtain information that the user has not authorized. In one embodiment, user characteristics may also include user behavior. User behavior refers to the behavior of the user in the client. If the client is an educational application, user behavior may include learning behavior, browsing behavior, etc. Exemplarily, when the current user uses the client, the client obtains the user characteristics of the current user and reports the user characteristics to the backend server so that the backend server obtains the user's behavioral characteristics.

[0041] Step 120: Input the user features into the first sub-neural network model to obtain a first feature vector of the current user. The first sub-neural network model is obtained through knowledge distillation training.

[0042] In an embodiment, the first sub-neural network model is a deep neural network model, which is used to extract the first feature vector of the current user based on the user features. The first feature vector can be considered as the feature vector extracted after the first sub-neural network model learns the user features. At this time, the user features can be considered as the data input into the first sub-neural network model, and the first feature vector can be considered as the result of the first sub-neural network model extracting the data. The specific structure of the first sub-neural network model can be set according to actual conditions. In one embodiment, the first sub-neural network model is obtained through knowledge distillation training, that is, the first sub-neural network model belongs to the student model in the knowledge distillation and has a relatively simple structure. However, because it has learned the "knowledge" of the teacher model, it has better performance. The first sub-neural network model is deployed in the background server.

[0043] Step 130: Determine the item recommendation result for the current user from the multiple candidate items based on the first feature vector and the second feature vectors of the multiple candidate items, each candidate item corresponds to a pre-saved second feature vector, the second feature vector is obtained by processing the item features of the candidate item by the second sub-neural network model, the second sub-neural network model is obtained by knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model.

[0044] Exemplarily, the specific type of the item is not limited in the embodiment. For example, when the client is an educational application, the item is a course for the user to learn. In the embodiment, each item appearing in the client has a corresponding second feature vector, and the second feature vector is a vectorized representation of the item feature. The second feature vector can be obtained by processing the corresponding item feature by the second sub-neural network model. Among them, the item feature refers to the information of the item, and the content of the item information can be set according to the actual situation. For example, if the item is a course, then the item information can include at least one content such as the course name, knowledge point, subject, grade, etc. After the item feature is input into the second sub-neural network model, the second feature vector of the item can be obtained. The second sub-neural network model is a deep neural network model, and the specific structure of the second sub-neural network model can be set according to the actual situation. In one embodiment, the second sub-neural network model is obtained by knowledge distillation training, that is, the second sub-neural network model belongs to the student model in knowledge distillation, and has a relatively simple structure, but because it has learned the "knowledge" of the teacher model, it has better performance.

[0045] In one embodiment, since the first feature vector and the second feature vector are important parameters for determining the result of item recommendation, that is, the recommended item is selected from the candidate items in combination with the user characteristics and the item characteristics to meet the user's needs, therefore, the first sub-neural network model and the second sub-neural network model correspond to the same teacher model, that is, the two sub-neural network models learn "knowledge" from the same teacher model to ensure the accuracy of the first feature vector and the second feature vector, thereby ensuring the accuracy of item recommendation. In one embodiment, the student model adopts a double-tower structure, that is, the student model includes the first sub-neural network model and the second sub-neural network model, wherein the first sub-neural network model is used to extract the first feature vector, and the second sub-neural network model is used to extract the second feature vector, and then the function used in the student model combines the two to obtain the recommended item prediction result of the student model. Since only the user's feature vector and the item's feature vector are used in the application stage. Therefore, the first sub-neural network model and the second sub-neural network model can be separated from the student model and deployed separately. In one embodiment, since the item characteristics generally do not change, for example, when the item is a course, its course name, knowledge points, subjects, grades, etc. will not change. At this time, the second feature vector obtained by the second sub-neural network model generally does not change. Therefore, in the embodiment, after obtaining the second feature vector of each item through the second sub-neural network model in the training phase, the second feature vector can be associated with the corresponding item and saved, so that the second feature vector of the item can be directly obtained during the application process, reducing the data processing amount of the backend server. When a new item appears, the second feature vector of the new item can be calculated offline by the second sub-neural network model and saved for subsequent use. Correspondingly, only the first sub-neural network model can be deployed when the student model is deployed on the backend server.

[0046] Exemplarily, multiple items that may be liked by the user are selected in advance from the item library. In an embodiment, the selected items are recorded as candidate items, wherein the selection method of the candidate items is set according to the actual situation, such as selecting multiple similar items as candidate items from the item library based on the items selected by the user in the past, or selecting multiple items that meet the user's characteristics from the item library as candidate items based on the user's characteristics. In one embodiment, multiple items are selected by recall. At this time, before step 130, it also includes: recalling multiple candidate items from at least one item library. Generally speaking, the recommendation system commonly used in the industry includes two cascade processes: recall and sorting, wherein the recall link is to quickly screen out items that some users may be interested in from a massive item library, that is, to obtain candidate items, and the sorting link is to determine the items recommended to the user from the items that some users may be interested in, that is, step 130 belongs to the sorting link. For example, Figure 2 A schematic diagram of a recommendation system provided in an embodiment of the present application, referring to Figure 2, the recommendation system includes a recall link and a sorting link. Specifically, for the recall link, a large number of items are first selected from at least one item library, wherein the number of selected items is generally in the thousands, and then a rough selection is made among the thousands of items to select a small number of items, and the small number of items are merged and sent to the sorting link. At this time, the number of items sent to the sorting link is ten levels, that is, the number of items is reduced from the thousand level to the ten level in the recall stage to reduce the amount of data processing in the sorting link. It can be understood that the specific implementation process of the recall link is not limited to the embodiment, such as being implemented by machine learning. Generally speaking, the recall process also refers to user characteristics (which may include user behavior) and item characteristics, that is, ten levels of candidate items are screened out from the thousands of items through user characteristics and item characteristics.

[0047] Further, after obtaining the candidate items, the second feature vectors pre-saved for each candidate item are obtained. Afterwards, the items recommended to the current user are determined according to the first feature vector and the second feature vector, that is, the item recommendation result is determined. At this time, the sorting link also uses user characteristics and behavioral characteristics. Exemplarily, the item recommendation result includes at least one item recommended to the current user, which is pushed by the background server to the client used by the current user and displayed in the client, wherein the display rule embodiment is not limited, such as displaying the column in the form of a pop-up window in the client, and displaying the recommended items in the column, wherein one item corresponds to one column. In one embodiment, a recommendation algorithm with a click-through rate as the target is used to predict the probability of each item being clicked by the current user, that is, to predict the click-through rate of each item, and then determine the items recommended to the current user according to the click-through rate. Among them, the method of calculating the click-through rate can be selected according to the actual situation. In one embodiment, in order to avoid delays in the calculation process, the inner product model is used to calculate the click-through rate, wherein the inner product model is used to calculate the inner product between the second feature vector of each item and the first feature vector of the current user, so as to reflect the click-through rate of each item through the inner product, and the inner product model is deployed in the sorting link. At this time, step 130 may specifically include steps 131 and 132:

[0048] Step 131: Calculate the inner product between the first feature vector and the second feature vector of each candidate item, with each candidate item corresponding to one inner product.

[0049] Exemplarily, the inner product can also be recorded as the dot product of two vectors. In the embodiment, the inner product of the first feature vector and each second feature vector is calculated to obtain multiple inner products, each inner product corresponds to a second feature vector, and each inner product can represent the probability that the item corresponding to the current second feature vector is clicked by the current user.

[0050] In one embodiment, the calculation model of the inner product is:

[0051]

[0052] Among them, X u represents the user characteristics of the current user, X i represents the item features of the candidate items, W u represents the model parameters of the first sub-neural network model, W i represents the model parameters of the second sub-neural network model, Φ W (·) represents a nonlinear mapping with model parameters, represents the first feature vector of the current user, The second eigenvector representing the candidate items, express and The inner product of each second eigenvector and the first eigenvector can be calculated by the above calculation model. It can be understood that the above calculation model can also be understood as a function used by the inner product model. Among them, the second eigenvector can be used directly after being calculated in the training process.

[0053] Step 132: Determine the item recommendation result for the current user according to the inner product corresponding to each candidate item.

[0054] Exemplarily, the click-through rate of an item can be evaluated by the inner product corresponding to each item, and then the item with a high click-through rate can be selected as the item recommendation result. In one embodiment, when the item recommendation result includes one item, the item with the highest click-through rate can be selected as the item recommendation result and displayed in the corresponding column. If multiple columns are displayed in the client, each column can determine the corresponding item recommendation result according to the item recommendation method provided in this embodiment. At this time, the items displayed in each column can be distinguished by category (such as classification by subject or knowledge point), that is, each column displays items of different categories. In another embodiment, when the item recommendation result includes two or more items, the two or more items with the highest click-through rate can be selected as the item recommendation result, and each item corresponds to a column for display in the client.

[0055] Optionally, the item recommendation time is pre-set, and only when the item recommendation time is met, the item recommendation method of this embodiment is executed to recommend items to the current user. The item recommendation time can be set according to actual conditions. For example, Tuesday and Thursday are set as the item recommendation time. At this time, if the current user uses the client on Tuesday or Thursday, the background server will recommend items. Optionally, the item recommendation cycle is pre-set, and when the item recommendation cycle is met, the item recommendation method provided in this embodiment is executed to recommend items to the current user. The item recommendation cycle can be set according to actual conditions. For example, if the item recommendation cycle is every three days, the background server will recommend items every three days.

[0056] The following is an exemplary description of the item recommendation method provided in the embodiment of the present application. In this example, an exemplary description is given using items as courses. Figure 3 This is an example framework diagram of a recommendation system provided in an embodiment of the present application, which is a recommendation system used by the item recommendation method, refer to Figure 3 ,The recommendation system includes a recall phase and a sorting phase.

[0057] Exemplarily, the item recommendation time is set to Tuesday and Thursday, and the same user can be recommended a course at most twice through a pop-up window on the same day. If the user uses the client during the item recommendation time, the client sends a user request to the backend server, which includes user features and an item recommendation request. After receiving the user request, the backend server obtains thousands of courses from the course library through the recall link and selects ten alternative courses from them and sends them to the sorting link. Then, the sorting link obtains user features, obtains the first feature vector of the current user through the first sub-neural network model, and obtains each second feature vector corresponding to the alternative course from the second feature vectors corresponding to each pre-stored course. Then, the inner product of each second feature vector and the first feature vector is calculated to estimate the click-through rate of each course through the inner product. Then, the course with the highest click-through rate is selected as the item recommendation result and returned to the client to be displayed to the user in the client.

[0058] In the above, the first feature vector of the current user is determined by the first sub-neural network model, and then the item recommendation result is determined according to the first feature vector and the second feature vector of each candidate item, and the second feature vector is obtained by pre-processing the second sub-neural network model, and the second sub-neural network model and the first sub-neural network model are obtained by knowledge distillation training and correspond to the same teacher model. The technical means solves the technical problem that a simple neural network model in the related art cannot achieve high-performance item recommendation. By using the technical means of knowledge distillation, a complex neural network model can be used as a teacher model to learn a large number of features, and then the "knowledge" of the teacher model can be transferred to a simple student model (i.e., the first sub-neural network model and the second sub-neural network model), which ensures that the simple student model has high performance, and by using the substantially unchanged characteristics of the second feature vector, the second feature vector is directly saved and reused after it is obtained, without the need to deploy the second sub-neural network model online, and only the first sub-neural network model needs to be deployed online, which further simplifies the complexity of the online deployment of the neural network model, and still ensures the accuracy of item recommendation, thereby improving the monetization ability of item recommendation.

[0059] Figure 4 A flowchart of another method for recommending items provided in an embodiment of the present application. This embodiment describes the training process of the neural network model based on the above embodiment. In this embodiment, knowledge distillation is used to train the neural network model. It should be noted that the training process can be performed in the background server or in other devices. When performed in other devices, the neural network model can be deployed in the background server after the training is completed.

[0060] In the embodiment, knowledge distillation can be used to implement model distillation and feature distillation. In the model distillation, both the teacher model and the student model adopt deep neural network models, and the volume of the teacher model is larger than that of the student model. For the deep neural network model, the volume of the model can be reflected by the depth, width and number of models. The more complex the model, the larger the volume of the model. For example, Figure 5 This is a schematic diagram of the model distillation provided in the embodiments of this application. Figure 5 , the model depth of the teacher model is 5, and the model depth of the student model is 3. At this time, the depth of the teacher model is greater than the depth of the student model. Furthermore, both the teacher model and the student model are trained with common features, and when the student model is trained with common features, the "knowledge" of the teacher model is also transferred to the student model, so that the student model with a simple structure has the performance of the teacher model. At this time, for the student model, its objective function is:

[0061]

[0062] Wherein, X represents common features, which are user features and item features in the embodiment, and f s represents the function used by the student model, f s (X; W s ) represents the output of the student model, W s represents the model parameters of the student model, y represents the true result corresponding to the output result, and L s (y,f s (X; W s )) represents the loss function between the output result of the student model and the true result, that is, the student model loss. This loss function can reflect the difference between the output result of the student model and the true result. t represents the function used by the teacher model, f t (X; W t ) represents the output of the student model, W t represents the model parameters of the teacher model, L d (f t (X; W t ),f s (X; W s )) represents the loss function of the output results of the teacher model and the student model, namely the distillation loss. This loss function can reflect the difference between the output results of the teacher model and the output results of the student model. The output results of the teacher model can be considered as the soft labels referenced during the training process of the student model, and the real results can be considered as the hard labels referenced during the training process of the student model. For example, when the student model predicts the recommended items, the output results of the student model are specifically the item recommendation prediction results of the student model. The item recommendation prediction results refer to the prediction results of the student model on whether each item is recommended to the user. The soft labels refer to the prediction results of the teacher model on whether each item is recommended to the user. The soft labels can be considered as the item recommendation prediction results output by the teacher model, and the hard labels are the real results of whether each item is recommended to the user. λ represents a hyperparameter, and its specific value can be set according to the actual situation. The two loss functions can be balanced through the hyperparameter so that the student model outputs the expected results.

[0063] Exemplarily, the training process of model distillation is described by taking the case that both the teacher model and the student model are used for item recommendation. Specifically, the teacher model is trained first, and then the student model is trained using the above objective function. The training process of the teacher model is specifically as follows: a set of common features (such as user features and item features in the embodiment) is input into the teacher model to obtain the output result of the teacher model, which is specifically the item recommendation prediction result obtained by the user features and the item features, and then the output result and the corresponding true result are substituted into the loss function of the teacher model to adjust the model parameters of the teacher model according to the loss function, wherein the true result is the true result of item recommendation, and the true result of item recommendation is used to describe whether the corresponding item is recommended to the corresponding user. After that, another set of common features is input into the teacher model, and the model parameters of the teacher model are adjusted again according to the loss function, and the above process is repeated until the loss function converges (that is, the loss function of the consecutive times is within the set range), then the teacher model is considered to be stable, and the accuracy of its output result meets expectations. Further, after the teacher model training is completed, the student model is trained. The training process of the student model is similar to that of the teacher model. However, the soft labels output by the teacher model are also used in the training process of the student model so that the student model can learn the "knowledge" of the teacher model through the soft labels. At this time, the loss function of the student model is the above-mentioned objective function, so as to adjust the model parameters of the student model through the loss function, thereby converging the objective function. Furthermore, it can be seen from the above-mentioned objective function that during the training process of the student model, not only the loss function between its own output results and the true results is considered, but also the loss function between its own output results and the soft labels is considered, and the two loss functions are balanced through hyperparameters to improve the performance of the student model. It should be noted that since the teacher model has been trained, the model parameters of the teacher model are fixed in the objective function.

[0064] Feature distillation is used to enable the student model to have the ability to learn advantageous features. In feature distillation, both the teacher model and the student model are deep neural network models, and the teacher model and the student model can have the same network structure. For example, Figure 6 This is a characteristic distillation schematic diagram provided in the embodiment of this application. Figure 6 , the network structure of the teacher model and the student model is the same. During the training process, the teacher model is trained using the dominant features, and the student model is trained using the common features. When the student model is trained, the "knowledge" of the teacher model is transferred to the student model so that the trained student model can obtain the information of the dominant features, that is, the learning method using privileged information (LUPI) is used to improve the performance of the student model. At this time, for the student model, its objective function is:

[0065]

[0066] Wherein, X represents common features, which are user features and item features in the embodiment. * represents the dominant feature, which is the interaction feature between the user and the item in the embodiment, f represents the function used by the student model and the teacher model, and in the embodiment, the teacher model and the student model use the same function, f(X; W s ) represents the output of the student model, W s represents the model parameters of the student model, y represents the true result corresponding to the output result, and L s (y,f(X;W s )) represents the loss function between the output result of the student model and the true result, that is, the student model loss. This loss function can reflect the difference between the output result and the true result. t represents the model parameters of the teacher model, f(X * ; W t ) represents the output of the teacher model, L d (f(X * ; W t ),f(X;W s )) represents the loss function of the output results of the teacher model and the student model, namely the distillation loss. This loss function can reflect the difference between the output results of the teacher model and the output results of the student model. λ represents a hyperparameter, and its specific value can be set according to the actual situation. The hyperparameter can balance the two loss functions so that the output of the student model meets the expected results. The difference between the objective function of feature distillation and the objective function of model distillation is that in feature distillation, the teacher model and the student model have the same structure and use the same function, and the teacher model is trained using the dominant features. At the same time, the student model loss function refers to the output results of the teacher model so that the student model can obtain information about the dominant features. It should be noted that the training process of the teacher model and the student model is similar to that of the teacher model and the student model during model distillation, and will not be repeated here.

[0067] It can be seen that through model distillation, a student model with a simple structure can have the performance of a teacher model with a complex structure, and through feature distillation, a student model can have the performance of learning advantageous features. Therefore, in the embodiment, a combination of model distillation and feature distillation is used to train the neural network model used in the item recommendation method. It can be understood that both the teacher model and the student model are used for item recommendation. For example, Figure 7 This is a schematic diagram of the combination of model distillation and feature distillation provided in the embodiment of this application. Figure 7, the depth of the teacher model is greater than the depth of the student model. When training the teacher model, both dominant features and common features are used. When training the student model, common features are used so that the student model can have the performance of the teacher model at scale and can obtain information about dominant features. In one embodiment, in order to avoid the impact of feature sparsity on the performance of the teacher model and the student model, when training the teacher model, not only dominant features are added, but also data from other fields are added, that is, item features, user features, and interaction features from multiple fields are used to train the teacher model. Other fields can be fields that display items in scenarios other than the item recommendation scenario in the client. At this time, refer to Figure 4 When training is performed in combination with model distillation and feature distillation, the item recommendation method provided in this embodiment specifically includes:

[0068] Step 210: Obtain a first training feature set, and train a teacher model for item recommendation using the first training feature set.

[0069] The first training feature set refers to the set of features used when training the teacher model, and the first training feature set is pre-collected features. The types of features and the number of features included in the first training feature set can be set according to actual conditions. In one embodiment, the first training feature set includes: multiple user features and multiple item features. Optionally, user features of currently registered users are collected as user features in the first training feature set, or user features of active users (such as users who have logged in to the client within one year) are collected as user features in the first training feature set, or user features are obtained by other means. Optionally, item features of each item in the item library are collected as item features in the first training feature set.

[0070] In one embodiment, the first training feature set also includes a plurality of pre-collected interaction features, and the interaction features are interaction features between users and items. Optionally, the interaction features between users and items corresponding to each user feature in the first training feature set are collected as interaction features in the first training feature set. Among them, the interaction features are learned by the teacher model as the dominant features in the training process. The interaction features between users and items are constantly changing, and with the increase of usage time, the number of users and items, the interaction features will gradually increase. Adding interaction features can improve the accuracy of the teacher model. However, due to the gradual increase of interaction features, the delay of the application process will increase. Moreover, if the interaction features are added to the student model, the item recommendation results need to be determined by calculating the nonlinear mapping between users and items in the online deployment stage, and the computational complexity of the nonlinear mapping is higher than the complexity of calculating the inner product. Therefore, the interaction features are determined as dominant features. Optionally, in addition to the above features, the first training feature set may also include the real results of item recommendation between users and items. The real results of item recommendation can be considered as the labels used in the teacher model training, which can be used as a reference for the item recommendation prediction results output by the teacher model. It is understandable that there is a real result of item recommendation between each user and each item. Optionally, the first training feature set may also include item features of other fields, user features using the fields, and interaction features between users and items.

[0071] It is understandable that the training process of the teacher model is similar to the training process of the teacher model during the above-mentioned model distillation and feature distillation, and will not be described in detail here. Optionally, when training the teacher model, a user feature, an item feature, and an interaction feature between the user and the item can be used as a set of training features, and then at least one set of training features is input into the teacher model so that the teacher model simultaneously predicts the item recommendation prediction results corresponding to at least one set of training features, and the item recommendation prediction results simultaneously consider the dominant features and the common features.

[0072] Step 220: obtain a second training feature set, and train a student model for item recommendation based on the second training feature set and the teacher model, wherein the volume of the student model is smaller than the volume of the teacher model, and the student model includes a first sub-neural network model and a second sub-neural network model, and the first training feature set and the second training feature set both include a plurality of pre-collected user features and a plurality of item features.

[0073] The second training feature set refers to the set of features used when training the student model, and the second training feature set is pre-collected features. The types of features and the number of features contained in the second training feature set can be set according to actual conditions. In one embodiment, the first training feature set includes: multiple user features and multiple item features, that is, the second training feature set includes common features in the first training feature set. It can be understood that in addition to the above-mentioned training features, the second training feature set can also include the real results of item recommendations between users and items. The real results of item recommendations are the same as the real results of item recommendations contained in the first training feature set, and can be used as hard labels in the student model training process. At this time, since the first training feature set includes dominant features and common features, and the second training feature set includes common features, feature distillation can be achieved.

[0074] In one embodiment, the student model includes a first sub-neural network model and a second sub-neural network model, wherein the first sub-neural network model and the second sub-neural network model can be the same neural network model or different neural network models. Optionally, the student model is a double-tower structure, that is, the first sub-neural network model and the second sub-neural network model form a double-tower structure. After setting the double-tower structure, the first sub-neural network model and the second sub-neural network model can be independently split out. Typically, the volume of the student model is smaller than the volume of the teacher model, that is, the teacher model is more complex than the student model to achieve model distillation.

[0075] It is understandable that after the teacher model training is completed, the student model can be trained. In one embodiment, the objective function of the student model is:

[0076]

[0077] Among them, X represents the user features and item features input into the student model, W s represents the model parameters of the student model, f s represents the function used by the student model, f s (X; W s ) represents the item recommendation prediction result of the student model, y represents the actual result of item recommendation, and L s (y,f s (X; W s )) represents y and f s (X; W s ), i.e., the student model loss, X1 represents the user features and item features input into the teacher model, represents the interactive features input to the teacher model, W t represents the model parameters of the teacher model, f t represents the function used by the teacher model, represents the item recommendation prediction result of the teacher model, express and f s (X; W s ), i.e., distillation loss, where λ represents a hyperparameter. From the above objective function, we can see that during the student model training process, the soft label output by the teacher model is used, and the soft label can be determined by the dominant features, common features, model parameters of the teacher model, and the function used. Therefore, feature distillation and model distillation can be achieved.

[0078] In one embodiment, the user features, item features, and interaction features input into the teacher model are user features, item features, and interaction features corresponding to multiple columns, and the multiple columns include a column for displaying recommended items and other columns; the user features and item features input into the student model are user features and item features corresponding to the column for displaying recommended items. Exemplarily, the teacher model inputs user features, item features, and interaction features of multiple columns, wherein the multiple columns include a column for displaying recommended items and other columns, and the student model inputs user features, item features, and interaction features of one column, wherein one column refers to a column for displaying recommended items. At this time, when training the teacher model, the input X1 and It can contain multiple common features and multiple recommended features. Later, when training the student model, the input X is included in X1.

[0079] It should be noted that continuous training of the teacher model and the student model will take a long time, but this time consumption is acceptable during the training process.

[0080] After the student model training is completed, the student model can be deployed online on the background server to perform item recommendation prediction through the student model. In one embodiment, after step 220, it also includes: deploying the first sub-neural network model online; saving each second feature vector obtained by the second sub-neural network model, each second feature vector corresponding to an item feature.

[0081] Exemplarily, since the item features of each item remain basically unchanged, the second feature vector obtained through the item features will not change. Accordingly, in the embodiment, after the student model training is completed, the second feature vector of each item can be obtained through the second sub-neural network model, and the second feature vector can be saved to directly obtain the required second feature vector in the application stage. At the same time, the first sub-neural network model is extracted, and the first sub-neural network model is deployed online, so that only the first sub-neural network model is used in the application process. Among them, the specific implementation method of the online deployment is not limited in the embodiment.

[0082] The training process and the application process are described below in an exemplary manner.

[0083] Figure 8 The neural network model diagram for the training phase provided in the embodiment of the present application is shown in FIG. Figure 8 , Act. represents the activation function layer set in the neural network model. It can be understood that the neural network model can also include convolutional layers in addition to the activation function layer. Figure 8 The depth of the neural network model is represented by the number of activation function layers, which is represented by Figure 8 It can be seen that the depth of the teacher model is significantly higher than that of the student model. Afterwards, the training features input into the teacher model include user features, item features, and interaction features corresponding to multiple columns. The user features input into the teacher model are represented as The item features input into the teacher model are represented as The teacher features input into the teacher model are expressed as After the teacher model training is completed, the student model is trained. The training features input into the student model include user features and item features of a column (the column showing recommended items). The user features input into the student model are represented as X U , the item features input into the teacher model are represented as X i The student model is a double-tower structure, and the part that processes user features is the first sub-neural network model, and the part that processes item features is the second sub-neural network model. Among them, the first sub-neural network model can obtain the first feature vector The second sub-neural network model can obtain the second eigenvector Afterwards, the student model constructs an objective function through the student model loss and the distillation loss, and adjusts the model parameters of the student model through the objective function to train the student model.

[0084] Furthermore, Fig. 9 A schematic diagram of a neural network model in the application phase provided in an embodiment of the present application. Fig. 9 , which is right Figure 8 Schematic diagram of the student model shown in Figure 1. Fig. 9 It can be seen that the first sub-neural network model for calculating the first feature vector is deployed in the background server to obtain the first feature vector of the current user through the user characteristics of the current user in the application stage. At the same time, the second feature vector of each item is calculated offline using the second sub-neural network model, and the second feature vector of the candidate item is directly obtained in the server, so as to calculate the inner product of the first feature vector and the second feature vector through the inner product calculation model, and then predict the click rate of each candidate item. After that, the item with the highest click rate is selected as the item recommendation result, and the client used by the current user is fed back.

[0085] In the above, the teacher model is trained by the first training feature set, and the student model with a relatively small volume is trained by the second training feature set and the teacher model. By applying the student model with a small volume, the student model can have the performance of the teacher model, and the technical problem that the simple neural network model cannot achieve high-performance item recommendation is solved. Further, the student model includes a first sub-neural network model and a second sub-neural network model. After that, the second feature vector of each item is obtained by the second sub-neural network model and the second feature vector is saved so that the second feature vector can be directly used in the application. At the same time, the first sub-neural network model is deployed online, which can ensure the prediction accuracy, reduce the complexity of the online deployment model, and provide the model running speed. Further, when training the teacher model, user features, item features and interaction features are used, and the learning ability of the teacher model can be improved through more features. In addition, when training the student model, only user features and item features are used, so that the student model can obtain the information of the dominant features, and avoid the dominant features misleading the training of the student model. Further, when training the teacher model, user features, item features and interaction features of other columns are also used, which can better enrich the feature content, improve the learning ability of the teacher model, and avoid the problem of sparse features.

[0086] Fig.10 This is a schematic diagram of the structure of an item recommendation device provided in an embodiment of the present application, referring to Fig.10 The item recommendation device includes a feature acquisition module 301, a vector determination module 302 and a result determination module 303.

[0087] Among them, the feature acquisition module 301 is used to obtain the user features of the current user; the vector determination module 302 is used to input the user features into the first sub-neural network model to obtain the first feature vector of the current user, and the first sub-neural network model is obtained through knowledge distillation training; the result determination module 303 is used to determine the item recommendation result of the current user from multiple candidate items based on the first feature vector and the second feature vectors of multiple candidate items, each of the candidate items corresponds to a pre-saved second feature vector, and the second feature vector is obtained by processing the item features of the candidate items by the second sub-neural network model, and the second sub-neural network model is obtained through knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model.

[0088] Based on the above embodiment, the result determination module 303 includes an inner product calculation unit, which is used to calculate the inner product between the first feature vector and the second feature vector of each candidate item, and each candidate item corresponds to an inner product; an item determination unit, which is used to determine the item recommendation result of the current user according to the inner product corresponding to each candidate item.

[0089] Based on the above embodiment, the calculation model of the inner product is:

[0090]

[0091] Among them, X u represents the user characteristics of the current user, X i represents the item features of the candidate items, W u represents the model parameters of the first sub-neural network model, W i represents the model parameters of the second sub-neural network model, represents the first feature vector of the current user, The second eigenvector representing the candidate items, express and The inner product of .

[0092] On the basis of the above embodiment, it also includes: a first training module, used to obtain a first training feature set, and train a teacher model for item recommendation through the first training feature set; a second training module, used to obtain a second training feature set, and train a student model for item recommendation based on the second training feature set and the teacher model, the volume of the student model is smaller than the volume of the teacher model, the student model includes a first sub-neural network model and a second sub-neural network model, and the first training feature set and the second training feature set both include multiple user features and multiple item features collected in advance.

[0093] On the basis of the above embodiment, it also includes: a deployment module, which is used to deploy the first sub-neural network model online after training the student model according to the second feature training set and the teacher model; a vector saving module, which is used to save the second feature vectors obtained by the second sub-neural network model, each of which corresponds to an item feature.

[0094] Based on the above embodiment, the first training feature set also includes pre-collected multiple interaction features, where the interaction features are interaction features between the user and the object.

[0095] Based on the above embodiment, the objective function of the student model is:

[0096]

[0097] Among them, X represents the user features and item features input into the student model, W s represents the model parameters of the student model, f s represents the function used by the student model, f s (X; W s) represents the item recommendation prediction result of the student model, y represents the actual result of item recommendation, and L s (y,f s (X; W s )) represents y and f s (X; W s ), X1 represents the user features and item features input to the teacher model, represents the interactive features input to the teacher model, W t represents the model parameters of the teacher model, f t represents the function used by the teacher model, represents the item recommendation prediction result of the teacher model, express and f s (X; W s ), λ represents the hyperparameter.

[0098] Based on the above embodiment, the user features, item features and interaction features input into the teacher model are user features, item features and interaction features corresponding to multiple fields; the user features and item features input into the student model are user features and item features corresponding to one field among the multiple fields.

[0099] Based on the above embodiment, it further includes: a recall module, which is used to recall multiple candidate items from at least one item library before determining the item recommendation result for the current user from the multiple candidate items based on the first feature vector and the second feature vectors of the multiple candidate items.

[0100] The item recommendation device provided above can be used to execute the item recommendation method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0101] It is worth noting that in the embodiment of the above-mentioned item recommendation device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0102] Fig.11 This is a schematic diagram of the structure of an item recommendation device provided in an embodiment of the present application. Fig.11 As shown, the item recommendation device includes a processor 40, a memory 41, an input device 42 and an output device 43; the number of processors 40 in the item recommendation device can be one or more. Fig.11 A processor 40 is taken as an example. The processor 40, the memory 41, the input device 42 and the output device 43 in the item recommendation device can be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0103] The memory 41 is a computer-readable storage medium that can be used to store software programs, computer executable programs and modules, such as program instructions / modules corresponding to the item recommendation method in the embodiment of the present application (for example, the feature acquisition module 301, the vector determination module 302 and the result determination module 303 in the item recommendation device). The processor 40 executes the various functional applications and data processing of the item recommendation device by running the software programs, instructions and modules stored in the memory 41, that is, realizing the above-mentioned item recommendation method.

[0104] The memory 41 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the item recommendation device, etc. In addition, the memory 41 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 41 may further include a memory remotely arranged relative to the processor 40, and these remote memories may be connected to the item recommendation device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] The input device 42 may be used to receive input digital or character information and generate key signal input related to user settings and function control of the item recommendation device. The output device 43 may include a display device such as a display screen.

[0106] The above-mentioned item recommendation device includes an item recommendation device, which can be used to execute any item recommendation method and has corresponding functions and beneficial effects.

[0107] In addition, an embodiment of the present application also provides a storage medium containing computer executable instructions, which, when executed by a computer processor, are used to perform relevant operations in the item recommendation method provided in any embodiment of the present application, and have corresponding functions and beneficial effects.

[0108] Those skilled in the art should understand that the embodiments of the present application may be provided as methods, systems, or computer program products.

[0109] Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0110] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory. The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0111] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0112] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0113] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. An item recommendation method, characterized in that: include: Get the user characteristics of the current user; Inputting the user feature into a first sub-neural network model to obtain a first feature vector of the current user, wherein the first sub-neural network model is obtained through knowledge distillation training; According to the first feature vector and second feature vectors of multiple candidate items, determine the item recommendation result of the current user from the multiple candidate items, each of the candidate items corresponds to a second feature vector that is pre-saved and can be reused, the second feature vector is obtained by processing the item features of the candidate items offline by a second sub-neural network model, the second sub-neural network model is obtained by knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model; Determining the item recommendation result for the current user from the plurality of candidate items according to the first feature vector and the second feature vectors of the plurality of candidate items comprises: Calculate the inner product between the first feature vector and the second feature vector of each candidate item, each candidate item corresponding to one inner product; The item recommendation result for the current user is determined according to the inner products corresponding to the candidate items.

2. The item recommendation method according to claim 1, characterized in that: The calculation model of the inner product is: Among them, X u represents the user characteristics of the current user, X i represents the item features of the candidate items, W u represents the model parameters of the first sub-neural network model, W i represents the model parameters of the second sub-neural network model, represents the first feature vector of the current user, The second eigenvector representing the candidate items, express and The inner product of .

3. The item recommendation method according to claim 1, characterized in that: Also includes: Obtaining a first training feature set, and training a teacher model for item recommendation using the first training feature set; A second training feature set is obtained, and a student model for item recommendation is trained based on the second training feature set and the teacher model, wherein the volume of the student model is smaller than the volume of the teacher model, the student model includes a first sub-neural network model and a second sub-neural network model, and the first training feature set and the second training feature set both include a plurality of pre-collected user features and a plurality of item features.

4. The item recommendation method according to claim 3, characterized in that: After training the student model for item recommendation according to the second training feature set and the teacher model, the method further includes: Deploy the first neural network model online; The second feature vectors obtained by the second sub-neural network model are saved, each of which corresponds to an item feature.

5. The item recommendation method according to claim 3, characterized in that: The first training feature set also includes a plurality of pre-collected interaction features, where the interaction features are interaction features between a user and an object.

6. The item recommendation method according to claim 5, characterized in that: The objective function of the student model is: Among them, X represents the user features and item features input into the student model, W s represents the model parameters of the student model, f s represents the function used by the student model, f s (X; W s ) represents the item recommendation prediction result of the student model, y represents the actual result of item recommendation, and L s (y,f s (X; W s )) represents y and f s (X; W s ), X1 represents the user features and item features input to the teacher model, represents the interactive features input to the teacher model, W t represents the model parameters of the teacher model, f t represents the function used by the teacher model, represents the item recommendation prediction result of the teacher model, express and f s (X; W s ), λ represents the hyperparameter.

7. The item recommendation method according to claim 5 or 6, characterized in that: The user features, item features, and interaction features input into the teacher model are user features, item features, and interaction features corresponding to a plurality of fields, and the plurality of fields include a field for displaying recommended items and other fields.

8. The item recommendation method according to claim 1, characterized in that: Before determining the item recommendation result for the current user from the plurality of candidate items according to the first feature vector and the second feature vectors of the plurality of candidate items, the method further includes: A plurality of candidate items are recalled from at least one item library.

9. An item recommendation device, characterized in that: include: A feature acquisition module is used to acquire user features of the current user; A vector determination module, used for inputting the user feature into a first sub-neural network model to obtain a first feature vector of the current user, wherein the first sub-neural network model is obtained through knowledge distillation training; A result determination module is used to determine the item recommendation result of the current user from multiple candidate items according to the first feature vector and second feature vectors of multiple candidate items, each of the candidate items corresponds to a pre-saved and reusable second feature vector, the second feature vector is obtained by offline processing of the item features of the candidate items by a second sub-neural network model, the second sub-neural network model is obtained by knowledge distillation training, and the second sub-neural network model and the first sub-neural network model correspond to the same teacher model; The result determination module includes: an inner product calculation unit, which is used to calculate the inner product between the first feature vector and the second feature vector of each candidate item, each candidate item corresponding to one inner product; The item determination unit is used to determine the item recommendation result of the current user according to the inner product corresponding to each of the candidate items.

10. An item recommendation device, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the item recommendation method as described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the item recommendation method as described in any one of claims 1 to 8 is implemented.

Images