Pre-estimation method based on pre-estimation model, recommendation method and model training method

By using a low-rank network layer to replace the fully connected layer in the estimation model, the problem of inaccurate prediction of scenarios with small sample size is solved, and the accuracy of the estimation model is improved.

CN119067745APending Publication Date: 2024-12-03HANGZHOU ALIBABA INT INTERNET IND CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410978008.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-19
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

During training, existing prediction models are flooded due to the difference in sample data volumes in different scenarios, making it difficult to accurately estimate scenarios with small sample sizes, resulting in low prediction accuracy.

Method used

Using a method based on the prediction model, the discrete features are converted into dense vectors through the embedding layer, and the low-rank network layer is used to replace the fully connected layer, including the first low-rank subnet and the second low-rank subnet, respectively, and the vector is reduced in dimension and Increase to realize vector transformation, reduce parameter quantity, and improve the model's ability to estimate scenarios with small sample size.

Benefits of technology

The weight parameter volume of the estimated model is effectively reduced, so that samples with small sample size can be fully trained, improving the prediction accuracy of scenarios with small sample size, and improving the accuracy of the estimated model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119067745A_ABST
    Figure CN119067745A_ABST
Patent Text Reader

Abstract

The invention discloses a pre-estimation method based on a pre-estimation model, a recommendation method, a training method, an electronic device and a computer readable storage medium, the pre-estimation model comprises an embedded layer and a low-rank network layer, and a low-rank network of the low-rank network layer comprises a first low-rank sub-network and a second low-rank sub-network. The estimation method comprises the following steps: inputting to-be-estimated information into an embedded layer to obtain an m-dimensional to-be-estimated vector; inputting the to-be-estimated vector into the first low-rank sub-network to obtain a dimension reduction vector of which the dimension is reduced to r dimension, r being smaller than m, and the rank of the first low-rank sub-network being not greater than r; inputting the dimension reduction vector into a second low-rank sub-network to obtain an output vector of which the dimension is increased to n dimension, n being a preset output dimension value, n being greater than r, and the rank of the second low-rank sub-network being not greater than r; and obtaining an estimation result according to the output vector. According to the scheme provided by the invention, the information of the scene with a small sample size can be accurately estimated through the estimation model, and the estimation accuracy is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and specifically to an estimation method based on an estimation model, a recommendation method, a training method of an estimation model, an estimation model, an electronic device and a computer-readable storage medium. Background Art

[0002] In order to match products that users are interested in, e-commerce platforms usually estimate the possibility of users operating products, such as the click-through rate (CTR) and conversion rate (CVR) of the products. Among them, the possibility of users operating products is usually estimated through trained estimation models.

[0003] When using the estimation model for estimation, the related technology usually calculates and processes the input features through the fully connected layer of the estimation model to obtain the analysis results for the input features. The estimation model usually needs to be applicable to estimation in different scenarios. Therefore, the training samples during the estimation model training usually include samples of multiple dimensions corresponding to multiple scenarios, for example, samples corresponding to different countries, samples corresponding to different industries, samples corresponding to different user stratifications, samples corresponding to different terminal types, etc. In order to make more accurate data estimates, the dimension of the weight matrix of the fully connected layer of the estimation model is usually relatively large, that is, the fully connected layer has more weight parameters, which requires a large sample size corresponding to each scenario during the training of the estimation model, so that the model can fully learn the data in different scenarios, so that the trained estimation model can accurately estimate the information of each scenario.

[0004] However, due to the different amounts of sample data that can be collected in different dimensions, for example, the sample size of large countries is larger than that of small countries, and the sample size of young and middle-aged users is larger than that of elderly users, etc., the sample data corresponding to some scenarios will be smaller, and there will be large differences between the amounts of sample data in different dimensions for training the estimation model. This may easily lead to information flooding of the differential features during the learning process of the fully connected layer. For example, it is easy for differential categories with large amounts of data (such as large countries) to dominate small differential categories (such as small countries), ultimately making it difficult for the model to learn data for scenarios with small sample sizes, making it impossible for the trained estimation model to make relatively accurate estimates for scenarios with small sample sizes, and resulting in low estimation accuracy. Summary of the invention

[0005] The present application provides an estimation method based on an estimation model, an estimation model training method, an estimation model, an estimation model, an apparatus, an electronic device, and a computer-readable storage medium, which can accurately estimate the information of scenes with a small number of samples through the estimation model, thereby improving the estimation accuracy. The specific scheme is as follows:

[0006] In a first aspect, an embodiment of the present application provides an estimation method based on an estimation model, wherein the estimation model includes an embedding layer and a low-rank network layer, wherein the low-rank network layer includes a low-rank network, wherein the low-rank network includes a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence, and wherein the method includes:

[0007] Obtain information to be estimated;

[0008] Inputting the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, where m is a preset input dimension value;

[0009] Inputting the vector to be estimated into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector into an r-dimensional reduced-dimensional vector, where r is less than m, and the rank of the first low-rank sub-network is not greater than r;

[0010] Inputting the dimension-reduced vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector, where n is a preset output dimension value, n is greater than r, and the rank of the second low-rank sub-network is not greater than r;

[0011] The information to be estimated is estimated according to the output vector to obtain an estimation result.

[0012] Optionally, the first low-rank sub-network is an m×r weight matrix, and the second low-rank sub-network is an r×n weight matrix.

[0013] Optionally, the low-rank network layer includes a plurality of the low-rank networks, and each of the low-rank networks is used to perform calculations on information corresponding to different scenarios;

[0014] The step of inputting the vector to be estimated into the first low-rank sub-network comprises:

[0015] Inputting the vector to be estimated into a first low-rank subnetwork of a target low-rank network, wherein the target low-rank network is a low-rank network corresponding to the scene corresponding to the information to be estimated among the multiple low-rank networks;

[0016] The step of inputting the dimension-reduced vector into the second low-rank sub-network comprises:

[0017] The dimensionally reduced vector is input into a second low-rank sub-network of the target low-rank network.

[0018] Optionally, the estimation model further includes a shared network layer;

[0019] Before estimating the information to be estimated according to the output vector, the method further includes:

[0020] Inputting the vector to be estimated into the shared network layer so that the shared network layer outputs an n-dimensional shared vector, wherein the shared network layer is used to convert an m-dimensional vector into an n-dimensional vector;

[0021] The estimating the information to be estimated according to the output vector to obtain an estimation result includes:

[0022] The information to be estimated is estimated according to the output vector and the shared vector to obtain an estimation result.

[0023] Optionally, the low-rank network layer includes low-rank network groups of various dimensions corresponding to data of different dimensions, and each of the low-rank network groups includes each of the low-rank networks for processing information corresponding to different scenarios;

[0024] The target low-rank network is a low-rank network corresponding to the scenario corresponding to the information to be estimated among the low-rank networks included in each of the low-rank network groups.

[0025] Optionally, the information to be estimated includes information of multiple dimensions, and the output vector includes corresponding output vectors of multiple dimensions;

[0026] The estimating the information to be estimated according to the output vector and the shared vector to obtain an estimation result includes:

[0027] Performing vector multiplication on the output vectors of the multiple dimensions to obtain a multiplied vector;

[0028] The information to be estimated is estimated according to the sum of the multiplied vector and the shared vector to obtain an estimation result.

[0029] Optionally, the estimation model further includes an output layer;

[0030] The estimating the information to be estimated according to the sum of the multiplied vector and the shared vector to obtain an estimation result includes:

[0031] The sum of the multiplied vector and the shared vector is input into the output layer to obtain an estimated result, where the estimated result is a score corresponding to the information to be estimated.

[0032] Optionally, the shared network layer is an m×n weight matrix.

[0033] Optionally, the estimation model includes multiple layers of the low-rank network layers arranged in sequence;

[0034] The step of inputting the vector to be estimated into the first low-rank sub-network comprises:

[0035] Determine the input data of the i-th low-rank network layer according to the i-1th n-dimensional output vector, and input the input data into the first low-rank subnetwork of the i-th low-rank network layer to obtain an i-th r-dimensional dimensionality reduction vector, wherein the 0th n-dimensional output vector is the vector to be estimated;

[0036] The step of inputting the dimension-reduced vector into the second low-rank sub-network comprises:

[0037] Inputting the i-th r-dimensional dimension reduction vector into the second low-rank subnetwork of the i-th low-rank network layer to obtain an i-th n-dimensional output vector;

[0038] Wherein, i traverses 1 to N, and N is the number of low-rank network layers included in the estimation model;

[0039] The estimating the information to be estimated according to the output vector to obtain an estimation result includes:

[0040] The information to be estimated is estimated according to the N-th n-dimensional output vector to obtain an estimation result.

[0041] Optionally, the information to be estimated includes: content information to be recommended and user characteristic information, and the estimation result is a probability value of the user operating the content information to be recommended.

[0042] Optionally, each of the low-rank networks is respectively provided with a corresponding weight parameter table, and each weight parameter of the corresponding low-rank network is recorded in the weight parameter table;

[0043] Before inputting the vector to be estimated into the first low-rank sub-network of the target low-rank network, the method further includes:

[0044] Obtain a target weight parameter table corresponding to the target low-rank network, and determine each weight parameter of the first low-rank sub-network and the second low-rank sub-network corresponding to the target low-rank network from the target weight parameter table.

[0045] In a second aspect, an embodiment of the present application further provides a recommendation method, the method comprising:

[0046] Obtaining user feature information and information about content to be recommended;

[0047] Using the user feature information and the information of the content to be recommended as information to be estimated, and estimating a parameter of the possibility that the user will operate on the content to be recommended by using the estimation method according to any one of the first aspects;

[0048] The content to be recommended is recommended to the user according to the possibility parameter.

[0049] In a third aspect, an embodiment of the present application further provides a method for training an estimation model, the method comprising:

[0050] Obtain a model to be trained, the model to be trained comprising an embedding layer and a low-rank network layer, the low-rank network layer comprising a low-rank network, and the low-rank network comprising a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence;

[0051] Get training samples;

[0052] Input the training sample into the embedding layer of the model to be trained to obtain an m-dimensional sample vector corresponding to the training sample, where m is a preset input dimension value;

[0053] Inputting the sample vector into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional sample dimension reduction vector, where r is less than m, and the first low-rank sub-network is used to convert the m-dimensional vector into an r-dimensional vector;

[0054] Inputting the sample dimension reduction vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional sample output vector, where n is a preset output dimension value, n is greater than r, and the second low-rank sub-network is used to convert the r-dimensional vector into an n-dimensional vector;

[0055] Predicting the training sample according to the sample output vector to obtain a sample prediction result;

[0056] According to the sample estimation results and the labeling results corresponding to the training samples, the model parameters of each layer of the to-be-trained model are adjusted to obtain a trained estimation model.

[0057] Optionally, the first low-rank sub-network is an m×r weight matrix, and the second low-rank sub-network is an r×n weight matrix.

[0058] Optionally, the low-rank network layer includes a plurality of the low-rank networks, each of the low-rank networks is used to perform calculations on information of different dimensions corresponding to different scenarios, and the training samples include training samples corresponding to different scenarios;

[0059] The step of inputting the sample vector into the first low-rank sub-network comprises:

[0060] Inputting each sample vector corresponding to the training sample of each scene into the first low-rank sub-network of the corresponding target low-rank network, respectively, wherein the target low-rank network is a low-rank network corresponding to the scene corresponding to the training sample among the multiple low-rank networks;

[0061] The step of reducing the sample dimension into a vector and inputting it into the second low-rank sub-network comprises:

[0062] The sample dimension reduction vector is input into the second low-rank sub-network of the target low-rank network corresponding to the vector.

[0063] Optionally, the model to be trained also includes a shared network layer;

[0064] Before estimating the training sample according to the sample output vector, the method further includes:

[0065] Inputting the sample vector into the shared network layer so that the shared network layer outputs an n-dimensional sample shared vector, wherein the shared network layer is used to convert an m-dimensional vector into an n-dimensional vector;

[0066] The step of estimating the training sample according to the sample output vector to obtain a sample estimation result includes:

[0067] The training sample is estimated according to the sample output vector and the sample sharing vector to obtain a sample estimation result.

[0068] In a third aspect, the present application also provides an estimation model, including:

[0069] The embedding layer is used to convert the input data into an m-dimensional vector, where m is the preset input dimension value;

[0070] A low-rank network layer, the low-rank network layer comprising a low-rank network, the low-rank network comprising a first low-rank sub-network and a second low-rank sub-network arranged in sequence, the first low-rank sub-network being used to convert an m-dimensional vector into an r-dimensional reduced-dimensional vector, the second low-rank sub-network being used to convert the r-dimensional reduced-dimensional vector into an n-dimensional output vector, where r is less than m, n is a preset output dimension value, and n is greater than r;

[0071] The output layer is used to obtain an estimated result for the input data according to the output vector of the second low-rank sub-network.

[0072] In a fourth aspect, the present application provides an electronic device comprising: a processor, a memory, and computer program instructions stored in the memory and executable on the processor; when the processor executes the computer program instructions, it implements the method described in any one of the first aspect or the second aspect.

[0073] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in any one of the first aspect or the second aspect.

[0074] In a sixth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method described in any one of the first aspect or the second aspect.

[0075] Compared with the prior art, this application has the following advantages:

[0076] An embodiment of the present application provides an estimation method based on an estimation model, wherein the estimation model includes an embedding layer and a low-rank network layer, wherein the low-rank network layer includes a low-rank network, and the low-rank network includes a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence. The estimation method obtains information to be estimated, and then inputs the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, where m is a preset input dimension value. The information to be estimated is usually discrete features such as user features and product features. Since the efficiency of calculating discrete features is low and it is difficult for discrete features to capture the connection between features, converting the discrete information to be estimated into a dense vector through an embedding layer can improve the calculation efficiency and more effectively capture the relationship between features. The dense vector can also be better input into the neural network for calculation, thereby improving the performance and accuracy of the model. After obtaining the m-dimensional vector to be estimated, the vector to be estimated is input into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional dimension-reduced vector. Since the rank of the first low-rank sub-network is not greater than r, and r is less than m, in practical applications, r can be set much smaller than m. Therefore, the number of parameters of the first low-rank sub-network is much smaller than the number of parameters of the original fully connected layer. Then, the r-dimensional dimension-reduced vector is input into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector. The information to be estimated is estimated according to the output vector to obtain an estimation result, where n is a preset output dimension value. Since the rank of the second low-rank sub-network is not greater than r, and n is greater than r, in practical applications, r can be set much smaller than m. Therefore, The number of parameters of the second low-rank sub-network is also much smaller than the number of parameters of the original fully connected layer. In this way, the present application reduces the dimension and restores the m-dimensional vector to be estimated through the first low-rank sub-network and the second low-rank sub-network respectively, so as to realize the function of the original full connection to convert the m-dimensional vector calculation into an n-dimensional output vector, and the sum of the parameters of the first low-rank sub-network and the second low-rank sub-network, two networks with ranks less than r, is much smaller than the parameters of the original fully connected layer. Therefore, for training samples with a relatively small number of samples, since the number of parameters of the low-rank network layer is relatively small, the model can also learn the data of scenes with a small number of samples through relatively small training samples, so that the trained estimation model can well make relatively accurate estimates for scenes with a small number of samples, thereby improving the estimation accuracy.

[0077] It can be seen that the application reduces the dimension of the m-dimensional vector through the first low-rank sub-network and restores the reduced vector by increasing the dimension through the second low-rank sub-network. The two low-rank sub-networks can convert the m-dimensional vector into an n-dimensional vector, thereby achieving a vector transformation with the same function as the original fully connected layer vector transformation. The solution provided by the present application replaces the original fully connected layer with a low-rank network layer composed of the first low-rank sub-network and the second low-rank sub-network, which can greatly reduce the amount of weight parameters of the estimation model, so that samples with a small sample size can also train the model more fully and accurately, so that the trained model can make a relatively accurate estimate of the information corresponding to the scene with a small sample size, thereby improving the accuracy of the estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of an application scenario of an estimation solution based on an estimation model provided in this application;

[0079] Figure 2 It is a schematic diagram of the model processing of the related technology through the fully connected layer for estimation;

[0080] Figure 3 It is a flowchart of an example of an estimation method based on an estimation model provided in an embodiment of the present application;

[0081] Figure 4 It is a schematic diagram of a model processing of an example of an estimation model in an embodiment of the present application;

[0082] Figure 5 is a schematic diagram of model processing of another example of estimation by the estimation model in the embodiment of the present application;

[0083] Figure 6 It is a flowchart of an example of a recommended method provided in an embodiment of the present application;

[0084] Figure 7 It is a flowchart of an example of a training method provided in an embodiment of the present application;

[0085] Figure 8 It is a structural block diagram of the electronic device provided in this application. DETAILED DESCRIPTION

[0086] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described clearly and completely below in conjunction with the drawings in the embodiments of the present application. However, the present application can be implemented in many other ways different from the following description. Therefore, based on the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative work should fall within the scope of protection of the present application.

[0087] It should be noted that the terms "first", "source domain", "third", etc. in the claims, description and drawings of the present application are used to distinguish similar objects and are not used to describe a specific order or sequence. The data used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including", "having" and their variants are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0088] In order to facilitate understanding of the various embodiments of the present application, the application background of the embodiments is described.

[0089] The recommendation system is an important functional module of the e-commerce platform and is deployed on multiple pages of the e-commerce platform, such as the dynamic shopping guide floor on the home page, the shopping guide floor takeover page, the home page guess you like, the shopping cart guess you like, etc. The task of the recommendation system is to match the products that users may be interested in from the hundreds of millions of product pools based on the user's characteristics and historical product behavior records (such as browsing, clicking, inquiring, and transactions), and recommend them to the user, helping the user to find the products of interest faster and complete the conversion (for example, the user purchases the product to complete the conversion). In order to match the products that users are interested in, e-commerce platforms usually estimate the possibility of users operating the products, such as the click-through rate (CTR) and conversion rate (CVR) of the products. Among them, the possibility of users operating the products is usually estimated through a trained estimation model.

[0090] When using an estimation model for estimation, the related technology usually calculates and processes the input features through the fully connected layer of the estimation model to obtain the analysis results for the input features. The estimation model usually needs to be applicable to estimation in different scenarios. Therefore, the training samples during the estimation model training usually include samples of multiple dimensions corresponding to multiple scenarios, for example, samples corresponding to different countries, samples corresponding to different industries, samples corresponding to different user stratifications, samples corresponding to different terminal types, etc. Samples of different dimensions vary greatly in user needs and data distribution, and multiple dimensions form intersections.

[0091] In order to make more accurate data estimates, the dimension of the weight matrix of the fully connected layer of the estimation model is usually large, that is, the fully connected layer has more weight parameters. This requires a large sample size for each scene during training so that the model can fully learn the data in different scenes, so that the trained estimation model can accurately estimate the information of each scene.

[0092] In the related art, when training a model based on multi-dimensional samples, samples of each dimension are usually uniformly input into the fully connected layer of the model to be trained for deep feature cross-pollination, so that the fully connected layer can learn the data corresponding to samples of different dimensions.

[0093] However, due to the different amounts of sample data that can be collected in different dimensions, for example, the sample size of large countries is larger than that of small countries, and the sample size of young and middle-aged users is larger than that of elderly users, etc., the sample data corresponding to some scenarios will be smaller, and there will be large differences between the sample data amounts of different dimensions for training the estimation model. This can easily lead to information flooding of the difference features during the learning process of the fully connected layer. For example, it is easy for difference categories with large data volumes (such as large countries) to dominate small difference categories (such as small countries), which ultimately makes it difficult for the model to learn data for scenarios with small sample sizes, making it impossible for the trained estimation model to make relatively accurate estimates for scenarios with small sample sizes, and it is also difficult to learn the true differences, so that the trained model cannot make relatively accurate estimates for different scenarios, and the estimation accuracy is low.

[0094] For example, Figure 2 As shown, the estimation model in the related technology usually includes an embedding layer, a field pooling layer and a multi-layer perceptron (MLP) (i.e., a fully connected layer). The Embedding Layer is used to map sparse high-dimensional features to a low-dimensional dense vector space. The Field-wise Pooling Layer is used to merge the vectors output by the Embedding Layer according to the feature domain. The fully connected layer is used to analyze and calculate the vectors merged by the Field-wise Pooling Layer to obtain an output vector. The output vector is used to determine the estimation result, such as the click-through rate estimation result or the conversion rate estimation result. Figure 2 It can be seen that the relevant technology uses a fully connected layer for deep feature cross-pollination and lacks the learning of multi-dimensional differences. The difference features are prone to information drowning in the MLP. In addition, due to the different data volumes, large difference categories (such as large countries) are prone to dominate small difference categories (such as small countries). Ultimately, it is difficult to learn the true differences, which affects the accuracy of the model.

[0095] In order to solve the above problems, the embodiments of the present application provide an estimation method, device, electronic device and computer-readable storage medium based on an estimation model, aiming to accurately estimate the information of scenes with a small number of samples through the estimation model, thereby improving the estimation accuracy.

[0096] The estimation method based on the estimation model provided in this application can be used to estimate the click-through rate or conversion rate of users for products, articles, search results, etc. For example, the solution provided in this application can be applied to the article recommendation scenario to estimate the click-through rate of users for each article to be recommended, so as to recommend articles with high click-through rates to users based on the estimated click-through rates; or, the solution provided in this application can also be applied to the e-commerce scenario to estimate the click-through rate or conversion rate (i.e., the probability of purchasing the product) of users for each product to be sold, so as to recommend products with high click-through rates or conversion rates to users based on the estimated click-through rates or conversion rates.

[0097] In order to facilitate understanding of the method embodiments of the present application, the application scenarios are introduced. Figure 1 , Figure 1 The following is a schematic diagram of an application scenario of the solution provided in the embodiment of the present application. It can be applied to a recommendation system. This application scenario is a schematic illustration and is not intended to be a specific description of its application scenario. Figure 1 As shown, in this application scenario, a server and a client are provided for implementing information recommendation (including product recommendation, article recommendation, news recommendation, audio and video recommendation, friend recommendation, scenic spot recommendation, etc.). In this embodiment, the client 101 and the server 102 are connected via network communication. For ease of description, the following describes the embodiment of this application by taking product recommendation as an example.

[0098] The client 101 can be an electronic device with display and data processing functions, such as a mobile phone, a tablet computer (pad), a smart watch, a desktop computer, a smart TV, a VR device, a vehicle-mounted device, a wearable device, a laptop computer, etc. The client 101 is used to log in to a user account, send the user account to the server, display recommended products, allow users to browse or purchase recommended products, obtain product query instructions entered by the user, etc.

[0099] A specific communication connection needs to be established between the client 101 and the server 102 to perform data transmission.

[0100] The server 102 has high computing power. The server 102 can be a server, which has high-speed processor (central processing unit, CPU) computing power, long-term reliable operation, powerful input / output (input / output, I / O) external data throughput capacity and better scalability. The server 102 can be a single server or a server cluster. The server is used to receive product query instructions and user accounts from the client, and determine candidate products that match the product query instructions, and determine the user characteristics of the user according to the user account, estimate the user click rate or user purchase rate of each candidate product, determine the recommended product according to the estimated click rate or purchase rate, and send the information of determining the recommended product to the client, so that the client displays the recommended product. The server 102 can also provide other specific services to the client, such as user information access, website access, application access, etc., which are not specifically limited in this application.

[0101] The client 101 can communicate with the server 102. The client 101 and the server 102 can communicate using various communication systems, such as a wired communication system or a wireless communication system. The wireless communication system can be, for example, a global system for mobile communications (GSM) system, a code division multiple access (CDMA) system, a wideband code division multiple access (WCDMA) system, a general packet radio service (GPRS), a long term evolution (LTE) system, a LTE frequency division duplex (FDD) system, a LTE time division duplex (TDD) system, a universal mobile telecommunication system (UMTS), a worldwide interoperability for microwave access (WiMAX) communication system, a future fifth generation (5G) system or a new radio (NR), a satellite communication system, etc.

[0102] Embodiment 1

[0103] The first embodiment of the present application provides an estimation method based on an estimation model, which is applied to a server. The method can be applied to an electronic device, which is a server. The server can be a server of a recommendation system, for example, a server of an e-commerce website, a server of a reading platform, a server of a dating website, a server of a travel website, etc., but is not limited thereto. The server can be a single server or a server cluster, which is not specifically limited in the present application.

[0104] The electronic device serving as the server is deployed with a trained estimation model, which may be a neural network model, so that the estimation model has a higher ability to process high-dimensional data, higher flexibility and scalability, and higher ability to process sparse data, thereby improving the accuracy and applicability of the model estimation. The estimation model may also be other models, such as a linear model, a factor decomposition model, etc., and those skilled in the art may flexibly select the model type according to different estimation scenarios.

[0105] The estimation model in the embodiment of the present application is a multi-dimensional low-rank adaptive network (Multi-Dimensional Low-Rank Adaptive Network, M-LoRA for short), which is a trained model, that is, the weight parameters of each layer of the estimation model have been adjusted through training. The training process of the estimation model will be introduced later, and the structure of the estimation model will be introduced here first. Figure 5 As shown, the estimation model includes an embedding layer and a low-rank network layer. The low-rank network layer includes one or more low-rank networks. Each low-rank network includes a first low-rank sub-network and a second low-rank sub-network that are sequentially arranged. In the same low-rank network, the output of the first low-rank sub-network is used as the input of the second low-rank sub-network. Figure 4 As shown, the low-rank network layer may include a low-rank network consisting of a first low-rank sub-network A1 and a second low-rank sub-network B1 and a low-rank network consisting of a first low-rank sub-network A2 and a second low-rank sub-network B2. Figure 5 As shown, the low-rank network layer may also include more low-rank networks consisting of a first low-rank sub-network A and a second low-rank sub-network B.

[0106] The embedding layer is used to convert the discrete information to be estimated into a dense vector through the embedding layer, that is, into an m-dimensional vector to be estimated, so as to improve the computational efficiency and more effectively capture the relationship between features. The dense vector can also be better input into the neural network for calculation, thereby improving the performance and accuracy of the model.

[0107] The low-rank network layer is used to calculate and process the m-dimensional vector to be estimated after the conversion of the embedding layer to obtain an n-dimensional output vector. Specifically, the first low-rank sub-network is used to reduce the dimension of the m-dimensional vector to be estimated to an r-dimensional reduced-dimensional vector, and the second low-rank sub-network is used to increase the dimension of the r-dimensional reduced-dimensional vector to an n-dimensional output vector. Among them, the ranks of the first low-rank sub-network and the second low-rank sub-network are not greater than r. Specifically, the first low-rank sub-network can be an m×r weight matrix, and the second low-rank sub-network can be an r×n weight matrix. Since the rank of the matrix is ​​the rank of the row vector or column vector of these matrices, that is, the number of vectors contained in the maximal independent group of the rows or columns of the matrix, the ranks of the first low-rank sub-network and the second low-rank sub-network are not greater than r. r is an integer value less than m and less than n, r can be much smaller than m and n, for example, r can be less than a preset multiple of m, the preset multiple can be any multiple of 5 to 50, the smaller the r value is, the smaller the weight parameter amount of the low-rank network is, and the trained estimation model can more accurately learn the data corresponding to the scene with a small sample size; the larger the r value is, the larger the weight parameter amount of the low-rank network is, and the trained estimation model can more accurately learn the data corresponding to the scene with a large sample size. Technical personnel in this field can select the size of the r value according to the sample size of the scene corresponding to the data to be estimated, and this application does not specifically limit it.

[0108] The value of the dimension m of the above-mentioned vector to be estimated should fully reflect the amount of information of the input information to be estimated. Therefore, the value of m should not be set too small to better reflect the amount of information of the information to be estimated. The value of m does not need to be set too large to cause complex calculations and redundant invalid calculations. Technical personnel in this field can set it according to actual needs, and this application does not specifically limit it.

[0109] The value of the dimension n of the above-mentioned output vector should contain the amount of information required for the estimation task. Therefore, the value of n should not be set too small so as to better perform the estimation task through sufficient vector dimensions. Technical personnel in this field can set the value of n according to the specific estimation task.

[0110] The above is an introduction to the embedding layer and low-rank network layer of the estimation model. The other specific structures of the estimation model will be introduced in detail later.

[0111] like Figure 3 As shown, the estimation method based on the estimation model provided in the first embodiment of the present application includes the following steps S110 to S150.

[0112] Step S110: Obtain information to be estimated.

[0113] The above-mentioned information to be estimated is information input into the estimation model for the estimation model to make an estimation based on the information to be estimated, and the information to be estimated corresponds to the content used by the estimation model for estimation. Exemplarily, when the estimation method provided in the embodiment of the present application is used to estimate the probability value of a user operating the content to be recommended, for example, to estimate the user's click rate or purchase rate for candidate products, to estimate the user's click rate for candidate articles, etc., the information to be estimated may include the content information to be recommended and user feature information.

[0114] User characteristic information is the characteristic information of the user who needs recommended content. The user characteristic information may include the user's basic information, the user's historical behavior data, the user's device information, etc. The user's basic information may include the user's identification, user's age, gender, geographic location, etc. The user's historical behavior data may include the user's historical purchase records, user's historical click records, user's historical search records, user's browsing records, etc. The user's device information may include the type of device used by the user, the operating system, browsing software, etc.

[0115] The information of the content to be recommended may include identification information of the content to be recommended (specific information used to associate the product), the type of the content to be recommended (including clothing, electronic products, household items, etc.), attributes of the product to be recommended (including price, brand, description information, corresponding pictures, etc.), and popularity of the content to be recommended (including product popularity, sales volume, etc.).

[0116] The information to be estimated may also include other information related to the estimated user click rate, such as the page currently browsed by the user, the product currently browsed by the user, the time when the user performed the operation, etc., and technicians in this field may flexibly set it according to needs.

[0117] When the solution provided by the present application is used to estimate the possibility of a user operating on a candidate content, the server can obtain a user ID from the client, and then determine the user feature information corresponding to the user ID from the stored user information library, and determine each candidate content from the content library, and determine the determined user feature information and the determined candidate content as the information to be estimated. That is, the server can determine the information to be estimated from the content stored in the database. In the embodiments of the present application, the estimated content is different, and the way the server obtains the information to be estimated is also different, and those skilled in the art can flexibly choose.

[0118] The information to be estimated in the embodiments of the present application can be understood as network behavior data stored on the Internet. The information to be estimated exists objectively during the operation of the Internet. The information to be estimated is data that is not subject to personal will and conforms to the laws of nature. The information to be estimated is information that exists objectively. For example, the identification of the product to be recommended, the attributes of the product to be recommended, and the sales volume of the product to be recommended are all information that the product to be recommended itself objectively possesses. The user's device information, the user's historical behavior data, the user's identification, and other information about the user's characteristic information to be estimated are also information that the user itself objectively already has, and are not subject to a person's subjective will. That is, a certain individual cannot modify the user's characteristic information because the user's characteristic information has been objectively and truly generated, and the user's characteristic information is information that complies with the laws of nature.

[0119] Step S120: input the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, wherein m is a preset input dimension value.

[0120] In the embodiment of the present application, the value of m and the relevant content of the embedding layer have been described in detail above and will not be repeated here. The information to be estimated usually includes various discrete feature information, such as user age, gender, location, purchase record, candidate content type, candidate content image..., that is, the content of the information to be estimated usually includes a large number of discrete and sparse features. The amount of these discrete and sparse feature data is large, which leads to large computational and storage overheads during model operation, and sparse discrete data cannot effectively represent the correlation between features. Therefore, converting the information to be estimated into a low-dimensional dense vector through the embedding layer can reduce computational and storage overheads, and can also effectively characterize the correlation between features, thereby improving the accuracy of model estimation.

[0121] Step S130: input the above-mentioned vector to be estimated into the above-mentioned first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional reduced-dimensional vector.

[0122] r is less than m, and the rank of the first low-rank subnetwork is not greater than r.

[0123] Step S140: Input the dimension-reduced vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector.

[0124] n is a preset output dimension value, n is greater than r, and the rank of the second low-rank subnetwork is not greater than r.

[0125] The value of r, the value of n, the first low-rank subnetwork and the second low-rank subnetwork have been described in detail above and will not be repeated here. When the low-rank network layer includes a low-rank network, the vector to be estimated can be directly input into the first low-rank subnetwork included in the low-rank network.

[0126] In order to enable the estimation model to estimate the data of different scenarios more accurately, for example, the estimation model estimates the click rate of users in different countries, and users in different countries correspond to different scenarios. For another example, the estimation model estimates the click rate of different types of goods, and different types of goods correspond to different scenarios. In view of this situation, if the low-rank network layer only includes one low-rank network, due to the relatively small number of parameters of the low-rank network, the low-rank network may not be able to effectively estimate the data in different scenarios. Therefore, in one embodiment, the low-rank network layer may include multiple low-rank networks, each of which is used to calculate and process the information corresponding to different scenarios. For example, the low-rank network layer may include a low-rank network for processing users in country B, a low-rank network for processing users in country A, and a low-rank network for processing users in country C. The low-rank network layer may also include multiple low-rank network groups corresponding to data of different dimensions, for example, including a low-rank network group for the dimension of the region to which the user belongs, and a low-rank network group including the dimension of the product type. The low-rank network groups of each dimension may include low-rank networks for different scenarios. For example, the low-rank network group of the commodity type dimension may include a low-rank network for processing daily necessities, a low-rank network for processing food commodities, a low-rank network for processing furniture commodities, etc. The low-rank network group for the user's region dimension may include a low-rank network for processing users in country B, a low-rank network for processing users in country A, and a low-rank network for processing users in country C. Those skilled in the art can set the number of low-rank networks included in the low-rank network layer according to the different scenarios involved in the actual use process and the different data dimensions. In the process of estimating model training, the low-rank networks corresponding to different scenarios are obtained by training samples corresponding to different scenarios. The specific training process will be introduced later.

[0127] When the low-rank network layer includes multiple low-rank networks, step S130 can be implemented according to the following step S131.

[0128] Step S131: input the vector to be estimated into the first low-rank sub-network of the target low-rank network, where the target low-rank network is a low-rank network corresponding to the scene corresponding to the information to be estimated among multiple low-rank networks.

[0129] For example, if the user characteristics included in the information to be estimated are users from country A, then the target low-rank network includes the low-rank network corresponding to the scenario of users from country A; if the type of candidate products included in the information to be estimated are daily necessities, then the target low-rank network includes the low-rank network corresponding to the scenario of daily necessities.

[0130] For example, Figure 5 As shown, the low-rank network includes a low-rank network group Domain1 corresponding to dimension one and a low-rank network group Domain2 corresponding to dimension two. If the scene corresponding to the information to be estimated is the i scene in dimension one and the j scene in dimension two, then the target low-rank network includes the low-rank network Domain corresponding to the i scene in dimension one. 1i The low-rank network Domain corresponding to the j scene of dimension 2 2j Among them, Figure 5 As shown, the low-rank network group Domain1 corresponding to dimension 1 contains Domain 11 ...Domain 1x The low-rank network corresponding to these x scenarios is similar. The low-rank network group Domain2 corresponding to dimension 2 contains Domain 21 ...Domain 2y The low-rank network corresponding to these y scenarios can analyze the data more accurately by dividing the low-rank network groups in various dimensions.

[0131] In a specific embodiment, the above-mentioned low-rank network layer includes low-rank network groups of different dimensions corresponding to data of different dimensions, and each of the above-mentioned low-rank network groups includes low-rank networks for processing information corresponding to different scenarios; in this case, the above-mentioned target low-rank network is a low-rank network corresponding to the scenario corresponding to the above-mentioned information to be estimated among the low-rank networks included in the above-mentioned low-rank network groups.

[0132] Alternatively, the network dimension may not be divided, and the low-rank network includes low-rank networks of multiple scenarios corresponding to one dimension, for example, Figure 4 As shown, it includes an A1B1 low-rank network corresponding to scene one in one dimension and an A2B2 low-rank network corresponding to scene two. By processing various data in different scenes through low-rank networks of multiple scenes corresponding to one dimension, the model structure can be simpler.

[0133] Each low-rank network is a structure including a first low-rank sub-network and a second low-rank sub-network.

[0134] Step S140 can be implemented according to the following step S141.

[0135] Step S141: Input the dimension reduction vector into the second low-rank sub-network of the target low-rank network.

[0136] In step S141, when the low-rank network layer contains low-rank network groups of different dimensions, the target low-rank network may include multiple ones. In this case, the first low-rank subnetwork that inputs the dimensionality reduction vector and outputs the dimensionality reduction vector can be input to a second low-rank subnetwork belonging to the same low-rank network, that is, the data flow of the model is the vector output by the first low-rank subnetwork input to the second low-rank subnetwork belonging to the same low-rank network.

[0137] This embodiment sets up multiple low-rank networks to process information of different scenes respectively, so that the data of different scenes can be estimated through the estimation model, that is, the estimation model can be applicable to the data estimation of different scenes, making the solution provided by the present application more applicable. During the training process, the parameters of the low-rank networks corresponding to different scenes can be trained through samples of different scenes, and the number of parameters of each low-rank network is relatively small, so that the parameters of the low-rank network can be accurately determined under the premise of a small sample size. Different low-rank networks are used to adaptively learn the differences of different scenes, which improves the accuracy of the model's estimation of the data of each scene, and multiple scene dimensions can be modeled synchronously to improve modeling efficiency.

[0138] In an embodiment of the present application, a low-rank network layer is used to replace the fully connected layer in the related art. The low-rank network layer obtains an output vector by performing a linear transformation on its own low-rank weight matrix and the input vector to be estimated.

[0139] Step S150: estimating the information to be estimated according to the output vector to obtain an estimation result.

[0140] Specifically, the output vector can be processed by an activation function (such as ReLU, Sigmoid, Softmax function, etc.) to obtain an estimated result of the information to be estimated. The estimated result can be the user's click rate or conversion rate for the candidate product, the user's complete viewing probability of the candidate video, and other possibility values ​​of the user's operation on the content information to be recommended. It can also be other estimated results, which are not specifically limited in this application.

[0141] An embodiment of the present application provides an estimation method based on an estimation model, wherein the estimation model includes an embedding layer and a low-rank network layer, wherein the low-rank network layer includes a low-rank network, and the low-rank network includes a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence. The estimation method obtains information to be estimated, and then inputs the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, where m is a preset input dimension value. The information to be estimated is usually discrete features such as user features and product features. Since the efficiency of calculating discrete features is low and it is difficult for discrete features to capture the connection between features, converting the discrete information to be estimated into a dense vector through an embedding layer can improve the calculation efficiency and more effectively capture the relationship between features. The dense vector can also be better input into the neural network for calculation, thereby improving the performance and accuracy of the model. After obtaining the m-dimensional vector to be estimated, the vector to be estimated is input into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional dimension-reduced vector. Since the rank of the first low-rank sub-network is not greater than r, and r is less than m, in practical applications, r can be set much smaller than m. Therefore, the number of parameters of the first low-rank sub-network is much smaller than the number of parameters of the original fully connected layer. Then, the r-dimensional dimension-reduced vector is input into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector. The information to be estimated is estimated according to the output vector to obtain an estimation result, where n is a preset output dimension value. Since the rank of the second low-rank sub-network is not greater than r, and n is greater than r, in practical applications, r can be set much smaller than m. Therefore, The number of parameters of the second low-rank sub-network is also much smaller than the number of parameters of the original fully connected layer. In this way, the present application reduces the dimension and restores the m-dimensional vector to be estimated through the first low-rank sub-network and the second low-rank sub-network respectively, so as to realize the function of the original full connection to convert the m-dimensional vector calculation into an n-dimensional output vector, and the sum of the parameters of the first low-rank sub-network and the second low-rank sub-network, two networks with ranks less than r, is much smaller than the parameters of the original fully connected layer. Therefore, for training samples with a relatively small number of samples, since the number of parameters of the low-rank network layer is relatively small, the model can also learn the data of scenes with a small number of samples through relatively small training samples, so that the trained estimation model can well make relatively accurate estimates for scenes with a small number of samples, thereby improving the estimation accuracy.

[0142] For example, the number of parameters of the first low-rank sub-network is m×r parameters, and the number of parameters of the second low-rank sub-network is r×n parameters, then the number of parameters of a low-rank network is r×(m+n). Compared with the m×n parameters of the fully connected layer in the related technology, since r can be set much smaller than m and n, r×(m+n) is also much smaller than m×n, and the number of parameters is greatly reduced.

[0143] It can be seen that the application reduces the dimension of the m-dimensional vector through the first low-rank sub-network and restores the reduced vector by increasing the dimension through the second low-rank sub-network. The two low-rank sub-networks can convert the m-dimensional vector into an n-dimensional vector, thereby achieving a vector transformation with the same function as the original fully connected layer vector transformation. The solution provided by the present application replaces the original fully connected layer with a low-rank network layer composed of the first low-rank sub-network and the second low-rank sub-network, which can greatly reduce the amount of weight parameters of the estimation model, so that samples with a small sample size can also train the model more fully and accurately, so that the trained model can make a relatively accurate estimate of the information corresponding to the scene with a small sample size, thereby improving the accuracy of the estimation.

[0144] In one embodiment, Figure 4 , Figure 5 As shown, the above estimation model may also include a shared network layer, which is used to convert an m-dimensional vector into an n-dimensional vector. The shared network layer may specifically be an m×n weight matrix. During the training process, the shared network layer can learn the common information between samples of different scenes. The shared network layer and the low-rank network layer may be arranged in parallel, that is, the shared network layer and the low-rank network layer may be located at the same layer position, for example, both are located at the second layer or both are located at other layer positions.

[0145] Before step S150, the above method may further include the following step S150a.

[0146] Step S150a: input the above-mentioned vector to be estimated into the shared network layer, so that the shared network layer outputs an n-dimensional shared vector.

[0147] Step S150 can be implemented according to the following step S151.

[0148] Step S151: Estimate the information to be estimated according to the output vector and the shared vector to obtain an estimation result.

[0149] Specifically, the output vector and the shared vector may be connected and combined, and the information to be estimated may be estimated according to the combined vector to obtain an estimation result.

[0150] The shared network layer provided in this embodiment enables the estimation model to extract common features of the input information, improves the generalization ability of the estimation model, and prevents overfitting, thereby improving the efficiency of the estimation model in reasoning using data and resources, and improving effectiveness and prediction accuracy.

[0151] In a specific embodiment, the information to be estimated may include information of multiple dimensions, and the output vector may include output vectors of corresponding multiple dimensions. For example, the information to be estimated includes information of user feature dimensions and information of candidate item information dimensions, and the output vector includes a user feature vector corresponding to the user feature dimensions and a candidate item information vector corresponding to the candidate item information dimensions.

[0152] Correspondingly, the above step S151 can be implemented according to the following step S151a.

[0153] Step S151a: Fusing the output vectors of the multiple dimensions with the shared vector to obtain a fused vector, and estimating the information to be estimated according to the fused vector to obtain an estimation result.

[0154] Among them, the output vectors of multiple dimensions can be vector-multiplied to obtain a multiplied vector, and then the sum of the multiplied vector and the shared vector is determined as a fusion vector. That is, the present application can estimate the information to be estimated based on the sum of the multiplied vector and the shared vector to obtain an estimated result.

[0155] The embodiment of the present application may also use the Hadamard product method to fuse the output vectors of the multiple dimensions with the shared vector. Specifically, the output vectors of the multiple dimensions and the shared vector may be fused using the following formula (1).

[0156] h ij =Wx+(B 1i A 1i x)×(B 2j A 2j x) (1)

[0157] In formula (1), h ij represents the fusion vector corresponding to the i scene of dimension one and the j scene of dimension two, W represents the parameter matrix of the shared network layer, x represents the vector to be estimated, Wx represents the above shared vector, A 1i represents the parameter matrix corresponding to the first low-rank sub-network corresponding to the i-th scene of dimension one, B 1i Represents the parameter matrix corresponding to the second low-rank sub-network corresponding to the i-th scene of dimension one, A 2j represents the parameter matrix corresponding to the first low-rank sub-network corresponding to the j-th scene of dimension 2, B 2j Represents the parameter matrix corresponding to the second low-rank sub-network corresponding to the j-th scene of dimension two.

[0158] In this embodiment, by performing vector multiplication on output vectors of multiple dimensions, the obtained multiplied vector can integrate the vector information of each dimension and capture the interactive relationship between elements of different dimensions. The summation with the shared vector can reflect the information to be estimated in different dimensions and the relationship between them as a whole, capture the commonalities between different information, and improve the estimation accuracy.

[0159] In a specific embodiment, the above-mentioned estimation model may further include an output layer, and the output layer is used to generate an estimation result according to the vector output by the low-rank network layer, and the estimation result may be a category label, a probability distribution, etc. In the embodiment of the present application, the output layer may include an activation function, and the activation function refers to the description of the activation function above, and will not be repeated here.

[0160] Correspondingly, the information to be estimated is estimated based on the sum of the multiplied vector and the shared vector to obtain an estimated result, including: inputting the sum of the multiplied vector and the shared vector into the output layer to obtain an estimated result, and the estimated result is a score corresponding to the information to be estimated.

[0161] This embodiment can conveniently obtain the estimation result through the output layer of the estimation model. When the estimation result is a score, it can make it more convenient to judge whether the information to be estimated meets expectations according to the score.

[0162] In one embodiment, the above estimation model may include multiple low-rank network layers arranged in sequence, such as Figure 5 The estimation model shown includes two layers of low-rank network layers arranged in sequence. The estimation model may also include three, four or more layers of low-rank network layers arranged in sequence. Those skilled in the art may set the number of layers of the low-rank network according to the complexity of the estimation task, the accuracy of the required estimation result, etc., which is not specifically limited in this application. Optionally, as Figure 5 As shown, a corresponding shared network layer can be set at the same layer position of each low-rank network layer.

[0163] In a specific embodiment, the above step S130 can input the vector to be estimated into the first low-rank sub-network according to the following step S132.

[0164] Step S132: Determine the input data of the i-th low-rank network layer according to the i-1th n-dimensional output vector, and input the input data into the first low-rank subnetwork of the i-th low-rank network layer to obtain the i-th r-dimensional dimensionality reduction vector.

[0165] The 0th n-dimensional output vector is the vector to be estimated.

[0166] Specifically, the i-1th n-dimensional output vector can be determined as the input data of the i-th low-rank network layer, or the i-1th n-dimensional output vector can be processed to obtain the input data of the i-th low-rank network layer, for example, Figure 5 As shown, when the i-1th n-dimensional output vector includes the i-1th n-dimensional output vector of multiple dimensions corresponding to the scenes of multiple dimensions, the i-1th n-dimensional output vectors of multiple dimensions can be fused, for example, by vector multiplication, and the fused vector is determined as the input data of the i-th low-rank network layer; for another example, when each low-rank network layer corresponds to a shared network layer, the i-1th n-dimensional output vector of multiple dimensions and the i-1th shared vector can be fused, and the fused vector is determined as the input data of the i-th low-rank network layer, wherein the 0th shared vector is the above-mentioned vector to be estimated, and the i-1th shared vector is the vector output by the i-1th shared network layer.

[0167] Correspondingly, step S140 can input the dimension-reduced vector into the second low-rank sub-network according to the following step S142.

[0168] Step S142: Input the i-th r-dimensional dimension reduction vector into the second low-rank subnetwork of the i-th low-rank network layer to obtain the i-th n-dimensional output vector.

[0169] Among them, i traverses 1 to N, and N is the number of low-rank network layers included in the estimated model.

[0170] Exemplarily, when i is 1, since the 0th n-dimensional output vector and the 0th shared vector are both the vector to be estimated, the vector to be estimated can first be input into the first low-rank subnetwork of the first low-rank network layer to obtain the first r-dimensional dimension reduction vector, and then the first r-dimensional dimension reduction vector is input into the second low-rank subnetwork of the first low-rank network layer to obtain the first n-dimensional output vector; since i traverses 1 to N, i becomes 2 at this time, and vector fusion can be performed according to the first n-dimensional output vectors of multiple dimensions and the first shared vector output by the first shared network layer, and the fused vector is determined as the input data of the second low-rank network layer, and the input data is input into the first low-rank subnetwork of the second low-rank network layer to obtain the second r-dimensional dimension reduction vector, and then the second r-dimensional dimension reduction vector is input into the second low-rank subnetwork of the second low-rank network layer to obtain the second n-dimensional output vector; ..., until the Nth n-dimensional output vector is obtained.

[0171] Correspondingly, step S150 may estimate the information to be estimated according to the following step S152 to obtain an estimated result.

[0172] Step S152: Estimate the information to be estimated according to the N-th n-dimensional output vector to obtain an estimation result.

[0173] When each low-rank network layer corresponds to a shared network layer, the output of the previous shared network layer can be used as the input of the next shared network layer.

[0174] This embodiment can improve the expressiveness and nonlinearity of the estimation model by setting up multiple layers of low-rank network layers and multiple layers of shared network layers. Through layer-by-layer learning and feature combination, the model can capture the complex patterns and features in the data, making the estimation model more accurate.

[0175] In one implementation, each of the low-rank networks is provided with a corresponding weight parameter table, and the weight parameter table records each weight parameter of the corresponding low-rank network.

[0176] Before step S131, the above method may further include the following step S120a.

[0177] Step S120a: Obtain a target weight parameter table corresponding to the target low-rank network, and determine the respective weight parameters of the first low-rank sub-network and the second low-rank sub-network corresponding to the target low-rank network from the target weight parameter table.

[0178] In this implementation manner, each low-rank network can be a network represented by unknown parameters, and the weight parameters of each low-rank network are stored in a weight parameter table. The weight parameters of each low-rank network can be efficiently stored and obtained by table lookup, which is convenient for maintenance if parameter tuning and modification are performed later. After adding low-rank networks corresponding to new scenarios and dimensions, the corresponding weight parameter table can be added immediately, making the solution provided in this application more scalable.

[0179] Alternatively, the weight parameters of each low-rank network can also be pre-stored in the respective low-rank networks, that is, each low-rank network is a low-rank network with set parameters, which facilitates the rapid use of the low-rank network and improves the efficiency of model estimation.

[0180] Embodiment 2

[0181] The second embodiment of the present application also provides a recommendation method, which is applied to a server. The method can be applied to an electronic device, which is a server. The server can be a server of a recommendation system, for example, a server of an e-commerce website, a server of a reading platform, a server of a dating website, a server of a travel website, etc., but is not limited to this. The server can be a single server or a server cluster, which is not specifically limited in the present application.

[0182] like Figure 6 As shown, the recommended method provided in the second embodiment of the present application includes the following steps S210 to S230.

[0183] Step S210: Obtain user characteristic information of the user and information of content to be recommended.

[0184] The user characteristic information can refer to the relevant description in the first embodiment and will not be repeated here. The information of the content to be recommended is similar to the information of the content to be recommended in the first embodiment, for example, it may include identification information of the content to be recommended (specific information used to associate the product), the type of the content to be recommended (including clothing, electronic products, household items, e-books, etc.), the attributes of the content to be recommended (including price, brand, description information, corresponding pictures, etc.), the popularity of the content to be recommended (including product popularity, sales volume, click-through rate, etc.), etc. The content to be recommended can be a product, website, audio and video, application, article, etc., but is not limited to this.

[0185] The specific process of the server acquiring the user characteristic information and the information of the content to be recommended may refer to the process of acquiring the information to be estimated in the first embodiment, which will not be described in detail in this embodiment.

[0186] Step S220: taking the user characteristic information and the information of the content to be recommended as information to be estimated, and estimating a parameter of the possibility that the user will operate on the content to be recommended by using the estimation method described in any one of the first embodiments.

[0187] For example, it is possible to estimate the click-through rate or conversion rate (i.e., purchase rate) of the user for the recommended content, or estimate the probability that the user will watch the recommended video for more than a preset time, estimate the probability that the user will read the recommended article completely, etc. The possibility parameter may also include an identifier for whether the user operates the recommended content, which may be a binary identifier, for example, 1 indicates that the user will operate the recommended content, and 0 indicates that the user will not operate the recommended content. Those skilled in the art may flexibly set the representation method of the estimation result according to the actual application scenario, and this application does not specifically limit it.

[0188] Step S230: recommending content to the user based on the likelihood value of the user operating the recommended product.

[0189] The server can recommend the content to be recommended to the user when the probability value of the user operating the recommended product is greater than a preset threshold, or recommend the content to be recommended to the user when the identifier evaluated for the content to be recommended is an identifier that the user will operate the content to be recommended.

[0190] The server recommends the content to be recommended to the user, that is, the server sends the content to be recommended to the client, so that the client displays the received content to be recommended.

[0191] This embodiment applies the estimation method of the first embodiment through the application scenario of content recommendation. The relevant content and beneficial effects can refer to the first embodiment, and the similarities will not be described in detail in this embodiment.

[0192] Embodiment 2

[0193] The third embodiment of the present application also provides a method for training an estimation model, wherein the method is performed by an electronic device, which may be a desktop computer, a laptop computer, a server, a mobile terminal, a gateway or other electronic device with data processing capabilities. Figure 7 As shown, the training method of the estimation model provided in this embodiment includes the following steps S310 to S370.

[0194] Step S310: Obtain a model to be trained, where the model to be trained includes an embedding layer and a low-rank network layer, where the low-rank network layer includes a low-rank network, and where the low-rank network includes a first low-rank sub-network and a second low-rank sub-network that are sequentially arranged.

[0195] The model structure of the model to be trained is the same as the model structure of the estimation model in the first embodiment. The specific structure refers to the relevant description of the first embodiment and will not be repeated here.

[0196] The initial parameters of each network in the model to be trained can be pre-set values ​​and continuously optimized in the subsequent training process. In the embodiment of the present application, the initial weight parameters of the first low-rank sub-network can satisfy the normal distribution N(0,σ 2 ), the initial weight parameter of the second low-rank sub-network can be 0, or can be other initialized parameters, which is not specifically limited in this application.

[0197] Step S320: Obtain training samples.

[0198] The training samples can be various historical data, and the training samples can include the annotation results corresponding to the sample content (such as sample products) (i.e., the labels of the sample content operated by the sample user), the user characteristic information of the sample user, the content information of the sample content, etc. The training samples can include samples of different dimensions, and the samples of each dimension include samples corresponding to different scenarios. For example, the training samples can include samples of the sample user dimension and samples of the product content dimension. Samples of different dimensions correspond to samples of different scenarios. For example, the samples of the user dimension include samples of Chinese users and samples of German users, and the samples of the product content dimension include samples of electronic products, samples of daily necessities, etc. The specific dimensional division and scene division can be flexibly set by those skilled in the art.

[0199] Step S330: input the training sample into the embedding layer of the model to be trained to obtain an m-dimensional sample vector corresponding to the training sample, where m is a preset input dimension value.

[0200] The execution process of step S330 is similar to the execution process of step S120 in the first embodiment, and will not be described in detail here.

[0201] Step S340: Input the sample vector into the first low-rank sub-network so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional sample dimensionality reduction vector, where r is less than m, and the first low-rank sub-network is used to convert the m-dimensional vector into an r-dimensional vector.

[0202] Step S350: Input the sample dimension reduction vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional sample output vector, where n is a preset output dimension value, n is greater than r, and the second low-rank sub-network is used to convert the r-dimensional vector into an n-dimensional vector.

[0203] Step S360: Estimate the training sample according to the sample output vector to obtain a sample estimation result.

[0204] The execution process of step S330 to step S360 is similar to the execution process of step S120 to step S150 of the first embodiment, and will not be described in detail here.

[0205] Step S370: According to the sample estimation result and the labeling result corresponding to the training sample, the model parameters of each layer of the to-be-trained model are adjusted to obtain a trained estimation model.

[0206] In an embodiment of the present application, the model parameters of each layer of the model to be trained can be adjusted using a supervised learning algorithm. Specifically, the electronic device can stop training and obtain a trained estimation model when the accuracy of the sample estimation result and the annotation result corresponding to the training sample is higher than a preset accuracy.

[0207] When adjusting the model parameters, step S370 adjusts and optimizes the various parameters included in the training model. Specifically, the parameters in the first low-rank subnetwork and the second low-rank subnetwork, the shared network layer, the output layer, the embedding layer and other network layers included in the low-rank network layer can be adjusted to obtain parameters that meet the optimization conditions.

[0208] Optionally, the first low-rank sub-network of the model to be trained is an m×r weight matrix, and the second low-rank sub-network is an r×n weight matrix.

[0209] Optionally, the low-rank network layer includes a plurality of the low-rank networks, each of the low-rank networks is used to perform calculations on information of different dimensions corresponding to different scenarios, and the training samples include training samples corresponding to different scenarios;

[0210] Step S340 may input the sample vector into the first low-rank sub-network according to the following step S341.

[0211] Step S341: input the sample vectors corresponding to the training samples of each scene into the first low-rank subnetwork of the corresponding target low-rank network, where the target low-rank network is a low-rank network corresponding to the scene corresponding to the training sample among the multiple low-rank networks.

[0212] Step S350 can input the sample dimension reduction vector into the second low-rank sub-network according to the following step S351.

[0213] Step S351: Input the sample dimension reduction vector into the corresponding second low-rank sub-network of the target low-rank network.

[0214] In this embodiment, the sample vector is input into a low-rank network corresponding to the scene to which it belongs for calculation and processing.

[0215] In one implementation, the model to be trained may further include a shared network layer, and the shared network layer may be an m×n weight matrix.

[0216] Before step S360, the above estimation method may further include the following step S360a.

[0217] Step S360a: input the sample vector into the shared network layer so that the shared network layer outputs an n-dimensional sample shared vector, wherein the shared network layer is used to convert an m-dimensional vector into an n-dimensional vector.

[0218] Step S360 can be implemented according to the following step S361.

[0219] Step S361: Estimate the training sample according to the sample output vector and the sample sharing vector to obtain a sample estimation result.

[0220] In one implementation, the low-rank network layer may include low-rank network groups of various dimensions corresponding to data of different dimensions, and each of the low-rank network groups includes each of the low-rank networks for processing information corresponding to different scenarios;

[0221] The target low-rank network corresponding to the sample vector is a low-rank network corresponding to the scene corresponding to the training sample corresponding to the sample vector, among the low-rank networks included in each of the low-rank network groups.

[0222] In one implementation, the training samples include samples of multiple dimensions, and the sample output vectors include sample output vectors of multiple dimensions.

[0223] Step S361 can be implemented by the following steps: vector multiplying sample output vectors of multiple dimensions to obtain a multiplied sample vector, and estimating the training sample according to the sum of the multiplied sample vector and the sample shared vector to obtain a sample estimation result.

[0224] In a specific embodiment, the model to be trained may further include an output layer;

[0225] The method of estimating the training sample according to the sum of the multiplied sample vector and the sample shared vector to obtain a sample estimation result includes: inputting the sum of the multiplied sample vector and the sample shared vector into the output layer to obtain a sample estimation result, wherein the sample estimation result is a score corresponding to the training sample.

[0226] In a specific embodiment, the model to be trained includes multiple layers of the low-rank network layers arranged in sequence;

[0227] The step of inputting the sample vector into the first low-rank sub-network comprises:

[0228] Determine the input data of the i-th low-rank network layer according to the i-1th n-dimensional sample output vector, and input the input data into the first low-rank subnetwork of the i-th low-rank network layer to obtain the i-th r-dimensional sample dimension reduction vector, wherein the 0th n-dimensional sample output vector is the sample vector;

[0229] The step of reducing the sample dimension into a vector and inputting it into the second low-rank sub-network comprises:

[0230] Input the i-th r-dimensional sample dimension reduction vector into the second low-rank subnetwork of the i-th low-rank network layer to obtain the i-th n-dimensional sample output vector;

[0231] Among them, i traverses 1 to N, and N is the number of low-rank network layers included in the model to be trained.

[0232] The estimating the training sample according to the sample output vector to obtain a sample estimation result includes: estimating the training sample according to the Nth n-dimensional sample output vector to obtain an estimation result.

[0233] This embodiment introduces the training process of the estimation model from the perspective of model training. The processing process of the training samples by the to-be-trained model during the training process is similar to the processing process of the estimation model on the estimation information during the estimation process. Therefore, the relevant content and beneficial effects of the specific implementation process of this embodiment may refer to the first embodiment, and the similarities will not be described in detail in this embodiment.

[0234] The fourth embodiment of the present application also provides an estimation device based on an estimation model corresponding to the estimation method embodiment based on an estimation model provided in the first embodiment, and the estimation device based on the estimation model is applied to a server, and an estimation model is deployed on the server, and the estimation model includes an embedding layer and a low-rank network layer, and the low-rank network layer includes a low-rank network, and the low-rank network includes a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence. Since the device embodiment is basically similar to the method embodiment, the description is relatively simple. For the details of the relevant technical features and the effects achieved, please refer to the corresponding description of the estimation method embodiment based on the estimation model provided above. The estimation device based on the estimation model provided in this embodiment includes:

[0235] An acquisition unit, used for acquiring information to be estimated;

[0236] A model operation unit is used to input the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, wherein m is a preset input dimension value; input the vector to be estimated into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional reduced-dimensional vector, r is less than m, and the rank of the first low-rank sub-network is not greater than r; input the reduced-dimensional vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector, n is a preset output dimension value, n is greater than r, and the rank of the second low-rank sub-network is not greater than r; estimate the information to be estimated according to the output vector to obtain an estimation result.

[0237] Optionally, the first low-rank sub-network is an m×r weight matrix, and the second low-rank sub-network is an r×n weight matrix.

[0238] Optionally, the low-rank network layer includes a plurality of the low-rank networks, and each of the low-rank networks is used to perform calculations on information corresponding to different scenarios;

[0239] The model operation unit is specifically used to: input the vector to be estimated into the first low-rank sub-network of the target low-rank network, and the target low-rank network is a low-rank network corresponding to the scene corresponding to the information to be estimated among the multiple low-rank networks; and input the dimensionality reduction vector into the second low-rank sub-network of the target low-rank network.

[0240] Optionally, the estimation model further includes a shared network layer;

[0241] The model operation unit is further used to input the vector to be estimated into the shared network layer, so that the shared network layer outputs an n-dimensional shared vector, wherein the shared network layer is used to convert an m-dimensional vector into an n-dimensional vector;

[0242] The model operation unit is specifically used to estimate the information to be estimated according to the output vector and the shared vector to obtain an estimation result.

[0243] Optionally, the low-rank network layer includes low-rank network groups of various dimensions corresponding to data of different dimensions, and each of the low-rank network groups includes each of the low-rank networks for processing information corresponding to different scenarios;

[0244] The target low-rank network is a low-rank network corresponding to the scenario corresponding to the information to be estimated among the low-rank networks included in each of the low-rank network groups.

[0245] Optionally, the information to be estimated includes information of multiple dimensions, and the output vector includes corresponding output vectors of multiple dimensions;

[0246] The model operation unit is specifically used to: perform vector multiplication on the output vectors of the multiple dimensions to obtain a multiplied vector; and estimate the information to be estimated according to the sum of the multiplied vector and the shared vector to obtain an estimation result.

[0247] Optionally, the estimation model further includes an output layer;

[0248] The model operation unit is specifically used to: input the sum of the multiplied vector and the shared vector into the output layer to obtain an estimated result, where the estimated result is a score corresponding to the information to be estimated.

[0249] Optionally, the shared network layer is an m×n weight matrix.

[0250] Optionally, the estimation model includes multiple layers of the low-rank network layers arranged in sequence;

[0251] The model operation unit is specifically used to: determine the input data of the i-th low-rank network layer according to the i-1th n-dimensional output vector, and input the input data into the first low-rank subnetwork of the i-th low-rank network layer to obtain the i-th r-dimensional dimensionality reduction vector, wherein the 0th n-dimensional output vector is the vector to be estimated; input the i-th r-dimensional dimensionality reduction vector into the second low-rank subnetwork of the i-th low-rank network layer to obtain the i-th n-dimensional output vector; wherein i traverses 1 to N, and N is the number of low-rank network layers included in the estimation model; estimate the information to be estimated according to the N-th n-dimensional output vector to obtain an estimation result.

[0252] Optionally, the information to be estimated includes: content information to be recommended and user characteristic information, and the estimation result is a probability value of the user operating the content information to be recommended.

[0253] Optionally, each of the low-rank networks is respectively provided with a corresponding weight parameter table, and each weight parameter of the corresponding low-rank network is recorded in the weight parameter table;

[0254] The model operation unit is used to obtain a target weight parameter table corresponding to the target low-rank network, and determine the respective weight parameters of the first low-rank sub-network and the second low-rank sub-network corresponding to the target low-rank network from the target weight parameter table.

[0255] The fifth embodiment of the present application also provides an electronic device embodiment corresponding to the estimation method based on the estimation model provided in the first embodiment, the electronic device is a server, the estimation model is deployed on the server, the estimation model includes an embedding layer and a low-rank network layer, the low-rank network layer includes a low-rank network, and the low-rank network includes a first low-rank sub-network and a second low-rank sub-network arranged in sequence; the following description of the electronic device embodiment is only exemplary. The electronic device embodiment is as follows:

[0256] Please refer to Figure 8 Understand the above electronic devices, Figure 8 The electronic device provided in this embodiment includes: a processor 1001, a memory 1002, a communication bus 1003, and a communication interface 1004;

[0257] The memory 1002 is used to store computer instructions for data processing. When the computer instructions are read and executed by the processor 1001, the following steps are performed:

[0258] Obtain information to be estimated;

[0259] Inputting the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, where m is a preset input dimension value;

[0260] Inputting the vector to be estimated into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector into an r-dimensional reduced-dimensional vector, where r is less than m, and the rank of the first low-rank sub-network is not greater than r;

[0261] Inputting the dimension-reduced vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector, where n is a preset output dimension value, n is greater than r, and the rank of the second low-rank sub-network is not greater than r;

[0262] The information to be estimated is estimated according to the output vector to obtain an estimation result.

[0263] The sixth embodiment of the present application also provides an electronic device embodiment corresponding to the recommendation method provided in the second embodiment, the electronic device is a server, and the electronic device embodiment is as follows:

[0264] The electronic device provided in this embodiment includes: a processor, a memory, a communication bus, and a communication interface;

[0265] The memory is used to store computer instructions for data processing, and when the computer instructions are read and executed by the processor, the following steps are performed:

[0266] Obtaining user feature information and information about content to be recommended;

[0267] The user characteristic information and the information of the content to be recommended are used as information to be estimated, and a probability parameter of the user operating the content to be recommended is estimated by using the estimation method described in any one of the first embodiments;

[0268] The content to be recommended is recommended to the user according to the possibility parameter.

[0269] The seventh embodiment of the present application also provides an electronic device embodiment corresponding to the training method provided in the third embodiment, and the electronic device embodiment is as follows:

[0270] The electronic device provided in this embodiment includes: a processor, a memory, a communication bus, and a communication interface;

[0271] The memory is used to store computer instructions for data processing, and when the computer instructions are read and executed by the processor, the following steps are performed:

[0272] Obtain a model to be trained, the model to be trained comprising an embedding layer and a low-rank network layer, the low-rank network layer comprising a low-rank network, and the low-rank network comprising a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence;

[0273] Get training samples;

[0274] Input the training sample into the embedding layer of the model to be trained to obtain an m-dimensional sample vector corresponding to the training sample, where m is a preset input dimension value;

[0275] Inputting the sample vector into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional sample dimension reduction vector, where r is less than m, and the first low-rank sub-network is used to convert the m-dimensional vector into an r-dimensional vector;

[0276] Inputting the sample dimension reduction vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional sample output vector, where n is a preset output dimension value, n is greater than r, and the second low-rank sub-network is used to convert the r-dimensional vector into an n-dimensional vector;

[0277] Predicting the training sample according to the sample output vector to obtain a sample prediction result;

[0278] According to the sample estimation results and the labeling results corresponding to the training samples, the model parameters of each layer of the to-be-trained model are adjusted to obtain a trained estimation model.

[0279] The eighth embodiment of the present application also provides a computer-readable storage medium for implementing the method described in the first embodiment. The computer-readable storage medium embodiment provided in the present application is described relatively simply. For the relevant parts, please refer to the corresponding description of the above method embodiment. The following described embodiment is only illustrative.

[0280] The computer-readable storage medium provided in this embodiment stores computer instructions, which can be applied to a server, and an estimation model is deployed on the server, wherein the estimation model includes an embedding layer and a low-rank network layer, wherein the low-rank network layer includes a low-rank network, and wherein the low-rank network includes a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence; when the instruction is executed by a processor, the following steps are implemented:

[0281] Obtain information to be estimated;

[0282] Inputting the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, where m is a preset input dimension value;

[0283] Inputting the vector to be estimated into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector into an r-dimensional reduced-dimensional vector, where r is less than m, and the rank of the first low-rank sub-network is not greater than r;

[0284] Inputting the dimension-reduced vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector, where n is a preset output dimension value, n is greater than r, and the rank of the second low-rank sub-network is not greater than r;

[0285] The information to be estimated is estimated according to the output vector to obtain an estimation result.

[0286] The ninth embodiment of the present application also provides a computer program product for implementing the method described in the first embodiment. The computer program product embodiment provided in this application is described relatively simply, and the relevant parts can refer to the corresponding description of the above method embodiment. The embodiment described below is only illustrative.

[0287] The computer program product provided in this embodiment includes a computer program, which, when executed by a processor, implements the following steps based on the estimation model:

[0288] Obtain information to be estimated;

[0289] Input the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, wherein m is a preset input dimension value; the estimation model includes an embedding layer and a low-rank network layer, the low-rank network layer includes a low-rank network, and the low-rank network includes a first low-rank sub-network and a second low-rank sub-network arranged in sequence;

[0290] Inputting the vector to be estimated into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector into an r-dimensional reduced-dimensional vector, where r is less than m, and the rank of the first low-rank sub-network is not greater than r;

[0291] Inputting the dimension-reduced vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector, where n is a preset output dimension value, n is greater than r, and the rank of the second low-rank sub-network is not greater than r;

[0292] The information to be estimated is estimated according to the output vector to obtain an estimation result.

[0293] The tenth embodiment of the present application further provides an estimation model, which includes:

[0294] The embedding layer is used to convert the input data into an m-dimensional vector, where m is the preset input dimension value;

[0295] A low-rank network layer, the low-rank network layer comprising a low-rank network, the low-rank network comprising a first low-rank sub-network and a second low-rank sub-network arranged in sequence, the first low-rank sub-network being used to convert an m-dimensional vector into an r-dimensional reduced-dimensional vector, the second low-rank sub-network being used to convert the r-dimensional reduced-dimensional vector into an n-dimensional output vector, where r is less than m, n is a preset output dimension value, and n is greater than r;

[0296] The output layer is used to obtain an estimated result for the input data according to the output vector of the second low-rank sub-network.

[0297] The various computing layers of the large language model provided in this embodiment have been described in detail in the first embodiment and will not be described in detail here.

[0298] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0299] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0300] 1. Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0301] 2. Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0302] Although the present application is disclosed as above in the form of a preferred embodiment, it is not intended to limit the present application. Any technical personnel in this field may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application.

Claims

1. An estimation method based on an estimation model, It is characterized in that The estimation model includes an embedding layer and a low-rank network layer, the low-rank network layer includes a low-rank network, the low-rank network includes a first low-rank sub-network and a second low-rank sub-network arranged in sequence, and the method includes: Obtain information to be estimated; Inputting the information to be estimated into the embedding layer of the estimation model to obtain an m-dimensional vector to be estimated corresponding to the information to be estimated, where m is a preset input dimension value; Inputting the vector to be estimated into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector into an r-dimensional reduced-dimensional vector, where r is less than m, and the rank of the first low-rank sub-network is not greater than r; Inputting the dimension-reduced vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional output vector, where n is a preset output dimension value, n is greater than r, and the rank of the second low-rank sub-network is not greater than r; The information to be estimated is estimated according to the output vector to obtain an estimation result.

2. The estimation method according to claim 1, It is characterized in that The first low-rank sub-network has an m×r weight matrix, and the second low-rank sub-network has an r×n weight matrix.

3. The estimation method according to claim 1, It is characterized in that The low-rank network layer includes a plurality of the low-rank networks, each of which is used to calculate and process information corresponding to different scenarios; The step of inputting the vector to be estimated into the first low-rank sub-network comprises: Inputting the vector to be estimated into a first low-rank subnetwork of a target low-rank network, wherein the target low-rank network is a low-rank network corresponding to the scene corresponding to the information to be estimated among the multiple low-rank networks; The step of inputting the dimension-reduced vector into the second low-rank sub-network comprises: The dimensionally reduced vector is input into a second low-rank sub-network of the target low-rank network.

4. The estimation method according to claim 3, It is characterized in that The estimation model also includes a shared network layer; Before estimating the information to be estimated according to the output vector, the method further includes: Inputting the vector to be estimated into the shared network layer so that the shared network layer outputs an n-dimensional shared vector, wherein the shared network layer is used to convert an m-dimensional vector into an n-dimensional vector; The estimating the information to be estimated according to the output vector to obtain an estimation result includes: The information to be estimated is estimated according to the output vector and the shared vector to obtain an estimation result.

5. The estimation method according to claim 4, It is characterized in that The low-rank network layer includes low-rank network groups of various dimensions corresponding to data of different dimensions, and each of the low-rank network groups includes each of the low-rank networks for processing information corresponding to different scenarios; The target low-rank network is a low-rank network corresponding to the scenario corresponding to the information to be estimated among the low-rank networks included in each of the low-rank network groups.

6. The estimation method according to claim 5, It is characterized in that The information to be estimated includes information of multiple dimensions, and the output vector includes corresponding output vectors of multiple dimensions; The estimating the information to be estimated according to the output vector and the shared vector to obtain an estimation result includes: Performing vector multiplication on the output vectors of the multiple dimensions to obtain a multiplied vector; The information to be estimated is estimated according to the sum of the multiplied vector and the shared vector to obtain an estimation result.

7. The estimation method according to claim 6, It is characterized in that The estimation model also includes an output layer; The estimating the information to be estimated according to the sum of the multiplied vector and the shared vector to obtain an estimation result includes: The sum of the multiplied vector and the shared vector is input into the output layer to obtain an estimated result, where the estimated result is a score corresponding to the information to be estimated.

8. The estimation method according to claim 4, It is characterized in that The shared network layer is an m×n weight matrix.

9. The estimation method according to claim 1, It is characterized in that The estimation model includes multiple layers of low-rank network layers arranged in sequence; The step of inputting the vector to be estimated into the first low-rank sub-network comprises: Determine the input data of the i-th low-rank network layer according to the i-1th n-dimensional output vector, and input the input data into the first low-rank subnetwork of the i-th low-rank network layer to obtain an i-th r-dimensional dimensionality reduction vector, wherein the 0th n-dimensional output vector is the vector to be estimated; The step of inputting the dimension-reduced vector into the second low-rank sub-network comprises: Inputting the i-th r-dimensional dimension reduction vector into the second low-rank subnetwork of the i-th low-rank network layer to obtain an i-th n-dimensional output vector; Wherein, i traverses 1 to N, and N is the number of low-rank network layers included in the estimation model; The estimating the information to be estimated according to the output vector to obtain an estimation result includes: The information to be estimated is estimated according to the N-th n-dimensional output vector to obtain an estimation result.

10. The estimation method according to claim 1, It is characterized in that The information to be estimated includes: content information to be recommended and user characteristic information, and the estimation result is a probability value of the user operating the content information to be recommended.

11. The estimation method according to claim 3, It is characterized in that Each of the low-rank networks is respectively provided with a corresponding weight parameter table, and each weight parameter of the corresponding low-rank network is recorded in the weight parameter table; Before inputting the vector to be estimated into the first low-rank sub-network of the target low-rank network, the method further includes: Obtain a target weight parameter table corresponding to the target low-rank network, and determine each weight parameter of the first low-rank sub-network and the second low-rank sub-network corresponding to the target low-rank network from the target weight parameter table.

12. A recommendation method, It is characterized in that The method comprises: Obtaining user feature information and information about content to be recommended; Using the user feature information and the information of the content to be recommended as information to be estimated, and estimating a parameter of the possibility of the user operating the content to be recommended by the estimation method according to any one of claims 1 to 11; The content to be recommended is recommended to the user according to the possibility parameter.

13. A training method for an estimation model, It is characterized in that The method comprises: Obtain a model to be trained, the model to be trained comprising an embedding layer and a low-rank network layer, the low-rank network layer comprising a low-rank network, and the low-rank network comprising a first low-rank subnetwork and a second low-rank subnetwork arranged in sequence; Get training samples; Input the training sample into the embedding layer of the model to be trained to obtain an m-dimensional sample vector corresponding to the training sample, where m is a preset input dimension value; Inputting the sample vector into the first low-rank sub-network, so that the first low-rank sub-network reduces the dimension of the input vector to an r-dimensional sample dimension reduction vector, where r is less than m, and the first low-rank sub-network is used to convert the m-dimensional vector into an r-dimensional vector; Inputting the sample dimension reduction vector into the second low-rank sub-network, so that the second low-rank sub-network increases the dimension of the input vector to an n-dimensional sample output vector, where n is a preset output dimension value, n is greater than r, and the second low-rank sub-network is used to convert the r-dimensional vector into an n-dimensional vector; Predicting the training sample according to the sample output vector to obtain a sample prediction result; According to the sample estimation results and the labeling results corresponding to the training samples, the model parameters of each layer of the to-be-trained model are adjusted to obtain a trained estimation model.

14. The training method according to claim 13, It is characterized in that The first low-rank sub-network has an m×r weight matrix, and the second low-rank sub-network has an r×n weight matrix.

15. The training method according to claim 10, It is characterized in that The low-rank network layer includes a plurality of the low-rank networks, each of the low-rank networks is used to calculate and process information of different dimensions corresponding to different scenarios, and the training samples include training samples corresponding to different scenarios; The step of inputting the sample vector into the first low-rank sub-network comprises: Inputting each sample vector corresponding to the training sample of each scene into the first low-rank sub-network of the corresponding target low-rank network, respectively, wherein the target low-rank network is a low-rank network corresponding to the scene corresponding to the training sample among the multiple low-rank networks; The step of reducing the sample dimension into a vector and inputting it into the second low-rank sub-network comprises: The sample dimension reduction vector is input into the second low-rank sub-network of the target low-rank network corresponding to the vector.

16. The training method according to claim 15, It is characterized in that The model to be trained also includes a shared network layer; Before estimating the training sample according to the sample output vector, the method further includes: Inputting the sample vector into the shared network layer so that the shared network layer outputs an n-dimensional sample shared vector, wherein the shared network layer is used to convert an m-dimensional vector into an n-dimensional vector; The step of estimating the training sample according to the sample output vector to obtain a sample estimation result includes: The training sample is estimated according to the sample output vector and the sample sharing vector to obtain a sample estimation result.

17. A prediction model, It is characterized in that include: The embedding layer is used to convert the input data into an m-dimensional vector, where m is the preset input dimension value; A low-rank network layer, the low-rank network layer comprising a low-rank network, the low-rank network comprising a first low-rank sub-network and a second low-rank sub-network arranged in sequence, the first low-rank sub-network being used to convert an m-dimensional vector into an r-dimensional reduced-dimensional vector, the second low-rank sub-network being used to convert the r-dimensional reduced-dimensional vector into an n-dimensional output vector, where r is less than m, n is a preset output dimension value, and n is greater than r; The output layer is used to obtain an estimated result for the input data according to the output vector of the second low-rank sub-network.

18. An electronic device, It is characterized in that include: A processor, a memory, and computer program instructions stored on the memory and executable on the processor; When the processor executes the computer program instructions, the method according to any one of claims 1 to 16 is implemented.

19. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method described in any one of claims 1 to 16 when executed by a processor.

Citation Information

Cited By

  • Text processing method and training method based on hierarchical vector approximation

    CN122021656A