Ranking method, device and equipment of recommendation system and readable storage medium
By using multi-scale local neighborhood nonlinear aggregation and convolutional neural network processing, the problem of inaccurate ranking caused by inductive bias in recommendation systems is solved, and ranking results that better meet user needs are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- IFLYTEK CO LTD
- Filing Date
- 2022-12-05
- Publication Date
- 2026-08-04
AI Technical Summary
In existing recommendation system ranking methods, inductive bias caused by the crossover of first- or second-order linear combinations leads to ranking results that do not meet user needs.
A multi-scale local neighborhood nonlinear aggregation technique is adopted. By constructing feature data pairs, multi-scale local neighborhood nonlinear aggregation is performed, and a convolutional neural network is used to process the feature data to obtain confidence scores and sort them.
This avoids inductive bias and improves the accuracy of the sorting results and their relevance to user needs.
Smart Images
Figure CN115757973B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of recommender system technology, and more specifically, to a ranking method, apparatus, device, and readable storage medium for a recommender system. Background Technology
[0002] With the continuous development of social technology and the rapid growth of various industries, a massive amount of information is generated daily. Recommendation systems can provide personalized recommendations for users, allowing them to quickly obtain truly useful information from a vast amount of data. Recommendation systems can be broadly divided into two stages: recall and ranking. The recall stage selects information from a large pool of data that the user is highly likely to be interested in. The ranking stage sorts the recalled information according to the user's potential interest, from highest to lowest, and then presents it to the user.
[0003] The interaction between user features and the features of the recommended object influences the probability that a user prefers a particular recommended object (for example, users of the same age but different genders may prefer different phone styles, and users of the same gender but different age groups may also prefer different phone styles). Currently, the common ranking method in recommendation systems is to first perform a first- or second-order linear combination interaction between user features and the features of the recommended object, and then process the resulting linear combination features to obtain the ranking result. However, this method of performing a first- or second-order linear combination interaction between user features and the features of the recommended object has a strong inductive bias, which may lead to ranking results that do not meet user needs.
[0004] Therefore, how to provide a ranking method for recommendation systems that avoids ranking results that do not meet user needs due to strong inductive bias has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention
[0005] In view of the above problems, this application proposes a ranking method, apparatus, device, and readable storage medium for a recommender system. The specific solution is as follows:
[0006] A ranking method for a recommendation system, the method comprising:
[0007] The feature data is determined, including the user's feature data and the feature data of multiple objects to be recommended corresponding to the user;
[0008] Based on the feature data, feature data pairs are constructed. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended.
[0009] For each feature data pair, multi-scale local neighborhood nonlinear aggregation is performed on the feature data pair to obtain aggregated feature data, and the aggregated feature data is processed to obtain the confidence score of the feature data pair.
[0010] The multiple objects to be recommended are sorted based on the confidence scores of each feature data pair.
[0011] Optionally, the determined feature data includes:
[0012] Acquire profile data, which includes the user's profile data and the profile data of multiple objects to be recommended corresponding to the user; the profile data includes continuous data, single-valued discrete data, and multi-valued discrete data;
[0013] The continuous data in the portrait data is subjected to long-tail removal processing to obtain long-tail removed continuous data, and the long-tail removed continuous data is normalized to obtain normalized continuous data.
[0014] The normalized continuous data, the single-valued discrete data, and the multi-valued discrete data are respectively standardized to obtain the feature data.
[0015] Optionally, the step of removing long-tailed data from the continuous data in the portrait data to obtain long-tailed continuous data includes:
[0016] The continuous data in the portrait data is processed by taking the nth power or the logarithm to obtain the long-tailed continuous data, where n is an integer greater than or equal to 2.
[0017] Optionally, the step of performing multi-scale local neighborhood nonlinear aggregation on the feature data pairs to obtain aggregated feature data, and processing the aggregated feature data to obtain the confidence score of the feature data pairs, includes:
[0018] The feature data pairs are input into a pre-trained ranking model. The ranking model performs multi-scale local neighborhood nonlinear aggregation on the feature data pairs to obtain aggregated feature data, and processes the aggregated feature data to obtain the confidence score of the feature data pairs. The ranking model is trained using the training feature data pairs as training samples and the labels of the training feature data pairs as sample labels.
[0019] Optionally, the ranking model includes a feature splicing module, a multi-scale local neighborhood nonlinear aggregation module, and a confidence scoring module;
[0020] The feature splicing module splices the feature data of the user in the feature data pair and the feature data of the object to be recommended to obtain the spliced feature data;
[0021] The multi-scale local neighborhood nonlinear aggregation module performs multi-scale local neighborhood nonlinear aggregation on the spliced feature data to obtain aggregated feature data.
[0022] The confidence scoring module scores the aggregated feature data to obtain the confidence score of the feature data pair.
[0023] Optionally, the multi-scale local neighborhood nonlinear aggregation module includes: a plurality of multi-scale random combination convolutional layers cascaded in sequence;
[0024] Each multi-scale random combination convolutional layer includes multiple convolutional kernels. Among these kernels, the minimum scale is 2, and the scales of the remaining kernels are determined by selecting them at equal intervals from a preset scale list. Each scale in the preset scale list is an odd number, with the minimum scale being 3 and the maximum scale being 2int(N / 4)-1, where N is the length of the concatenated feature data.
[0025] The depth of each multi-scale random combination convolutional layer is determined based on N.
[0026] Optionally, when each convolutional kernel traverses the feature data, it maintains normal convolution with a preset probability, and performs random convolution in other cases. The normal convolution is to slide the convolution window sequentially from the start position to the end position of the feature data, and the random convolution is to keep the central feature unchanged each time and randomly select other features from the features other than the central feature.
[0027] Optionally, the confidence scoring module includes a pooling layer and an activation layer, wherein the pooling layer is connected to the last multi-scale random combination convolutional layer, and the activation layer is connected to the pooling layer.
[0028] A ranking apparatus for a recommendation system, the apparatus comprising:
[0029] A feature data determination unit is used to determine feature data, which includes the user's feature data and the feature data of multiple objects to be recommended corresponding to the user;
[0030] The feature data pair construction unit is used to construct feature data pairs based on the feature data. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended.
[0031] The confidence score determination unit is used to perform multi-scale local neighborhood nonlinear aggregation on each feature data pair to obtain aggregated feature data, and to process the aggregated feature data to obtain the confidence score of the feature data pair.
[0032] The sorting unit is used to sort the multiple objects to be recommended based on the confidence scores of each feature data pair.
[0033] Optionally, the feature data determining unit includes:
[0034] A profile data acquisition unit is used to acquire profile data, which includes the user's profile data and the profile data of multiple objects to be recommended corresponding to the user; the profile data includes continuous data, single-value discrete data and multi-value discrete data.
[0035] The long-tail removal and normalization processing unit is used to perform long-tail removal processing on the continuous data in the portrait data to obtain long-tail removed continuous data, and to perform normalization processing on the long-tail removed continuous data to obtain normalized continuous data.
[0036] The standardization processing unit is used to standardize the normalized continuous data, the single-valued discrete data, and the multi-valued discrete data respectively to obtain the feature data.
[0037] Optionally, the long-tail removal and normalization processing unit is specifically used for:
[0038] The continuous data in the portrait data is processed by taking the nth power or the logarithm to obtain the long-tailed continuous data, where n is an integer greater than or equal to 2.
[0039] Optionally, the confidence score determination unit is specifically used for:
[0040] The feature data pairs are input into a pre-trained ranking model. The ranking model performs multi-scale local neighborhood nonlinear aggregation on the feature data pairs to obtain aggregated feature data, and processes the aggregated feature data to obtain the confidence score of the feature data pairs. The ranking model is trained using the training feature data pairs as training samples and the labels of the training feature data pairs as sample labels.
[0041] Optionally, the ranking model includes a feature splicing module, a multi-scale local neighborhood nonlinear aggregation module, and a confidence scoring module;
[0042] The feature splicing module splices the feature data of the user in the feature data pair and the feature data of the object to be recommended to obtain the spliced feature data;
[0043] The multi-scale local neighborhood nonlinear aggregation module performs multi-scale local neighborhood nonlinear aggregation on the spliced feature data to obtain aggregated feature data.
[0044] The confidence scoring module scores the aggregated feature data to obtain the confidence score of the feature data pair.
[0045] Optionally, the multi-scale local neighborhood nonlinear aggregation module includes: a plurality of multi-scale random combination convolutional layers cascaded in sequence;
[0046] Each multi-scale random combination convolutional layer includes multiple convolutional kernels. Among these kernels, the minimum scale is 2, and the scales of the remaining kernels are determined by selecting them at equal intervals from a preset scale list. Each scale in the preset scale list is an odd number, with the minimum scale being 3 and the maximum scale being 2int(N / 4)-1, where N is the length of the concatenated feature data.
[0047] The depth of each multi-scale random combination convolutional layer is determined based on N.
[0048] Optionally, when each convolutional kernel traverses the feature data, it maintains normal convolution with a preset probability, and performs random convolution in other cases. The normal convolution is to slide the convolution window sequentially from the start position to the end position of the feature data, and the random convolution is to keep the central feature unchanged each time and randomly select other features from the features other than the central feature.
[0049] Optionally, the confidence scoring module includes a pooling layer and an activation layer, wherein the pooling layer is connected to the last multi-scale random combination convolutional layer, and the activation layer is connected to the pooling layer.
[0050] A ranking device for a recommendation system includes a memory and a processor;
[0051] The memory is used to store programs;
[0052] The processor is used to execute the program to implement the various steps of the ranking method of the recommendation system as described above.
[0053] A readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the various steps of the ranking method of the recommendation system as described above.
[0054] By employing the above technical solution, this application discloses a ranking method, apparatus, device, and readable storage medium for a recommendation system. In this solution, firstly, the user's feature data and the feature data of multiple objects to be recommended corresponding to the user are determined; then, feature data pairs are constructed, each pair including the user's feature data and the feature data of one object to be recommended, with the number of feature data pairs being the same as the number of objects to be recommended; next, for each feature data pair, multi-scale local neighborhood nonlinear aggregation is performed to obtain aggregated feature data, and the aggregated feature data is processed to obtain a confidence score for the feature data pair; finally, the multiple objects to be recommended are ranked based on the confidence scores of each feature data pair. In this solution, the multi-scale local neighborhood nonlinear aggregation method for each feature data pair does not have a strong inductive bias, thus avoiding situations where a strong inductive bias leads to ranking results that do not meet user needs. Attached Figure Description
[0055] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:
[0056] Figure 1 This is a flowchart illustrating a ranking method for a recommendation system disclosed in an embodiment of this application;
[0057] Figure 2 This is a flowchart illustrating a method for determining feature data disclosed in an embodiment of this application;
[0058] Figure 3 This is a schematic diagram illustrating the visualization of a user viewing time profile dataset disclosed in an embodiment of this application.
[0059] Figure 4 This is a schematic diagram illustrating a user viewing time profile dataset after square root removal and long tail removal, as disclosed in an embodiment of this application, through visualization.
[0060] Figure 5 This is a schematic diagram illustrating the visualization of a user viewing time profile dataset after logarithmic processing and long-tail removal, as disclosed in an embodiment of this application.
[0061] Figure 6 This is a schematic diagram of the structure of a sorting model disclosed in an embodiment of this application;
[0062] Figure 7 This is a schematic diagram illustrating the traversal of a convolutional kernel on feature data as disclosed in an embodiment of this application;
[0063] Figure 8 This is a schematic diagram of the sorting device structure of a recommendation system disclosed in an embodiment of this application;
[0064] Figure 9 This is a hardware structure block diagram of a ranking device for a recommendation system disclosed in an embodiment of this application. Detailed Implementation
[0065] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0066] To facilitate understanding of the technical solution of this application, the inventors of this case will first give a brief introduction to the ranking algorithms of recommendation systems in the prior art.
[0067] With the development of machine learning technology, recommender systems generally rank the objects to be recommended using ranking models. Current ranking models (such as FiBiNet, DeepFM, and Wide & Deep models) typically consist of a feature cross-layer plus a fully connected layer. The feature cross-layer performs a first- or second-order linear combination of user features and features of the objects to be recommended, while the fully connected layer processes the resulting linearly combined features to obtain the ranking result. However, this method of performing a first- or second-order linear combination of user features and features of the objects to be recommended has a strong inductive bias, which may lead to ranking results that do not meet user needs.
[0068] In addition, current ranking models directly normalize and standardize the portrait data during the feature engineering stage when constructing feature data based on the portrait data. This results in the constructed feature data lacking good discriminative power, which in turn leads to poor robustness of the ranking model.
[0069] For example, some profile data exhibits a long-tail phenomenon, meaning that the values in this profile data are distributed according to a power-law distribution across the entire dataset. This means that a large number of data points are concentrated in smaller values, and the positive or negative relationship between the samples and these values is significantly positive. This results in an uneven distribution of positive and negative samples, with most positive samples concentrated in smaller values. For example, the distribution of a certain field's score and the proportion of positive samples is shown in the table below:
[0070] 0-497 80％ 498-1989 16％ 1990-4475 3％ 4476-7956 1％ 7957-12432 0％ 12433-17902 0％ 17903-24367 0％ 24368-31826 0％ 31827-40280 0％ 40281-50000 0％
[0071] As can be seen from the table, positive samples are mainly concentrated after 1989, accounting for 96%. If normalization is applied directly to the score of this field, 1989 / 50000 = 0.03978, meaning that a large number of positive samples are concentrated in a narrow interval [0, 0.03978]. Because the positive samples are concentrated in a narrow interval, the model needs to distinguish between positive and negative samples with a value of 0.03978. If the model fluctuates even slightly, it will affect the classification. Such a model is too sensitive and lacks robustness.
[0072] In view of the problems with existing ranking methods in recommender systems, the inventors of this case conducted in-depth research and finally proposed a ranking method for recommender systems.
[0073] The following embodiments will be used to describe the ranking method of the recommendation system provided in this application.
[0074] Reference Figure 1 , Figure 1 This is a flowchart illustrating a ranking method for a recommendation system disclosed in an embodiment of this application. The method may include:
[0075] Step S101: Determine feature data, which includes the user's feature data and the feature data of multiple objects to be recommended corresponding to the user.
[0076] In this application, the multiple objects to be recommended corresponding to the user can be obtained during the recall phase of the recommendation system. As one possible implementation, in this application, feature data can be determined based on profile data. Specifically, the user's feature data can be determined based on the user's profile data, and the feature data of multiple objects to be recommended can be determined based on the profile data of multiple objects to be recommended. Specific implementation methods will be described in detail through later embodiments and will not be elaborated here.
[0077] Step S102: Based on the feature data, construct feature data pairs, each feature data pair including the user's feature data and the feature data of the object to be recommended, the number of feature data pairs being the same as the number of objects to be recommended.
[0078] Step S103: For each feature data pair, perform multi-scale local neighborhood nonlinear aggregation on the feature data pair to obtain aggregated feature data, and process the aggregated feature data to obtain the confidence score of the feature data pair.
[0079] In this application, multi-scale local neighborhood nonlinear aggregation of the feature data pairs can be performed based on a convolutional neural network to obtain aggregated feature data, and the aggregated feature data can be processed to obtain the confidence score of the feature data pairs. The specific implementation will be described in detail through the following embodiments, and will not be described here.
[0080] Step S104: Sort the multiple objects to be recommended based on the confidence scores of each feature data pair.
[0081] In this application, the confidence score of the feature data pair is used to characterize the user's level of interest in the recommended object. The higher the confidence score, the higher the user's level of interest in the recommended object and the higher the ranking. The lower the confidence score, the lower the user's level of interest in the recommended object and the lower the ranking.
[0082] This embodiment discloses a ranking method for a recommendation system. The scheme first determines the user's feature data and the feature data of multiple objects to be recommended corresponding to the user; then, feature data pairs are constructed, each pair including the user's feature data and the feature data of one object to be recommended, with the number of feature data pairs being the same as the number of objects to be recommended; next, for each feature data pair, multi-scale local neighborhood nonlinear aggregation is performed to obtain aggregated feature data, and the aggregated feature data is processed to obtain a confidence score for the feature data pair; finally, the multiple objects to be recommended are ranked based on the confidence scores of each feature data pair. In this scheme, the multi-scale local neighborhood nonlinear aggregation method for each feature data pair does not have a strong inductive bias, thus avoiding situations where a strong inductive bias leads to ranking results that do not meet user needs.
[0083] In another embodiment of this application, the specific implementation of step S101, which determines the feature data, will be described.
[0084] Reference Figure 2 , Figure 2 This is a flowchart illustrating a method for determining feature data disclosed in an embodiment of this application. The method may include:
[0085] Step S201: Obtain profile data, which includes the user's profile data and the profile data of multiple objects to be recommended corresponding to the user; the profile data includes continuous data, single-valued discrete data and multi-valued discrete data.
[0086] In this application, profile data can be divided into short-term profile data and long-term profile data. User profile data can be constructed by combining the user's basic information and behavioral data, and profile data of multiple objects to be recommended corresponding to the user can be constructed by combining the basic information and behavioral data of multiple objects to be recommended.
[0087] It's important to note that continuous data refers to data with floating-point values, whose range is a continuous interval. Examples include image pixels, popularity, and online duration. Single-valued discrete data means that the field can only take one value, and the value range is an enumeration type; examples include gender and age. Multi-valued discrete data can have multiple values simultaneously, with no limit on the number, and the value range is the same enumeration type; examples include actors and product composition.
[0088] Step S202: Perform long-tail removal processing on the continuous data in the portrait data to obtain long-tail removed continuous data, and perform normalization processing on the long-tail removed continuous data to obtain normalized continuous data.
[0089] As one possible implementation, the continuous data in the portrait data can be processed by taking the nth power to obtain the long-tailed continuous data, where n is an integer greater than or equal to 2.
[0090] As another possible implementation, the continuous data in the portrait data can be logarithmically processed to obtain continuous data after removing the long tail.
[0091] To facilitate understanding, we will use continuous data such as user viewing time as an example to explain the long-tail removal process.
[0092] In most cases, only those videos viewed by users for around 10 minutes can be considered positive samples. Let's assume this user viewing time profile dataset is visualized as follows... Figure 3 As shown, the horizontal axis represents time (in 4.5 seconds for ease of display), and the vertical axis represents the number of users. Since most users are concentrated within 10 minutes, that is... Figure 1 If we directly use normalization operations, such as dividing by 1600, within the range of 0 to 133, 133 becomes 0.083, and it becomes less easy to find the boundary between positive and negative samples within the range of 0 to 0.083.
[0093] Assuming that the viewing time is raised to the second power (i.e., taken as a square root) to remove the long tail, the user viewing time profile dataset after square root removal can be visualized as follows: Figure 4 As shown, after removing the long tail, the square root of 133 is 11.53, and the square root of 1600 is 40. Then, after normalization, 133 becomes 0.2875. Now most positive samples are distributed between 0 and 0.2875, which is much easier to distinguish between positive and negative samples than 0.083.
[0094] Assuming that the viewing duration is processed by taking the natural logarithm (ln) to remove the long tail, the user viewing duration profile dataset after logarithmic long tail removal can be visualized as follows: Figure 5As shown, after removing the long tail, ln(133) is 4.89, while ln(1600) is 7.38. After normalization, a large number of positive samples are between 0 and 0.66, which makes the discrimination higher.
[0095] Step S203: Standardize the normalized continuous data, the single-valued discrete data, and the multi-valued discrete data respectively to obtain the feature data.
[0096] In this application, different standardization processes can be applied to different types of data.
[0097] As one possible implementation, normalized continuous data can be mapped to an E-dimensional space using a linear transformation, where the value of E can be determined based on the scenario requirements. For both single-valued and multi-valued discrete data, an embedding layer (with corresponding APIs in TensorFlow and PyTorch) can be used. Specifically, the corresponding encoding index can be used to retrieve the data from the corresponding weight matrix row, or the one-hot vector of the discrete value can be multiplied by the weight matrix.
[0098] As mentioned in the foregoing embodiments of this application, multi-scale local neighborhood nonlinear aggregation of the feature data pairs can be performed based on a convolutional neural network to obtain aggregated feature data. The aggregated feature data is then processed to obtain a confidence score for the feature data pairs. Specifically, a ranking model can be pre-constructed, using the training feature data pairs as training samples and the labels of the training feature data pairs as sample labels. The model is then trained. Based on the trained ranking model, the feature data pairs can be input into the pre-trained ranking model. The ranking model performs multi-scale local neighborhood nonlinear aggregation of the feature data pairs to obtain aggregated feature data, and the aggregated feature data is then processed to obtain a confidence score for the feature data pairs.
[0099] It should be noted that in this application, the following strategy can be adopted during the training sample generation process: First, a relatively large threshold and a relatively small threshold, such as 0.8, are set; samples with scores higher than 0.8 are defined as positive samples, and those with scores lower than 0.2 are defined as negative samples. Then, clustering is used to find the positive sample cluster centers and the negative sample cluster centers, and the remaining samples are filtered. Samples close to positive samples are classified as positive samples, and those close to negative samples are classified as negative samples. This approach serves to remove noise.
[0100] In another embodiment of this application, the structure of the sorting model will be described in detail.
[0101] Reference Figure 6 , Figure 6This is a schematic diagram of the structure of a ranking model disclosed in an embodiment of this application. The ranking model may include: a feature splicing module, a multi-scale local neighborhood nonlinear aggregation module, and a confidence scoring module.
[0102] The feature splicing module splices the feature data of the user in the feature data pair and the feature data of the object to be recommended to obtain the spliced feature data;
[0103] The multi-scale local neighborhood nonlinear aggregation module performs multi-scale local neighborhood nonlinear aggregation on the spliced feature data to obtain aggregated feature data.
[0104] The confidence scoring module scores the aggregated feature data to obtain the confidence score of the feature data pair.
[0105] As one possible implementation, the multi-scale local neighborhood nonlinear aggregation module includes: at least two multi-scale random combination convolutional layers cascaded in sequence;
[0106] Each multi-scale random combination convolutional layer includes multiple (typically 5 to 7, too many would be unsuitable for training and result in a bloated model) convolutional kernels. Among these kernels, the minimum scale is 2, because 1×1 is ineffective for feature aggregation and because no convolution operation is performed in the embedding layer dimension (experiments have shown it to be ineffective). The scales of the remaining kernels are determined by selecting them at equal intervals from a preset scale list. Each scale in the preset scale list is an odd number, with a minimum scale of 3 and a maximum scale of 2int(N / 4)-1, where N is the length of the concatenated feature data.
[0107] It should be noted that, theoretically, the maximum scale of the convolutional kernel is equal to the length of the concatenated feature data. However, if it equals the length of the concatenated feature data, then only one convolution can be performed to aggregate the features. This one-time aggregation of all features is not as effective as multi-layer aggregation (multi-layer concatenation with small receptive fields has comparable results to a large single layer, but the performance is excellent because multi-layer non-linear cascaded representation is stronger than a single layer). Therefore, in the ranking model of this application, to ensure at least two convolutional layers are used, the maximum size for the concatenated feature data of length N is int(N / 2). Furthermore, odd-numbered convolutional kernels are easier to pad, so odd numbers are always chosen.
[0108] In addition, the depth of each multi-scale random combination convolutional layer is determined based on N.
[0109] As one possible implementation, the depth of each multi-scale random combination convolutional layer needs to satisfy the formula Layers = int(logN+1) ± 1.
[0110] Each multi-scale randomized convolutional layer also includes a pooling layer (max pooling or average pooling), and the last multi-scale randomized convolutional layer uses adaptive pooling to 1×1 to connect the confidence scoring module.
[0111] It should be noted that convolution operations have a strong spatial inductive bias, as convolution is implemented by sequentially sliding the convolution window from the start to the end position on the feature data. However, in recommendation systems, there is no sequential relationship between features. To avoid this spatial inductive bias, in this application, each convolution kernel maintains normal convolution with a preset probability when traversing the feature data, and performs random convolution in other cases. The normal convolution is the sequential sliding of the convolution window from the start to the end position of the feature data, and the random convolution is that the central feature remains unchanged each time, and other features are randomly selected from the features other than the central feature.
[0112] For ease of understanding, let's assume the preset probability is α. Then, when each convolutional kernel traverses the feature data, it performs normal convolution with probability α and random convolution with probability 1-α, as shown below. Figure 7 As shown.
[0113] Additionally, it's important to note that in current neural network models and in the process of algorithm developers building neural networks, the logic is the same: convolutional layer + activation + pooling; this has become a common paradigm. However, from a mathematical perspective, as long as the activation function is monotonic, this structure can be transformed in the following way without affecting the result: convolutional layer + pooling + activation, i.e., the following equation holds true.
[0114] max(σ(g(X)))=σ(max(g(X)))
[0115] The proof of the above equation is relatively easy:
[0116] For a monotonically increasing function f, if x1 <= x2 in its domain, then f(x1) <= f(x2).
[0117] For a set X consisting of n sampled data points a1, a2, ..., an, where a_max = max(a1, a2, ..., an)
[0118] For any ai∈{a1,a2,…,an},
[0119] If ai <= a_max, then f(ai) <= f(a_max).
[0120] The function values of each element in set X constitute set Y.
[0121] Then Y = {f(a1), f(a2), ..., f(an)}, and Y_max represents the maximum value in set Y. That is, for any ai ∈ X, f(ai) ∈ Y, and therefore f(ai) <= Y_max.
[0122] a_max∈X, f(a_max)∈Y, then f(a_max)<=Y_max
[0123] Let a* = argmax(Y), which is the a corresponding to Y_max.
[0124] a*∈X, then a*<=a_max, then Y_max=f(a*)<=f(a_max)
[0125] Therefore, f(a_max) = Y_max, and the above equation holds true.
[0126] Therefore, for monotonic activation functions (such as sigmoid and softplus), pooling before activation requires far less computation than activation before pooling.
[0127] Therefore, as one possible implementation, the confidence scoring module includes a pooling layer and an activation layer, wherein the pooling layer is connected to the last multi-scale random combination convolutional layer, and the activation layer is connected to the pooling layer.
[0128] The following describes the sorting apparatus of the recommendation system disclosed in the embodiments of this application. The sorting apparatus of the recommendation system described below can be referred to in correspondence with the sorting method of the recommendation system described above.
[0129] Reference Figure 8 , Figure 8 This is a schematic diagram of the sorting device structure of a recommendation system disclosed in an embodiment of this application. Figure 8 As shown, the sorting device of the recommendation system may include:
[0130] The feature data determination unit 11 is used to determine feature data, which includes the user's feature data and the feature data of multiple objects to be recommended corresponding to the user.
[0131] Feature data pair construction unit 12 is used to construct feature data pairs based on the feature data. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended.
[0132] The confidence score determination unit 13 is used to perform multi-scale local neighborhood nonlinear aggregation on each feature data pair to obtain aggregated feature data, and to process the aggregated feature data to obtain the confidence score of the feature data pair.
[0133] The sorting unit 14 is used to sort the multiple objects to be recommended based on the confidence scores of each feature data pair.
[0134] As one possible implementation, the feature data determining unit includes:
[0135] A profile data acquisition unit is used to acquire profile data, which includes the user's profile data and the profile data of multiple objects to be recommended corresponding to the user; the profile data includes continuous data, single-value discrete data and multi-value discrete data.
[0136] The long-tail removal and normalization processing unit is used to perform long-tail removal processing on the continuous data in the portrait data to obtain long-tail removed continuous data, and to perform normalization processing on the long-tail removed continuous data to obtain normalized continuous data.
[0137] The standardization processing unit is used to standardize the normalized continuous data, the single-valued discrete data, and the multi-valued discrete data respectively to obtain the feature data.
[0138] As one possible implementation, the long-tail removal and normalization processing unit is specifically used for:
[0139] The continuous data in the portrait data is processed by taking the nth power or the logarithm to obtain the long-tailed continuous data, where n is an integer greater than or equal to 2.
[0140] As one possible implementation, the confidence score determination unit is specifically used for:
[0141] The feature data pairs are input into a pre-trained ranking model. The ranking model performs multi-scale local neighborhood nonlinear aggregation on the feature data pairs to obtain aggregated feature data, and processes the aggregated feature data to obtain the confidence score of the feature data pairs. The ranking model is trained using the training feature data pairs as training samples and the labels of the training feature data pairs as sample labels.
[0142] As one possible implementation, the ranking model includes a feature splicing module, a multi-scale local neighborhood nonlinear aggregation module, and a confidence scoring module;
[0143] The feature splicing module splices the feature data of the user in the feature data pair and the feature data of the object to be recommended to obtain the spliced feature data;
[0144] The multi-scale local neighborhood nonlinear aggregation module performs multi-scale local neighborhood nonlinear aggregation on the spliced feature data to obtain aggregated feature data.
[0145] The confidence scoring module scores the aggregated feature data to obtain the confidence score of the feature data pair.
[0146] As one possible implementation, the multi-scale local neighborhood nonlinear aggregation module includes: a plurality of multi-scale random combination convolutional layers cascaded in sequence;
[0147] Each multi-scale random combination convolutional layer includes multiple convolutional kernels. Among these kernels, the minimum scale is 2, and the scales of the remaining kernels are determined by selecting them at equal intervals from a preset scale list. Each scale in the preset scale list is an odd number, with the minimum scale being 3 and the maximum scale being 2int(N / 4)-1, where N is the length of the concatenated feature data.
[0148] The depth of each multi-scale random combination convolutional layer is determined based on N.
[0149] As one possible implementation, each convolutional kernel maintains normal convolution with a preset probability when traversing the feature data, and performs random convolution in other cases. The normal convolution is to slide the convolution window sequentially from the start position to the end position of the feature data, and the random convolution is to keep the central feature unchanged each time and randomly select other features from the features other than the central feature.
[0150] In one possible implementation, the confidence scoring module includes a pooling layer and an activation layer, wherein the pooling layer is connected to the last multi-scale random combination convolutional layer, and the activation layer is connected to the pooling layer.
[0151] Reference Figure 9 , Figure 9 A hardware structure block diagram of a ranking device for a recommendation system provided in this application embodiment is shown below. Figure 9 The hardware structure of the ranking device of the recommendation system may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
[0152] In this embodiment of the application, the number of processor 1, communication interface 2, memory 3, and communication bus 4 is at least one, and processor 1, communication interface 2, and memory 3 communicate with each other through communication bus 4;
[0153] Processor 1 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
[0154] Memory 3 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device;
[0155] The memory stores a program, which the processor can call. The program is used for:
[0156] The feature data is determined, including the user's feature data and the feature data of multiple objects to be recommended corresponding to the user;
[0157] Based on the feature data, feature data pairs are constructed. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended.
[0158] For each feature data pair, multi-scale local neighborhood nonlinear aggregation is performed on the feature data pair to obtain aggregated feature data, and the aggregated feature data is processed to obtain the confidence score of the feature data pair.
[0159] The multiple objects to be recommended are sorted based on the confidence scores of each feature data pair.
[0160] Optionally, the refined and extended functions of the program can be found in the description above.
[0161] This application embodiment also provides a readable storage medium that can store a program suitable for execution by a processor, the program being used for:
[0162] The feature data is determined, including the user's feature data and the feature data of multiple objects to be recommended corresponding to the user;
[0163] Based on the feature data, feature data pairs are constructed. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended.
[0164] For each feature data pair, multi-scale local neighborhood nonlinear aggregation is performed on the feature data pair to obtain aggregated feature data, and the aggregated feature data is processed to obtain the confidence score of the feature data pair.
[0165] The multiple objects to be recommended are sorted based on the confidence scores of each feature data pair.
[0166] Optionally, the refined and extended functions of the program can be found in the description above.
[0167] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0168] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0169] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A ranking method of a recommendation system, characterized by, The method includes: The feature data is determined, including the user's feature data and the feature data of multiple objects to be recommended corresponding to the user; Based on the feature data, feature data pairs are constructed. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended. For each feature data pair, a multi-scale local neighborhood nonlinear aggregation is performed on the feature data pair based on a ranking model to obtain aggregated feature data. The aggregated feature data is then processed to obtain a confidence score for the feature data pair. The ranking model includes a multi-scale local neighborhood nonlinear aggregation module, which comprises multiple cascaded multi-scale random combination convolutional layers. Each multi-scale random combination convolutional layer includes multiple convolutional kernels. Among these kernels, the minimum scale is 2, and the scales of the remaining kernels are determined by equally spaced selections from a preset scale list. Each scale in the preset scale list is an odd number, with a minimum scale of 3 and a maximum scale of 2int(N / 4)-1, where N is the length of the concatenated feature data. The depth of each multi-scale random combination convolutional layer is determined based on N. The concatenated feature data is obtained by concatenating the user's feature data from the feature data pair with the feature data of the object to be recommended. The multiple objects to be recommended are sorted based on the confidence scores of each feature data pair.
2. The method of claim 1, wherein, The determined feature data includes: Acquire profile data, which includes the user's profile data and the profile data of multiple objects to be recommended corresponding to the user; the profile data includes continuous data, single-valued discrete data, and multi-valued discrete data; The continuous data in the portrait data is subjected to long-tail removal processing to obtain long-tail removed continuous data, and the long-tail removed continuous data is normalized to obtain normalized continuous data. The normalized continuous data, the single-valued discrete data, and the multi-valued discrete data are respectively standardized to obtain the feature data.
3. The method according to claim 2, characterized in that, The step of removing long-tailed data from the continuous data in the portrait data to obtain long-tailed continuous data includes: The continuous data in the portrait data is processed by taking the nth power or the logarithm to obtain the long-tailed continuous data, where n is an integer greater than or equal to 2.
4. The method according to claim 1, characterized in that, The process of performing multi-scale local neighborhood nonlinear aggregation on the feature data pairs to obtain aggregated feature data, and then processing the aggregated feature data to obtain the confidence score of the feature data pairs, includes: The feature data pairs are input into the pre-trained ranking model. The ranking model performs multi-scale local neighborhood nonlinear aggregation on the feature data pairs to obtain aggregated feature data, and processes the aggregated feature data to obtain the confidence score of the feature data pairs. The ranking model is trained using the training feature data pairs as training samples and the labels of the training feature data pairs as sample labels.
5. The method according to claim 4, characterized in that, The ranking model also includes a feature concatenation module and a confidence scoring module; The feature splicing module splices the feature data of the user in the feature data pair and the feature data of the object to be recommended to obtain the spliced feature data; The multi-scale local neighborhood nonlinear aggregation module performs multi-scale local neighborhood nonlinear aggregation on the spliced feature data to obtain aggregated feature data. The confidence scoring module scores the aggregated feature data to obtain the confidence score of the feature data pair.
6. The method according to claim 1, characterized in that, When each convolutional kernel traverses the feature data, it maintains normal convolution with a preset probability, and performs random convolution in other cases. The normal convolution is to slide the convolution window sequentially from the start position to the end position of the feature data, and the random convolution is to keep the central feature unchanged each time and randomly select other features from the features other than the central feature.
7. The method according to claim 5, characterized in that, The confidence scoring module includes a pooling layer and an activation layer. The pooling layer is connected to the last multi-scale random combination convolutional layer, and the activation layer is connected to the pooling layer.
8. A sorting device for a recommendation system, characterized in that, The device includes: A feature data determination unit is used to determine feature data, which includes the user's feature data and the feature data of multiple objects to be recommended corresponding to the user; The feature data pair construction unit is used to construct feature data pairs based on the feature data. Each feature data pair includes the feature data of the user and the feature data of the object to be recommended. The number of feature data pairs is the same as the number of objects to be recommended. The confidence score determination unit is used to perform multi-scale local neighborhood nonlinear aggregation on each feature data pair based on a ranking model to obtain aggregated feature data, and to process the aggregated feature data to obtain the confidence score of the feature data pair. The ranking model includes a multi-scale local neighborhood nonlinear aggregation module, which includes: multiple multi-scale random combination convolutional layers cascaded in sequence; each multi-scale random combination convolutional layer includes multiple convolutional kernels, among which the minimum scale is 2, and the scales of the remaining convolutional kernels are selected at equal intervals from a preset scale list, where each scale in the preset scale list is an odd number, the minimum scale is 3, and the maximum scale is 2int(N / 4)-1, where N is the length of the concatenated feature data; the depth of each multi-scale random combination convolutional layer is determined based on N; the concatenated feature data is obtained by concatenating the user's feature data and the feature data of the object to be recommended in the feature data pair. The sorting unit is used to sort the multiple objects to be recommended based on the confidence scores of each feature data pair.
9. A sorting device for a recommendation system, characterized in that, Including memory and processor; The memory is used to store programs; The processor is configured to execute the program to implement the various steps of the ranking method of the recommendation system as described in any one of claims 1 to 7.
10. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the various steps of the ranking method of the recommendation system as described in any one of claims 1 to 7.