Recommendation method, apparatus and computing device cluster

Abstract

A recommendation method, comprising: on the basis of a historical behavior sequence of a user, screening out N candidate items from an item library; performing attention computation on the N candidate items and the historical behavior sequence, so as to obtain a first matrix, wherein during the attention computation, a Q matrix is obtained by performing a linear transformation on the N candidate items, and a first K matrix and a first V matrix are obtained by performing different linear transformations on the historical behavior sequence; gradually performing vector dot product and matrix dot multiplication operations on the Q matrix, a second K matrix and a second V matrix, so as to obtain a second matrix, the second K matrix and the second V matrix being obtained by performing different linear transformations on the N candidate items; performing a matrix addition operation on the first matrix and the second matrix to obtain a third matrix, one row in the third matrix being used for representing a behavioral weighted representation of one candidate item; on the basis of the third matrix, screening out K candidate items from amongst the N candidate items; and recommending the K candidate items to the user. The method improves the recommendation efficiency of recommendation systems.

Description

A recommended method, apparatus and computing device cluster

[0001] This application claims priority to Chinese Patent Application No. 202510474682.8, filed on April 15, 2025, entitled “A Recommended Method, Apparatus and Computing Device Cluster”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and more particularly to a recommendation method, apparatus, and cluster of computing devices. Background Technology

[0003] Recommendation systems, as a crucial means for internet platforms to accumulate traffic and achieve monetization, are undergoing a paradigm shift driven by generative AI technology. Generative recommendation systems are a new type of recommendation system that utilizes generative AI technology to generate personalized recommendations. These systems can not only recommend existing items based on users' historical behavior and preferences, but also creatively propose new items or content, thereby providing a richer and more personalized user experience.

[0004] In generative recommendation architectures, attention calculations are required between user historical behavior sequences and candidate items. Since the computational complexity of self-attention mechanisms increases quadratically with sequence length, and user historical behavior sequences are often very long, this makes the attention calculation between user behavior sequences and candidate items extremely time-consuming in generative recommendation architectures. This time consumption affects the real-time performance and recommendation effectiveness of the recommendation system, becoming a key problem that generative recommendation systems need to solve. Summary of the Invention

[0005] This application provides a recommendation method, apparatus, computing device cluster, computer-readable storage medium, and computer program product containing instructions, which can improve the real-time performance and recommendation effectiveness of the recommendation system.

[0006] In a first aspect, this application provides a recommendation method, comprising: selecting N candidate items from an item library based on a user's historical behavior sequence, where N≥2, and the candidate items are goods, content, advertisements, or services; performing attention calculation on the N candidate items and the historical behavior sequence to obtain a first matrix, wherein in the attention calculation process, the query vector Q matrix is ​​obtained by linear transformation of the N candidate items, and the first key vector K matrix and the first value vector V matrix are obtained by different linear transformations of the historical behavior sequence; performing vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix stepwise to obtain a second matrix, wherein the second K matrix and the second V matrix are obtained by different linear transformations of the N candidate items; performing matrix addition operations on the first matrix and the second matrix to obtain a third matrix, wherein a row in the third matrix is ​​used to represent the behavior weighted representation of a candidate item; selecting K candidate items from the N candidate items based on the third matrix; and recommending the K candidate items to the user.

[0007] In this way, by separating the attention calculation between candidate items and historical behavior sequences from the attention calculation between candidate items in the recommender system, and converting the attention calculation between candidate items into vector dot product and matrix dot product, redundant calculations in the attention calculation process of the recommender system are effectively eliminated, the computation time and computational resource consumption are reduced, and the computational efficiency is improved, thus greatly improving the recommendation efficiency of the recommender system.

[0008] In one possible implementation, the first matrix is ​​calculated using a first type of computing unit on the AI ​​accelerator; the second matrix is ​​calculated using a second type of computing unit on the AI ​​accelerator. This allows computation to be performed using different computing resources on the AI ​​accelerator, thereby improving computational efficiency.

[0009] In one possible implementation, the first type of computing unit is a computing unit used for matrix operations, and the second type of computing unit is a general-purpose computing unit.

[0010] In one possible implementation, the first type of computing unit and the second type of computing unit execute tasks in parallel. This allows for parallel pipeline masking using two different computing resources, thereby accelerating parallel computing and further improving computational efficiency.

[0011] In one possible implementation, attention is calculated on N candidate items and the historical behavior sequence to obtain a first matrix. This involves performing matrix multiplication operations on the Q matrix, the first K matrix, and the first V matrix sequentially. In this way, attention calculation between candidate items and the historical behavior sequence can be achieved through matrix multiplication.

[0012] In one possible implementation, vector and matrix dot products are performed sequentially on the Q matrix, the second K matrix, and the second V matrix to obtain the second matrix. This includes: performing vector dot products on the Q matrix and the second K matrix to obtain the fourth matrix; performing column copying and expansion on the fourth matrix based on the number of columns in the second V matrix to obtain the fifth matrix, where the number of columns in the fifth matrix is ​​the same as the number of columns in the second V matrix; and performing matrix dot products on the fifth matrix and the second V matrix to obtain the second matrix. This allows for computation within the candidate items.

[0013] In one possible implementation, K candidate items are selected from N candidate items based on a third matrix. This includes: predicting the user's interest probability for each candidate item based on the third matrix; and selecting K candidate items from the N candidate items based on the user's interest probability for each candidate item, where the user's interest probability for the K items is greater than the user's interest probability for the remaining candidate items in the N candidate items. This allows for fine-grained ranking of the candidate items, thereby obtaining the items to be recommended to the user.

[0014] In one possible implementation, before selecting N candidate projects from the project library based on the user's historical behavior sequence, the process includes: obtaining the user's user identifier; retrieving the user's user logs from the log database based on the user identifier; and extracting features from the user logs to obtain the historical behavior sequence. In this way, the user's historical behavior sequence can be obtained through the user identifier.

[0015] Secondly, this application provides a recommendation device, including a filtering module and a processing module. The filtering module is used to filter N candidate items (N≥2) from an item library based on a user's historical behavior sequence, where the candidate items are goods, content, advertisements, or services. The processing module is used to perform attention calculations on the N candidate items and the historical behavior sequence to obtain a first matrix. During the attention calculation, the query vector Q matrix is ​​obtained by linear transformation of the N candidate items, and the first key vector K matrix and the first value vector V matrix are obtained by different linear transformations of the historical behavior sequence. The processing module is also used to perform vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix sequentially to obtain a second matrix, where the second K matrix and the second V matrix are obtained by different linear transformations of the N candidate items. The processing module is also used to perform matrix addition operations on the first and second matrices to obtain a third matrix, where a row in the third matrix represents a weighted representation of the behavior of a candidate item. The processing module is also used to filter K candidate items from the N candidate items based on the third matrix and to recommend the K candidate items to the user.

[0016] In one possible implementation, the first matrix is ​​calculated using a first type of computing unit on an AI accelerator; the second matrix is ​​calculated using a second type of computing unit on an AI accelerator.

[0017] In one possible implementation, the first type of computing unit is a computing unit used for matrix operations, and the second type of computing unit is a general-purpose computing unit.

[0018] In one possible implementation, the first type of computing unit and the second type of computing unit execute tasks in parallel.

[0019] In one possible implementation, when the processing module performs attention calculations on N candidate items and historical behavior sequences to obtain the first matrix, it specifically performs matrix multiplication operations on the Q matrix, the first K matrix, and the first V matrix in sequence to obtain the first matrix.

[0020] In one possible implementation, when the processing module performs vector dot products and matrix dot products on the Q matrix, the second K matrix, and the second V matrix to obtain the second matrix, it specifically performs the following: performs vector dot products on the Q matrix and the second K matrix to obtain the fourth matrix; performs column copying and expansion on the fourth matrix based on the number of columns of the second V matrix to obtain the fifth matrix, wherein the number of columns of the fifth matrix is ​​the same as the number of columns of the second V matrix; and performs matrix dot products on the fifth matrix and the second V matrix to obtain the second matrix.

[0021] In one possible implementation, when the processing module selects K candidate items from N candidate items based on the third matrix, it specifically performs the following: predicts the user's interest probability for each candidate item based on the third matrix; and selects K candidate items from N candidate items based on the user's interest probability for each candidate item, wherein the user's interest probability for the K items is greater than the user's interest probability for the remaining candidate items in the N candidate items.

[0022] In one possible implementation, before the filtering module selects N candidate items from the item library based on the user's historical behavior sequence, it is also used to: obtain the user's user identifier; retrieve the user's user log from the log database based on the user identifier; and extract features from the user log to obtain the historical behavior sequence.

[0023] Thirdly, embodiments of this application provide a recommended apparatus, including at least one processor and an interface; the at least one processor obtains program instructions through the interface; the at least one processor is used to execute program line instructions to implement the method described in the first aspect or any possible implementation of the first aspect. Exemplarily, the recommended apparatus may be, but is not limited to, a chip.

[0024] Fourthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect.

[0025] Fifthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, perform the method described in the first aspect or any possible implementation thereof. Exemplarily, the computing device cluster may include one or more computing devices.

[0026] Sixthly, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0027] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0028] Figure 1 is a schematic diagram of a self-attention calculation process provided in an embodiment of this application;

[0029] Figure 2 is a schematic diagram of a generative recommendation system architecture provided in an embodiment of this application;

[0030] Figure 3 is a schematic diagram of the reasoning structure of a sorting model provided in an embodiment of this application;

[0031] Figure 4 is a schematic diagram of the attention calculation process in a ranking model provided in an embodiment of this application;

[0032] Figure 5 is a schematic diagram of a technical concept for attention calculation in a recommendation system provided by an embodiment of this application;

[0033] Figure 6 is a flowchart illustrating a recommended method provided in an embodiment of this application;

[0034] Figure 7 is a comparative schematic diagram of an attention calculation process provided in an embodiment of this application;

[0035] Figure 8 is a schematic diagram of the structure of a recommendation system provided in an embodiment of this application;

[0036] Figure 9 is a schematic diagram of the working process of a feature engineering module provided in an embodiment of this application;

[0037] Figure 10 is a schematic diagram of a recommended device provided in an embodiment of this application;

[0038] Figure 11 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0039] Figure 12 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0040] Figure 13 is a schematic diagram of another computing device cluster provided in an embodiment of this application;

[0041] Figure 14 is a schematic diagram of another recommended device provided in an embodiment of this application. Detailed Implementation

[0042] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0043] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0044] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0045] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0046] First, the relevant technical terms used in this application will be introduced.

[0047] (1) Historical Behavior Sequence

[0048] In recommender systems, historical behavior sequences refer to records of a user's past interactions with the system. These records include user actions such as clicking on, purchasing, and rating items, as well as the items associated with these actions. These actions are arranged chronologically, forming a sequence. This sequence not only contains the user's behavior but also implicitly reveals the user's interests and preferences. Recommender systems can use this information to predict other items the user might be interested in and generate a recommendation list accordingly. For a specific user, their historical behavior sequence is a list of multiple elements, where each element represents a specific action the user took on an item. This sequence is usually arranged chronologically and can be represented as: Historical Behavior Sequence = [(I1, B1), (I2, B2), ..., (I... n B n Among them: I i It is the i-th item (such as product label, movie title, etc.), B i It represents the user's action on the i-th item (such as clicking, purchasing, rating, etc.), and n is the total number of actions in the sequence.

[0049] (2) Candidate Projects

[0050] In a recommendation system, candidate items refer to products, content, or services that the system might recommend to a user. These items are selected from a larger pool of items as potential recommendations. The number of candidate items can also be large, as platforms typically aim to choose the most suitable options for the user from a wide range of choices.

[0051] (3) Behavioral weighted representation of candidate items

[0052] The behavior-weighted representation of candidate items describes the feature representation obtained after weighting candidate items through an attention mechanism in a recommender system. This representation incorporates information from the user's historical behavior, enabling each candidate item to receive personalized feature enhancements based on the user's behavioral patterns.

[0053] (4) Matrix cross product and matrix dot product

[0054] Matrix cross product, also known as matrix multiplication, is an operation between two matrices where the number of columns in the left matrix must equal the number of rows in the right matrix. The result of the matrix cross product is obtained by performing a dot product of each row of the left matrix with each column of the right matrix, and then summing the results. For example:

[0055] Matrix dot product, also known as Hadamard product, is an operation between two matrices. The two matrices must have the same shape, meaning they have the same number of rows and columns. The result of the matrix dot product is obtained by multiplying corresponding elements of the two matrices. For example:

[0056] (5) Self-attention mechanism of Transformer

[0057] The Transformer's self-attention mechanism captures global dependencies between sequence elements by dynamically assigning weights. As shown in Figure 1, the self-attention calculation process can include: In S11, the input sequence X and the attention mask (attention_mask) are used. The attention_mask indicates which input elements in sequence X should be considered and which should be ignored. Elements in the attention_mask are typically Boolean (0 or 1), where 1 indicates that an element should be considered and 0 indicates that an element should be ignored. In S12, the Q matrix, K matrix, and V matrix are calculated. Where Q = X × W Q K = X × W K V = X × W V W Q W K and W V These are all learned weight matrices used for linear transformations. For each token in the input sequence X, the corresponding Q, K, and V can be obtained through linear transformations. In S13, the Q×K matrix cross product is performed and divided by a scaling factor to avoid gradient vanishing or exploding problems caused by excessive dimensionality, thus obtaining matrix P′. d k Let Q and K be the dimensions. In S14, a matrix dot product is performed between matrix P′ and attention_mask, and the result is processed by the softmax function to obtain matrix P, i.e., P = softmax(P′*attention_mask). The purpose of this step is to convert the attention score into a probability distribution representing the attention weight of each element. In S15, a matrix cross product of P×V is performed to obtain the output. The entire process is efficiently completed through matrix parallel computation, enabling the model to simultaneously focus on semantic associations at different locations and capture long-distance dependencies.

[0058] In the attention mechanism, the Q-matrix is ​​used to find the other inputs most relevant to the current input, calculating the attention score and determining which information the model should focus on in the current step. The K-matrix, a vector matched with the Q-matrix, determines the allocation of attention weights. Therefore, comparing the Q-matrix and K-matrix through their dot product reveals their relevance. The attention score generated by the dot product of the Q-matrix and K-matrix reflects the similarity between them. Furthermore, the V-matrix contains the actual content or information, selectively aggregated based on the attention weights. Thus, the V-matrix, weighted and combined according to the attention weights calculated from the Q-matrix and K-matrix, forms the final representation of the attention output.

[0059] (6)attention_mask

[0060] The attention mask plays a crucial role in self-attention mechanisms. It specifies which sequence elements the model should ignore during attention computation, helping to handle padding issues in variable-length sequences, prevent future information leakage, ignore irrelevant elements, and improve computational efficiency. Furthermore, attention masks support processing sequences of varying lengths in batches and can be used for security and privacy protection, ensuring the model does not access sensitive information, thus enhancing the model's flexibility and applicability. In self-attention computation, the attention mask can be a multi-dimensional matrix, such as a two-dimensional or three-dimensional matrix. When the attention mask is a two-dimensional matrix, its size can be represented as (m×n), where m is the number of rows and n is the number of columns. When the attention mask is a three-dimensional matrix, its size can be represented as (B×m×n), where B is the depth or layer of the attention mask, m is the number of rows, and n is the number of columns.

[0061] The technical solution provided in this application will be described below.

[0062] For example, Figure 2 illustrates a schematic diagram of a generative recommendation system architecture provided in an embodiment of this application. As shown in Figure 2, in this generative recommendation system architecture, when a user initiates a request on the platform (i.e., an internet service or application providing recommendation services, which can be web-based, mobile application, or other forms of digital service, etc.), the system responds quickly. The AI ​​generator immediately generates personalized recommendations based on the user's instructions and historical feedback. These recommendations are provided directly to the user without undergoing a ranking process, achieving rapid response. Simultaneously, human uploaders continuously add new items to the item library, which contains all recommendable items, including AI-generated and human-uploaded content. The recommendation engine can further analyze the user's historical behavior and contextual information to rank candidate items in the item library to determine the most suitable content to recommend to the user. The recommendation engine selects the top K ranked items to ensure the relevance and attractiveness of the recommendations. Finally, the system generates a comprehensive recommendation list that integrates recommendations directly provided by the AI ​​generator and candidate items ranked and filtered by the recommendation engine, providing users with personalized recommendation results. Users, as the core service recipients of the recommendation system, provide data through their interactions with the system (such as clicks, browsing, and feedback). This data not only optimizes recommendation results but also helps the system better understand user needs and preferences. User interactions and contextual information provide valuable data for the system, enabling it to understand user behavior patterns and provide more personalized recommendations. After providing recommendations to users, generative recommendation systems optimize their algorithms based on user feedback. Through this closed-loop, continuous learning and adaptation process, the system can accurately make personalized recommendations, ensuring that recommended items evolve as user preferences change, thereby improving user satisfaction and system efficiency.

[0063] In Figure 2, the recommendation engine mainly consists of a recall model and a ranking model. The recall model is primarily responsible for quickly filtering highly relevant items from a large item library as a candidate set. Its algorithm design is usually simple and efficient, aiming to reduce the size of the candidate set to hundreds to thousands through initial screening and dimensionality reduction. Recall algorithms typically employ various strategies, including rule-based recall (such as popular leaderboards, key operational categories, etc.), recall based on simple algorithms (such as similarity calculation, user profile and item profile matching, etc.), and recall based on complex algorithms (such as deep learning, reinforcement learning, etc.). These recall algorithms filter items that users may be interested in from different dimensions, similar to ensemble learning, to ensure the diversity and relevance of the candidate set. The ranking model is mainly responsible for refining the ranking of the candidate set selected by the recall model to determine the items ultimately recommended to the user. Ranking algorithms can be simple, such as logistic regression, factorization machines, gradient boosting decision trees, etc., or more complex deep learning models. The ranking model uses machine learning algorithms to perform secondary scoring and ranking of the recall results, providing a unified evaluation of the candidate set, thereby achieving efficient and accurate recommendations. In recommendation engines, recall and ranking models work together. The recall model quickly narrows down the candidate set, and the ranking model refines the candidate set, thus achieving efficient and accurate recommendations.

[0064] As described above, the goal of ranking models is to accurately rank candidate items obtained in the recall phase to determine the final content recommended to the user. To achieve this goal, ranking models need to fully capture the complex interaction relationships between users and items and efficiently process large-scale data. The Transformer architecture, due to its unique self-attention mechanism, can capture global dependencies in user behavior sequences, while its parallel computing capabilities significantly improve processing efficiency. Furthermore, the Transformer architecture offers flexible scalability, adapting to different recommendation scenarios. Therefore, the Transformer architecture has become one of the preferred architectures for ranking models and is widely used in recommendation systems to improve the accuracy and efficiency of recommendations.

[0065] The ranking model using the Transformer architecture can have, but is not limited to, the structure shown in Figure 3. In Figure 3, the ranking model can utilize the micro-batched-fast attention leveraging cacheable operations (M-FALCON) algorithm to divide the m candidate items recalled by the recall model into multiple micro-batches. Each micro-batch is b in size. mThis micro-batch processing approach makes better use of computing resources and can leverage encoder-level key-value (KV) caching to share computation results within micro-batches and across requests. For example, in an online recommendation system, when different users request recommendations, some users may have the same historical behavior sequences. KV caching can avoid repeatedly calculating these identical sequences, reducing computational redundancy. It should be understood that the steps for dividing into micro-batches can be chosen based on the actual situation and are not limited here.

[0066] Furthermore, the ranking model can perform parallel attention calculations on the candidate items in each micro-batch and the user's historical behavior sequence to predict the specific behaviors the user might take towards the candidate items in each micro-batch. Referring again to Figure 3, taking the prediction of the user's possible behaviors towards each candidate item in MicroBatch0 as an example, through the embedding layer, the ranking model converts each element in the historical behavior sequence into an embedded representation. Similarly, it also converts the candidate items in MicroBatch0 into embedded representations. These embedded representations serve as input to the attention mechanism, used to calculate the correlation between the historical behavior sequence and the candidate items in MicroBatch0. As shown in region 31 of Figure 3, no attention calculation is performed between candidate items. After completing the attention calculation, the ranking model generates an attention score for each candidate item. These scores are then further processed by the prediction layer to predict the specific behaviors the user might take towards each candidate item in MicroBatch0. Finally, the ranking model evaluates the predicted behaviors the user might take towards each candidate item across all micro-batches and selects the most suitable or relatively suitable candidate item to recommend to the user based on these prediction results.

[0067] It should be understood that since candidate items are generally considered independent, and the main goal of ranking models is to predict a user's interest or preference for each individual candidate item, rather than the relationship between candidate items, attention calculations are not required between candidate items in recommender systems. To achieve this, an attention mask is typically introduced to avoid unnecessary attention calculations, ensuring that the ranking model only focuses on relevant candidate items, thereby improving computational efficiency and recommendation accuracy. The attention calculation process in ranking models can be seen in the example shown in Figure 4. Figure 4 illustrates this in detail using two candidate items and two elements from a historical behavior sequence as an example. It should be noted that the process in Figure 4 also applies to other numbers of candidate items and other numbers of elements in a historical behavior sequence; therefore, for more general cases, the attention calculation process can be analogous to the steps in Figure 4. Specifically, as shown in Figure 4, firstly, the two candidate items Φ′3 and Φ′4 are converted into embedded representations through an embedding layer, and the two embedded representations are concatenated along the sequence length dimension to obtain the input sequence.

[0068] Then, using the input sequence and the corresponding weight matrix, the Q matrix, K matrix (hereinafter referred to as "K′ matrix") and V matrix (hereinafter referred to as "V′ matrix") associated with the input sequence are calculated respectively.

[0069] Next, the K′ matrix and the K matrix associated with the two elements in the historical behavior sequence are concatenated along the sequence length dimension to obtain the K″ matrix; and the V′ matrix and the V matrix associated with the two elements in the historical behavior sequence are concatenated along the sequence length dimension to obtain the V″ matrix. The K and V matrices associated with the two elements in the historical behavior sequence can be obtained through pre-processing or real-time processing, depending on the specific circumstances, and are not limited here. Furthermore, the calculation process of the K and V matrices associated with the two elements in the historical behavior sequence can refer to the previously described calculation process for the K and V matrices, and will not be repeated here. It should be understood that in the attention mechanism, the Q matrix represents the feature embedding of candidate items, used to query the parts most relevant to the user's historical behavior. It determines which content the model should focus on in the current step by calculating attention scores. The K matrix is ​​the key matrix for matching with the Q matrix, determining the allocation of attention weights. Attention scores can be generated between the Q and K matrices through matrix multiplication, thereby determining their correlation. These attention scores reflect the similarity between the Q and K matrices. In ranking models, the goal is to select candidate items similar to historical behavior sequences. To achieve this, the K-matrix related to the historical behavior sequence and the K-matrix related to the candidate items need to be concatenated. This concatenation operation allows the model to consider features of both historical behavior and candidate items simultaneously, thus more accurately assessing their similarity and selecting the most relevant candidate items. Furthermore, in recommender systems, to more comprehensively evaluate a user's interest in candidate items, it's necessary to consider not only the user's historical behavior sequence but also the features of the candidate items themselves. Therefore, the V-matrix related to the historical behavior sequence and the V-matrix related to the candidate items can be concatenated to fuse their feature information. This concatenation operation allows the model to consider both the user's historical interests and the content of the candidate items simultaneously, thus more accurately calculating attention weights and generating more precise user behavior predictions, providing users with more personalized recommendation results.

[0070] After calculating the Q matrix, K matrix (i.e., the "K″ matrix"), and V matrix (i.e., the "V″ matrix") required for attention calculation, subsequent calculations can be performed. As shown in S44 of Figure 4, matrix P′ can be obtained by performing matrix multiplication on the Q matrix and the K″ matrix. In matrix P′, QjKi = q j1 *k i1 +q j2 *k i2 +q j3 *k i3 +q j4 *k i4As can be seen from this calculation formula, the elements Q3K4 and Q4K3 in the P′ matrix represent calculations between two different candidate items. However, in a recommendation system, different candidate items are independent and should not be correlated. Therefore, an attention mask is needed to perform a matrix dot product operation with the P′ matrix to mask these two elements, resulting in the P matrix shown in S45. Finally, a matrix multiplication operation is performed between the V″ matrix and the P matrix to obtain the output result shown in S46. The O in the output matrix... 11 =Q3K1*v 11 +Q3K2*v 21 +Q3K3*v 31 .

[0071] As can be seen from the attention calculation process of the ranking model described above, there are redundant calculations in the Q-matrix and K″-matrix calculations. With these redundant calculations, an attention mask is needed to perform additional processing on the calculation results. Therefore, this increases computation time, reduces computational efficiency, and requires more computational resources.

[0072] In view of this, the embodiments of this application provide an attention calculation method in a recommendation system, which can effectively eliminate all redundant calculations in the attention calculation process of the ranking model, reduce calculation time and consumption of computing resources, and improve computational efficiency, thereby significantly improving the recommendation efficiency of the recommendation system.

[0073] For example, Figure 5 illustrates a schematic diagram of the technical concept for attention calculation in a recommender system according to an embodiment of this application. Figure 5 provides a detailed explanation using two candidate items and two elements from a historical behavior sequence as an example. It should be noted that the process in Figure 5 is also applicable to other numbers of candidate items and other numbers of elements in a historical behavior sequence; therefore, for more general cases, the attention calculation process can be performed analogously to the steps in Figure 5. Specifically, as shown in Figure 5, similar to S41 to S42 in Figure 4, S51 to S52 in Figure 5 also require first calculating the Q matrix, K′ matrix, and V′ matrix associated with the input sequence.

[0074] Then, in S53, each row of the Q matrix and the K′ matrix is ​​treated as a vector, and a vector dot product operation is performed on the same rows to obtain the r matrix. The r matrix is ​​a 2x1 matrix, and its two elements are the elements Q3K3 and Q4K4 in the P matrix in Figure 4, respectively. Further, in S54, the r matrix is ​​first expanded by copying the columns of the V′ matrix to obtain the r′ matrix with the same number of columns as the V′ matrix. Then, a matrix dot product operation is performed on the r′ matrix and the V′ matrix to obtain the s matrix. In addition, in S55, a matrix product operation can be performed on the Q matrix and the K matrix of two elements in the historical behavior sequence to obtain the h matrix. Further, in S56, a matrix product operation is performed on the h matrix and the V matrix of two elements in the historical behavior sequence to obtain the t matrix.

[0075] Finally, in S57, performing matrix addition on the s and t matrices yields the same output matrix as obtained in S46 of Figure 4.

[0076] As shown in Figure 5, this technical concept primarily addresses the non-overlapping nature of attention calculations in candidate sets within recommendation systems. It transforms matrix multiplication between candidate items into vector and matrix dot products, thereby eliminating all redundant computations. Furthermore, by converting matrix multiplication between candidate items into vector and matrix dot products, this computation can be parallelized with the computation of candidate items and historical behavior sequences, thus improving computational efficiency. Additionally, as shown in Figure 7, comparing the computation flow shown in Figure 4 (the original flow in Figure 7) with the computation flow shown in Figure 5 (the flow of this embodiment shown in Figure 7), it can be seen that the computation flow shown in Figure 5, by dividing the computation flow in Figure 4 into two parts, successfully eliminates the redundant computations described in Figure 4, and the results of the two computations are identical. Moreover, parallel computation is also achieved by dividing the computation into two parts.

[0077] The following describes the specific implementation process of the above technical concepts in a recommendation system.

[0078] For example, Figure 6 shows a flowchart of a recommendation method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. For example, the method can be executed by a recommendation device, which can be implemented by software and / or hardware, and can be, but is not limited to, configured in an electronic device or server. For ease of description, the following description uses a computing device as the execution subject. As shown in Figure 6, the recommendation method may include the following steps:

[0079] S601. Based on the user's historical behavior sequence, select N candidate items from the item library, where N≥2, and the candidate items are goods, content, advertisements or services.

[0080] In this embodiment, the computing device can utilize a recall model to analyze and process the user's historical behavior sequences. By identifying the user's behavioral patterns, preferences, and similarities with other users, the recall model can quickly filter out N (N≥2) candidate items highly relevant to the user's interests from a large item library, where N≥2. These candidate items can be, but are not limited to, goods, content, advertisements, or services. For example, content can include, but is not limited to, social content, knowledge content, text content, and audio-visual content. Social content refers to various forms of information related to interpersonal communication, social interaction, and social networks, such as social media updates, comments, likes, shares, and displays of social relationships (e.g., friend lists, followed individuals) and social activities and events (e.g., online and offline gatherings, event notifications), aiming to promote communication and interaction among users. Knowledge content refers to various forms of content intended to transmit knowledge, information, and education, such as online courses, academic articles, learning materials, news reports, industry trends, research reports, Q&A communities, and forum discussions, aiming to help users acquire professional knowledge and skills, understand the latest information and trends, and promote the dissemination and understanding of knowledge. Text-based content refers to information conveyed through text, such as articles, books, and news reports. Audio-visual content refers to information conveyed through sound and / or images, such as videos, audio recordings, movies, and music.

[0081] Before filtering candidate items from the item library, computing devices can first obtain user identifiers through user access operations. For example, when a user logs into a product application (APP), the computing device can use the user's registration information (such as username) to determine their unique user identifier. Subsequently, the computing device uses this user identifier to retrieve data reflecting the user's profile, such as historical behavior records, from a log database to form a user log. This data includes the user's browsing history, purchasing behavior, search keywords, and dwell time, helping the recommendation system better understand the user's interests and preferences, thereby supporting subsequent personalized recommendations. Finally, the computing device extracts features from the user log, analyzes the user's browsing, purchasing, and searching behavior patterns, and generates the user's historical behavior sequence, providing a basis for subsequent recommendations. It should be understood that computing devices should obtain the user's authorization before acquiring user-related data and ensure that related operations comply with the laws and regulations of the region where the computing device is located, in order to protect the user's privacy and rights, and to ensure the legality and transparency of the personalized recommendation system.

[0082] S602. Perform attention calculation on N candidate items and historical behavior sequences to obtain the first matrix. In the attention calculation process, the Q matrix is ​​obtained by linear transformation of the N candidate items, and the first K matrix and the first V matrix are obtained by different linear transformations of the historical behavior sequences.

[0083] In this embodiment, the computing device can perform attention calculations on N candidate items and a historical behavior sequence using the attention algorithm described above to obtain a first matrix. During the attention calculation process, the Q matrix is ​​obtained by linearly transforming the N candidate items using a pre-learned weight matrix, and the first K matrix and the first V matrix are obtained by performing different linear transformations on the historical behavior sequence using different pre-learned weight matrices. For example, when performing attention calculations, the computing device can first perform matrix multiplication on the Q matrix and the first K matrix, and then sequentially perform matrix multiplication on the calculation result and the first V matrix, i.e., sequentially perform matrix multiplication on the Q matrix, the first K matrix, and the first V matrix to obtain the first matrix.

[0084] S603. Perform vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix step by step to obtain the second matrix. The second K matrix and the second V matrix are obtained by performing different linear transformations on N candidate items.

[0085] In this embodiment, the computing device can perform different linear transformations on N candidate items using pre-learned weight matrices to obtain a second K matrix and a second V matrix. Then, the computing device can progressively perform vector dot products and matrix dot products on the Q matrix, the second K matrix, and the second V matrix to obtain the second matrix. Specifically, the computing device can first perform vector dot products on the Q matrix and the second K matrix to obtain a fourth matrix. Then, based on the number of columns in the second V matrix, the computing device performs column copying and expansion on the fourth matrix to obtain a fifth matrix. The fifth matrix has the same number of columns as the second V matrix. Thus, the fifth matrix becomes a matrix with the same number of rows and columns as the second V matrix, allowing for subsequent calculations. Finally, the computing device performs matrix dot products on the fifth matrix and the second V matrix to obtain the second matrix.

[0086] S604. Perform matrix addition on the first and second matrices to obtain the third matrix, where a row in the third matrix represents the weighted representation of the behavior of a candidate item.

[0087] In this embodiment, after obtaining the first matrix and the second matrix, the computing device can perform matrix addition on the first matrix and the second matrix to obtain the third matrix. One row in the third matrix represents a weighted representation of the behavior of a candidate item.

[0088] S605. Based on the third matrix, select K candidate projects from N candidate projects.

[0089] In this embodiment, the computing device can predict the user's interest probability for each candidate item based on the third matrix, thus obtaining the user's interest probability for each candidate item. For example, the third matrix can be processed by a network layer in a ranking model. Then, the computing device can sort the N candidate items based on the user's interest probability for each candidate item, such as sorting them from largest to smallest or smallest to largest, to select K candidate items from the N candidate items. The user's interest probability for the K items is greater than the user's interest probability for the remaining candidate items in the N candidate items. In this way, the computing device can select K candidate items from the N candidate items based on the third matrix.

[0090] S606. Recommend K candidate items to the user.

[0091] In this embodiment, the computing device can recommend K selected candidate items to the user.

[0092] In this way, by separating the attention calculation between candidate items and historical behavior sequences from the attention calculation between candidate items in the recommender system, and converting the attention calculation between candidate items into vector dot product and matrix dot product, redundant calculations in the attention calculation process of the recommender system are effectively eliminated, reducing computation time and computational resource consumption, and improving computational efficiency, thus significantly improving the recommendation efficiency of the recommender system. It can be understood that, as shown in Figure 6, the N candidate items can also be divided into multiple micro-batches, and the attention between each micro-batch and the historical behavior sequence can be calculated separately (i.e., the calculation step in S602), and calculations can also be performed within each micro-batch separately (i.e., the calculation step in S603) to calculate the third matrix corresponding to each micro-batch.

[0093] For example, the computing device may be configured with an AI accelerator (such as a graphics processing unit (GPU), tensor processing unit (TPU), neural processing unit (NPU), etc.). The AI ​​accelerator may be configured with at least two types of computing units. In this case, S602 described in Figure 6 may be executed, but is not limited to, by a first type of computing unit on the AI ​​accelerator, and S603 may be executed, but is not limited to, by a second type of computing unit on the AI ​​accelerator. For example, the first type of computing unit may be a computing unit for performing matrix operations, such as a cube core of an NPU or a tensor core of a GPU. The second type of computing unit may be a general-purpose computing unit, such as a vector core of an NPU or a CUDA core of a GPU. In some embodiments, the first type of computing unit and the second type of computing unit can execute tasks in parallel. This allows for parallel pipeline masking using two different computing resources, achieving parallel computing acceleration and further improving computational efficiency.

[0094] For example, as shown in Figure 8, a recommendation system can generally include: a user request parsing module, a feature engineering module, an embedding module, an offline recall module, and an online ranking module.

[0095] The user request parsing module is mainly used to verify the request format, extract key information such as user identifier and context when a user initiates a recommendation request, and pass the parsed content to subsequent modules in a standardized format.

[0096] The feature engineering module primarily receives data from the user request parsing module and cleans raw user logs from the log database. It extracts behavioral and attribute features from multiple dimensions and combines them into composite features, outputting a concatenated sequence of user historical behavior. As shown in Figure 9, the feature engineering module can transform raw data into structured data that can be effectively utilized by machine learning models. This process typically includes steps such as data classification, data transformation, and structured data classification. The data classification step categorizes the data contained in the raw data to obtain unstructured and structured data. The data transformation step converts unstructured data into structured data. This step can include text segmentation, image feature extraction, and audio signal processing, with the aim of extracting quantifiable and comparable features. The structured data classification step divides the data into qualitative and quantitative data. Qualitative data describes the attributes or characteristics of things and is typically used for classification problems. Qualitative data can be further divided into nominal-level data (representing unordered data) and sequence-level data (representing ordered data). Quantitative data allows for mathematical operations and is typically used for regression problems. Quantitative data can be further divided into: interval level (representing ordered data with equal intervals, but without an absolute zero point) and proportion level (representing ordered data with equal intervals and an absolute zero point).

[0097] The Embedding module is mainly used to transform historical behavior sequences into dense vectors, retain key information, and input them into the offline recall module to help subsequent models better capture the potential relationship between users and items.

[0098] The offline recall module is mainly used to quickly filter out candidate items related to user interests from the project library offline, providing highly relevant candidate items for the subsequent sorting module.

[0099] The online ranking module is mainly used to comprehensively consider factors such as the relevance of candidate items to users, item quality, popularity, and real-time user preferences. It scores candidate items in parallel using parallel attention calculations, re-ranks the candidate items, and provides a higher-quality personalized recommendation list. The execution process of the online ranking module can be described using the method shown in Figure 6.

[0100] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments and features described in the above embodiments can be combined according to actual conditions, and the combined solutions are still within the protection scope of this application.

[0101] Next, based on the methods in the above embodiments, a recommended apparatus provided in the embodiments of this application will be introduced.

[0102] For example, Figure 10 shows a schematic diagram of a recommendation device provided in an embodiment of this application. As shown in Figure 10, the recommendation device 1000 includes a filtering module 1001 and a processing module 1002. The filtering module 1001 is used to filter N candidate items from an item library based on the user's historical behavior sequence, where N≥2, and the candidate items are goods, content, advertisements, or services. The processing module 1002 is used to perform attention calculation on the N candidate items and the historical behavior sequence to obtain a first matrix. During the attention calculation, the Q matrix is ​​obtained by linear transformation of the N candidate items, and the first K matrix and the first V matrix are obtained by different linear transformations of the historical behavior sequence. The processing module 1002 is also used to perform vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix stepwise to obtain a second matrix. The second K matrix and the second V matrix are obtained by different linear transformations of the N candidate items. Processing module 1002 is further configured to perform matrix addition on the first and second matrices to obtain a third matrix, wherein a row in the third matrix represents a weighted representation of the behavior of a candidate item. Processing module 1002 is also configured to, based on the third matrix, select K candidate items from N candidate items, and recommend K candidate items to the user.

[0103] In some embodiments, the first matrix is ​​calculated using a first type of computing unit on an AI accelerator; the second matrix is ​​calculated using a second type of computing unit on an AI accelerator.

[0104] In some embodiments, the first type of computing unit is a computing unit for performing matrix operations, and the second type of computing unit is a general-purpose computing unit.

[0105] In some embodiments, the first type of computing unit and the second type of computing unit perform tasks in parallel.

[0106] In some embodiments, when the processing module 1002 performs attention calculations on N candidate items and historical behavior sequences to obtain a first matrix, it specifically performs matrix multiplication operations on the Q matrix, the first K matrix, and the first V matrix in sequence to obtain the first matrix.

[0107] In some embodiments, when the processing module 1002 performs vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix step by step to obtain the second matrix, it is specifically used to: perform vector dot product operations on the Q matrix and the second K matrix to obtain the fourth matrix; perform column copying expansion on the fourth matrix based on the number of columns of the second V matrix to obtain the fifth matrix, wherein the number of columns of the fifth matrix is ​​the same as the number of columns of the second V matrix; and perform matrix dot product operations on the fifth matrix and the second V matrix to obtain the second matrix.

[0108] In some embodiments, when the processing module 1002 selects K candidate items from N candidate items based on the third matrix, it is specifically used to: predict the user's interest probability for each candidate item based on the third matrix; and select K candidate items from N candidate items based on the user's interest probability for each candidate item, wherein the user's interest probability for the K items is greater than the user's interest probability for the remaining candidate items in the N candidate items.

[0109] In some embodiments, before filtering N candidate items from the item library based on the user's historical behavior sequence, the filtering module 1001 is further configured to: obtain the user's user identifier; retrieve the user's user log from the log database based on the user identifier; and extract features from the user log to obtain the historical behavior sequence.

[0110] In some embodiments, both the filtering module 1001 and the processing module 1002 shown in FIG. 10 can be implemented in software or in hardware. For example, the implementation of the filtering module 1001 will be described below. Similarly, the implementation of the processing module 1002 can refer to the implementation of the filtering module 1001.

[0111] As an example of a software functional unit, the filtering module 1001 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, the filtering module 1001 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0112] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0113] As an example of a hardware functional unit, the screening module 1001 may include at least one computing device, such as a server. Alternatively, the screening module 1001 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0114] The multiple computing devices included in the filtering module 1001 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the filtering module 1001 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the filtering module 1001 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0115] It should be noted that, in other embodiments, the filtering module 1001 can be used to execute any step in the recommendation method described in the above embodiments, and the processing module 1002 can also be used to execute any step in the recommendation method described in the above embodiments. Furthermore, the steps implemented by the filtering module 1001 and the processing module 1002 can be specified as needed. By implementing different steps in the recommendation method described in the above embodiments through the filtering module 1001 and the processing module 1002, all the functions of the recommendation device 1000 shown in FIG10 can be achieved.

[0116] This application also provides a computing device 1100. As shown in FIG11, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate with each other via the bus 1102. The computing device 1100 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 1100.

[0117] Bus 1102 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 11, but this does not imply that there is only one bus or one type of bus. Bus 1104 can include pathways for transmitting information between various components of computing device 1100 (e.g., memory 1106, processor 1104, communication interface 1108).

[0118] The processor 1104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0119] The memory 1106 may include volatile memory, such as random access memory (RAM). The processor 1104 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0120] The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the functions of the filtering module 1001 and the processing module 1002 shown in FIG. 10, thereby implementing the recommended method described in the above embodiments. That is, the memory 1106 stores instructions for executing the recommended method described in the above embodiments.

[0121] Alternatively, the memory 1106 stores executable code, which the processor 1104 executes to implement the functions of the recommended device 1000 shown in FIG10, thereby implementing the recommended method described in the above embodiments. That is, the memory 1106 stores instructions for executing the recommended method described in the above embodiments.

[0122] The communication interface 1108 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the computing device 1100 and other devices or communication networks.

[0123] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0124] As shown in Figure 12, the computing device cluster includes at least one computing device 1100. The memory 1106 of one or more computing devices 1100 in the computing device cluster may store the same instructions for performing the recommended methods described in the above embodiments.

[0125] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the recommended methods described in the above embodiments. In other words, a combination of one or more computing devices 1100 can jointly execute instructions for executing the recommended methods described in the above embodiments.

[0126] It should be noted that the memory 1106 in different computing devices 1100 within the computing device cluster can store different instructions, which are used to execute some of the functions of the recommendation device 1000 shown in Figure 10. That is, the instructions stored in the memory 1106 of different computing devices 1100 can implement the functions of one or more modules in the filtering module 1001 and the processing module 1002.

[0127] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 13 illustrates one possible implementation. As shown in Figure 13, two computing devices 1100A and 1100B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 1106 in computing device 1100A stores instructions for executing the functions of the filtering module 1001. Simultaneously, the memory 1106 in computing device 1100B stores instructions for executing the functions of the processing module 1002.

[0128] It should be understood that the functions of computing device 1100A shown in Figure 13 can also be performed by multiple computing devices 1100. Similarly, the functions of computing device 1100B can also be performed by multiple computing devices 1100.

[0129] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 12 and 13. The difference is that the memory 1106 of one or more computing devices 1100 in this computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0130] In some possible implementations, the memory 1106 of one or more computing devices 1100 in the computing device cluster may also store partial instructions for executing the aforementioned recommended method. In other words, a combination of one or more computing devices 1100 can jointly execute the instructions for executing the aforementioned recommended method.

[0131] It should be understood that each step of the above method embodiments can be accomplished by hardware logic circuits or software instructions in a processor.

[0132] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0133] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices containing at least one computing device, cause the cluster of computing devices to perform the methods in the above embodiments.

[0134] Based on the methods in the above embodiments, this application also provides a recommendation device. Please refer to FIG14, which is a schematic diagram of another recommendation device provided in this application. As shown in FIG14, the recommendation device 1400 includes one or more processors 1401 and interface circuits 1402. Optionally, the recommendation device 1400 may further include a bus 1403. Wherein:

[0135] Processor 1401 can be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed through integrated logic circuits in the hardware of processor 1401 or through software instructions. Processor 1401 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods and steps disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor.

[0136] The interface circuit 1402 can be used to send or receive data, instructions or information. The processor 1401 can use the data, instructions or other information received by the interface circuit 1402 to process the data, instructions or other information, and can send the processed information out through the interface circuit 1402.

[0137] Optionally, the recommended device 1400 also includes a memory, which may include read-only memory and random access memory, and provides operation instructions and data to the processor. A portion of the memory may also include non-volatile random access memory (NVRAM).

[0138] Optionally, the memory stores executable software modules or data structures, and the processor can execute corresponding operations by calling the operation instructions stored in the memory (which may be stored in the operating system).

[0139] Optionally, the interface circuit 1402 can be used to output the execution results of the processor 1401.

[0140] It should be noted that the functions of the processor 1401 and the interface circuit 1402 can be implemented through hardware design, software design, or a combination of hardware and software; no restrictions are imposed here.

[0141] It should be understood that each step of the above method embodiments can be completed by hardware logic circuits or software instructions in a processor.

[0142] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0143] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0144] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0145] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A recommendation method, characterized in that, The method includes: Based on the user's historical behavior sequence, N candidate items are selected from the item library, where N≥2, and the candidate items are goods, content, advertisements or services; Attention is calculated on the N candidate items and the historical behavior sequence to obtain a first matrix. In the attention calculation process, the query vector Q matrix is ​​obtained by linear transformation of the N candidate items, and the first key vector K matrix and the first value vector V matrix are obtained by different linear transformations of the historical behavior sequence. The Q matrix, the second K matrix, and the second V matrix are successively subjected to vector dot product and matrix dot product operations to obtain the second matrix, wherein the second K matrix and the second V matrix are obtained by performing different linear transformations on the N candidate items; Perform matrix addition on the first matrix and the second matrix to obtain a third matrix, wherein a row in the third matrix is ​​used to represent the behavioral weighted representation of a candidate item; Based on the third matrix, K candidate projects are selected from the N candidate projects; The K candidate items are recommended to the user.

2. The method according to claim 1, characterized in that, The first matrix is ​​calculated using the first type of computing unit on the artificial intelligence (AI) accelerator. The second matrix is ​​calculated using the second type of computing unit on the AI ​​accelerator.

3. The method according to claim 2, characterized in that, The first type of computing unit is a computing unit used for matrix operations, and the second type of computing unit is a general-purpose computing unit.

4. The method according to claim 2 or 3, characterized in that, The first type of computing unit and the second type of computing unit execute tasks in parallel.

5. The method according to any one of claims 1-4, characterized in that, The attention calculation performed on the N candidate items and the historical behavior sequence to obtain the first matrix includes: Perform matrix multiplication operations sequentially on the Q matrix, the first K matrix, and the first V matrix to obtain the first matrix.

6. The method according to any one of claims 1-5, characterized in that, The stepwise vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix to obtain the second matrix include: Perform a vector dot product operation on the Q matrix and the second K matrix to obtain the fourth matrix; Based on the number of columns of the second V matrix, the fourth matrix is ​​expanded by column copying to obtain the fifth matrix, wherein the number of columns of the fifth matrix is ​​the same as the number of columns of the second V matrix; Perform a matrix dot product operation on the fifth matrix and the second V matrix to obtain the second matrix.

7. The method according to any one of claims 1-6, characterized in that, The step of selecting K candidate projects from the N candidate projects based on the third matrix includes: Based on the third matrix, the probability of the user's interest in each of the candidate items is predicted. Based on the user's probability of interest in each of the candidate items, K candidate items are selected from the N candidate items, wherein the user's probability of interest in the K items is greater than the user's probability of interest in the remaining candidate items in the N candidate items.

8. The method according to any one of claims 1-7, characterized in that, Before selecting N candidate projects from the project library based on the user's historical behavior sequence, the process also includes: Obtain the user's user identifier; Based on the user identifier, the user's user logs are retrieved from the log database; Feature extraction is performed from the user logs to obtain the historical behavior sequence.

9. A recommended device, characterized in that, include: The filtering module is used to filter N candidate items from the item library based on the user's historical behavior sequence, where N≥2, and the candidate items are goods, content, advertisements or services; The processing module is used to perform attention calculation on the N candidate items and the historical behavior sequence to obtain a first matrix. In the attention calculation process, the query vector Q matrix is ​​obtained by linear transformation of the N candidate items, and the first key vector K matrix and the first value vector V matrix are obtained by different linear transformations of the historical behavior sequence. The processing module is further configured to perform vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix step by step to obtain the second matrix, wherein the second K matrix and the second V matrix are obtained by performing different linear transformations on the N candidate items; The processing module is further configured to perform matrix addition on the first matrix and the second matrix to obtain a third matrix, wherein a row in the third matrix is ​​used to represent a weighted representation of the behavior of a candidate item; The processing module is further configured to, based on the third matrix, select K candidate projects from the N candidate projects, and recommend the K candidate projects to the user.

10. The apparatus according to claim 9, characterized in that, The first matrix is ​​calculated using the first type of computing unit on the artificial intelligence (AI) accelerator. The second matrix is ​​calculated using the second type of computing unit on the AI ​​accelerator.

11. The apparatus according to claim 10, characterized in that, The first type of computing unit is a computing unit used for matrix operations, and the second type of computing unit is a general-purpose computing unit.

12. The apparatus according to claim 10 or 11, characterized in that, The first type of computing unit and the second type of computing unit execute tasks in parallel.

13. The apparatus according to any one of claims 9-12, characterized in that, When the processing module performs attention calculations on the N candidate items and the historical behavior sequence to obtain the first matrix, it is specifically used for: Perform matrix multiplication operations sequentially on the Q matrix, the first K matrix, and the first V matrix to obtain the first matrix.

14. The apparatus according to any one of claims 9-13, characterized in that, When the processing module performs vector dot product and matrix dot product operations on the Q matrix, the second K matrix, and the second V matrix step by step to obtain the second matrix, it is specifically used for: Perform a vector dot product operation on the Q matrix and the second K matrix to obtain the fourth matrix; Based on the number of columns of the second V matrix, the fourth matrix is ​​expanded by column copying to obtain the fifth matrix, wherein the number of columns of the fifth matrix is ​​the same as the number of columns of the second V matrix; Perform a matrix dot product operation on the fifth matrix and the second V matrix to obtain the second matrix.

15. The apparatus according to any one of claims 9-14, characterized in that, When the processing module selects K candidate items from the N candidate items based on the third matrix, it is specifically used for: Based on the third matrix, the probability of the user's interest in each of the candidate items is predicted. Based on the user's probability of interest in each of the candidate items, K candidate items are selected from the N candidate items, wherein the user's probability of interest in the K items is greater than the user's probability of interest in the remaining candidate items in the N candidate items.

16. The method according to any one of claims 9-15, characterized in that, Before filtering N candidate items from the item library based on the user's historical behavior sequence, the filtering module is also used for: Obtain the user's user identifier; Based on the user identifier, the user's user logs are retrieved from the log database; Feature extraction is performed from the user logs to obtain the historical behavior sequence.

17. A recommendation device, characterized in that, Includes at least one processor and interface; The at least one processor obtains program instructions through the interface; The at least one processor is configured to execute the program line instructions to implement the method as described in any one of claims 1-8.

18. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-8.

19. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-8, wherein the cluster of computing devices includes at least one computing device.

20. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-8, wherein the computing device cluster includes at least one computing device.

Images