Content recommendation method, content feature extraction method, and apparatus

Abstract

A content recommendation method, relating to the technical field of artificial intelligence. The method comprises: acquiring a semantic embedding vector, a positional embedding vector, and a temporal embedding vector of each behavior item in a historical behavior sequence of a user; in a neural network model, performing attention processing on a first channel feature, and performing linear processing on a second channel feature to obtain a user feature, wherein the first channel feature comprises the semantic embedding vector, and the second channel feature comprises at least one of the positional embedding vector and the temporal embedding vector; and on the basis of the user feature, recommending to the user content that matches preferences of the user. In this way, by separating at least one of semantic information, positional information, and temporal information of each behavior item in the historical behavior sequence, the limitation of positional and / or temporal relationships can be removed, thereby achieving better efficiency and enabling finer-grained modeling of interactions among items in different dimensions, thus achieving better accuracy.

Description

A content recommendation method, a content feature extraction method and apparatus

[0001] This application claims priority to Chinese Patent Application No. 202510083075.9, filed on January 17, 2025, entitled “A Content Recommendation Method, Content Feature Extraction Method and Apparatus”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence (AI) technology, and in particular to a content recommendation method, a content feature extraction method, and an apparatus. Background Technology

[0003] Recommender systems are information filtering systems designed to predict and recommend products, services, or content that a user may be interested in. They provide personalized recommendations by analyzing data such as the user's historical behavior, preferences, and contextual information. Neural network models play a crucial role in recommender systems; they are typically built using convolutional neural networks (CNNs) or recurrent neural networks (RNNs). While these models excel at handling sequential data, they often lack the scaling law, meaning they struggle to maintain performance on large datasets, limiting their applicability to large-scale datasets.

[0004] In the technological evolution of recommender systems, attention-based techniques have been introduced and improved, leading to novel neural network models such as hierarchical sequential transduction units (HSTUs). These models optimize attention techniques, not only possessing scaling laws but also being able to model feature interactions more effectively. However, despite these advancements in feature interactions, they still have limitations when dealing with complex feature interaction problems in recommender systems, which may affect the overall recommendation performance. Summary of the Invention

[0005] This application provides a content recommendation method, a content feature extraction method, an apparatus, a computing device cluster, a computer storage medium, and a computer product, which can improve the overall recommendation effect of the recommendation system.

[0006] In a first aspect, this application provides a content recommendation method, comprising: obtaining semantic embedding vectors, positional embedding vectors, and temporal embedding vectors of each behavior item in a user's historical behavior sequence; performing attention processing on first channel features in a neural network model, and performing linear processing on second channel features to obtain user features, wherein the first channel features include semantic embedding vectors, and the second channel features include at least one of positional embedding vectors and temporal embedding vectors; and recommending content that matches the user's preferences to the user based on the user features.

[0007] Thus, in the content recommendation process, by separating at least one of the semantic, location, and time information of each behavior item in a user's historical behavior sequence, the limitations of location and / or time relationships can be removed, resulting in better efficiency. Furthermore, separating at least one of the semantic, location, and time information of each behavior item in the historical behavior sequence also allows for finer-grained modeling of the interaction relationships between items across different dimensions, leading to better accuracy.

[0008] In one possible implementation, within the neural network model, attention processing is applied to the first channel features, and linear processing is applied to the second channel features to obtain user features. This includes: in the neural network model, attention processing is applied to the first channel features, linear processing is applied to the second channel features, and linear processing is applied to the third channel features to obtain user features; wherein, the first channel features are semantic embedding vectors, the second channel features are positional embedding vectors, and the third channel features are temporal embedding vectors. In this way, by separating the semantic, positional, and temporal information of each behavior item in the historical behavior sequence, the limitations of positional and temporal relationships can be removed, resulting in better efficiency and enabling finer-grained modeling of the interaction relationships between items in different dimensions, leading to better accuracy.

[0009] In one possible implementation, within the neural network model, attention processing is applied to the first channel features, linear processing is applied to the second channel features, and linear processing is applied to the third channel features to obtain user features. This includes: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain the first feature; performing linear matrix operations on the second channel features and the V matrix to obtain the second feature; performing linear matrix operations on the third channel features and the V matrix to obtain the third feature; and obtaining the user features based on the first, second, and third features. In this way, the user's historical behavior sequence can be processed using a neural network model to obtain user features.

[0010] In one possible implementation, user features are obtained based on a first feature, a second feature, and a third feature. This includes: concatenating the first, second, and third features to obtain concatenated features; applying an information fusion matrix to the concatenated features to obtain fused features, wherein the weights of the first, second, and third features are different; and processing the fused features using a feedforward neural network layer with a SwiGLU activation function to obtain the user features. In this way, by modeling the fusion of multi-channel information and the implicit relationships between items in two independent stages, better modeling results can be achieved.

[0011] In one possible implementation, the first channel features include semantic embedding vectors and positional embedding vectors, and the second channel features are temporal embedding vectors.

[0012] In one possible implementation, the first channel feature includes a semantic embedding vector and a temporal embedding vector, and the second channel feature is a positional embedding vector.

[0013] In one possible implementation, the first channel feature is a semantic embedding vector, and the second channel feature includes a positional embedding vector and a temporal embedding vector.

[0014] In one possible implementation, within the neural network model, attention processing is performed on the first channel features, and linear processing is performed on the second channel features to obtain user features. This includes: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain the first feature; performing linear matrix operations on the second channel features and the V matrix to obtain the second feature; and obtaining the user features based on the first and second features.

[0015] Secondly, this application provides a content feature extraction method, comprising: obtaining semantic embedding vectors, positional embedding vectors, and temporal embedding vectors of each content item in a content sequence; performing attention processing on the first channel features in a neural network model, and performing linear processing on the second channel features to obtain content features of each content item in the content sequence, wherein the first channel features include semantic embedding vectors, and the second channel features include at least one of positional embedding vectors and temporal embedding vectors; and storing the content features in a storage medium.

[0016] Thus, during content feature extraction, by separating at least one of the semantic, positional, and temporal information of each content item in the content sequence, the limitations of positional and / or temporal relationships can be removed, resulting in better efficiency. Furthermore, separating at least one of the semantic, positional, and temporal information of each content item in the content sequence also allows for finer-grained modeling of the interaction relationships between items in different dimensions, leading to better accuracy.

[0017] In one possible implementation, in the neural network model, attention processing is performed on the first channel features, and linear processing is performed on the second channel features to obtain user features, including: in the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain user features; wherein, the first channel features are semantic embedding vectors, the second channel features are position embedding vectors, and the third channel features are temporal embedding vectors.

[0018] In one possible implementation, within the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain user features. This includes: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain a first feature; performing linear matrix operations on the second channel features and the V matrix to obtain a second feature; performing linear matrix operations on the third channel features and the V matrix to obtain a third feature; and obtaining content features based on the first, second, and third features.

[0019] Thirdly, this application provides a content recommendation apparatus, comprising: an acquisition module and a processing module. The acquisition module is used to acquire semantic embedding vectors, positional embedding vectors, and temporal embedding vectors of each behavior item in a user's historical behavior sequence. The processing module is used to perform attention processing on the first channel features and linear processing on the second channel features in a neural network model to obtain user features, wherein the first channel features include semantic embedding vectors, and the second channel features include at least one of positional embedding vectors and temporal embedding vectors. The processing module is also used to recommend content that matches the user's preferences based on the user features.

[0020] In one possible implementation, when the processing module performs attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain user features, it specifically performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features; wherein the first channel features are semantic embedding vectors, the second channel features are position embedding vectors, and the third channel features are temporal embedding vectors.

[0021] In one possible implementation, when the processing module performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features, it specifically performs the following steps: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain a first feature; performing linear matrix operations on the second channel features and the V matrix to obtain a second feature; performing linear matrix operations on the third channel features and the V matrix to obtain a third feature; and obtaining user features based on the first, second, and third features.

[0022] In one possible implementation, when the processing module obtains user features based on the first feature, the second feature, and the third feature, it specifically performs the following steps: concatenating the first feature, the second feature, and the third feature to obtain concatenated features; applying an information fusion matrix to the concatenated features to obtain fused features, wherein the weights of the first feature, the second feature, and the third feature are different; and processing the fused features using a feedforward neural network layer with a SwiGLU activation function to obtain user features.

[0023] In one possible implementation, the first channel features include semantic embedding vectors and positional embedding vectors, and the second channel features are temporal embedding vectors.

[0024] In one possible implementation, the first channel feature includes a semantic embedding vector and a temporal embedding vector, and the second channel feature is a positional embedding vector.

[0025] In one possible implementation, the first channel feature is a semantic embedding vector, and the second channel feature includes a positional embedding vector and a temporal embedding vector.

[0026] In one possible implementation, when the processing module performs attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain user features, it specifically performs the following: applying a parameter matrix to the first channel features to obtain a query vector Q matrix, a key vector K matrix, and a value vector V matrix; and performing self-attention calculation based on the Q matrix, K matrix, and V matrix to obtain the first feature.

[0027] Perform linear matrix operations on the second channel features and the V matrix to obtain the second feature; based on the first and second features, obtain the user features.

[0028] Fourthly, this application provides a content feature extraction apparatus, comprising: an acquisition module and a processing module. The acquisition module is used to acquire the semantic embedding vector, positional embedding vector, and temporal embedding vector of each content item in a content sequence. The processing module is used to perform attention processing on the first channel features and linear processing on the second channel features in a neural network model to obtain the content features of each content item in the content sequence, wherein the first channel features include a semantic embedding vector, and the second channel features include at least one of a positional embedding vector and a temporal embedding vector. The processing module is also used to store the content features in a storage medium.

[0029] In one possible implementation, when the processing module performs attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain user features, it specifically performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features; wherein the first channel features are semantic embedding vectors, the second channel features are position embedding vectors, and the third channel features are temporal embedding vectors.

[0030] In one possible implementation, when the processing module performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features, it specifically performs the following steps: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain a first feature; performing linear matrix operations on the second channel features and the V matrix to obtain a second feature; performing linear matrix operations on the third channel features and the V matrix to obtain a third feature; and obtaining content features based on the first, second, and third features.

[0031] Fifthly, this application provides a computing device cluster, including at least one computing device, each computing device including a processor and a memory; the processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the method described in the first aspect or any possible implementation of the first aspect, or performs the method described in the second aspect or any possible implementation of the second aspect.

[0032] Sixthly, this application provides a computer-readable storage medium including computer program instructions that, when executed by a cluster of computing devices, enable the cluster to perform the method described in the first aspect or any possible implementation thereof, or to perform the method described in the second aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0033] In a seventh aspect, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method described in the first aspect or any possible implementation thereof, or to perform the method described in the second aspect or any possible implementation thereof. Exemplarily, the cluster of computing devices may include one or more computing devices.

[0034] It is understood that the beneficial effects of the second to seventh aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0035] Figure 1 is a schematic diagram of the architecture of a recommendation model provided in an embodiment of this application;

[0036] Figure 2 is a schematic diagram of several possible structures of the adaptive multi-channel self-attention layer in the recommendation model architecture shown in Figure 1;

[0037] Figure 3 is a schematic diagram of the application process of the recommendation model architecture shown in Figure 1;

[0038] Figure 4 is a flowchart illustrating a content recommendation method provided in an embodiment of this application;

[0039] Figure 5 is a flowchart illustrating a content feature extraction method provided in an embodiment of this application;

[0040] Figure 6 is a schematic diagram of a content recommendation device provided in an embodiment of this application;

[0041] Figure 7 is a schematic diagram of the structure of a computing device provided in an embodiment of this application;

[0042] Figure 8 is a schematic diagram of the structure of a computing device cluster provided in an embodiment of this application;

[0043] Figure 9 is a schematic diagram of another computing device cluster provided in an embodiment of this application. Detailed Implementation

[0044] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0045] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0046] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0047] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0048] For example, this application aims to address the problem of feature interaction modeling in recommender systems, particularly when dealing with explicit and / or implicit feature interactions. In this application, by introducing a feature interaction-enhanced recommender model architecture, combined with adaptive multi-channel self-attention layers and / or multi-stage divide-and-conquer feedforward layers, the aim is to model the interaction relationships between items of different dimensions with finer granularity, while improving the accuracy and efficiency of the recommender system. The recommender model architecture is described below.

[0049] For example, Figure 1 shows a schematic diagram of the architecture of a recommendation model provided in an embodiment of this application. As shown in Figure 1, the recommendation model architecture mainly includes: an adaptive multi-channel self-attention layer 110 and a multi-stage feed-forward network (Multi-stage FFN) 120. Of course, the recommendation model architecture may also include embedding layers, hidden layers, or MLPs, etc., depending on the actual situation, and is not limited here.

[0050] In this recommendation model architecture, the input sequence can be processed through an embedding layer to obtain the semantic embedding vector of each item in the input sequence. The input sequence can be, but is not limited to, the user's historical behavior data.

[0051] The adaptive multi-channel self-attention layer 110 may include a semantic information channel, a positional information channel, and a temporal information channel. The semantic information channel is primarily used to perform attention calculations on the semantic embedding vectors of each item in the input sequence, enabling feature cross-referencing between the semantic information of each item in the input sequence. The positional information channel is primarily used to calculate the positional embedding vectors of each item in the input sequence, enabling feature cross-referencing between the positional information of each item in the input sequence. For example, the positional embedding vectors can be obtained by processing the positional information of each item in the input sequence through an embedding layer. The temporal information channel is primarily used to calculate the temporal embedding vectors of each item in the input sequence, enabling feature cross-referencing between the temporal information of each item in the input sequence. For example, the temporal embedding vectors can be obtained by processing the temporal information of each item in the input sequence through an embedding layer. In this way, by performing feature cross-referencing on different information in different channels, the loss of information caused by treating this information as a whole for feature cross-referencing can be avoided, thereby achieving more fine-grained modeling of the interaction relationships between items of different dimensions.

[0052] In the adaptive multi-channel self-attention layer 110, a parameter matrix can be applied to the semantic embedding vector of each item in the input sequence to obtain the query vector matrix Q, the key vector matrix K, and the value vector matrix V. Where Q = X × W Q K = X × W K V = X × W V X is the semantic embedding vector of each item in the input sequence. W Q W K and W V These are all learned weight matrices used for linear transformation. After calculating the Q, K, and V matrices, the results can be transmitted to the semantic information channel for attention calculation. For example, the calculation formula may be, but is not limited to, as follows: In some embodiments, the features obtained through attention calculation can also be referred to as "first features".

[0053] In addition, in the adaptive multi-channel self-attention layer 110, the positional information of each item in the input sequence can be transformed through an embedding layer to obtain the positional embedding feature A. pFurthermore, the temporal information of each item in the input sequence is transformed through an embedding layer to obtain the temporal embedding feature A. t Furthermore, the location can be embedded in feature A. p The V matrix is ​​transmitted to the location information channel for linear processing, such as location embedding feature A. p The V matrix can be used for linear matrix operations, etc.; and the time-embedded feature A can be used. t The V matrix is ​​transmitted to the time information channel for linear processing, such as the time-embedded feature A. t The V matrix can be used for linear matrix operations, etc. In some embodiments, the feature obtained by linear processing through the position information channel can also be called the "second feature". The feature obtained by linear processing through the time information channel can also be called the "third feature".

[0054] Furthermore, in the adaptive multi-channel self-attention layer 110, the features output from the semantic information channel, the positional information channel, and the temporal information channel can be concatenated to obtain concatenated features. For example, to model higher-order features in the input sequence, further feature interactions can be performed on the concatenated features to obtain higher-order features. For instance, a parameter matrix W can be applied to the semantic embedding vector of each item in the input sequence. u Then, the applied parameter matrix W u The obtained features and the concatenated features are used to perform matrix operations to further perform feature crossing. Of course, it is also possible to choose not to model the higher-order features in the input sequence, depending on the actual situation, and no restrictions are imposed here.

[0055] The multi-stage divide-and-conquer feedforward layer 120 is mainly divided into two processing stages. In the first stage (Stage 1), the multi-channel information features output by the adaptive multi-channel self-attention layer 110 are weighted using a weight matrix W1 to fuse the multi-channel information features, thereby better interacting important features and obtaining fused features. For example, the weight matrix W1 can also be called the "information fusion matrix". Then, a residual network is used to calculate the residuals of the semantic embedding vectors of each item in the input sequence and the fused features to better learn the information between features. In the second stage (Stage 2), the implicit interactions in the features are modeled by the SwiGLU-activated FFN deep network (learned by the FFN itself). For example, the features output by Stage 1 can be dimensionality-increasing using weight matrices W2 and W3 to increase the feature dimension. Then, the features obtained by dimensionality-increasing using the SwiGLU activation function are processed to generate a value between 0 and 1, which will be used as a gating signal. Next, the result of the SwiGLU activation function is combined with the features obtained by dimensionality upscaling using the weight matrix W3 to perform a Hadamard product, implementing a gating mechanism to control the flow of information. Finally, the result obtained by the Hadamard product is dimensionality-reduced using the weight matrix W4 to restore it to the original dimension or the dimension required by the model, thus obtaining the processing result for this stage. Further, the result of Stage 2 is compared with the semantic embedding vectors of each item in the input sequence X to calculate the residual, thus obtaining the final processing result of the multi-stage divide-and-conquer feedforward layer 120. In this way, in Stage 2, the activation level is dynamically adjusted according to the input, which can better model the implicit interactions in the features, thereby bringing better modeling results. For example, in the multi-stage divide-and-conquer feedforward layer 120, it can be understood, but is not limited to, as follows: first, the spliced ​​features from the adaptive multi-channel self-attention layer 110 are fused in multiple channels, and then the fused features are processed using a feedforward neural network layer with the SwiGLU activation function to obtain user features.

[0056] As can be seen from the above description, in the adaptive multi-channel self-attention layer 110, by separating semantic information (QK) from positional and temporal information, the limitations of positional and temporal relationships are removed, resulting in better efficiency. Furthermore, separating QK from position and time allows for finer-grained modeling of the interaction relationships between items in different dimensions, leading to better accuracy. In the multi-stage divide-and-conquer feedforward layer 120, by modeling the fusion of multi-channel information and the implicit relationships between items in two independent stages, better modeling results are achieved.

[0057] It should be understood that in the adaptive multi-channel self-attention layer 110, in addition to splitting semantics, time and location and constructing semantic information channels, time information channels and location information channels, one of the three can be arbitrarily split.

[0058] For example, as shown in Figure 2(A), only the location information can be stripped away, thus including both semantic and temporal information channels, as well as the location information channel. In this case, in the adaptive multi-channel self-attention layer 110, parameter matrices can be applied to the semantic and temporal embedding vectors of each item in the input sequence to obtain the Q matrix, K matrix, and V matrix. Where Q = X′ × W Q K = X′ × W K V = X′ × W V X′ represents the semantic embedding vector and temporal embedding vector of each item in the input sequence. W Q W K and W V These are all learned weight matrices. After calculating the Q, K, and V matrices, the results can be transmitted to the semantic and temporal information channels for attention calculation. The position embedding vector A for each item in the input sequence... p The location can be embedded in feature A p The calculated V matrix is ​​transmitted to the location information channel for linear processing. Further, the features output from the semantic & temporal information channels and the location information channel can be concatenated to obtain concatenated features, which are then processed further. For example, in the case shown in Figure 2(A), the feature obtained through attention calculation can be called the "first feature." The feature obtained through linear processing can be called the "second feature."

[0059] Alternatively, as shown in Figure 2(B), only the temporal information can be extracted. In this case, it can include both semantic and positional information channels, as well as a temporal information channel. Then, in the adaptive multi-channel self-attention layer 110, parameter matrices can be applied to the semantic embedding vector and positional embedding vector of each item in the input sequence to obtain the Q matrix, K matrix, and V matrix. Where Q = X″ × W Q K = X″ × W K V = X″ × W V X″ represents the semantic embedding vector and positional embedding vector of each item in the input sequence. W Q W K and W V These are all learned weight matrices. After calculating the Q, K, and V matrices, the results can be transmitted to the semantic and location information channels for attention calculation. As for the temporal embedding vector A of each item in the input sequence... tThe time-embedded feature A can be used. t The calculated V matrix is ​​transmitted to the time information channel for linear processing. Furthermore, the features output from the semantic & location information channels and the time information channel can be concatenated to obtain concatenated features, which are then processed further. For example, in the case shown in Figure 2(B), the feature obtained through attention calculation can also be called the "first feature." The feature obtained through linear processing can also be called the "second feature."

[0060] Alternatively, as shown in Figure 2(C), only the semantic information is stripped away. This can include two channels: a time information channel, a location information channel, and a semantic information channel. In this case, in the adaptive multi-channel self-attention layer 110, parameter matrices can be applied to the semantic embedding vector and location embedding vector of each item in the input sequence to obtain the Q matrix, K matrix, and V matrix. Where Q = X × W Q K = X × W K V = X × W V X is the semantic embedding vector of each item in the input sequence. W Q W K and W V These are all learned weight matrices. After calculating the Q, K, and V matrices, the results can be transmitted to the semantic information channel for attention calculation. As for the temporal embedding vector A of each item in the input sequence... t and position embedding vector A p The time-embedded feature A can be used. t Position embedding vector A p The calculated V matrix is ​​transmitted to the time and location information channels for linear processing. Further, the features output from the semantic information channel and the time and location information channels can be concatenated to obtain concatenated features, which are then processed further. For example, in the case shown in Figure 2(C), the feature obtained through attention calculation can be called the "first feature." The feature obtained through linear processing can be called the "second feature."

[0061] In addition, in the multi-stage divide-and-conquer feedforward layer 120 described above, it is also possible to choose to use only one stage, such as Stage 1 or Stage 2, and use it in combination with the adaptive multi-channel self-attention layer 110. The specific choice depends on the actual situation and is not limited here.

[0062] It should be understood that the recommendation model architecture described above can be applied to e-commerce platforms, video streaming services, social media, music streaming services, or educational platforms. Specifically, in e-commerce platforms, recommendation systems can be used for personalized product recommendations, improving user satisfaction and purchase rates. In video streaming services, video platforms can use recommendation systems to recommend content of interest to users, enhancing the user experience. In social media, social media platforms can use recommendation systems to recommend news, articles, or users that users may be interested in. In music streaming services, music platforms can use recommendation systems to recommend new songs or similar genres, improving the user's listening experience. In educational platforms, educational platforms can use recommendation systems to recommend relevant learning resources to users, helping them improve learning efficiency. In these application scenarios, the recommendation model architecture described above can effectively improve the performance of recommendation systems, providing users with more accurate and personalized recommendation services.

[0063] Furthermore, the recommendation model architecture described above can be applied to both offline and online phases. The following sections describe the process of using this recommendation model architecture in both phases.

[0064] (1) Offline stage

[0065] The offline phase primarily involves training a generative model built upon the aforementioned model architecture, and then using this trained model to infer user or item features. As shown in Figure 3, during training, data matching the model's intended use scenario can be selected from long-sequence data in recall, click-through rate (CTR), shallow conversion rate (CVR), or deep CVR scenarios. During this selection process, data cleaning, standardization, and feature extraction are performed to ensure data quality. Training data is then constructed from the selected data. For example, training data may include users' historical behavior data, etc. The generative model is then trained using this training data. After training, the generative model can be used to extract user or item features, which are then stored and provided to the downstream recommendation engine for use in the online phase. For instance, in an advertising recommendation scenario, the trained model can extract features from advertisements provided by advertisers to obtain the features of each advertisement to be delivered, which are then stored and provided to the downstream recommendation engine for online advertising delivery.

[0066] (2) Online phase

[0067] Referring again to Figure 3, the online phase primarily utilizes the generative model trained in the offline phase to process the user's historical behavior data, extracting user features and providing them to the downstream recommendation engine. The downstream recommendation engine then uses these online user features to filter content matching the user's needs from the offline user or item features, thus providing more accurate and personalized recommendation services. Of course, the online phase can also directly use the generative model trained in the offline phase for predictions; the specific approach depends on the actual situation and is not limited here.

[0068] The following describes the specific implementation process of the above recommendation model architecture.

[0069] For example, Figure 4 shows a flowchart of a content recommendation method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. For example, this method can be executed by a content recommendation device, which can be implemented by software and / or hardware, and can be configured in, but is not limited to, an electronic device or a server; typically, it can be configured on a server. As shown in Figure 4, the content recommendation method may include the following steps:

[0070] S401. Obtain the semantic embedding vector, location embedding vector, and time embedding vector of each behavior item in the user's historical behavior sequence.

[0071] In this embodiment, a user's historical behavior sequence can be a series of interactions and activities recorded by the user over time on a specific platform or service. These behaviors may include, but are not limited to: browsing records of products, articles, videos, or web pages; search records of the user within a search engine or application; purchase records of products or services purchased by the user; click and selection records of the user on advertisements, recommendation lists, or other interfaces; ratings and text reviews given by the user for products, services, or content; or playback records of the user on music or video streaming services, etc. The user's historical behavior can only be obtained in accordance with legal regulations and with the user's authorization and consent.

[0072] After obtaining the user's historical behavior sequence, features can be extracted from the user's historical behavior sequence using a neural network model to obtain the semantic embedding vector, positional embedding vector, and temporal embedding vector of each behavior item in the historical behavior sequence.

[0073] S402. In the neural network model, attention processing is performed on the first channel features, and linear processing is performed on the second channel features to obtain user features, wherein the first channel features include semantic embedding vectors, and the second channel features include at least one of position embedding vectors and time embedding vectors.

[0074] In this embodiment, attention processing can be performed on the first channel feature, which includes a semantic embedding vector, and linear processing can be performed on the second channel feature, which includes at least one of a positional embedding vector and a temporal embedding vector, to obtain user features in the neural network model. For example, the first channel feature can be a semantic embedding vector, and the second channel feature can include a positional embedding vector and a temporal embedding vector, as shown in Figure 2(C). Alternatively, the first channel feature can include a semantic embedding vector and a positional embedding vector, and the second channel feature can be a temporal embedding vector, as shown in Figure 2(B). Furthermore, the first channel feature can include a semantic embedding vector and a temporal embedding vector, and the second channel feature can be a positional embedding vector, as shown in Figure 2(A). In some embodiments, the first channel feature can be a semantic embedding vector, the second channel feature can be a positional embedding vector, and the third channel feature can be a temporal embedding vector, as shown in Figure 1. In this case, attention processing can be performed on the first channel feature, linear processing can be performed on the second channel feature, and linear processing can be performed on the third channel feature in the neural network model to obtain user features. For the attention processing and linear processing involved in S402, please refer to the relevant descriptions in Figure 1 or Figure 2 above, which will not be repeated here.

[0075] S403. Based on user characteristics, recommend content that matches the user's preferences.

[0076] In this embodiment, after determining user characteristics, content matching the user's preferences can be filtered from the content to be recommended based on the correlation between the user characteristics and the content to be recommended, and the filtered content can then be recommended to the user. For example, the content to be recommended can be advertisements, products, articles, or services, etc., and is not limited here.

[0077] Thus, in the content recommendation process, by separating at least one of the semantic, location, and time information of each behavior item in a user's historical behavior sequence, the limitations of location and / or time relationships can be removed, resulting in better efficiency. Furthermore, separating at least one of the semantic, location, and time information of each behavior item in the historical behavior sequence also allows for finer-grained modeling of the interaction relationships between items across different dimensions, leading to better accuracy.

[0078] For example, Figure 5 shows a schematic flowchart of a content feature extraction method provided in an embodiment of this application. It is understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. For example, this method can be executed by a content feature extraction device, which can be implemented by software and / or hardware, and can be configured in, but is not limited to, an electronic device or a server; typically, it can be configured on a server. S502 in Figure 5 can be referred to the relevant description of S402 in Figure 4 above, and will not be repeated here. As shown in Figure 5, the content feature extraction method may include the following steps:

[0079] S501. Obtain the semantic embedding vector, positional embedding vector, and temporal embedding vector of each content item in the content sequence.

[0080] In this embodiment, the content sequence may be, but is not limited to, user-provided content. The content in the content sequence may be advertisements, images, products, articles, or services, etc., and is not limited here. After obtaining the content sequence, features can be extracted from the content sequence using a neural network model to obtain the semantic embedding vector, positional embedding vector, and temporal embedding vector of each content item in the content sequence.

[0081] S502. In the neural network model, attention processing is performed on the first channel features, and linear processing is performed on the second channel features to obtain the content features of each content item in the content sequence. The first channel features include semantic embedding vectors, and the second channel features include at least one of position embedding vectors and time embedding vectors.

[0082] S503. Store the content features in the storage medium.

[0083] In this embodiment, after obtaining the content features, the content feature storage value can be stored in the storage medium.

[0084] Thus, during content feature extraction, by separating at least one of the semantic, positional, and temporal information of each content item in the content sequence, the limitations of positional and / or temporal relationships can be removed, resulting in better efficiency. Furthermore, separating at least one of the semantic, positional, and temporal information of each content item in the content sequence also allows for finer-grained modeling of the interaction relationships between items in different dimensions, leading to better accuracy.

[0085] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. In addition, the various embodiments and features described in the above embodiments can be combined according to actual conditions, and the combined solutions are still within the protection scope of this application.

[0086] Next, based on the methods in the above embodiments, a content recommendation device provided in this application will be introduced.

[0087] For example, Figure 6 shows a schematic diagram of the structure of a content recommendation device provided in an embodiment of this application. As shown in Figure 6, the content recommendation device 600 includes an acquisition module 601 and a processing module 602. The acquisition module 601 is used to acquire the semantic embedding vector, position embedding vector, and temporal embedding vector of each behavior item in the user's historical behavior sequence. The processing module 602 is used to perform attention processing on the first channel features and linear processing on the second channel features in a neural network model to obtain user features, wherein the first channel features include a semantic embedding vector, and the second channel features include at least one of a position embedding vector and a temporal embedding vector. The processing module 602 is also used to recommend content that matches the user's preferences based on the user features.

[0088] In one possible implementation, when the processing module 602 performs attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain user features, it is specifically used to: perform attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features; wherein the first channel features are semantic embedding vectors, the second channel features are position embedding vectors, and the third channel features are temporal embedding vectors.

[0089] In one possible implementation, when processing module 602 performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features, it specifically performs the following steps: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain a first feature; performing linear matrix operations on the second channel features and the V matrix to obtain a second feature; performing linear matrix operations on the third channel features and the V matrix to obtain a third feature; and obtaining user features based on the first, second, and third features.

[0090] In one possible implementation, when the processing module 602 obtains user features based on the first feature, the second feature, and the third feature, it specifically performs the following steps: concatenating the first feature, the second feature, and the third feature to obtain concatenated features; applying an information fusion matrix to the concatenated features to obtain fused features, wherein the weights of the first feature, the second feature, and the third feature are different; and processing the fused features using a feedforward neural network layer with a SwiGLU activation function to obtain user features.

[0091] In one possible implementation, the first channel features include semantic embedding vectors and positional embedding vectors, and the second channel features are temporal embedding vectors.

[0092] In one possible implementation, the first channel feature includes a semantic embedding vector and a temporal embedding vector, and the second channel feature is a positional embedding vector.

[0093] In one possible implementation, the first channel feature is a semantic embedding vector, and the second channel feature includes a positional embedding vector and a temporal embedding vector.

[0094] In one possible implementation, when processing module 602 performs attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain user features, it specifically performs the following steps: applying a parameter matrix to the first channel features to obtain a query vector matrix Q, a key vector matrix K, and a value vector matrix V; performing self-attention calculation based on the Q, K, and V matrices to obtain the first feature; performing linear matrix operations on the second channel features and the V matrix to obtain the second feature; and obtaining the user features based on the first and second features.

[0095] In some embodiments, both the acquisition module 601 and the processing module 602 shown in FIG6 can be implemented in software or in hardware. For example, the implementation of the acquisition module 601 will be described below. Similarly, the implementation of the processing module 602 can refer to the implementation of the acquisition module 601.

[0096] As an example of a software functional unit, module 601 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, or a container. Further, the aforementioned computing instance may be one or more. For example, module 601 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed within the same region or in different regions. Further, the multiple hosts / virtual machines / containers used to run the code may be distributed within the same availability zone (AZ) or in different AZs, each AZ including one or more geographically proximate data centers. Typically, a region may include multiple AZs.

[0097] Similarly, multiple hosts / virtual machines / containers used to run this code can be distributed within the same Virtual Private Cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Communication between two VPCs within the same region, as well as between VPCs in different regions, requires a communication gateway to be set up within each VPC to enable interconnection between VPCs.

[0098] As an example of a hardware functional unit, the acquisition module 601 may include at least one computing device, such as a server. Alternatively, the acquisition module 601 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be implemented using a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0099] The multiple computing devices included in the acquisition module 601 can be distributed in the same region or in different regions. Similarly, the multiple computing devices included in the acquisition module 601 can be distributed in the same Availability Zone (AZ) or in different AZs. Likewise, the multiple computing devices included in the acquisition module 601 can be distributed in the same Virtual Private Cloud (VPC) or in multiple VPCs. These multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0100] It should be noted that, in other embodiments, the acquisition module 601 can be used to execute any step in the content recommendation method described in the above embodiments, and the processing module 602 can also be used to execute any step in the content recommendation method described in the above embodiments. Furthermore, the steps implemented by the acquisition module 601 and the processing module 602 can be specified as needed. By having the acquisition module 601 and the processing module 602 respectively implement different steps in the content recommendation method described in the above embodiments, all the functions of the content recommendation device 600 shown in FIG. 6 can be achieved.

[0101] This application also provides a computing device 700. As shown in FIG7, the computing device 700 includes: a bus 702, a processor 704, a memory 706, and a communication interface 708. The processor 704, the memory 706, and the communication interface 708 communicate with each other via the bus 702. The computing device 700 may be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the computing device 700.

[0102] Bus 702 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 7, but this does not imply that there is only one bus or one type of bus. Bus 704 can include pathways for transmitting information between various components of computing device 700 (e.g., memory 706, processor 704, communication interface 708).

[0103] Processor 704 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0104] The memory 706 may include volatile memory, such as random access memory (RAM). The processor 704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0105] The memory 706 stores executable program code, and the processor 704 executes this executable program code to implement the functions of the acquisition module 601 and the processing module 602 shown in FIG. 6, thereby implementing the content recommendation method described in the above embodiments. That is, the memory 706 stores instructions for executing the content recommendation method described in the above embodiments.

[0106] Alternatively, the memory 706 stores executable code, which the processor 704 executes to implement the functions of the content recommendation device 600 shown in FIG. 6, thereby implementing the content recommendation method described in the above embodiments. That is, the memory 706 stores instructions for executing the content recommendation method described in the above embodiments.

[0107] The communication interface 708 uses transceiver modules, such as, but not limited to, network interface cards and transceivers, to enable communication between the computing device 700 and other devices or communication networks.

[0108] This application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0109] As shown in Figure 8, the computing device cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the content recommendation method described in the above embodiments.

[0110] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the content recommendation method described in the above embodiments. In other words, a combination of one or more computing devices 700 can jointly execute instructions for executing the content recommendation method described in the above embodiments.

[0111] It should be noted that the memories 706 in different computing devices 700 within the computing device cluster can store different instructions, which are used to execute some of the functions of the content recommendation device 600 shown in Figure 6. That is, the instructions stored in the memories 706 of different computing devices 700 can implement the functions of one or more modules in the acquisition module 601 and the processing module 602.

[0112] In some possible implementations, one or more computing devices in a computing device cluster can be connected via a network. This network can be a wide area network (WAN) or a local area network (LAN), etc. Figure 9 illustrates one possible implementation. As shown in Figure 9, two computing devices 700A and 700B are connected via a network. Specifically, they are connected to the network through communication interfaces in each computing device. In this type of possible implementation, the memory 706 in computing device 700A stores instructions for executing the functions of the acquisition module 601. Simultaneously, the memory 706 in computing device 700B stores instructions for executing the functions of the processing module 602.

[0113] It should be understood that the functions of computing device 700A shown in Figure 9 can also be performed by multiple computing devices 700. Similarly, the functions of computing device 700B can also be performed by multiple computing devices 700.

[0114] This application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similarly referred to the connection method of the computing device cluster described in Figures 8 and 9. The difference is that the memory 706 of one or more computing devices 700 in this computing device cluster can store the same instructions for executing the methods in the above embodiments.

[0115] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the aforementioned content recommendation method. In other words, a combination of one or more computing devices 700 can jointly execute the instructions for executing the aforementioned content recommendation method.

[0116] It should be understood that each step of the above method embodiments can be accomplished by hardware logic circuits or software instructions in a processor.

[0117] Based on the methods in the above embodiments, this application also provides a content recommendation device. The content recommendation device includes: an acquisition module, configured to acquire semantic embedding vectors, positional embedding vectors, and temporal embedding vectors of each content item in a content sequence; a processing module, configured to perform attention processing on the first channel features and linear processing on the second channel features in a neural network model to obtain content features of each content item in the content sequence, wherein the first channel features include semantic embedding vectors, and the second channel features include at least one of positional embedding vectors and temporal embedding vectors; the processing module is further configured to store the content features in a storage medium.

[0118] In some embodiments, when the processing module performs attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain user features, it is specifically used to: perform attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features; wherein, the first channel features are semantic embedding vectors, the second channel features are position embedding vectors, and the third channel features are temporal embedding vectors.

[0119] In some embodiments, when the processing module performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features in the neural network model to obtain user features, it specifically performs the following steps: applying a parameter matrix to the first channel features to obtain a query vector Q matrix, a key vector K matrix, and a value vector V matrix; performing self-attention calculation based on the Q matrix, K matrix, and V matrix to obtain a first feature; performing linear matrix operations on the second channel features and the V matrix to obtain a second feature; performing linear matrix operations on the third channel features and the V matrix to obtain a third feature; and obtaining content features based on the first feature, the second feature, and the third feature.

[0120] It should be understood that the above-described device is used to execute the content feature extraction method in the above embodiments. The implementation principle and technical effect of the corresponding program modules in the device are similar to those described in the above method. The working process of the device can be referred to the corresponding process in the above method, and will not be repeated here.

[0121] Based on the methods in the above embodiments, this application provides a computer-readable storage medium including computer program instructions. When executed by a cluster of computing devices including at least one computing device, the computer program instructions cause the cluster of computing devices to perform the methods in the above embodiments. Exemplarily, the computer-readable storage medium can be any available medium that the computing device can store, or a data storage device such as a data center containing one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives).

[0122] Based on the methods in the above embodiments, this application provides a computer program product containing instructions that, when executed by a cluster of computing devices containing at least one computing device, cause the cluster of computing devices to perform the methods in the above embodiments.

[0123] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0124] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0125] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0126] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of this application.

Claims

1. A content recommendation method, characterized in that, The method includes: Obtain the semantic embedding vector, location embedding vector, and time embedding vector of each behavior item in the user's historical behavior sequence; In the neural network model, attention processing is performed on the first channel features, and linear processing is performed on the second channel features to obtain user features, wherein the first channel features include the semantic embedding vector, and the second channel features include at least one of the location embedding vector and the time embedding vector; Based on the user characteristics, content that matches the user's preferences is recommended to the user.

2. The method according to claim 1, characterized in that, In the neural network model, attention processing is applied to the first channel features, and linear processing is applied to the second channel features to obtain user features, including: In the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain the user features; The first channel feature is a semantic embedding vector, the second channel feature is a positional embedding vector, and the third channel feature is a temporal embedding vector.

3. The method according to claim 2, characterized in that, In the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain the user features, including: Apply a parameter matrix to the first channel feature to obtain the query vector Q matrix, the key vector K matrix, and the value vector V matrix; Self-attention calculation is performed based on the Q matrix, the K matrix, and the V matrix to obtain the first feature; Perform linear matrix operations on the second channel feature and the V matrix to obtain the second feature; Perform linear matrix operations on the third channel feature and the V matrix to obtain the third feature; The user characteristics are obtained based on the first feature, the second feature, and the third feature.

4. The method according to claim 3, characterized in that, The process of obtaining the user characteristics based on the first feature, the second feature, and the third feature includes: The first feature, the second feature, and the third feature are concatenated to obtain the concatenated feature; An information fusion matrix is ​​applied to the splicing features to obtain fused features, wherein the weights of the first feature, the second feature, and the third feature are different; The fused features are processed using a feedforward neural network layer with a Swiglu activation function to obtain the user features.

5. The method according to claim 1, characterized in that, The first channel feature includes the semantic embedding vector and the positional embedding vector, and the second channel feature is the temporal embedding vector.

6. The method according to claim 1, characterized in that, The first channel feature includes the semantic embedding vector and the temporal embedding vector, and the second channel feature is the positional embedding vector.

7. The method according to claim 1, characterized in that, The first channel feature is the semantic embedding vector, and the second channel feature includes the position embedding vector and the time embedding vector.

8. The method according to any one of claims 1, 5-7, characterized in that, In the neural network model, attention processing is applied to the first channel features, and linear processing is applied to the second channel features to obtain user features, including: Apply a parameter matrix to the first channel feature to obtain the query vector Q matrix, the key vector K matrix, and the value vector V matrix; Self-attention calculation is performed based on the Q matrix, the K matrix, and the V matrix to obtain the first feature; Perform linear matrix operations on the second channel feature and the V matrix to obtain the second feature; The user characteristics are obtained based on the first feature and the second feature.

9. A content feature extraction method, characterized in that, The method includes: Obtain the semantic embedding vector, positional embedding vector, and temporal embedding vector for each content item in the content sequence; In the neural network model, attention processing is performed on the first channel features, and linear processing is performed on the second channel features to obtain the content features of each content item in the content sequence. The first channel features include the semantic embedding vector, and the second channel features include at least one of the position embedding vector and the time embedding vector. The content features are stored in a storage medium.

10. The method according to claim 9, characterized in that, In the neural network model, attention processing is applied to the first channel features, and linear processing is applied to the second channel features to obtain user features, including: In the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain the user features; The first channel feature is a semantic embedding vector, the second channel feature is a positional embedding vector, and the third channel feature is a temporal embedding vector.

11. The method according to claim 10, characterized in that, In the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain the user features, including: Apply a parameter matrix to the first channel feature to obtain the query vector Q matrix, the key vector K matrix, and the value vector V matrix; Self-attention calculation is performed based on the Q matrix, the K matrix, and the V matrix to obtain the first feature; Perform linear matrix operations on the second channel feature and the V matrix to obtain the second feature; Perform linear matrix operations on the third channel feature and the V matrix to obtain the third feature; The content features are obtained based on the first feature, the second feature, and the third feature.

12. A content recommendation device, characterized in that, include: The acquisition module is used to acquire the semantic embedding vector, location embedding vector, and time embedding vector of each behavior item in the user's historical behavior sequence; The processing module is used to perform attention processing on the first channel features and linear processing on the second channel features in a neural network model to obtain user features, wherein the first channel features include the semantic embedding vector, and the second channel features include at least one of the position embedding vector and the time embedding vector; The processing module is also used to recommend content that matches the user's preferences to the user based on the user characteristics.

13. The apparatus according to claim 12, characterized in that, In the neural network model, when the processing module performs attention processing on the first channel features and linear processing on the second channel features to obtain user features, it is specifically used for: In the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain the user features; The first channel feature is a semantic embedding vector, the second channel feature is a positional embedding vector, and the third channel feature is a temporal embedding vector.

14. The apparatus according to claim 13, characterized in that, In the neural network model, when the processing module performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features to obtain the user features, it is specifically used for: Apply a parameter matrix to the first channel feature to obtain the query vector Q matrix, the key vector K matrix, and the value vector V matrix; Self-attention calculation is performed based on the Q matrix, the K matrix, and the V matrix to obtain the first feature; Perform linear matrix operations on the second channel feature and the V matrix to obtain the second feature; Perform linear matrix operations on the third channel feature and the V matrix to obtain the third feature; The user characteristics are obtained based on the first feature, the second feature, and the third feature.

15. The apparatus according to claim 14, characterized in that, When the processing module obtains the user feature based on the first feature, the second feature, and the third feature, it is specifically used for: The first feature, the second feature, and the third feature are concatenated to obtain the concatenated feature; An information fusion matrix is ​​applied to the splicing features to obtain fused features, wherein the weights of the first feature, the second feature, and the third feature are different; The fused features are processed using a feedforward neural network layer with a Swiglu activation function to obtain the user features.

16. The apparatus according to claim 12, characterized in that, The first channel feature includes the semantic embedding vector and the positional embedding vector, and the second channel feature is the temporal embedding vector.

17. The apparatus according to claim 12, characterized in that, The first channel feature includes the semantic embedding vector and the temporal embedding vector, and the second channel feature is the positional embedding vector.

18. The apparatus according to claim 12, characterized in that, The first channel feature is the semantic embedding vector, and the second channel feature includes the position embedding vector and the time embedding vector.

19. The apparatus according to any one of claims 12, 16-18, characterized in that, In the neural network model, when the processing module performs attention processing on the first channel features and linear processing on the second channel features to obtain user features, it is specifically used for: Apply a parameter matrix to the first channel feature to obtain the query vector Q matrix, the key vector K matrix, and the value vector V matrix; Self-attention calculation is performed based on the Q matrix, the K matrix, and the V matrix to obtain the first feature; Perform linear matrix operations on the second channel feature and the V matrix to obtain the second feature; The user characteristics are obtained based on the first feature and the second feature.

20. A content feature extraction device, characterized in that, include: The acquisition module is used to acquire the semantic embedding vector, positional embedding vector, and temporal embedding vector of each content item in the content sequence; The processing module is used to perform attention processing on the first channel features and linear processing on the second channel features in the neural network model to obtain the content features of each content item in the content sequence, wherein the first channel features include the semantic embedding vector, and the second channel features include at least one of the position embedding vector and the time embedding vector; The processing module is also used to store the content features in a storage medium.

21. The apparatus according to claim 20, characterized in that, In the neural network model, when the processing module performs attention processing on the first channel features and linear processing on the second channel features to obtain user features, it is specifically used for: In the neural network model, attention processing is performed on the first channel features, linear processing is performed on the second channel features, and linear processing is performed on the third channel features to obtain the user features; The first channel feature is a semantic embedding vector, the second channel feature is a positional embedding vector, and the third channel feature is a temporal embedding vector.

22. The apparatus according to claim 21, characterized in that, In the neural network model, when the processing module performs attention processing on the first channel features, linear processing on the second channel features, and linear processing on the third channel features to obtain the user features, it is specifically used for: Apply a parameter matrix to the first channel feature to obtain the query vector Q matrix, the key vector K matrix, and the value vector V matrix; Self-attention calculation is performed based on the Q matrix, the K matrix, and the V matrix to obtain the first feature; Perform linear matrix operations on the second channel feature and the V matrix to obtain the second feature; Perform linear matrix operations on the third channel feature and the V matrix to obtain the third feature; The content features are obtained based on the first feature, the second feature, and the third feature.

23. A computing device cluster, characterized in that, It includes at least one computing device, each computing device including a processor and memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device to cause the cluster of computing devices to perform the method as described in any one of claims 1-8, or to perform the method as described in any one of claims 9-11.

24. A computer-readable storage medium, characterized in that, The method includes computer program instructions that, when executed by a cluster of computing devices, cause the cluster of computing devices to perform the method as described in any one of claims 1-8, or to perform the method as described in any one of claims 9-11, wherein the cluster of computing devices includes at least one computing device.

25. A computer program product containing instructions, characterized in that, When the instruction is executed by the computing device cluster, the computing device cluster causes the computing device cluster to perform the method as described in any one of claims 1-8, or to perform the method as described in any one of claims 9-11, wherein the computing device cluster includes at least one computing device.

Images