Auto-regression type end-to-end personalized content recommendation method and system

Abstract

The invention provides an autoregressive end-to-end personalized content recommendation method and system, and the method comprises the following steps: S1, extracting a specific field and a universal field which are used for reflecting feature information, and then forming subject matter structured data; s2, converting the structured data of the subject matter into natural language representation, and then generating a subject matter feature vector; s3, constructing a user behavior sequence; s4, forming a user behavior feature vector sequence, analyzing the user behavior feature vector sequence by using an autoregression model, and generating an interest feature vector of the user at the current moment; step S5, generating a candidate subject matter sequence; and step S6, updating the user behavior sequence according to the selection result of the subject matter in the candidate subject matter sequence by the user. Experiments prove that the method is based on the deep learning technology, user behavior preference information can be effectively combined, the end-to-end personalized content recommendation effect is achieved, and different business scenes can be effectively served.

Description

Technical Field

[0001] The present invention relates to, in particular, an autoregressive end-to-end personalized content recommendation method and system. Background Art

[0002] As an important information service, personalized recommendation technology has been widely used in many platforms, such as e-commerce, social media, and news aggregation. However, traditional personalized recommendation systems have certain limitations and urgently need to be improved and optimized through new technologies.

[0003] Traditional recommendation algorithms, such as collaborative filtering and content-based recommendations, are unable to effectively utilize the rich information in interactive data, which limits the system's in-depth exploration of user behaviors and preferences, resulting in low accuracy. Traditional recommendation algorithms need to be retrained or adjust parameters when faced with changes in user behavior patterns, and their generalization capabilities are weak. When dealing with new users or new objects, it is necessary to collect sufficient long-term data for recommendations, but the real-time performance is poor. Usually, the recommended objects are those that the user has already come into contact with or are similar, making the recommendation results homogenized and lacking in diversity. New users or new objects are difficult to generate effective recommendations due to the lack of sufficient historical data, and there is a cold start problem.

[0004] With the development of artificial intelligence technology, deep neural network algorithms are gradually applied to recommendation systems. Compared with traditional recommendation algorithms, recommendation algorithms based on deep learning have significant advantages. In terms of accuracy, deep learning models can automatically learn complex features of data, handle more complex user behaviors and target attributes, and improve recommendation accuracy and relevance. In terms of robustness, there is no need to rely on manually designed features, and it has strong adaptability to noisy data, can extract useful information from noise, and improve the overall performance of the recommendation system. It can also capture the subtle differences in user behavior and provide more personalized and diversified recommendations.

[0005] The development of personalized recommendation technology not only improves user experience, but also helps enterprises and subject matter providers provide more targeted services. After the introduction of deep learning technology, personalized recommendation systems can analyze and understand user behaviors and preferences more deeply and achieve more accurate recommendations. And with the development of big data and cloud computing technology, personalized recommendation systems can process larger-scale data and provide a wider range of recommendation services. Recommendation systems not only create direct economic value for users and enterprises, but also have an impact on social informatization, personalized services, and knowledge dissemination, and have great research value and application value.

[0006] Through searching patent documents, it is found that CN108550068A provides a personalized product recommendation method and system based on user behavior analysis. The personalized product recommendation method based on user behavior analysis in this patent includes the following steps: extracting the attribute information of the product, extracting the user's access behavior to the product; obtaining the attenuation factor and product attribute information through the user's access behavior to the product based on the time dimension, and establishing a user model; obtaining the user's interest attribute information in each attribute dimension according to the user model; establishing a regression tree model based on product attributes according to the user's interest attribute information, product attribute information and the user's access behavior to the product; obtaining the user's candidate recommended product list according to the user's access record; using the regression tree model of product attributes to predict the user's candidate recommended product score; sorting the candidate recommended products according to the user's predicted score, and recommending the higher-ranked products to the user as candidate recommendations. This patent method is relatively complex and cumbersome, such as the need to calculate the attenuation factor in multiple steps to establish a user model.

[0007] In summary, in view of the above problems of the prior art, studying an autoregressive end-to-end personalized content recommendation method and system has become a task that needs to be solved urgently. Summary of the invention

[0008] In view of the defects in the prior art, the object of the present invention is to provide an autoregressive end-to-end personalized content recommendation method and system.

[0009] An autoregressive end-to-end personalized content recommendation method provided by the present invention comprises the following steps:

[0010] Step S1, extracting specific fields and general fields used to reflect characteristic information from the original subject matter, and then forming subject matter structured data;

[0011] Step S2, constructing a prompt word template based on the target object structured data, then converting the target object structured data into a natural language representation, and then processing the natural language representation using a text embedding model to generate a target object feature vector, and then storing the target object feature vector in a vector database;

[0012] Step S3, obtaining the user's historical interaction behavior data and constructing a user behavior sequence;

[0013] Step S4, based on the user behavior sequence and the processing method of the target object features in steps S1 and S2, extract the features of the target object in the user behavior sequence to form a user behavior feature vector sequence, analyze the user behavior feature vector sequence using an autoregressive model, and generate the user's interest feature vector at the current moment;

[0014] Step S5, based on the user's current interest feature vector, perform similarity search in the vector database to generate a candidate subject sequence;

[0015] Step S6: updating the user behavior sequence according to the user's selection result of the target object in the candidate target object sequence.

[0016] Preferably, in step S1, the general fields and specific fields of the original subject matter are first extracted. The general fields are general attributes of the subject matter, applicable to different scenarios, including three attributes: title, description, and category; the specific fields are further supplements and improvements to the information of the original subject matter, and the content of the specific fields depends on the actual recommendation scenario.

[0017] Preferably, in step S2, the content of the general field and the specific field is filled into the prompt word template <> to form a natural language representation, and the prompt word template is as follows:

[0018] Prompt_Template = [TITTLE]<>[DESC]<>[CLS]<>[META][F_1]<>[F_2]<>…, where [TITTLE], [DESC] and [CLS] are tags for general fields, [META] is a prefix tag for a specific field, and [F_1], [F_2], etc. are tags for specific fields.

[0019] Preferably, in step S2, the text embedding model is an embedding model using a BERT architecture, and the structured data represented by the natural language is input into the embedding model to generate a target feature vector.

[0020] Preferably, in step S3, the data is obtained by connecting to an external database and passing interface parameters.

[0021] Preferably, in step S4, based on the user behavior sequence s, s is divided into subsequences s according to the timestamp t. 1 and 2 , where s 1 is the behavior sequence before time t, s 2 is the behavior sequence after time t. 1 Sampling is performed to select objects with user satisfaction greater than m to form a sequence s p . Splicing p and 2 Construct the user's final behavior sequence s f , extract s f The characteristics of the winning bid are used to generate user behavior sequence characteristics through an autoregressive model.

[0022] Preferably, in step S4, the autoregressive model adopts a transformer architecture based on bidirectional attention and unidirectional attention mechanisms, wherein the bidirectional attention is used to model the user's long-term preferences, and the unidirectional attention is used to model the user's short-term behavior sequence.

[0023] Preferably, during the model training process, negative samples are constructed based on random sampling, and the parameters of the embedding model and the autoregressive model are optimized using the cross entropy loss function.

[0024] Preferably, in step S5, based on vector similarity, the top-k nearest target feature vectors are retrieved through the vector database, and linked to the corresponding target ID through the vector index, thereby generating a candidate target sequence of length k.

[0025] The present invention also provides an autoregressive end-to-end personalized content recommendation system, comprising:

[0026] Offline processing module: used to process the objects in the recommendation system, including extracting feature vectors of all objects in advance during the offline processing phase, and storing and subsequently retrieving them through the vector database;

[0027] Real-time recommendation module: By embedding the user's historical behavior sequence to form user behavior characteristics, the user behavior characteristics and preference characteristics are analyzed based on the autoregressive model to predict the subject matter that the current user is interested in. When the user generates new interaction data, the user behavior sequence is updated synchronously in real time, and a new prediction result is generated at the next moment;

[0028] The workflow of the offline processing module is decoupled from the workflow of the real-time recommendation module. When a new target appears in the recommendation system, the offline processing module executes the processing flow and applies the newly stored target to the workflow of the real-time recommendation module.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. The present invention improves the accuracy and diversity of personalized recommendations and designs a personalized content recommendation algorithm based on deep learning. It captures the semantic characteristics of the target object and the behavioral characteristics of the user through the embedding model and the autoregressive model, strengthens the modeling of user interest preferences and behavior sequences, thereby improving the personalized relevance and diversity of the recommendation results.

[0031] 2. The present invention improves the real-time performance of the recommendation system and proposes an end-to-end autoregressive recommendation framework, which aims to solve the computational complexity and real-time problems brought about by the recall-sorting-post-processing multi-stage processing pipeline in the mainstream recommendation method. Based on the autoregressive model, the computational complexity is greatly reduced, thereby improving the real-time performance of the recommendation system.

[0032] 3. The present invention solves the cold start problem for new targets and designs a target feature extraction module based on natural language embedding, namely, an offline processing module, which can independently process the data of new targets offline, thereby applying the new targets to the recommendation system in real time.

[0033] 4. The present invention first designs the recommendation system framework, divides the system into two parts: offline processing and real-time recommendation, then introduces the subject matter management and real-time recommendation functions, and adopts a personalized content recommendation algorithm process. Experiments have confirmed that this method is based on deep learning technology, can effectively combine user behavior preference information, achieve end-to-end personalized content recommendation effects, and can effectively serve different business scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0035] Figure 1 This is a process of an autoregressive end-to-end personalized content recommendation method in an embodiment of the present invention;

[0036] Figure 2 A framework diagram of an autoregressive end-to-end personalized content recommendation system in an embodiment of the present invention;

[0037] Figure 3 This is the behavior feature extraction process in the embodiment of the present invention;

[0038] Figure 4 This is the model training process in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0040] Personalized recommendations are mainly based on user behavior, preferences, historical data and other relevant information, using algorithms to predict products or services that users may be interested in, and recommend these contents to users. This kind of recommendation can be applied to a variety of scenarios, including but not limited to e-commerce, music and video streaming services, news aggregation, social media and search engines. The present invention aims to use a neural network algorithm based on deep learning to automatically learn users' long-term interest preferences and historical interaction information, predict users' potential interest characteristics, and use an end-to-end autoregressive model to push the content that users are most likely to be interested in.

[0041] Embodiment 1:

[0042] Figure 1 The present invention provides an end-to-end personalized content recommendation method according to an embodiment of the present invention.

[0043] like Figure 1 As shown, this embodiment provides an autoregressive end-to-end personalized content recommendation method, including the following steps:

[0044] Step S1, extracting specific fields and common fields used to reflect characteristic information from the original target object, and then forming target object structured data.

[0045] Specifically, the forms of the objects in different recommendation scenarios are diverse, and they need to be converted into a unified form of representation. First, the general fields and specific fields of the original objects are extracted. The general fields are general attributes of the objects and are applicable to different scenarios, including three attributes: title, description, and category. Specific fields are further supplemented and improved on the information of the objects. The content of specific fields depends on the actual recommendation scenario. For example, in product recommendations, specific fields include attributes such as price, brand, and sales volume; in video recommendations, specific fields include attributes such as duration, keywords, and playback volume.

[0046] Step S2, construct a prompt word template based on the target object structured data, then convert the target object structured data into a natural language representation, and then use the text embedding model to process the natural language representation to generate a target object feature vector, and then store the target object feature vector in a vector database.

[0047] Specifically, the content of the general field and the specific field is filled into the prompt word template <> to form a natural language representation. The prompt word template is as follows:

[0048] Prompt_Template = [TITTLE]<>[DESC]<>[CLS]<>[META][F_1]<>[F_2]<>…, where [TITTLE], [DESC] and [CLS] are tags for general fields, [META] is a prefix tag for a specific field, and [F_1], [F_2], etc. are tags for specific fields.

[0049] The tag itself has nothing to do with the data, but implies a specific purpose or intention, which is to help the embedding model better understand the input content.

[0050] In this embodiment, the text embedding model is an embedding model using the BERT architecture, and the structured data represented by the constructed natural language is input into the embedding model to generate a target feature vector.

[0051] Step S3, obtaining the user's historical interaction behavior data and constructing a user behavior sequence.

[0052] In this embodiment, the information is obtained through any method such as external database connection, interface parameter transmission, etc.

[0053] Step S4, based on the user behavior sequence and the processing method of the target object features in steps S1 and S2, extract the features of the target object in the user behavior sequence to form a user behavior feature vector sequence, use the autoregressive model to analyze the user behavior feature vector sequence, and generate the user's interest feature vector at the current moment.

[0054] Specifically, a method is adopted in which specific fields and common fields are extracted from the objects in the user behavior sequence and organized and integrated to form structured data of the objects, and a method is adopted in which a prompt word template is constructed based on the structured data of the objects, the structured data is converted into a natural language representation, and the natural language representation is processed using a text embedding model to generate a feature vector of the objects and store it in a vector database.

[0055] Figure 3 This is the behavior feature extraction process in the embodiment of the present invention.

[0056] like Figure 3 As shown, in this embodiment, based on the user behavior sequence s, s is divided into subsequences s according to the timestamp t 1 and 2 , where s 1 is the behavior sequence before time t, s 2 is the behavior sequence after time t. 1 Sampling is performed to select objects with user satisfaction greater than m (satisfaction is between 0 and 1, the higher the value, the more in line with the user's preference), forming a sequence s p . Splicing p and 2 Construct the user's final behavior sequence s f , extract s f The characteristics of the winning bid are used to generate user behavior sequence characteristics through an autoregressive model.

[0057] In step S4, the autoregressive model adopts the transformer architecture based on bidirectional attention and unidirectional attention mechanisms. The bidirectional attention is used to model the user's long-term preferences, and the unidirectional attention is used to model the user's short-term behavior sequence.

[0058] Figure 4 This is the model training process in an embodiment of the present invention.

[0059] like Figure 4As shown in the figure, during the model training process, negative samples are constructed based on random sampling, and the cross entropy loss function is used to optimize the parameters of the embedding model and the autoregressive model.

[0060] Step S5: Based on the user's current interest feature vector, similarity search is performed in the vector database to generate a candidate target sequence.

[0061] Specifically, based on vector similarity, the top-k nearest target feature vectors are retrieved through the vector database and linked to the corresponding target ID through the vector index, thereby generating a candidate target sequence of length k.

[0062] Step S6: updating the user behavior sequence according to the user's selection result of the target object in the candidate target object sequence.

[0063] Embodiment 2:

[0064] Figure 2 The present invention is a framework diagram of an autoregressive end-to-end personalized content recommendation system in an embodiment of the present invention.

[0065] like Figure 2 As shown, this embodiment provides an autoregressive end-to-end personalized content recommendation system, including:

[0066] Offline processing module: used to process the target objects in the recommendation system, including extracting feature vectors of all targets in advance during the offline processing stage, and storing and subsequently retrieving them through the vector database.

[0067] Real-time recommendation module: By embedding the user's historical behavior sequence to form user behavior characteristics, the user behavior characteristics and preference characteristics are analyzed based on the autoregressive model to predict the subject matter that the current user is interested in. When the user generates new interaction data, the user behavior sequence is updated synchronously in real time, and a new prediction result is generated at the next moment;

[0068] The workflow of the offline processing module is decoupled from the workflow of the real-time recommendation module. When new targets appear in the recommendation system, the offline processing module quickly and independently executes the processing flow and promptly applies the newly entered targets to the workflow of the real-time recommendation module.

[0069] The functions of the autoregressive end-to-end personalized content recommendation system of this embodiment are as follows:

[0070] 1. Object management, specifically, using a vector database to store the feature vectors of the object, and providing four methods for adding, deleting, rewriting, and searching for the structured information of the object. Among them, the adding method is used to store the data of the newly generated objects of the recommendation system; the deleting method is used to delete the existing data; the rewriting method is used to replace the original data with the rewritten object data; the searching method is used to search for the corresponding object information by specifying the id.

[0071] 2. Provide users with personalized real-time recommendation functions based on user behavior information. Specifically, based on the user's current behavior record, automatically analyze the user's potential preference information, parse the user's behavior feature vector, and based on vector similarity, generate a recommendation result sequence containing k objects for the user in real time.

[0072] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0073] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. An autoregressive end-to-end personalized content recommendation method, characterized in that: The following steps are involved: Step S1, extracting specific fields and general fields used to reflect characteristic information from the original subject matter, and then forming subject matter structured data; Step S2, constructing a prompt word template based on the target object structured data, then converting the target object structured data into a natural language representation, and then processing the natural language representation using a text embedding model to generate a target object feature vector, and then storing the target object feature vector in a vector database; Step S3, obtaining the user's historical interaction behavior data and constructing a user behavior sequence; Step S4, based on the user behavior sequence and the processing method of the target object features in steps S1 and S2, extract the features of the target object in the user behavior sequence to form a user behavior feature vector sequence, analyze the user behavior feature vector sequence using an autoregressive model, and generate an interest feature vector of the user at the current moment; Step S5, based on the user's current interest feature vector, perform similarity search in the vector database to generate a candidate subject sequence; Step S6: updating the user behavior sequence according to the user's selection result of the target object in the candidate target object sequence.

2. The autoregressive end-to-end personalized content recommendation method according to claim 1, characterized in that: In step S1, firstly, the general fields and specific fields of the original subject matter are extracted, wherein the general fields are general attributes of the subject matter, applicable to different scenarios, and include three attributes: title, description, and category; The specific fields further supplement and improve the information of the original subject matter, and the content of the specific fields depends on the actual recommendation scenario.

3. The autoregressive end-to-end personalized content recommendation method according to claim 1, characterized in that: In step S2, the content of the general field and the specific field is filled into the prompt word template <> to form a natural language representation. The prompt word template is as follows: Prompt_Template = [TITTLE]<>[DESC]<>[CLS]<>[META][F_1]<>[F_2]<>…, wherein [TITTLE], [DESC] and [CLS] are tags of the general field, [META] is the prefix tag of the specific field, and [F_1], [F_2], etc. are tags of specific fields.

4. The autoregressive end-to-end personalized content recommendation method according to claim 1, characterized in that: In step S2, the text embedding model is an embedding model using a BERT architecture, and the structured data represented by the natural language is input into the embedding model to generate a target feature vector.

5. The autoregressive end-to-end personalized content recommendation method according to claim 1, characterized in that: In the step S3, the data is obtained by connecting to an external database and passing interface parameters.

6. The autoregressive end-to-end personalized content recommendation method according to claim 1, characterized in that: In step S4, based on the user behavior sequence s, s is divided into subsequences s according to the timestamp t. 1 and 2 , where s 1 is the behavior sequence before time t, s 2 is the behavior sequence after time t, for s 1 Sampling is performed to select objects with user satisfaction greater than m to form a sequence s p , splicing p and 2 Construct the user's final behavior sequence s f , extract s f The characteristics of the winning bid are used to generate user behavior sequence characteristics through an autoregressive model.

7. The autoregressive end-to-end personalized content recommendation method according to claim 6, characterized in that: In step S4, the autoregressive model adopts a transformer architecture based on bidirectional attention and unidirectional attention mechanisms. The bidirectional attention is used to model the user's long-term preferences, and the unidirectional attention is used to model the user's short-term behavior sequence.

8. The autoregressive end-to-end personalized content recommendation method according to claim 7, characterized in that: During the model training process, negative samples are constructed based on random sampling, and the parameters of the embedding model and the autoregressive model are optimized using the cross entropy loss function.

9. The autoregressive end-to-end personalized content recommendation method according to claim 1, characterized in that: In step S5, based on the vector similarity, the top-k nearest target feature vectors are retrieved through the vector database, and linked to the corresponding target ID through the vector index, so as to generate a candidate target sequence with a length of k.

10. An autoregressive end-to-end personalized content recommendation system, using an autoregressive end-to-end personalized content recommendation method according to any one of claims 1 to 9, characterized in that: include: Offline processing module: used to process the objects in the recommendation system, including extracting feature vectors of all objects in advance during the offline processing phase, and storing and subsequently retrieving them through the vector database; Real-time recommendation module: By embedding the user's historical behavior sequence to form user behavior characteristics, the user behavior characteristics and preference characteristics are analyzed based on the autoregressive model to predict the subject matter that the current user is interested in. When the user generates new interaction data, the user behavior sequence is updated synchronously in real time, and a new prediction result is generated at the next moment; The workflow of the offline processing module is decoupled from the workflow of the real-time recommendation module. When a new target appears in the recommendation system, the offline processing module executes the processing flow and applies the newly stored target to the workflow of the real-time recommendation module.

Images