Interactive content prediction method, device, storage medium and electronic device

Abstract

The present invention provides an interactive content prediction method, device, storage medium and electronic device, which are applied to the field of artificial intelligence technology. The present invention collects the search and recommendation behavior data of target users on specified products, constructs a conversion relationship between the two to generate corresponding representations, and extracts search representation vectors and recommendation representation vectors from them. These vectors are combined with task description prompts to form interactive content prediction prompt vectors, which are input into a large language model to obtain personalized interactive content prediction results. The present invention realizes accurate and personalized user interactive content prediction by integrating the user behavior characteristics of the dual scenarios of search and recommendation, and then utilizing the generation and reasoning capabilities of the large language model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an interactive content prediction method, device, storage medium and electronic device. Background Art

[0002] In video platforms, search and recommendation functions are two basic application scenarios. Users actively enter queries to express their intentions through the search function, while through the recommendation function, they implicitly express their intentions through their feedback behavior. Although there are differences between these two behavior modes, the core issue is to understand the user's current intentions and show them appropriate content.

[0003] However, in the prior art, search and recommendation scenarios are usually modeled separately. Search involves steps such as query analysis, related content recall, sorting and refinement, etc. Recommendation involves real-time feature extraction, content recall, sorting and refinement, etc. Although this separate modeling method can achieve certain results in specific tasks, it still has significant limitations in achieving accurate personalized task reasoning output.

[0004] Therefore, how to effectively integrate the scene characteristics in the dual scenarios of search and recommendation to achieve accurate and personalized user interaction content prediction has become a technical problem that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of the above problems, the present invention provides an interactive content prediction method, device, storage medium and electronic device that overcome the above problems or at least partially solve the above problems. The technical solution is as follows:

[0006] An interactive content prediction method, comprising:

[0007] Obtaining user interaction behavior data of a target user within a preset period, wherein the user interaction behavior data includes user search behavior data and user recommendation behavior data of the target user on a specified Internet product;

[0008] Constructing a conversion relationship between the user search behavior data and the user recommendation behavior data to generate a search behavior representation and a recommendation behavior representation;

[0009] extracting a search representation vector from the search behavior representation and a recommendation representation vector from the recommendation behavior representation;

[0010] Combining the search representation vector and the recommendation representation vector as a context information vector and a task description prompt to generate an interactive content prediction prompt vector;

[0011] The interaction content prediction prompt vector is input into a large language model to obtain an interaction content prediction result of the target user output by the large language model.

[0012] An interactive content prediction device comprises: a user interactive behavior data acquisition unit, a behavior representation generation unit, a representation vector acquisition unit, an interactive content prediction prompt vector generation unit and an interactive content prediction result acquisition unit.

[0013] The user interaction behavior data acquisition unit is used to obtain the user interaction behavior data of the target user within a preset period, wherein the user interaction behavior data includes the user search behavior data and user recommendation behavior data of the target user on a specified Internet product;

[0014] The behavior representation generating unit is used to construct a conversion relationship between the user search behavior data and the user recommendation behavior data, and generate a search behavior representation and a recommendation behavior representation;

[0015] The representation vector obtaining unit is used to extract the search representation vector in the search behavior representation and the recommendation representation vector in the recommendation behavior representation;

[0016] The interactive content prediction prompt vector generating unit is used to combine the search representation vector and the recommendation representation vector as a context information vector and a task description prompt to generate an interactive content prediction prompt vector;

[0017] The interaction content prediction result obtaining unit is used to input the interaction content prediction prompt vector into a large language model to obtain the interaction content prediction result of the target user output by the large language model.

[0018] A computer-readable storage medium stores a program, which implements the interactive content prediction method when executed by a processor.

[0019] An electronic device comprises at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other via the bus; and the processor is used to call program instructions in the memory to execute the interactive content prediction method.

[0020] By means of the above technical scheme, the present invention provides an interactive content prediction method, device, storage medium and electronic device, which obtains the user interactive behavior data of the target user within a preset period, wherein the user interactive behavior data includes the user search behavior data and user recommendation behavior data of the target user on the specified Internet product; constructs a conversion relationship between the user search behavior data and the user recommendation behavior data, generates a search behavior representation and a recommendation behavior representation; extracts the search representation vector in the search behavior representation and the recommendation representation vector in the recommendation behavior representation; combines the search representation vector and the recommendation representation vector as the context information vector and the task description prompt to generate an interactive content prediction prompt vector; inputs the interactive content prediction prompt vector into the large language model to obtain the interactive content prediction result of the target user output by the large language model. The present invention learns user representation by combining the user behavior characteristics of the search and recommendation dual scenarios, and then uses the generation and reasoning capabilities of the large language model to effectively integrate the user behavior characteristics in the search and recommendation dual scenarios, and realize accurate and personalized user interactive content prediction.

[0021] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0023] Figure 1 A schematic diagram showing a flow chart of an implementation of an interactive content prediction method provided by an embodiment of the present invention;

[0024] Figure 2 A framework diagram of behavior representation pre-training provided by an embodiment of the present invention is shown;

[0025] Figure 3 A framework diagram of large language model reasoning provided by an embodiment of the present invention is shown;

[0026] Figure 4 A schematic diagram showing the structure of a behavior conversion module provided by an embodiment of the present invention is shown;

[0027] Figure 5 A schematic diagram showing the deployment position of the Gated xattn-dense network layer in large language model reasoning provided by an embodiment of the present invention is shown;

[0028] Figure 6 A schematic diagram illustrating a vLLM accelerated reasoning process provided by an embodiment of the present invention is shown;

[0029] Figure 7 A schematic diagram showing the structure of an interactive content prediction device provided by an embodiment of the present invention is shown;

[0030] Figure 8 A schematic structural diagram of an electronic device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0031] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0032] Existing technologies usually model search and recommendation scenarios separately. As basic functions, search and recommendation have been deeply integrated into major Internet platform products and have accumulated rich user behavior data. The task objectives of the two are similar, and both require accurate identification of user intent and rapid return of required content. The difference is that in the search scenario, users actively enter queries to express their search intent; while in the recommendation scenario, there is no explicit user input. In response to the problems of real-time feedback, multi-dimensional interest characterization, long-term and short-term interest modeling, multi-objective optimization, cold start exploration, etc. in these two scenarios, a series of data feature processing methods, model structure optimization methods, and traffic exploration methods have been proposed. Most of these methods model and optimize the two scenarios separately.

[0033] Recently, some joint modeling methods have emerged, trying to jointly model these two scenarios, construct two supervised learning subtasks, and learn from each other's feature information. For example: by learning the "inherent intent" derived from user behavior and the "demand intent" expressed in the search scenario, the accuracy of search and recommendation can be improved, or the three steps of extraction, alignment, and fusion can be connected to the prediction model of the search recommendation scenario to improve the prediction effect. On the other hand, combining user behavior modeling with the powerful generation and reasoning capabilities of the Large Language Model (LLM) and applying it to personalized recommendation tasks is also a hot research direction.

[0034] However, the above methods either do not consider the integration of the behavioral characteristics of the search and recommendation scenarios, or do not effectively combine user behavior modeling with LLM. In addition, the actual application of LLM to the recommendation system also faces problems such as response time not meeting real-time requirements, high concurrency pressure, and large resource consumption.

[0035] In summary, the existing technology cannot achieve accurate personalized task reasoning output, mainly due to the lack of comprehensive characterization and integration of user behavior characteristics in both search and recommendation scenarios, resulting in the model's insufficient understanding of user intent. At the same time, the high resource consumption and low real-time performance of the LLM reasoning process further limit its practical effect in large-scale applications. Therefore, innovative methods need to make breakthroughs in behavioral feature characterization, cross-scenario collaboration, and efficient application of large models to achieve more accurate and efficient personalized task reasoning output.

[0036] Based on this, an interactive content prediction method is provided in an embodiment of the present invention, which learns user representation by combining the user behavior characteristics of the search and recommendation scenes, and uses the cross-attention mechanism combined with the generation and reasoning capabilities of the large language model to better complete the personalized user interactive content reasoning task. In addition, in the large language model reasoning stage, the vLLM (Virtual Large Language Model) framework is used to accelerate the matrix multiplication calculation in the reasoning process and improve the reasoning speed of the large language model.

[0037] like Figure 1 As shown, a flowchart of an implementation of an interactive content prediction method provided by an embodiment of the present invention is provided. The method may include:

[0038] S100: Obtain user interaction behavior data of a target user within a preset period, wherein the user interaction behavior data includes user search behavior data and user recommendation behavior data of the target user on a specified Internet product.

[0039] The user interaction behavior data refers to the behavior data generated when the user interacts with the specified Internet product. In the embodiment of the present invention, the user interaction behavior data includes user search behavior data and user recommendation behavior data.

[0040] Among them, user search behavior data refers to the data generated when users search in Internet products, including search keywords, search time, and clicked search results, etc. For example, if a user searches for "AAAAA" at 18:00 on September 28, 2024, and then clicks on "AAAAA Season 4", then his search behavior is: (time="2024-09-28 18:00", uid=user id, click_content = "AAAAA Season 4" id, query=""AAAAA""). All these records are collected together to form user search behavior data.

[0041] Among them, user recommendation behavior data refers to the data generated when users interact with platform recommended content in Internet products, including viewing recommended content, clicking on recommended links, liking or sharing recommended content, etc. For example: the platform recommends "BBBBB Season 4", "CC", and "DDDD" to users at 18:00 on September 28, 2024, and then the user clicks on "BBBBB Season 4", then his recommendation behavior is: (time="2024-09-28 18:00", uid=user id, show_content=["BBBBB Season 4" id, "CC" id, "DDDD" id], click_content = ""BBBBB Season 4" id"), all these records are collected to form user recommendation behavior data.

[0042] Internet products refer to various services and applications provided through the Internet. These products can include software, platforms, websites and applications, designed to meet the different needs of users. Common categories of Internet products include: social media platforms, e-commerce websites, search engines, online entertainment, online education, financial technology products, instant messaging tools and online health services.

[0043] Optionally, the length of the preset period can be set according to specific application scenarios and requirements. For example, assuming the preset period is one month, all search behavior data and recommendation behavior data of the target user on the specified Internet product within this one month will be collected.

[0044] S110: Construct a conversion relationship between user search behavior data and user recommendation behavior data, and generate a search behavior representation and a recommendation behavior representation.

[0045] Among them, the conversion relationship refers to a mathematical or logical relationship that can be converted into each other after quantifying and modeling the association between user search behavior data and user recommendation behavior data using an attention algorithm.

[0046] Among them, the search behavior representation is the feature representation of the user search behavior data after fusing the associated features of the user recommendation behavior to the user search behavior through the attention algorithm, usually in the form of a vector or embedding, so as to concisely describe the user's search behavior characteristics.

[0047] Among them, the recommendation behavior representation is the feature representation of the user recommendation behavior data after fusing the correlation features of the user search behavior to the user recommendation behavior through the attention algorithm, usually in the form of a vector or embedding, so as to concisely describe the user's recommendation behavior characteristics.

[0048] Specifically, the embodiment of the present invention can construct a conversion relationship between user search behavior data and user recommendation behavior data based on the attention algorithm, and generate a search behavior representation and a recommendation behavior representation.

[0049] Among them, the Attention Mechanism is a method used to improve performance in deep learning and neural network models, including the Multi-head self-attention algorithm and the Multi-head Cross-Attention algorithm.

[0050] Furthermore, embodiments of the present invention can combine the use of a multi-head self-attention algorithm, a multi-head cross-attention algorithm, and a feedforward network algorithm to construct four conversion representation vectors between user search behavior data and user recommendation behavior data: search to recommendation, recommendation to search, search to search, and recommendation to recommendation. The recommendation to search conversion representation vector and the search to search conversion representation vector are combined to generate a search behavior representation. The search to recommendation conversion representation vector and the recommendation to recommendation conversion representation vector are combined to generate a recommendation behavior representation.

[0051] S120: Extract a search representation vector from the search behavior representation and a recommendation representation vector from the recommendation behavior representation.

[0052] Among them, the search representation vector is a high-dimensional vector selectively sampled from the search behavior representation.

[0053] Among them, the recommendation representation vector is a high-dimensional vector selectively sampled from the recommendation behavior representation.

[0054] The embodiment of the present invention can reduce the burden of subsequent model processing and improve the efficiency of the overall model by extracting the characterization vector.

[0055] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, the extracting of the search representation vector in the search behavior representation and the recommendation representation vector in the recommendation behavior representation may specifically include:

[0056] The search behavior representation and the recommendation behavior representation are respectively input into a perceptual resampler for data compression and feature extraction to obtain a search representation vector and a recommendation representation vector.

[0057] Among them, the Perceiver Resampler is a module for efficiently processing variable-length input sequences. Its purpose is to convert variable-length input sequences into fixed-length representations, making subsequent model processing more efficient and concise. The Perceiver Resampler uses the Attention Mechanism to select and aggregate relevant information in the input behavior representation sequence to generate a fixed-length representation vector, that is, the search representation vector is a vector form extracted from the search behavior representation by the Perceiver Resampler, and the recommendation representation vector is a vector form extracted from the recommendation behavior representation by the Perceiver Resampler.

[0058] S130 , combining the search representation vector and the recommendation representation vector as a context information vector and a task description prompt to generate an interactive content prediction prompt vector.

[0059] The context information vector refers to a combined vector constructed based on the search representation vector and the recommendation representation vector.

[0060] The task description prompt refers to the additional information provided to the model to clarify the goal and context of the current task and help the model make predictions or generate more accurately. For example: "What video will the user click next?", "What is the user's favorite video type?".

[0061] The interactive content prediction hint vector is a high-dimensional vector representation used to predict and generate content that users may be interested in or need. The interactive content prediction hint vector combines the user's search and recommendation behavior characteristics, as well as the context information vector and task description information to enhance the model's understanding of user needs, thereby providing more accurate predictions and recommendations.

[0062] The embodiment of the present invention can combine the search representation vector, the recommendation representation vector and the task description prompt through the cross-attention algorithm to generate an interactive content prediction prompt vector for input into the prediction model.

[0063] Among them, the Cross-Attention Algorithm is used to establish associations between different types of data representations to help the model better understand the relationship between them.

[0064] In an embodiment of the present invention, the context information vector provides the background of the user's historical interaction behavior, and the task description prompt provides the specific requirements of the current task. Combining the two through the cross-attention algorithm helps the large language model to more comprehensively understand and predict the user's next interaction content.

[0065] S140: Input the interactive content prediction prompt vector into the large language model to obtain the interactive content prediction result of the target user output by the large language model.

[0066] Among them, the interactive content prediction result refers to the output result after processing by the large language model, which is usually personalized prediction content for the target user, such as: "The next video the user clicks on is EEEE" and "The user's favorite video type is travel variety shows."

[0067] Specifically, the embodiment of the present invention may input the interactive content prediction hint vector into the decoder of the large language model, so that the decoder outputs a textually coherent interactive content prediction result according to the interactive content prediction hint vector.

[0068] Among them, the decoder of the large language model (LLM Decoder) is used to convert the internal representation or hidden state of the model into output text (such as words, sentences or paragraphs).

[0069] The present invention provides an interactive content prediction method, which includes: obtaining user interactive behavior data of a target user within a preset period, wherein the user interactive behavior data includes user search behavior data and user recommendation behavior data of the target user on a specified Internet product; constructing a conversion relationship between the user search behavior data and the user recommendation behavior data, generating a search behavior representation and a recommendation behavior representation; extracting a search representation vector in the search behavior representation and a recommendation representation vector in the recommendation behavior representation; combining the search representation vector and the recommendation representation vector as context information vectors and task description prompts to generate an interactive content prediction prompt vector; inputting the interactive content prediction prompt vector into a large language model to obtain an interactive content prediction result of the target user output by the large language model. The present invention learns user representation by combining the user behavior characteristics of the search and recommendation dual scenarios, and then utilizes the generation and reasoning capabilities of the large language model, which can effectively integrate the user behavior characteristics in the search and recommendation dual scenarios, and realize accurate and personalized user interactive content prediction.

[0070] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, the search representation vector and the recommendation representation vector are combined as the context information vector and the task description prompt to generate the interactive content prediction prompt vector, which may specifically include:

[0071] The search representation vector and the recommendation representation vector are input into the large language model, and the search representation vector and the recommendation representation vector are processed using the gating mechanism, cross-attention mechanism and dense network layer of the large language model to construct a context information vector; the task description prompt is vectorized and combined with the context information vector to generate an interactive content prediction prompt vector.

[0072] Specifically, the embodiment of the present invention may input the search representation vector and the recommendation representation vector into the gated cross-attention-dense network layer of the large language model for processing to construct a context information vector.

[0073] Among them, the gated cross-attention-dense network layer is a combination layer in the neural network, which uses the gated mechanism and the cross-attention mechanism to process the input and further transforms it through the dense network layer.

[0074] Among them, the gating mechanism is used to control the flow of information. By introducing a gating function (such as Sigmoid or Tanh), the model can selectively pass or suppress information.

[0075] Among them, the cross-attention mechanism is used to calculate the attention weights between different sequences, thereby combining information from different sources.

[0076] Among them, the dense network layer is used to further process and transform the feature representation from the attention mechanism.

[0077] Gated Cross-Attention-Dense Network Layer selectively passes information through the gating mechanism, fuses the contextual information of multiple inputs through the cross-attention mechanism, and further processes and transforms this information through the dense network layer. This combined structure enables the model to construct a more accurate representation of the input context, improving performance in natural language processing and multimodal tasks.

[0078] Specifically, the embodiment of the present invention can divide the task description prompt into smaller tokens. Optionally, the tokens can be words, subwords or characters. For example: assuming the task description prompt is "Natural language processing is interesting", the result of word segmentation is "["Natural language processing", "very", "interesting"]". The result of subword segmentation is "["natural", "language", "processing", "very", "interesting", "interesting"]". Then, the embodiment of the present invention can convert these tokens into vectors by lookup table or model embedding so that the computer can process them.

[0079] The embodiment of the present invention is based on inputting the search representation vector and the recommendation representation vector into the gated cross-attention-dense network layer of the large language model, and generating an interactive content prediction prompt vector in combination with the task description prompt. It can effectively screen and optimize the input information, capture the dependencies of complex search and recommendation dual-scene sequences, perform deep feature extraction, and comprehensively integrate user historical interaction behaviors and task requirements, thereby significantly improving the accuracy and personalization of interactive content prediction.

[0080] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, a conversion relationship between user search behavior data and user recommendation behavior data is constructed to generate search behavior representation and recommendation behavior representation, which may specifically include:

[0081] Based on the pre-built content embedding table and query embedding table, user interaction behavior vectors, user search behavior vectors and user recommendation behavior vectors are generated using user search behavior data and user recommendation behavior data.

[0082] Each row in the content embedding table represents a specific content item (such as an article, product, video, etc.), and each column represents a feature of the content item (usually a high-dimensional vector). These feature vectors are pre-trained through embedding algorithms (such as Word2Vec, GloVe, etc.) and are used to represent the position and relationship of content items in the semantic space.

[0083] The query embedding table stores the vector representation of user queries (such as search keywords, phrases, etc.). Each row corresponds to a query item, and each column is the embedding vector of the query item. The query embedding table helps convert the user's query items into a vector form that the model can process.

[0084] Among them, the user interaction behavior vector is a vector representation generated by integrating the user's search and recommendation interaction behaviors on the platform, which is used to capture the user's overall behavior patterns and preferences.

[0085] Among them, the user search behavior vector is a vector representation generated for the user's search behavior. These vectors contain the user's behavioral characteristics during the search process, such as search keywords, search time, search frequency, etc.

[0086] Among them, the user recommendation behavior vector is a vector representation generated for the user's behavior when recommending content on the platform. These vectors capture the user's interaction with the recommended content, such as recommended content, time, frequency, etc.

[0087] Specifically, the embodiments of the present invention can concatenate user interaction behaviors in user search behavior data and user recommendation behavior data according to the time of occurrence to generate a user interaction behavior sequence; use a pre-built content embedding table and a query embedding table to convert each user interaction behavior in the user interaction behavior sequence into a user interaction behavior mapping vector; and concatenate each user interaction behavior mapping vector to generate a user interaction behavior vector.

[0088] The user interaction behavior includes user search behavior and user recommendation behavior. The user interaction behavior sequence refers to a behavior sequence of at least one user search behavior and at least one user recommendation behavior generated by the target user within a preset period, sorted by occurrence time.

[0089] For ease of understanding, here is an example: Assume that the user's behavior record on a certain day is: 09:00: the user searched for the keyword "smartphone", 09:05: the user clicked on the recommended "latest smartphone review" article, 09:10: the user searched for the keyword "smartphone battery life", 09:20: the user clicked on the recommended "best battery life smartphone" video. Based on these behaviors, a user interaction behavior sequence can be generated in chronological order: "[("search","smartphone"), ("recommend","latest smartphone review"), ("search","smartphone battery life"), ("recommend","best battery life smartphone")]". If the embedding vector of "smartphone" in the query embedding table is [0.1, 0.2, 0.3], the embedding vector of "smartphone battery life" in the query embedding table is [0.4, 0.5, 0.6], the embedding vector of "latest smartphone reviews" in the content embedding table is [0.7, 0.8, 0.9], and the embedding vector of "best battery life smartphone" in the content embedding table is [1.0, 1.1, 1.2], then each user interaction behavior is converted into the corresponding user interaction behavior mapping vector "[("search",[0.1, 0.2, 0.3]),("recommend",[0.7, 0.8, 0.9]),("search",[0.4, 0.5, 0.6]), ("recommend",[1.0, 1.1, 1.2])]". Next, concatenate each user interaction behavior mapping vector to generate a comprehensive user interaction behavior vector: “[0.1, 0.2, 0.3, 0.7, 0.8, 0.9, 0.4, 0.5, 0.6, 1.0, 1.1, 1.2]”.

[0090] The embodiment of the present invention concatenates the interactive behaviors in the user search behavior data and the user recommendation behavior data in chronological order to generate a user interactive behavior sequence. Then, each interactive behavior is converted into a corresponding mapping vector using the pre-built content embedding table and query embedding table. Finally, these mapping vectors are concatenated together to generate a comprehensive user interactive behavior vector. This process captures the time series characteristics of user behavior and the relationship between different behaviors, providing basic data for further behavior analysis and recommendation systems.

[0091] Specifically, an embodiment of the present invention can concatenate user search behaviors in user search behavior data according to the time of occurrence to generate a user search behavior sequence; use a pre-built query embedding table to convert each user search behavior in the user search behavior sequence into a user search behavior mapping vector; and concatenate each user search behavior mapping vector to generate a user search behavior vector.

[0092] The user search behavior sequence refers to a behavior sequence of multiple user search behaviors generated by a target user within a preset period, sorted by occurrence time.

[0093] For ease of understanding, an example is given here: Assume that the user's search behavior record on a certain day is: 09:00: the user searched for the keyword "smart watch", 10:15: the user searched for the keyword "heart rate monitoring smart watch", 11:30: the user searched for the keyword "smart watch brand comparison. Based on these behaviors, a user search behavior sequence can be generated in chronological order: "["smart watch","heart rate monitoring smart watch","smart watch brand comparison"]". If the embedding vector of "smart watch" in the query embedding table is [0.1,0.2,0.3], the embedding vector of "heart rate monitoring smart watch" in the query embedding table is [0.4,0.5,0.6], and the embedding vector of "smart watch brand comparison" in the query embedding table is [0.7, 0.8,0.9], then each search behavior is converted into the corresponding search behavior mapping vector "[[0.1,0.2, Next, concatenate each user search behavior mapping vector to generate a comprehensive user search behavior vector: “[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]”.

[0094] The embodiment of the present invention concatenates the search behaviors in the user search behavior data in chronological order to generate a user search behavior sequence. Then, each search behavior is converted into a corresponding search behavior mapping vector using a pre-built query embedding table. Finally, these mapping vectors are concatenated together to generate a comprehensive user search behavior vector. This process captures the time series characteristics of user search behavior and provides basic data for further behavior analysis and recommendation systems.

[0095] Specifically, the embodiments of the present invention can concatenate user recommendation behaviors in user recommendation behavior data according to the time of occurrence to generate a user recommendation behavior sequence; use a pre-built content embedding table to convert each user recommendation behavior in the user recommendation behavior sequence into a user recommendation behavior mapping vector; and concatenate each user recommendation behavior mapping vector to generate a user recommendation behavior vector.

[0096] The user recommendation behavior sequence refers to a behavior sequence of multiple user recommendation behaviors generated for a target user within a preset period, sorted by occurrence time.

[0097] For ease of understanding, here is an example: Suppose the user's recommendation behavior record on a certain day is: 08:30: the platform recommended the article "How to choose a smart watch that suits you", and the user clicked on the recommended content; 09:45: the platform recommended the "Latest Smart Watch Reviews" video, and the user watched the recommended content; 11:00: the platform recommended the "Smart Watch and Health Monitoring" article, and the user clicked on the recommended content. Based on these behaviors, a user recommended behavior sequence can be generated in chronological order: "["How to choose a smart watch that suits you", "Latest smart watch reviews", "Smart watches and health monitoring"]". If the embedding vector of "How to choose a smart watch that suits you" in the content embedding table is [0.2, 0.3, 0.4], the embedding vector of "Latest smart watch reviews" in the content embedding table is [0.5, 0.6, 0.7], and the embedding vector of "Smart watches and health monitoring" in the content embedding table is [0.8, 0.9, 1.0], each recommended behavior in the user recommended behavior sequence is converted into a corresponding recommended behavior mapping vector "[[0.2, 0.3, 0.4], [0.5, 0.6, 0.7], [0.8, 0.9, 1.0]]". Next, concatenate each user recommendation behavior mapping vector to generate a comprehensive user recommendation behavior vector: “[0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]”.

[0098] The embodiment of the present invention concatenates the recommended behaviors in the user recommendation behavior data in chronological order to generate a user recommendation behavior sequence. Then, each recommended behavior is converted into a corresponding recommended behavior mapping vector using a pre-built content embedding table. Finally, these mapping vectors are concatenated together to generate a comprehensive user recommendation behavior vector. This process captures the time series characteristics of user recommendation behaviors and provides basic data for further behavior analysis and recommendation systems.

[0099] Based on the multi-head self-attention algorithm and the feedforward network algorithm, the user interaction behavior vector, the user search behavior vector and the user recommendation behavior vector are used to generate the search-to-recommendation conversion representation vector, the recommendation-to-search conversion representation vector, the search-to-search conversion representation vector and the recommendation-to-recommendation conversion representation vector.

[0100] Among them, the Multi-Head Self-Attention Algorithm uses multiple attention heads to calculate the correlation between different parts of the input, thereby capturing the dependencies between different positions in the input sequence. Each attention head works independently, and finally merges their output results. This mechanism can enhance the expressiveness of the model.

[0101] Among them, the feed-forward network algorithm is a basic form of neural network, in which information flows from the input layer through the hidden layer to the output layer without loops or feedback. Typically, the feed-forward network contains multiple layers of linear transformations and nonlinear activation functions to extract complex features from the input.

[0102] Among them, the search-to-recommendation (s2r) conversion representation vector is a representation vector obtained through multi-head self-attention and feedforward network conversion, which reflects the influence and conversion relationship of search behavior on recommendation behavior.

[0103] Among them, the recommendation-to-search (r2s) conversion representation vector is a representation vector obtained through multi-head self-attention and feedforward network transformation, which reflects the influence and conversion relationship of recommendation behavior on search behavior.

[0104] Among them, the search-to-search (s2s) conversion representation vector is a representation vector obtained through multi-head self-attention and feedforward network conversion, which reflects the dependency relationship between different search behaviors of users.

[0105] Among them, the recommendation-to-recommendation (r2r) conversion representation vector is obtained through multi-head self-attention and feedforward network transformation, which reflects the user's dependency relationship between different recommendation behaviors.

[0106] Specifically, an embodiment of the present invention can fuse the interaction behavior position code with the user interaction behavior vector and input the resultant into a multi-head self-attention module, and then connect it to a feedforward neural network module to generate a search-to-recommendation conversion representation vector and a recommendation-to-search conversion representation vector; fuse the search behavior position code with the user search behavior vector and input the resultant into a multi-head self-attention module, and then connect it to a feedforward neural network module to generate a search-to-search conversion representation vector; fuse the recommendation behavior position code with the user recommendation behavior vector and input the resultant into a multi-head self-attention module, and then connect it to a feedforward neural network module to generate a recommendation-to-recommendation conversion representation vector.

[0107] In natural language processing and sequence modeling tasks, positional encoding is a technique used to represent the position of elements in a sequence. Its role in deep learning models is to supplement the order information of elements in a sequence, because the usual embedding vectors do not contain position information.

[0108] In the embodiment of the present invention, position coding is used for vector representation of different types of behaviors (interaction behavior, search behavior, recommendation behavior), specifically, there are interaction behavior position coding, search behavior position coding and recommendation behavior position coding.

[0109] The interaction behavior position encoding is used to represent the position of each behavior in the user interaction behavior sequence. Interaction behaviors may include user search and recommendation behaviors. Position encoding can help the model understand the relative position of each behavior in the sequence.

[0110] Among them, search behavior position encoding is specifically used to represent the position information in the user's search behavior sequence. Position encoding of search behavior helps the model understand the position of each search query in the entire search sequence.

[0111] Among them, the recommendation behavior position encoding is specifically used to represent the position information in the user recommendation behavior sequence. Position encoding of the recommendation behavior helps the model understand the position of each recommendation content in the entire recommendation sequence.

[0112] After the embodiment of the present invention fuses the position encoding with the behavior vector, a transformation representation vector can be generated through a multi-head self-attention module and a feedforward neural network module. In this way, the model can better capture the relationship between the position information and the behavior in the sequence.

[0113] Based on the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm, the search behavior representation is generated using the recommendation-to-search conversion representation vector and the search-to-search conversion representation vector, and the recommendation behavior representation is generated using the search-to-recommendation conversion representation vector and the recommendation-to-recommendation conversion representation vector.

[0114] Among them, the Multi-Head Cross-Attention Algorithm involves the interaction between two different sequences. It calculates the correlation between one sequence and another sequence through multiple attention heads, allowing information to be exchanged between sequences.

[0115] Among them, search behavior representation refers to the comprehensive representation generated from search to recommendation conversion representation vector and search to search conversion representation vector through multi-head self-attention, multi-head cross-attention and feedforward network, which reflects the user's search behavior characteristics.

[0116] Among them, the recommendation behavior representation refers to the comprehensive representation generated from the recommendation-to-search conversion representation vector and the recommendation-to-recommendation conversion representation vector through multi-head self-attention, multi-head cross-attention and feedforward network, which reflects the user's recommendation behavior characteristics.

[0117] The embodiment of the present invention generates multiple behavior vectors by combining the user's search and recommendation behavior data through pre-built content embedding tables and query embedding tables, and processes these vectors using multi-head self-attention, feedforward network and multi-head cross-attention algorithms to generate different transformation representation vectors, and finally form the user's search behavior representation and recommendation behavior representation. This process can capture the complex dependencies and transformation patterns between user behaviors, thereby providing support for more accurate content recommendations and search optimization.

[0118] Optional, in the above Figure 1 On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, before constructing a conversion relationship between user search behavior data and user recommendation behavior data and generating search behavior representation and recommendation behavior representation, the method may further include:

[0119] Obtain user interaction training data, wherein the user interaction training data includes user search behavior training data and user recommendation behavior training data; based on the pre-constructed content embedding table and query embedding table, generate user interaction behavior training vectors, user search behavior training vectors and user recommendation behavior training vectors by using the user search behavior training data and the user recommendation behavior training data; based on the multi-head self-attention algorithm and the feedforward network algorithm, generate search-to-recommendation conversion representation training vectors, recommendation-to-search conversion representation training vectors, search-to-search conversion representation training vectors and recommendation-to-recommendation conversion representation training vectors by using the user interaction behavior training vectors, the user search behavior training vectors and the user recommendation behavior training vectors; based on the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm, generate search behavior training representations by using the recommendation-to-search conversion representation training vectors and the search-to-search conversion representation training vectors, and generate recommendation behavior training representations by using the search-to-recommendation conversion representation vectors and the recommendation-to-recommendation conversion representation vectors. The search behavior training representation and the recommendation behavior training representation are respectively input into the perceptual resampler to obtain the search representation training vector and the recommendation representation training vector; the search representation training vector, the recommendation representation training vector, the content feature vector of the user interaction training data, the user feature vector and the query feature vector are input into the multi-gated expert mixture model to obtain the next interaction prediction result output by the multi-gated expert mixture model; the next interaction prediction result is compared with the next interaction true result of the user interaction training data, and the loss function value is calculated; through the back propagation algorithm, the algorithm parameters of the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm are adjusted based on the loss function value, and the adjusted algorithm parameters are used to adjust the algorithm parameters for the next round until the algorithm parameter adjustment termination condition is reached.

[0120] Among them, the content feature vector (Item Embedding) represents the feature vector of the content (such as products, videos, articles, etc.). These feature vectors can include various attributes of the content, such as category, title, description, tag, etc. The content feature vector captures the semantic information of the content.

[0121] Among them, the user embedding represents the user's feature vector. These feature vectors can include various attributes of the user, such as user ID, age, gender, hobbies, historical behavior, etc. The user feature vector captures the user's preferences and behavior patterns.

[0122] The query embedding is a feature vector representing a query (e.g., a search keyword entered by a user). The query embedding captures information about the user’s current search intent.

[0123] Content feature vectors, user feature vectors, and query feature vectors are usually generated in advance before model training.

[0124] Among them, the Multi-gate Mixture-of-Experts (MMoE) model is a neural network architecture for multi-task learning. It aims to improve the efficiency and effectiveness of multi-task learning by sharing part of the model's parameters and assigning a dedicated module (expert network) to each task to better handle the specific needs of different tasks. The core idea of ​​the MMoE architecture is to add multiple expert networks and gating mechanisms on the basis of sharing the underlying network to dynamically select experts suitable for the current task.

[0125] Specifically, in each training iteration process, the embodiment of the present invention first calculates the interaction prediction results under the current algorithm parameters of the multi-head self-attention algorithm, the multi-head cross-attention algorithm, and the feedforward network algorithm, compares the interaction prediction results with the real interaction results, and calculates the loss function value. Next, the gradient of the loss function to each parameter is calculated by the back propagation algorithm. Then, the algorithm parameters of the multi-head self-attention, multi-head cross-attention, and feedforward network are updated according to the gradient information using an optimization algorithm (such as gradient descent or its variants). Through continuous iteration and optimization, the multi-head self-attention algorithm, the multi-head cross-attention algorithm, and the feedforward network algorithm gradually learn more effective feature representations and decision rules, thereby improving the overall performance. In practical applications, hyperparameters such as learning rate, the number of attention heads, and the depth of the feedforward network layer can be adjusted according to the needs of specific tasks to achieve the best results. In addition, regularization methods can be introduced to prevent overfitting and improve the robustness of the algorithm, so as to better adapt to different input data and task requirements.

[0126] Optionally, the algorithm parameter adjustment termination conditions may include: 1. After the preset number of training rounds is completed, the training process will end. 2. When the loss function value changes very little in multiple consecutive training rounds, it indicates that the model parameters have converged. 3. The preset performance target is achieved during the training process. 4. The set maximum training time or computing resource limit is reached.

[0127] The embodiment of the present invention pre-trains the algorithm parameters of the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm through a multi-gated expert mixture model, which can better learn and capture the complex relationship between interactive behaviors, thereby more deeply understanding the long-range dependency of sequence data and the interactive relationship between different data sources, improving the model's ability to capture complex behavior patterns, and facilitating the accurate integration and prediction of subsequent user interactive behaviors.

[0128] Optional, in the above Figure 1On the basis of one or more corresponding embodiments, in another optional embodiment provided by the embodiment of the present invention, the large language model adopts a vLLM reasoning framework.

[0129] Among them, the vLLM inference framework is a specially optimized large-scale language model inference and training framework. Its goal is to improve the efficiency and performance of large-scale language models (LLMs) during inference and training. vLLM usually includes various technologies and strategies to better utilize computing resources, reduce latency, improve throughput, and support larger-scale models and data.

[0130] By adopting the vLLM reasoning framework, the embodiments of the present invention can utilize various technologies and strategies to improve computing efficiency, reduce memory usage, support distributed computing and hardware acceleration, etc., thereby improving the performance and availability of large language models.

[0131] In order to facilitate understanding of the overall model reasoning process of the interactive content prediction method provided by the embodiment of the present invention, Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6 To explain: Figure 2 The figure shows a framework diagram of the behavior representation pre-training provided by an embodiment of the present invention. First, the user's behaviors in the two scenarios of search and recommendation are sorted in chronological order to form a behavior sequence. This sequence is input into the behavior conversion module to construct the behavior conversion features in the search and recommendation scenarios. The behavior conversion module outputs the search behavior representation and the recommendation behavior representation respectively. Next, the search behavior representation and the recommendation behavior representation are input into the Perceiver Resampler module, which can efficiently compress the variable-length sequence and generate a search representation training vector and a recommendation representation training vector of uniform length. Then, the search representation training vector Vs, the recommendation representation training vector Vr, and the content feature vector e are compressed. i , user feature vector e u and the query feature vector e q The input is fed into the multi-gated expert mixture model MMoE, which jointly models and predicts the next search and recommended click content.

[0132] Figure 3The figure shows a framework diagram of the large language model reasoning provided by an embodiment of the present invention. The reasoning process includes: inputting the search representation vector and the recommendation representation vector into the gated xattn-dense network layer, and combining these representation vectors with context information and task description prompts using the cross-attention mechanism in the gated xattn-dense network layer. The combined vector is input into the LLM Decoder (large language model decoder) to perform personalized task reasoning and generate output results. The reasoning process is accelerated through the vLLM framework to improve reasoning efficiency.

[0133] The structure of the behavior conversion module is as follows: Figure 4 As shown in Figure 1, the behavior conversion module uses the attention mechanism to model the conversion relationship between search and recommendation behaviors and generate representation vectors of search and recommendation behaviors. There are four types of conversion relationships: search to search (H(s2s)), search to recommendation (H(s2r)), recommendation to search (H(r2s)), and recommendation to recommendation (H(r2r)). First, two embedding tables are created in advance: content embedding table M i and query the embedded table M w . The user's search and recommendation behaviors are concatenated according to the occurrence time to form the user interaction behavior vector Eu. After the user's behavior is disassembled by type and concatenated according to the occurrence time, the user search behavior vector Es and the user recommendation behavior vector Er are generated. The interaction behavior position code Pu is added to the user interaction behavior vector Eu, the search behavior position code Ps is added to the user search behavior vector Es, and the recommendation behavior position code Pr is added to the user recommendation behavior vector Er. The recommendation behavior position code Pr is fused with the user recommendation behavior vector Er and input into the Multi-head self-attention (MSA) module, and then connected to the Feed-forward network (FFN) module to obtain the recommendation-to-recommendation conversion representation vector Hr2r. The search behavior position code Ps is fused with the user search behavior vector Es and input into the MSA module, and then connected to the FFN module to obtain the search-to-search conversion representation vector Hs2s. The interaction behavior position code Pu is fused with the user interaction behavior vector Eu and input into the MSA module, and then connected to the FFN module to obtain the recommendation-to-search conversion representation vector Hr2s and the search-to-recommendation conversion representation vector Hs2r. The recommendation-to-recommendation conversion representation vector Hr2r and the search-to-recommendation conversion representation vector Hs2r are fused through the MSA module, the Multi-head Cross-Attention (MCA) module, and the FFN module to generate the recommendation behavior representation Vr. The recommendation-to-search conversion representation vector Hr2s and the search-to-search conversion representation vector Hs2s are fused through the MSA module, the MCA module, and the FFN module to generate the search behavior representation Vs.

[0134] Among them, the deployment location of the gated xattn-dense network layer in the large language model inference is as follows Figure 5 As shown in the figure, the gated xattn-dense network layer receives the search representation vector and recommendation representation vector output by the Perceiver Resampler module, and uses the cross-attention method to construct the input context representation of the large model. The generated context representation is combined with the input task description prompt. The large language model is used to reason about the combined input and obtain the reasoning result of the corresponding task.

[0135] Among them, vLLM accelerates the reasoning process such as Figure 6 As shown in the figure, vLLM significantly improves the inference efficiency and performance of the model through innovative memory management strategies. The core points of the vLLM framework include the PagedAttention algorithm and memory optimization technology. The PagedAttention algorithm allows the tensors of the key and value of the input token to be stored in non-contiguous memory space, and manages memory by mapping logical blocks and physical blocks, thereby reducing memory waste and improving memory usage efficiency. The memory optimization technology uses quantization technologies such as FP16 to reduce memory usage and speed up calculations, and optimizes the CUDA kernel to achieve more efficient memory access and calculations.

[0136] Although operations are depicted in a particular order, this should not be understood as requiring that the operations be performed in the particular order shown or in a sequential order.Multitasking and parallel processing may be advantageous under certain circumstances.

[0137] It should be understood that the various steps described in the method embodiments of the present invention may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present invention is not limited in this respect.

[0138] Corresponding to the above method embodiment, the embodiment of the present invention further provides an interactive content prediction device, whose structure is as follows: Figure 7 As shown, it may include: a user interaction behavior data obtaining unit 10, a behavior representation generating unit 20, a representation vector obtaining unit 30, an interaction content prediction prompt vector generating unit 40 and an interaction content prediction result obtaining unit 50.

[0139] The user interaction behavior data acquisition unit 10 is used to obtain the user interaction behavior data of the target user within a preset period, wherein the user interaction behavior data includes the user search behavior data and user recommendation behavior data of the target user on a specified Internet product.

[0140] The behavior representation generating unit 20 is used to construct a conversion relationship between the user search behavior data and the user recommendation behavior data, and to generate a search behavior representation and a recommendation behavior representation.

[0141] The representation vector obtaining unit 30 is used to extract the search representation vector in the search behavior representation and the recommendation representation vector in the recommendation behavior representation.

[0142] The interactive content prediction hint vector generating unit 40 is used to combine the search representation vector and the recommendation representation vector as the context information vector and the task description hint to generate an interactive content prediction hint vector.

[0143] The interaction content prediction result obtaining unit 50 is used to input the interaction content prediction prompt vector into the large language model to obtain the interaction content prediction result of the target user output by the large language model.

[0144] Optionally, the interactive content prediction hint vector generating unit 40 may specifically include: a context information vector constructing subunit and an interactive content prediction hint vector subunit.

[0145] The context information vector construction subunit is used to input the search representation vector and the recommendation representation vector into the large language model, and use the gating mechanism, cross-attention mechanism and dense network layer of the large language model to process the search representation vector and the recommendation representation vector to construct the context information vector.

[0146] The interactive content prediction hint vector subunit is used to vectorize the task description hint and combine it with the context information vector to generate an interactive content prediction hint vector.

[0147] Optionally, the behavior representation generating unit 20 may specifically include: a behavior vector generating subunit, a conversion representation vector generating subunit and a behavior representation generating subunit.

[0148] The behavior vector generation subunit is used to generate user interaction behavior vectors, user search behavior vectors and user recommendation behavior vectors based on the pre-built content embedding table and query embedding table and using user search behavior data and user recommendation behavior data.

[0149] The conversion representation vector generation subunit is used to generate a search-to-recommendation conversion representation vector, a recommendation-to-search conversion representation vector, a search-to-search conversion representation vector and a recommendation-to-recommendation conversion representation vector based on a multi-head self-attention algorithm and a feedforward network algorithm, using user interaction behavior vectors, user search behavior vectors and user recommendation behavior vectors.

[0150] The behavior representation generation subunit is used to generate search behavior representation based on the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm, using the recommendation-to-search conversion representation vector and the search-to-search conversion representation vector, and to generate recommendation behavior representation using the search-to-recommendation conversion representation vector and the recommendation-to-recommendation conversion representation vector.

[0151] Optionally, the behavior vector generating subunit can be specifically used to concatenate user interaction behaviors in user search behavior data and user recommendation behavior data according to the occurrence time to generate a user interaction behavior sequence; use a pre-built content embedding table and a query embedding table to convert each user interaction behavior in the user interaction behavior sequence into a user interaction behavior mapping vector; concatenate each user interaction behavior mapping vector to generate a user interaction behavior vector; concatenate user search behaviors in the user search behavior data according to the occurrence time to generate a user search behavior sequence; use a pre-built query embedding table to convert each user search behavior in the user search behavior sequence into a user search behavior mapping vector; concatenate each user search behavior mapping vector to generate a user search behavior vector; concatenate user recommendation behaviors in the user recommendation behavior data according to the occurrence time to generate a user recommendation behavior sequence; use a pre-built content embedding table to convert each user recommendation behavior in the user recommendation behavior sequence into a user recommendation behavior mapping vector; concatenate each user recommendation behavior mapping vector to generate a user recommendation behavior vector.

[0152] Optionally, the conversion representation vector generating subunit can be specifically used to fuse the interaction behavior position encoding with the user interaction behavior vector and input the resultant into a multi-head self-attention module, and then connect to a feedforward neural network module to generate a search-to-recommendation conversion representation vector and a recommendation-to-search conversion representation vector; fuse the search behavior position encoding with the user search behavior vector and input the resultant into a multi-head self-attention module, and then connect to a feedforward neural network module to generate a search-to-search conversion representation vector; fuse the recommendation behavior position encoding with the user recommendation behavior vector and input the resultant into a multi-head self-attention module, and then connect to a feedforward neural network module to generate a recommendation-to-recommendation conversion representation vector.

[0153] Optionally, the interactive content prediction device may also include: a user interaction training data acquisition unit, an interactive behavior training vector generation unit, a conversion representation training vector generation unit, a behavior training representation generation unit, a representation training vector acquisition unit, a next interaction prediction result acquisition unit, a loss function value calculation unit and an algorithm parameter adjustment unit.

[0154] The user interaction training data acquisition unit is used for the behavior representation generation unit 20 to construct a conversion relationship between user search behavior data and user recommendation behavior data based on the attention algorithm, and obtain user interaction training data before generating search behavior representation and recommendation behavior representation, wherein the user interaction training data includes user search behavior training data and user recommendation behavior training data.

[0155] The interactive behavior training vector generating unit is used to generate user interactive behavior training vectors, user search behavior training vectors and user recommendation behavior training vectors based on pre-built content embedding tables and query embedding tables and using user search behavior training data and user recommendation behavior training data.

[0156] The conversion representation training vector generation unit is used to generate a search-to-recommendation conversion representation training vector, a recommendation-to-search conversion representation training vector, a search-to-search conversion representation training vector, and a recommendation-to-recommendation conversion representation training vector based on a multi-head self-attention algorithm and a feedforward network algorithm, using user interaction behavior training vectors, user search behavior training vectors, and user recommendation behavior training vectors.

[0157] The behavior training representation generation unit is used to generate a search behavior training representation based on a multi-head self-attention algorithm, a multi-head cross-attention algorithm and a feedforward network algorithm, using a recommendation-to-search conversion representation training vector and a search-to-search conversion representation training vector, and to generate a recommendation behavior training representation using a search-to-recommendation conversion representation vector and a recommendation-to-recommendation conversion representation vector.

[0158] The representation training vector obtaining unit is used to input the search behavior training representation and the recommendation behavior training representation into the perceptual resampler respectively to obtain the search representation training vector and the recommendation representation training vector.

[0159] The next interaction prediction result obtaining unit is used to input the search representation training vector, the recommendation representation training vector, the content feature vector of the user interaction training data, the user feature vector and the query feature vector into the multi-gated expert mixture model to obtain the next interaction prediction result output by the multi-gated expert mixture model.

[0160] The loss function value calculation unit is used to compare the next interaction prediction result with the next interaction actual result of the user interaction training data to calculate the loss function value.

[0161] The algorithm parameter adjustment unit is used to adjust the algorithm parameters of the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm based on the loss function value through the back propagation algorithm, and use the adjusted algorithm parameters to adjust the algorithm parameters for the next round until the algorithm parameter adjustment termination condition is reached.

[0162] Optionally, the representation vector obtaining unit 30 may be specifically used to input the search behavior representation and the recommendation behavior representation into a perceptual resampler for data compression and feature extraction, respectively, to obtain a search representation vector and a recommendation representation vector.

[0163] The present invention provides an interactive content prediction device, which is used to: obtain user interactive behavior data of a target user within a preset period, wherein the user interactive behavior data includes user search behavior data and user recommendation behavior data of the target user on a specified Internet product; construct a conversion relationship between the user search behavior data and the user recommendation behavior data, and generate a search behavior representation and a recommendation behavior representation; extract a search representation vector in the search behavior representation and a recommendation representation vector in the recommendation behavior representation; combine the search representation vector and the recommendation representation vector as context information vectors and task description prompts to generate an interactive content prediction prompt vector; input the interactive content prediction prompt vector into a large language model to obtain the interactive content prediction result of the target user output by the large language model. The present invention learns user representation by combining the user behavior characteristics of the search and recommendation dual scenarios, and then uses the generation and reasoning capabilities of the large language model to effectively integrate the user behavior characteristics in the search and recommendation dual scenarios, and realize accurate and personalized user interactive content prediction.

[0164] Regarding the device in the above embodiment, the specific manner in which each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0165] The interactive content prediction device includes a processor and a memory. The above-mentioned user interactive behavior data acquisition unit 10, behavior representation generation unit 20, representation vector acquisition unit 30, interactive content prediction prompt vector generation unit 40 and interactive content prediction result acquisition unit 50 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0166] The processor contains a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set. By adjusting the kernel parameters, the user behavior characteristics of the dual scenarios of comprehensive search and recommendation are combined, and the generation and reasoning capabilities of the large language model are used to achieve accurate and personalized prediction of user interaction content.

[0167] An embodiment of the present invention provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the interactive content prediction method is implemented.

[0168] An embodiment of the present invention provides a processor, which is used to run a program, wherein the interactive content prediction method is executed when the program is running.

[0169] like Figure 8 As shown, an embodiment of the present invention provides an electronic device 1000, which includes at least one processor 1001, at least one memory 1002 connected to the processor 1001, and a bus 1003; wherein the processor 1001 and the memory 1002 communicate with each other through the bus 1003; the processor 1001 is used to call the program instructions in the memory 1002 to execute the above-mentioned interactive content prediction method. The electronic device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0170] The present invention also provides a computer program product, which, when executed on an electronic device, is suitable for executing a program for initializing the steps of the interactive content prediction method.

[0171] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatuses, electronic devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processor of the computer or other programmable device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] In a typical configuration, an electronic device includes one or more processors (CPU), a memory, and a bus. The electronic device may also include an input / output interface, a network interface, and the like.

[0173] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip. The memory is an example of a computer-readable medium.

[0174] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0175] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the present invention are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0176] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0177] In the description of the present invention, it should be understood that the terms "up", "down", "front", "back", "left" and "right" etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the positions or elements referred to must have specific directions, be constructed and operate in specific directions. Therefore, they should not be understood as limitations of the present invention.

[0178] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. It should also be noted that the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0179] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0180] The above are only embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for predicting interactive content, characterized in that: include: Obtaining user interaction behavior data of a target user within a preset period, wherein the user interaction behavior data includes user search behavior data and user recommendation behavior data of the target user on a specified Internet product; Constructing a conversion relationship between the user search behavior data and the user recommendation behavior data to generate a search behavior representation and a recommendation behavior representation; extracting a search representation vector from the search behavior representation and a recommendation representation vector from the recommendation behavior representation; Combining the search representation vector and the recommendation representation vector as a context information vector and a task description prompt to generate an interactive content prediction prompt vector; Inputting the interaction content prediction prompt vector into a large language model to obtain an interaction content prediction result of the target user output by the large language model; The step of constructing a conversion relationship between the user search behavior data and the user recommendation behavior data to generate a search behavior representation and a recommendation behavior representation includes: Based on the pre-built content embedding table and query embedding table, using the user search behavior data and the user recommendation behavior data, generate a user interaction behavior vector, a user search behavior vector and a user recommendation behavior vector; Based on a multi-head self-attention algorithm and a feedforward network algorithm, the user interaction behavior vector, the user search behavior vector and the user recommendation behavior vector are used to generate a search-to-recommendation conversion representation vector, a recommendation-to-search conversion representation vector, a search-to-search conversion representation vector and a recommendation-to-recommendation conversion representation vector; Based on the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm, the search behavior representation is generated using the recommendation-to-search conversion representation vector and the search-to-search conversion representation vector, and the recommendation behavior representation is generated using the search-to-recommendation conversion representation vector and the recommendation-to-recommendation conversion representation vector.

2. The method according to claim 1, characterized in that The step of combining the search representation vector and the recommendation representation vector as a context information vector and a task description prompt to generate an interactive content prediction prompt vector includes: Inputting the search representation vector and the recommendation representation vector into the large language model, and processing the search representation vector and the recommendation representation vector using the gating mechanism, cross-attention mechanism, and dense network layer of the large language model to construct a context information vector; The task description prompt is vectorized and combined with the context information vector to generate an interaction content prediction prompt vector.

3. The method according to claim 1, characterized in that The method of generating a user interaction behavior vector, a user search behavior vector, and a user recommendation behavior vector based on the pre-built content embedding table and the query embedding table and utilizing the user search behavior data and the user recommendation behavior data comprises: Concatenate the user interaction behaviors in the user search behavior data and the user recommendation behavior data according to the occurrence time to generate a user interaction behavior sequence; use the pre-built content embedding table and query embedding table to convert each of the user interaction behaviors in the user interaction behavior sequence into a user interaction behavior mapping vector; concatenate each of the user interaction behavior mapping vectors to generate a user interaction behavior vector; Concatenate the user search behaviors in the user search behavior data according to the occurrence time to generate a user search behavior sequence; use the pre-built query embedding table to convert each of the user search behaviors in the user search behavior sequence into a user search behavior mapping vector; concatenate each of the user search behavior mapping vectors to generate a user search behavior vector; The user recommendation behaviors in the user recommendation behavior data are concatenated according to the occurrence time to generate a user recommendation behavior sequence; each of the user recommendation behaviors in the user recommendation behavior sequence is converted into a user recommendation behavior mapping vector using the pre-constructed content embedding table; each of the user recommendation behavior mapping vectors is concatenated to generate a user recommendation behavior vector.

4. The method according to claim 1, characterized in that: The method based on the multi-head self-attention algorithm and the feedforward network algorithm uses the user interaction behavior vector, the user search behavior vector and the user recommendation behavior vector to generate a search-to-recommendation conversion representation vector, a recommendation-to-search conversion representation vector, a search-to-search conversion representation vector and a recommendation-to-recommendation conversion representation vector, including: The interaction behavior position encoding is fused with the user interaction behavior vector and then input into the multi-head self-attention module, and then connected to the feedforward neural network module to generate a search-to-recommendation conversion representation vector and a recommendation-to-search conversion representation vector; The search behavior position encoding is fused with the user search behavior vector and then input into a multi-head self-attention module, and then connected to a feed-forward neural network module to generate a search-to-search conversion representation vector; The recommended behavior position encoding is fused with the user recommended behavior vector and input into the multi-head self-attention module, and then connected to the feedforward neural network module to generate a recommendation to recommendation conversion representation vector.

5. The method according to claim 1, characterized in that Before constructing the conversion relationship between the user search behavior data and the user recommendation behavior data to generate the search behavior representation and the recommendation behavior representation, the method further includes: Obtaining user interaction training data, wherein the user interaction training data includes user search behavior training data and user recommendation behavior training data; Based on the pre-built content embedding table and query embedding table, using the user search behavior training data and the user recommendation behavior training data, generate a user interaction behavior training vector, a user search behavior training vector and a user recommendation behavior training vector; Based on the multi-head self-attention algorithm and the feedforward network algorithm, the user interaction behavior training vector, the user search behavior training vector and the user recommendation behavior training vector are used to generate a search-to-recommendation conversion representation training vector, a recommendation-to-search conversion representation training vector, a search-to-search conversion representation training vector and a recommendation-to-recommendation conversion representation training vector; Based on the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm, the search behavior training representation is generated by using the recommendation-to-search conversion representation training vector and the search-to-search conversion representation training vector, and the recommendation behavior training representation is generated by using the search-to-recommendation conversion representation vector and the recommendation-to-recommendation conversion representation vector; Inputting the search behavior training representation and the recommendation behavior training representation into a perceptual resampler respectively to obtain a search representation training vector and a recommendation representation training vector; Inputting the search representation training vector, the recommendation representation training vector, the content feature vector of the user interaction training data, the user feature vector and the query feature vector into a multi-gated expert mixture model to obtain a next interaction prediction result output by the multi-gated expert mixture model; Compare the next interaction prediction result with the next interaction actual result of the user interaction training data, and calculate the loss function value; Through the back propagation algorithm, the algorithm parameters of the multi-head self-attention algorithm, the multi-head cross-attention algorithm and the feedforward network algorithm are adjusted based on the loss function value, and the adjusted algorithm parameters are used to adjust the algorithm parameters for the next round until the algorithm parameter adjustment termination condition is reached.

6. The method according to any one of claims 1 to 5, characterized in that The extracting the search representation vector in the search behavior representation and the recommendation representation vector in the recommendation behavior representation includes: The search behavior representation and the recommendation behavior representation are respectively input into a perceptual resampler for data compression and feature extraction to obtain a search representation vector and a recommendation representation vector.

7. An interactive content prediction device, characterized in that: include: A user interaction behavior data acquisition unit, a behavior representation generation unit, a representation vector acquisition unit, an interaction content prediction prompt vector generation unit, and an interaction content prediction result acquisition unit, The user interaction behavior data acquisition unit is used to obtain the user interaction behavior data of the target user within a preset period, wherein the user interaction behavior data includes the user search behavior data and user recommendation behavior data of the target user on a specified Internet product; The behavior representation generating unit is used to construct a conversion relationship between the user search behavior data and the user recommendation behavior data, and generate a search behavior representation and a recommendation behavior representation; The representation vector obtaining unit is used to extract the search representation vector in the search behavior representation and the recommendation representation vector in the recommendation behavior representation; The interactive content prediction prompt vector generating unit is used to combine the search representation vector and the recommendation representation vector as a context information vector and a task description prompt to generate an interactive content prediction prompt vector; The interaction content prediction result obtaining unit is used to input the interaction content prediction prompt vector into a large language model to obtain the interaction content prediction result of the target user output by the large language model; Wherein, the behavior representation generation unit includes: a behavior vector generation subunit, a conversion representation vector generation subunit and a behavior representation generation subunit; The behavior vector generating subunit is used to generate a user interaction behavior vector, a user search behavior vector and a user recommendation behavior vector based on the pre-built content embedding table and query embedding table and using the user search behavior data and the user recommendation behavior data; The conversion representation vector generating subunit is used to generate a search-to-recommendation conversion representation vector, a recommendation-to-search conversion representation vector, a search-to-search conversion representation vector, and a recommendation-to-recommendation conversion representation vector based on a multi-head self-attention algorithm and a feedforward network algorithm by using the user interaction behavior vector, the user search behavior vector, and the user recommendation behavior vector; The behavior representation generation subunit is used to generate a search behavior representation based on a multi-head self-attention algorithm, a multi-head cross-attention algorithm and a feedforward network algorithm, using the recommendation-to-search conversion representation vector and the search-to-search conversion representation vector, and to generate a recommendation behavior representation using the search-to-recommendation conversion representation vector and the recommendation-to-recommendation conversion representation vector.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the interactive content prediction method according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: The electronic device includes at least one processor, and at least one memory and a bus connected to the processor; wherein the processor and the memory communicate with each other through the bus; the processor is used to call program instructions in the memory to execute the interactive content prediction method as described in any one of claims 1 to 6.

Images