Recommendation information determination method and related equipment

Abstract

The invention provides a recommendation information determination method and device, electronic equipment, a computer program product and a computer readable storage medium. The method comprises the following steps: determining object attribute characteristics of a recommended object from attribute data of the recommended object; determining an interest map of the recommended object based on the behavior data of the recommended object and the first recommended content; determining behavior mode characteristics of the recommended target object based on the behavior data; based on the object attribute features, the interest map, the behavior pattern features and the content features of the to-be-recommended contents, determining recalled contents from the to-be-recommended contents; and based on the recall content, determining second recommendation content recommended to the recommendation object. Through the method, the accuracy of the recommended information can be improved.

Description

Technical Field

[0001] The present application relates to artificial intelligence technology, and in particular to a method for determining recommended information and related equipment. Background Art

[0002] Information recommendation refers to a service that determines the information items that the recommended object may be interested in based on the recommended object's interests, historical behaviors and other data, so that the recommended object can perform conversion operations on the information recommended to him or her, such as clicking to download content, enter a page, or place an order for goods, etc. Summary of the invention

[0003] The embodiments of the present application provide a method, device, electronic device, computer-readable storage medium, and computer program product for determining recommendation information, which can improve the accuracy of recommendation information.

[0004] The technical solution of the embodiment of the present application is implemented as follows:

[0005] The present application provides a method for determining recommendation information, the method comprising:

[0006] Determining object attribute characteristics of the recommended object from the attribute data of the recommended target object;

[0007] Determine an interest graph of the recommendation object based on the behavior data of the recommendation object and the first recommendation content, wherein the recommendation time of the first recommendation content is earlier than the current time;

[0008] Based on the behavior data, determining the behavior pattern characteristics of the recommended object;

[0009] Determining recalled content from the content to be recommended based on the object attribute characteristics, the interest graph, the behavior pattern characteristics, and the content characteristics of the content to be recommended;

[0010] Based on the recalled content, second recommended content is determined to be recommended to the recommendation object.

[0011] The present application provides a device for determining recommended information, including:

[0012] a feature determination module, configured to determine the object attribute features of the recommended object from the attribute data of the recommended object; determine the interest graph of the recommended object based on the behavior data of the recommended object and the first recommended content, wherein the recommendation time of the first recommended content is earlier than the current time; and determine the behavior pattern features of the recommended object based on the behavior data;

[0013] A content recall module, configured to determine recalled content from the content to be recommended based on the object attribute characteristics, the interest graph, the behavior pattern characteristics and the content characteristics of the content to be recommended;

[0014] An information determination module is used to determine second recommended content recommended to the recommendation object based on the recalled content.

[0015] An embodiment of the present application provides an electronic device, the electronic device comprising:

[0016] A memory for storing computer executable instructions or computer programs;

[0017] The processor is used to implement the method for determining recommended information provided in the embodiment of the present application when executing the computer executable instructions or computer program stored in the memory.

[0018] An embodiment of the present application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the method for determining recommendation information provided in the embodiment of the present application when executed by a processor.

[0019] An embodiment of the present application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, the method for determining recommendation information provided in the embodiment of the present application is implemented.

[0020] The embodiment of the present application has the following beneficial effects: for the recommended object, low-order object attribute features are extracted from its attribute data, and interest graphs that can represent its current interest preferences and behavior pattern features that represent its current behavior habits are extracted from its behavior data and the content data of the first recommended content recommended earlier than the current moment, so as to achieve the purpose of capturing more extensive interest features and behavior features of the recommended object, obtain higher-order object features, and at the same time combine the lower-order object attribute features obtained based on the attribute data, the higher-order object features (i.e., interest graphs and behavior pattern features) obtained based on the behavior data and the first recommended content recommended earlier, and the content features of the content to be recommended itself, so as to more accurately analyze the interest of the recommended object in different contents to be recommended, so as to more accurately select the recalled content, and finally generate the corresponding second recommended content for the recommended object based on the recalled content. Since the recalled content is the content that is more in line with the interests of the recommended object, the second recommended content obtained based on the recalled content will also be in line with the interests of the recommended object, so that the determined recommendation information will be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a schematic diagram of an application environment of the method for determining recommendation information provided in an embodiment of the present application;

[0022] Figure 2 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 1 ;

[0023] Figure 3 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 2 ;

[0024] Figure 4 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 3 ;

[0025] Figure 5 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 4 ;

[0026] Figure 6 is a processing flow chart of intelligent marketing provided by an embodiment of the present application;

[0027] Figure 7 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0029] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0030] In the following description, the terms "first\second\third" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second\third" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0031] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0032] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as those commonly understood by those skilled in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0033] The relevant data collection and processing in the embodiments of this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in examples, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0034] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0035] 1) Content to be recommended refers to the content stored in the content library that has promotion needs. Content to be recommended can be physical items (such as books, clothing and other physical goods) or non-physical online resources (such as articles, videos, games and other resources that do not have physical entities).

[0036] 2) The recommended object refers to the subject determined by the recommended information, that is, in the embodiment of the present application, the appropriate recommended content is determined for the recommended object.

[0037] 3) Recall content refers to the content that may be of interest to the recommended object that is initially selected from the content to be recommended. After sorting, filtering, and integrating external knowledge on the recalled content, the recommended content corresponding to the recommended object can be obtained (the advertisements mentioned below can be understood as recommended content).

[0038] 4) Query input refers to the content entered by the recommendation object during information retrieval or information query. This content reflects the query intention of the recommendation object, that is, it reflects the information that the recommendation target object hopes to display.

[0039] Content recommendation refers to a service that determines the information that the recommended object may be interested in based on the recommended object's interests, historical behaviors and other data, so that the recommended object can perform conversion operations on the information recommended to him or her, such as clicking to download content, enter a page, or place an order for goods, etc.

[0040] In the related art, there are some methods for recommending information based on the features of the recommended object. For example, feature extraction is performed on the historical data of the recommended object to obtain a first feature set, and feature extraction is performed on the advertising information data of the advertising platform to obtain a second feature set, and then the second feature set is filtered according to the first feature set to obtain a third feature set; the advertising platform places advertisements in the market according to the third feature set, so that advertisements that match the user's historical data and have not been placed can be placed as recommended information to the recommended target object.

[0041] Although this method screens the recommended content layer by layer through feature sets, it lacks an explanation of the model and does not explain how the feature sets are obtained from the data. As a result, it is difficult to ensure that the recommended content is in line with the interests of the recommended objects, which makes the accuracy of the determined recommended information low.

[0042] In addition, in order to further enhance the interest of the recommended object in the recommended content, the related technology will also obtain the query input of the recommended object and fine-tune the prompt to generate relevant copy for the recommended content to enrich the information content of the recommended content. For example, in the related technology, a large language model can be used to obtain the answer to the recommended object's question, and the conversion rate can be estimated based on the historical records of the recommended object. According to the conversion rate, it is determined whether to recommend an advertisement to the recommended object. If recommended, the recommended product is determined based on the historical records of the recommended object, and the prompt is obtained based on the question. The prompt and the recommended product are input into another large language model to obtain the recommended copy. Finally, the answer, recommended product and recommended copy are output to the recommended object together, so that the information provided to the recommended object is richer.

[0043] In order to ensure that the query input intention of the recommended object can be accurately understood and the copy generation can be more accurate and personalized, the prompt needs to be fine-tuned. However, the above method lacks a specific method for prompt fine-tuning, and it is difficult to ensure that the prompt can be fine-tuned successfully every time, so it is difficult to ensure the generation effect of the copy, and it is also difficult to ensure that the recommended content can be diversified, resulting in homogenization of the recommended content and failure to meet the diverse needs of the recommended objects.

[0044] At the same time, for some specific business activities, or new recommendation objects and new content to be recommended, the lack of corresponding historical data often causes the determined recommendation content to be inaccurate, resulting in poor recommendation results for specific activities.

[0045] In response to at least one of the above-mentioned problems existing in the related art, an embodiment of the present application provides a method, device, electronic device, computer-readable storage medium and computer program product for determining recommendation information. The method for determining recommendation information can not only determine basic object attribute characteristics for the recommendation object from the attribute data of the recommendation object, but also analyze the recommendation object in terms of interest preferences and behavioral habits in combination with the behavior data of the recommendation object and the first recommended content recommended before the current moment, so as to capture a wider range of interests and behavioral characteristics, thereby obtaining a higher-order interest map and behavior pattern characteristics for the recommendation object. Then, when determining the recall content, it can simultaneously consider the object attribute characteristics used to reflect the basic situation of the recommendation object, as well as the interest map that can reflect the current high-order characteristics of the recommendation object, that is, the interest preferences and behavioral habits. and behavioral pattern characteristics, so that in the process of determining the recommended information, not only the basic information of the recommended object can be considered, but also the more complex and changeable interest preferences and behavioral habits of the recommended object can be considered at the same time, so as to improve the accuracy of the recommended information; in addition, when generating copy for the recalled content, the query input of the recommended object can be rewritten to obtain the search keywords, and then based on the obtained search keywords, relevant supplementary content can be recalled from the external content library, and the supplementary content related to the recalled content can be screened from the recalled supplementary content, and the copy of the recalled content can be expanded to obtain a second recommended content with richer forms; and for the cold start problem of specific activities or new users and new content, representative seed objects can be selected as the training basis for model fine-tuning to obtain a customized model, so as to improve the cold start effect through a customized model.

[0046] In order to better understand a method, device, electronic device, computer-readable storage medium, and computer program product for determining recommended information provided by an embodiment of the present application, an application environment applicable to an embodiment of the present application is first described below.

[0047] See also Figure 1 , Figure 1 is a schematic diagram of an application environment of the method for determining recommended information provided in an embodiment of the present application. As an implementation mode, the method for determining recommended information provided in an embodiment of the present application can be applied to an electronic device, wherein the electronic device can be as follows: Figure 1 The server 110 shown in the figure can be connected to the terminal 120 via a network 130. The network 130 is used to provide a medium for a communication link between the server 110 and the terminal 120. The network 130 may include various connection types, such as a priority communication link, a wireless communication link, etc., which is not limited in this embodiment of the present application.

[0048] It should be understood that Figure 1The server 110, network 130 and terminal 120 are merely illustrative. Depending on the implementation requirements, any server, network and terminal may be used. For example, the server 110 may be a physical server or a server cluster composed of multiple servers, and the terminal 120 may be a smart phone, tablet computer, desktop computer, laptop computer, smart watch or other device. It is understood that in the embodiment of the present application, multiple terminals 120 may be allowed to access the server 110 at the same time.

[0049] In some embodiments, the server 110 can determine the object attribute characteristics of the recommended object from the basic data of the recommended object; determine the interest map of the recommended object based on the behavior data of the recommended object and the first recommended content, wherein the recommendation time of the first recommended content is earlier than the current time; then determine the behavior pattern characteristics of the recommended object based on the behavior data; then determine the recalled content from the content to be recommended based on the object attribute characteristics, the interest map, the behavior pattern characteristics and the content characteristics of the content to be recommended; finally, determine the second recommended content to be recommended to the recommended object based on the recalled content.

[0050] The following will describe the recommended information determination method provided by the embodiments of the present application. It should be noted that the recommended information determination method provided by each embodiment of the present application is executed by an electronic device, wherein the electronic device can be implemented as a server.

[0051] See also Figure 2 , Figure 2 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 1 . Will combine Figure 2 The steps shown are explained, Figure 2 The main body of the step is the electronic device.

[0052] Step 101: Determine the object attribute characteristics of the recommended object from the attribute data of the recommended object.

[0053] The embodiment of the present application is implemented in a scenario of determining appropriate recommended content for a recommended object. The recommended content can be understood as an advertisement for promoting physical items or online resources, or as media content that needs to be disseminated, information about social contacts, and the like. In the embodiment of the present application, the electronic device will first acquire data for the recommended object from a database or its own storage space to obtain attribute data of the recommended object. Next, the electronic device will perform feature encoding on the acquired attribute data to obtain object attribute features of the recommended object. In other words, the object attribute features are features obtained directly from the attribute data of the recommended object.

[0054] In the embodiment of the present application, the recommendation object refers to the subject determined by the recommendation information, which can be any user who has registered in the recommendation system, or any user who is online at the current time.

[0055] The attribute data of the recommended object may include static attribute information of the recommended object, such as age, gender, region, etc.

[0056] It should be noted that in the embodiments of the present application, a suitable feature encoding method can be selected according to the characteristics of the attribute data. Exemplarily, for numerical attribute data (such as age, etc.), normalization can be used to complete feature encoding, for categorical attribute data (such as gender, geographic location, etc.), one-hot encoding can be used to achieve feature encoding, and for sequence attribute data, an embedded network or a recurrent neural network can be used for feature encoding. After completing the feature encoding of the attribute data of the recommended object, the electronic device can continue to extract features from the obtained encoding features, and use the features extracted from the encoding features as object attribute features. Here, convolutional neural networks, recurrent neural networks, etc. can be used for feature extraction.

[0057] In some embodiments of the present application, the electronic device may also combine the attribute data of the recommended object with the content data of the content to be recommended to obtain the object attribute characteristics of the recommended object. Figure 2 Step 101 in the method, i.e., determining the object attribute characteristics of the recommended object from the attribute data of the recommended object, can be achieved by the following processing: performing feature encoding on the attribute data of the recommended object to obtain a first encoding feature; performing feature encoding on the content data of the content to be recommended to obtain a second encoding feature; and determining the feature interaction result of the first encoding feature and the second encoding feature as the object attribute characteristics of the recommended object.

[0058] That is to say, the electronic device can obtain content data of the content to be recommended, perform feature encoding on the obtained content data, and interact the obtained second encoding feature with the first encoding feature extracted from the attribute data to obtain the object attribute feature of the recommended object.

[0059] The content to be recommended refers to any content in the content library, which can be a physical object or an online resource without a physical entity. The content data of the content to be recommended can be attribute data of the content to be recommended, such as content ID, category, brand, price and other information, and can also be access data of the content to be recommended, such as access time, access device type and other data.

[0060] Similar to the feature encoding of attribute data, the electronic device can select a suitable encoding method according to the characteristics of the content data. For example, for numerical content data (such as the number of views, etc.), normalization can be used to complete the feature encoding, for category content data (such as the commodity category, etc.), one-hot encoding can be used to complete the feature encoding, and for sequence content data (such as a sequence composed of the access time of each access), an embedded network or a recurrent neural network can be used for feature encoding.

[0061] It should be noted that the electronic device can implement the feature interaction described above by using methods such as dot product, concatenation, and summation. For example, the electronic device can directly calculate the dot product of the first coding feature and the second coding feature obtained from the content data to obtain the object attribute feature, or concatenate the first coding feature obtained from the attribute data and the second coding feature obtained from the content data to obtain the object attribute feature.

[0062] It can be understood that by combining the first coding feature extracted from the attribute data with the second coding feature extracted from the content data, the complex content interaction features of the recommended object can be captured, which helps to enhance the expressiveness of the object attribute features.

[0063] Step 102: Determine the interest graph of the recommendation object based on the behavior data of the recommendation object and the first recommended content, and the recommendation time of the first recommended content is earlier than the current time.

[0064] The electronic device acquires the behavior data of the recommended object and the corresponding content data of the first recommended content, and builds an interest graph for the recommended object by combining the acquired behavior data and the content data of the first recommended content. It should be noted that the interest graph is used to characterize the current interest preferences of the recommended object, such as the content category of interest.

[0065] The behavior data of the recommended object may include the behavior information of the recommended object, such as information on historical clicks, purchases, evaluations, and other behaviors.

[0066] See also Figure 3 , Figure 3 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 2 In some embodiments of the present application, Figure 2 Step 102 in the method, i.e., determining the interest graph of the recommendation object based on the behavior data of the recommendation object and the first recommendation content, can be implemented by the following processing:

[0067] Step 1021: Determine the object cluster to which the recommended object belongs based on the behavior data of the recommended object, and determine the clustering features of the recommended object based on the object cluster.

[0068] The electronic device can classify the recommended object into a corresponding object cluster based on the behavior data of the recommended object. Different objects in the same object cluster have certain similarities in behavior data, so the electronic device classifies the recommended object into a cluster with similar interests. Afterwards, the electronic device can use the cluster center of the object cluster to which the recommended object belongs, or the average value of all features in the object cluster, as the cluster feature of the recommended object.

[0069] It should be noted that the electronic device can divide the object into clusters before the process of determining the recommended content. In this process, the electronic device can first determine some objects for clustering, extract features from the behavior data of these objects, and then use clustering algorithms such as k-means and hierarchical clustering to process the extracted features to obtain different object clusters.

[0070] Step 1022: construct an object content interaction graph of the recommended object based on the behavior data and the first recommended content.

[0071] The electronic device uses the recommended object and the first recommended content as graph nodes, and determines the operation of the recommended object on the first recommended content from the behavioral data, such as click, purchase, etc. If the recommended object has performed an operation on the first recommended content (such as click, forward, etc.), the electronic device will build an edge between the recommended object and the first recommended content. If the recommended object has not performed an operation on the first recommended content, the electronic device will not build an edge between the recommended target object and the first recommended content. In this way, an object-content interaction graph can be constructed for the recommended object to illustrate the operation of the recommended object on the first recommended content.

[0072] Step 1023: Perform attention optimization on the edges in the object content interaction graph to obtain an optimized interaction graph.

[0073] After obtaining the object content interaction graph, the electronic device will dynamically adjust the attention weights of the edges in the object content interaction graph according to the weights or importance of different operations, thereby obtaining a new interaction graph, which is the optimized interaction graph. Here, the electronic device can directly use the weights of different operations as the latest attention weights, or it can convert the weights of the operations, such as normalizing them, and use the normalized results as the attention weights of the edges to achieve attention optimization.

[0074] Step 1024: Perform neighborhood aggregation on the optimized interaction graph to obtain interaction graph features.

[0075] The electronic device calculates the corresponding node features for each node in the optimized interaction graph, and applies aggregation functions (such as average aggregation, pooling aggregation, maximum aggregation, etc.) to the calculated node features to generate a new feature representation of the node. During the aggregation process, the electronic device will consider optimizing the weight of each edge in the interaction graph, that is, weighting the feature values ​​of neighboring nodes with weights, and then summing them up to obtain new features, and replace the original node features with the new features. Finally, the electronic device can select the latest features of the node of the recommended object as the interaction graph feature.

[0076] Step 1025: Determine the interest graph of the recommended object based on the clustering features and the interaction graph features.

[0077] The electronic device can directly determine the clustering features and the interaction graph features as the interest graph of the recommended object, or can fuse the clustering features and the interaction graph features by concatenation, bitwise addition, etc., and determine the fusion result as the interest graph of the recommended object. In this way, the interest graph of the recommended object can be obtained.

[0078] It can be understood that by determining the object cluster to which the recommended object belongs, obtaining clustering features, constructing an object content interaction graph for the recommended object, and introducing an attention mechanism in the object content interaction graph, it is possible to capture the interests and needs that are not explicitly expressed by the recommended object, that is, to capture the implicit interests and potential needs of the recommended object, thereby more accurately determining the interest preferences of the recommended object, which is helpful to more accurately determine the recall content for the recommended object in the future.

[0079] Of course, in other embodiments of the present application, the electronic device can also extract the behavior information of the recommendation object with respect to the first recommended content from the behavior data, and determine the content that the recommendation object is interested in from the first recommended content based on the type of behavior information, and analyze the interest points such as themes, brands, categories, etc. of these contents to obtain an interest map.

[0080] Step 103: Determine the behavior pattern characteristics of the recommended object based on the behavior data.

[0081] The electronic device will also mine the behavior pattern of the recommended object from the behavior data of the recommended object, thereby obtaining the behavior pattern characteristics. It should be noted that the behavior pattern characteristics are used to characterize the current behavior habits of the recommended object, such as whether it is accustomed to swiping away the recommended content directly, or accustomed to clicking to view the recommended content.

[0082] In some embodiments, Figure 2Step 103 in the above, i.e., determining the behavior pattern characteristics of the recommended object based on the behavior data, can be achieved by the following processing: mining and obtaining frequent behavior sequences from the behavior data; and determining the behavior pattern characteristics from the frequent behavior sequences.

[0083] The electronic device can perform frequent behavior sequence mining on the behavior data to obtain the frequent behavior sequence of the recommended object, and then learn the frequent behavior sequence through the sequence parsing model to obtain the behavior pattern characteristics of the recommended object.

[0084] It should be noted that electronic devices can achieve frequent sequence mining through the following processing: determine the minimum support threshold for the behavior data, and the support threshold is used to determine whether the sequence pattern appears frequently; use the sequence representation method to convert the behavior data into a behavior sequence; generate all candidate sequences for the obtained behavior sequence, and this process can be implemented based on the prefix tree algorithm; for each candidate sequence, calculate its support, that is, the number of sequences that contain the candidate sequence; filter out the candidate sequences whose support is greater than or equal to the minimum support threshold as frequent behavior sequences.

[0085] In the embodiments of the present application, the sequence parsing model may be implemented as a long short-term memory network or a recurrent neural network, which is not limited in the embodiments of the present application.

[0086] In other embodiments, Figure 2 Step 103, i.e., determining the behavior pattern characteristics of the recommended object based on the behavior data, can also be achieved by the following processing: obtaining the operating habits of the recommended object from the behavior data; and performing feature encoding on the operating habit information to obtain the behavior pattern characteristics.

[0087] The operation habit information here is used to indicate the operation habits of the recommended object when accessing content. The electronic device can use the operations that appear more than a certain threshold in the behavior data as the operation habits of the recommended object, and obtain the content, time and other information of the operation habits to obtain the operation habit information. The electronic device can use models such as embedded networks and feature encoders to process the operation habit information and obtain corresponding behavior pattern features.

[0088] It can be understood that based on the interest graph and behavior pattern characteristics, the dynamic interest changes and behavior changes of the recommendation objects can be captured, which helps to respond to these changes quickly when determining the recall content, and thus helps to more accurately determine the recommended content for the recommendation objects.

[0089] It should be noted that steps 101 to 103 may be executed in an order other than the order described above, or may be executed simultaneously, which is not limited in the embodiment of the present application.

[0090] Step 104: Determine the recalled content from the content to be recommended based on the object attribute characteristics, interest graph, behavior pattern characteristics and content characteristics of the content to be recommended.

[0091] The electronic device can first extract the content features of the content to be recommended from the content data of the content to be recommended, and apply the content recommendation model to the object attribute features, interest map, behavior pattern features and content features of the content to be recommended to predict the interest of the recommendation object in each content to be recommended, and determine the recall content for the recommendation object from the content to be recommended in combination with the prediction results output by the content recommendation model.

[0092] It should be noted that the interest map and behavior pattern features here can be understood as high-order feature representations of the recommended objects, while the object attribute features can be understood as low-order feature representations of the recommended objects. Therefore, in this step, the low-order feature representation and high-order feature representation of the recommended objects are actually combined to jointly determine the interest of the recommended objects in the content to be recommended, so as to more accurately determine the content that the recommended objects are interested in from the content to be recommended, so as to obtain recalled content.

[0093] In some embodiments, the electronic device may first fuse the object attribute features, interest graphs, and behavior pattern features to obtain fused features, and then use a content recommendation model to process the fused features and content features to predict the interest of the recommended object in each content to be recommended and obtain a prediction result.

[0094] In other embodiments, the electronic device may also directly input the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics of the content to be recommended into the content recommendation model in sequence, so as to process these characteristics through the content recommendation model, thereby predicting the interest of the recommended object in each content to be recommended and obtaining a prediction result.

[0095] It should be noted that the processing performed by the content recommendation model on the input features depends on the specific structure of the content recommendation model. For example, when the content recommendation model includes a convolution layer, the input features will be convolution processed; when the content recommendation model includes an attention layer, the input features will be adjusted for attention weights, etc. In the embodiments of the present application, the content recommendation model can be implemented as a commonly used deep learning model, such as a convolutional neural network, or a recurrent neural network, etc. Of course, the structure of the content recommendation model can also be set by itself, and the embodiments of the present application are not limited here.

[0096] In an embodiment of the present application, the electronic device can use the recommended content whose interest level is greater than a threshold as recalled content, or it can sort the recommended content information in descending order of interest level and take the first K contents in the sequence (K is a positive integer) as recalled content.

[0097] It should be noted that content recommendation models can include general recommendation models used in general scenarios (i.e., scenarios without specific marketing activities, such as daily recommendations), and can also include customized recommendation models used for specific activities. Among them, customized recommendation models are models that are specially fine-tuned for specific activities and are suitable for processing recommendations, cold starts, etc. for specific activities.

[0098] In some embodiments of the present application, Figure 2 Step 104, i.e., determining the recalled content from the content to be recommended based on the object attribute characteristics, interest graph, behavior pattern characteristics and content characteristics of the content to be recommended, can be achieved by the following processing: predicting the interest of the recommendation object in the recommended content based on the object attribute characteristics, interest graph, behavior pattern characteristics and content characteristics of the content to be recommended; and determining the recalled content from the content to be recommended based on the interest.

[0099] Among them, the electronic device can calculate the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics through the content recommendation model to obtain the interest degree, so that the interest degree is used to characterize whether the recommended object has interest in the content to be recommended. Finally, the electronic device can recall the content to be recommended whose interest degree is greater than the interest threshold to obtain the recalled content, or recall the content to be recommended with the first several interest degrees to obtain the recalled content.

[0100] It should be noted that the electronic device can first fuse the object attribute characteristics, interest map and behavior pattern characteristics by splicing or summing, and input the fusion result and content characteristics into the content recommendation model to predict the interest level, or directly input the object attribute characteristics, interest map and behavior pattern characteristics into the content recommendation model to predict the interest level.

[0101] In the embodiment of the present application, the content recommendation model may include a general recommendation model and a customized recommendation model for a specific activity.

[0102] In the case where the content recommendation model includes: a customized recommendation model for a specific activity, predicting the interest of the recommendation object in the recommended content based on the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics of the content to be recommended can be achieved through the following processing: using the customized recommendation model to predict the interest of the content to be recommended from the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics of the content to be recommended.

[0103] For ordinary recommendation scenarios that do not require specific activities or cold starts, electronic devices will use a universal recommendation model to predict the interest of the recommended object in the content to be recommended from the object attribute characteristics, interest graph, behavior pattern characteristics and content characteristics, and obtain the interest level corresponding to the content to be recommended.

[0104] In the case where the content recommendation model includes: a customized recommendation model for a specific activity, predicting the interest of the recommendation object in the recommended content based on the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics of the content to be recommended can be achieved through the following processing: using the customized recommendation model to predict the interest of the content to be recommended from the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics of the content to be recommended.

[0105] That is to say, for special recommendation scenarios such as specific activities, the electronic device will obtain a customized recommendation model fine-tuned for the specific activity, and use the customized recommendation model to predict the interest of the recommended object in the content to be recommended from the object attribute characteristics, interest map, behavior pattern characteristics and content characteristics, and obtain the interest level corresponding to the content to be recommended.

[0106] It should be noted that since there is less data at the beginning of a specific activity, the recommendation of a specific activity generally requires a cold start. In order to achieve a better recommendation effect for a specific activity, the electronic device needs to select seed objects that meet the requirements of the specific activity, fine-tune the general recommendation model, and obtain a corresponding customized recommendation model. The generation process of the customized recommendation model will be described below.

[0107] Step 105: Determine second recommended content to be recommended to the recommendation object based on the recalled content.

[0108] After obtaining the recalled content, the electronic device can directly use the recalled content as the second recommended content to recommend it to the recommended object; of course, the electronic device can also combine the recalled content with supplementary knowledge to obtain the second recommended content. In this way, the second recommended content can be more diverse and help enhance the recommended object's interest in the second recommended content.

[0109] See also Figure 4 , Figure 4 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 3 In some embodiments of the present application, Figure 2 Step 105 in the above, i.e., determining the second recommended content to be recommended to the recommended object based on the recalled content, can be implemented by the following processing:

[0110] Step 1051: Generate search keywords based on the query input of the recommended object.

[0111] In an embodiment of the present application, the electronic device first obtains the content input by the recommended object during information retrieval or information query, obtains the query input of the recommended object, and rewrites the query input according to the keyword search format to obtain the search keyword.

[0112] In some embodiments, the electronic device can rewrite the query input through the large language model to obtain the corresponding search keyword. At this time, the electronic device will read the query input of the recommended object through the large language model, and use the output of the large language model as the search keyword. It should be noted that the large language model here can obtain the ability to generate corresponding search keywords according to the query input through training, and the training data can be a sample pair consisting of the training input and the training search term.

[0113] In other embodiments, the electronic device may also match the query input with multiple preset inputs corresponding to multiple preset keywords, and use the preset keywords corresponding to the matched preset inputs (for example, the similarity reaches a similarity threshold) as the search keywords corresponding to the query input to complete the generation of the search keywords.

[0114] Step 1052: Based on the search keyword, obtain the first supplementary content of the recalled content from the external knowledge base.

[0115] After obtaining the search keyword, the electronic device can call the search engine, and query the knowledge items in the external content library through the search engine, and recall the knowledge items hit by the search keyword to obtain the external knowledge items. It should be noted that the external content library here can be understood as a content library deployed outside the content library where the content to be recommended is located, that is, the content library other than the content library where the content to be recommended is located can be called an external content library.

[0116] The electronic device can use a search engine to recall content containing search keywords from an external content library to obtain the first supplementary content. It can also use a search engine to match the search keywords with keywords of content in the external content library and use content whose similarity reaches a similarity threshold as the first supplementary content to be recalled.

[0117] Step 1053: Filter the first supplementary content to obtain the second supplementary content related to the recalled content.

[0118] After obtaining the first supplementary content, the electronic device can filter the first supplementary content to determine the supplementary content associated with the recalled content, and record the filtered supplementary content as the second supplementary content of the recalled content. In other words, the second supplementary content contains content that can enrich the detailed description of the recalled content.

[0119] In some embodiments, the electronic device may extract information from the first supplementary content through an information extraction model, and compare the extracted information with the recalled content to determine whether the first supplementary content is related to the recalled content. For example, when the first supplementary content is a description of a certain type of headphones, if the recalled content is the headphones of this type, then the first supplementary content may be considered to be related to the recalled content, and the first supplementary content may be determined as the second supplementary content of the recalled content. When the first supplementary content is a description of a certain type of headphones, if the recalled content is a mobile phone, then the first supplementary content is not related to the recalled content, and the electronic device may remove the first supplementary content.

[0120] It should be noted that the information extraction model can acquire information extraction capability through training, and at this time, the training data can be training supplementary content and sample pairs consisting of information labels of the training supplementary content. The information extraction model can be implemented as a transformer model or a recurrent neural network, which is not limited in the embodiments of the present application.

[0121] In other embodiments, the electronic device may parse the first supplementary content according to predefined rules to extract information. For example, entity extraction is performed on the first supplementary content, and the extracted entity is the information contained in the first supplementary content. The electronic device compares it with the recalled content, and if it matches the recalled content, the first supplementary content is determined as the second supplementary content.

[0122] Step 1054: Determine the second recommended content to be recommended to the recommendation object based on the recalled content and the second supplementary content.

[0123] The electronic device may add the second supplementary content to the content description of the recalled content, and directly use the recalled content with the second supplementary content added as the second recommended content of the recommendation object. The electronic device may also perform a quality score on the content pair consisting of the recalled content and the second supplementary content, and select the content pair with higher quality as the final second recommended content.

[0124] More specifically, in other embodiments of the present application, Figure 3 Step 1054 in, i.e., determining the second recommended content to be recommended to the recommendation object based on the recalled content and the second supplementary content, can be achieved by the following processing: using the recalled content and the second supplementary content to generate a content pair, and generating a scoring feature for the content pair; extracting a query feature from the query input; predicting a matching value of the scoring feature and the query feature; based on the matching value, screening the content pairs, and determining the screened content pairs as the second recommended content of the recommendation object.

[0125] Here, the electronic device will first pair the recalled content with its corresponding second supplementary content to generate a content pair, and then construct a corresponding scoring feature for the content pair. At the same time, the electronic device will also extract features from the query input of the recommended object to obtain the corresponding query features. Next, the electronic device calls the quality assessment model, reads the obtained scoring features and query features through the quality assessment model, and predicts the matching of these two features to obtain the matching value of the scoring features and the query features. After that, the electronic device will filter the content pairs whose matching values ​​are greater than the score threshold, or filter the content pairs corresponding to the first several matching values, and use the filtered content pairs as the final second recommended content.

[0126] It should be noted that the electronic device can generate a scoring feature based on the features of the recalled content and the features of the second supplementary content in the content pair. For example, the electronic device can perform feature extraction on the second supplementary content to obtain a supplementary content feature, and perform feature extraction on the recalled content to obtain a content feature, and fuse the supplementary content feature and the content feature by concatenation or summation, and use the obtained fusion result as the scoring feature.

[0127] The electronic device makes a matching prediction on the scoring feature and the query feature, which is essentially to analyze whether the content pair consisting of the recalled content and its corresponding second supplementary content is consistent with the query input of the recommendation object, that is, whether the content pair meets the requirements of the recommendation object in information retrieval or information query. The content pairs with matching values ​​greater than the score threshold, or the content pairs corresponding to the first several matching values, meet the requirements of the recommendation object, so the electronic device will use these content pairs as the final second recommended content, so as to ensure that the recommendation object is provided with higher quality recommended content.

[0128] It is understandable that compared with the related art, since it is difficult to ensure the diversified effect of recommended information, the recommended information becomes homogenized and cannot meet the diversified needs of the recommended objects. In the embodiment of the present application, the electronic device can obtain the first supplementary content from the external content library in combination with the query input of the recommended object, and screen the content related to the recalled content as the second supplementary content of the recalled content, so as to be integrated into the detailed description of the recalled content. In this way, it is easier to generate better copy for the recalled content, thereby ensuring the diversified effect of the recommended information and meeting the diversified needs of the recommended objects.

[0129] At this point, the determination of the recommendation information for the recommendation object is completed, and the determined recommendation information can be sent to the recommendation object for the recommendation object to view.

[0130] It is understandable that, compared with the related art, due to the lack of description of the specific construction process of the features, it is easy to encounter the problem of difficulty in ensuring that the recommended information meets the interests of the recommended target object, which ultimately leads to the problem of inaccurate recommended information. In the embodiment of the present application, the electronic device extracts low-order object attribute features from the attribute data of the recommended object, and extracts the interest map that can represent its current interest preferences and the behavior pattern features that represent its current behavior habits from its behavior data and the content data of the first recommended content recommended earlier than the current moment, thereby achieving the purpose of capturing a wider range of interest features and behavior features of the recommended object, obtaining higher-order object features, and combining low-order object attribute features, high-order object features (i.e., interest maps and behavior pattern features) obtained based on behavior data and the first recommended content recommended earlier, and content features of the content to be recommended itself, it is possible to more accurately analyze the interest of the recommended object in different contents to be recommended, so as to more accurately select the recalled content, and finally generate the corresponding second recommended content for the recommended object based on the recalled content. Since the recalled content is content that is more in line with the interests of the recommendation object, the second recommended content obtained based on the recalled content will also be in line with the interests of the recommendation object. In this way, the determined recommendation information will be more accurate.

[0131] Next, the generation process of the customized recommendation model is described.

[0132] See also Figure 5 , Figure 5 This is a flow chart of the method for determining recommended information provided in the embodiment of the present application. Figure 4 In some embodiments of the present application, the method may further include the following processing:

[0133] Step 106: Select a corresponding seed training object for a specific activity.

[0134] For a specific activity, the electronic device will first select appropriate seed training objects from the many objects used for model training. These seed training objects are representative groups that meet the business requirements of the specific activity.

[0135] In some embodiments, Figure 4 Step 106, i.e., selecting a corresponding seed training object for a specific activity, can be achieved by the following processing: determining a seed selection criterion based on the business objective of the specific activity; and selecting an object that meets the seed selection criterion from each object cluster as a seed training object.

[0136] The business goal of a specific activity can be set by the operator according to the specific needs of the specific activity, or can be determined according to the historical goals of historical activities similar to the specific activity, for example, the historical goals of historical activities similar to the specific activity are used as the business goals of the specific activity. Then, the electronic device can select objects from the object clusters obtained by clustering the existing objects according to the seed selection criteria, and use the selected objects as seed training objects. In this way, the obtained seed training objects are objects that meet the business goals of the specific activity.

[0137] For example, for a specific promotional activity, the business requirement is to increase the purchasing power and activity of users as much as possible. Therefore, the electronic device can select training objects with purchasing power greater than a certain level and activity greater than a certain level from each object cluster as seed training objects.

[0138] In other embodiments, Figure 4 Step 106 in the above, i.e., selecting a corresponding seed training object for a specific activity, can also be achieved by the following process: determining historical participants of historical activities similar to the specific activity as seed training objects.

[0139] Since there are certain similarities between historical activities and specific activities, historical participants can also be regarded as potential participants of specific activities. Therefore, the electronic device can directly use the historical participants of historical activities as seed training objects.

[0140] Step 107: Use the data of the seed training object to fine-tune the general recommendation model to obtain a customized recommendation model for a specific activity.

[0141] After obtaining the seed training object, the electronic device can obtain the object data of the seed object, that is, obtain the data of the seed object, extract the features of the seed object from the data of the seed object, and use the features of the seed object (such as the basic object features of the seed training object, interest graph features and behavior pattern features, etc.) to retrain the general recommendation model, and the training result is the customized recommendation model. It should be noted that when fine-tuning the general recommendation model, the training recommendation content used can be the same as the training recommendation content used by the general recommendation model during training, or it can be the training recommendation content selected for a specific activity, which is not limited in the embodiments of the present application.

[0142] It is understandable that compared to the related art, for specific activities, the lack of corresponding historical data will cause the recommendation information to be inaccurate, which ultimately leads to poor recommendation effects for specific activities. In the embodiments of the present application, since the seed training object is an object that meets the business objectives of the specific activity, the general recommendation model is fine-tuned using the data of the seed training object, which can refine the preferences of specific groups, thereby more accurately determining the recommendation information for specific activities, thereby improving the recommendation effect for specific activities.

[0143] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0144] The embodiment of the present application is implemented in the scenario of intelligent marketing of goods. In this scenario, it is necessary to determine suitable goods (called recall content) and product copy (called first supplementary content) for users (called recommendation objects), and generate recommended content items (called second recommended content) for the goods and corresponding copy.

[0145] Figure 6 is a processing flow chart of intelligent marketing provided by the embodiment of the present application. Figure 6 , the process may include the following processing:

[0146] Step 201: Data preparation and feature processing.

[0147] First, data is collected for users and products. For example: basic user data (collecting static information such as user ID, age, gender, and address), user behavior data (collecting historical click, purchase, and evaluation information), product data (collecting product ID, category, brand, price, and label information), and context data (collecting dynamic information such as access time and device type).

[0148] Feature processing can be divided into the following categories:

[0149] Numerical feature processing: Normalize numerical features such as age and price.

[0150] Category feature processing: Uniquely encode category features such as gender, geographic location, product category, and brand.

[0151] Sequence feature processing: Process the user's historical behavior sequence, such as encoding through embedding or recurrent neural network.

[0152] Feature intersection and combination: Generate interactive features between users and products to enhance feature expression capabilities.

[0153] Step 202: training a general model (referred to as a general recommendation model).

[0154] Use the processed user features and product features to construct a feature vector. Use a deep learning model to train with the interaction data between users and products. At this time, the loss function can choose binary cross entropy loss or mean square error loss. The goal is to predict the dot product probability of the user for the product.

[0155] The general model obtained through training can be used to generate user feature representations (called object attribute features) and product feature representations (called content features), and these embedded vectors are used for subsequent recommendations.

[0156] Step 203: Learning high-level features of users.

[0157] Clustering algorithms and graph neural networks are used to construct interest graphs to capture users' potential interest preferences, and behavioral pattern analysis is used to capture users' high-frequency behavior patterns and interest transfer paths. The characteristics of the interest graph (called interest graph) and the characteristics of the behavior pattern (called behavior pattern characteristics) obtained are the high-order features of the user, which are used for subsequent recommendations.

[0158] When constructing the interest graph, the k-means clustering algorithm is first used to cluster the user's behavioral data features and identify the user's interest preferences in different dimensions. Thus, through k-means, users can be divided into several interest groups, and each group of users shows similar interests in certain categories.

[0159] Next, we construct a user-product graph (called an object content interaction graph). At this point, we can identify users and products as nodes in the graph, and use user behaviors (such as clicks and purchases) as edges between nodes to complete the construction of the user-product graph. The weight of the edge is determined based on the interaction frequency or rating factors.

[0160] After obtaining the user-product graph, an attention mechanism can be introduced into the graph to dynamically adjust the weight of neighborhood aggregation according to the weights and importance of different edges (the resulting graph is the optimized interaction graph); then a graph convolutional network is used to learn the embedding representation of the user-product graph, and through neighborhood aggregation, the user's high-order interest features and relationship patterns are captured.

[0161] After obtaining the clustering features and the embedded representation of the graph (called the interaction graph feature), the clustering features and the embedded representation of the graph can be fused to generate the user's interest graph, thereby describing the user's interest distribution and preferences in different categories of goods.

[0162] Next, we analyze the user's behavior patterns. At this point, we use frequent pattern mining to discover high-frequency behavior patterns and interest transfer paths from the user's behavior data, and use the long short-term memory network sequence model to model the user's behavior sequence to predict the user's potential interest transfer trend and preference changes.

[0163] After obtaining the characteristics of the behavior pattern and the characteristics of the interest graph, these two characteristics will be stored in the user's feature database as high-order characteristics of the user for subsequent recommendations.

[0164] Step 204: training a customized model for a seed user (referred to as a customized recommendation model).

[0165] For specific marketing activities, analyze the clustering results to understand the characteristics and behavior patterns of each group, including purchasing power, activity, interest preferences, etc. Then select representative user groups. At this time, you can select the standards for representative user groups based on business needs and goals. For example, judge the importance of users based on purchasing power and activity (called seed selection criteria), and select user groups with strong purchasing power and high activity as seed user groups (called seed training objects). For example, use statistical indicators to analyze the characteristics of users in each cluster (called object class cluster), including but not limited to purchase frequency, number of purchases, registration duration, etc.; for purchasing power, pay attention to the user's purchase amount, number of purchases, purchase type, etc.; for activity, pay attention to the user's dot product behavior, login frequency, browsing duration, etc. Characteristics. Select a representative seed user group based on at least one of purchasing power and activity, and determine a diverse seed user group.

[0166] The general model is then fine-tuned using the behavior and feature data of the seed user group to obtain a customized recommendation model that is tailored to specific marketing activities.

[0167] Step 205: recall of products and contents.

[0168] This is divided into internal knowledge recall and external knowledge integration. Among them, internal knowledge recall refers to using a general model or a customized model to predict the user's scores for all products, and select the top K products (called recalled content) for recall. External knowledge integration uses a large language model to rewrite the user's query (called query input) into a keyword search format (called search keywords), and then calls the external search engine API to recall the top K related content (called the first supplementary content).

[0169] Next, information is extracted from the content recalled from the outside to obtain knowledge items related to the candidate products (called associated knowledge items), and the related knowledge items are integrated into the candidate products to enrich the product details. At this time, there may be some products without related knowledge items, or knowledge items without related products. These products and knowledge items can be filtered out.

[0170] Step 206: Evaluate product content pairs (referred to as content pairs).

[0171] The products recalled from the internal and the knowledge items recalled from the external are quantized separately, and the features of the internal products and the features of the external knowledge items are fused through splicing to form a feature vector (called a scoring feature). The user query is paired with the external details that integrate the relevant knowledge items to form the input data of the evaluation model, and then the fused feature vector is scored using the trained evaluation model to obtain the matching score (called matching value) of each content pair.

[0172] The training data of the evaluation model here consists of query-content pairs and their relevance labels. More specifically, the training data includes query text feature vectors (used to represent the user's query, usually identified after text processing and vectorization), recalled content feature vectors (identifying content pairs recalled from internal and external sources, also identified after vectorization), and labels (labels that identify whether the query and recalled content pairs are correctly matched, usually binary labels, 1 for correct match, -1 for mismatch).

[0173] Step 207: extract knowledge and return recommendation results (called second recommended content).

[0174] According to the matching score results, high-quality content pairs are filtered and integrated, such as content pairs with a preset score threshold (0.5) as candidate recommendation results to generate the final recommendation results.

[0175] The above processing will be described below using a specific example.

[0176] Collect basic user data, such as user ID: U123, age: 28, gender: male, and geographic location: region A;

[0177] Collect user behavior data, such as historical clicks: Product A, Product B; historical purchases: Product C; reviews: Product D (5 stars);

[0178] Collect product data, such as:

[0179] Product A: ID = A, Category = Electronic Products, Brand = Brand X, Price = 1,000 yuan, Tag = Mobile Phone;

[0180] Product B: ID = B, Category = Electronic Products, Brand = Brand Y, Price = 800 yuan, Label = Tablet;

[0181] Product C: ID = C, Category = Household Appliances, Brand = Brand Z, Price = 500 yuan, Label = Rice Cooker;

[0182] Collect contextual data such as:

[0183] Visiting time: 8 p.m.;

[0184] Device type: Mobile phone.

[0185] After completing data collection, perform feature processing. For example:

[0186] Numerical feature processing: normalize the user's age (for example, the age range is 0-100, and the normalized value is 0.28);

[0187] Category feature processing: convert gender, geographic location, product category, etc. into one-hot encoding (e.g. male = [1,0], female = [0,1]);

[0188] Sequence feature processing: embed the user’s historical behavior sequence into code (e.g. click history product A = [0.1, 0.3, 0.2], product B = [0.2, 0.4, 0.1]);

[0189] Feature intersection and combination: Generate interactive features between users and products, such as the combination features of age and product price.

[0190] For general model training, first process the user's features and the product's features to obtain the user's feature vector, such as [0.28 (age), 1,0 (male), 0,1 (Beijing)], and the product's feature vector, such as [1,0 (electronic products), 1,0 (brand X), 0,1 (1000 yuan)]. Then select the deep neural network model and input the user's feature vector and the product's feature vector.

[0191] Training data example: user click behavior (user ID: U123, product A, label: 1) and user non-click behavior (user ID: U123, product D, label: 0). The goal is to train through the interaction data of users and products, and to predict the probability of users clicking on products. Finally, the embedding vectors of users and products are output, for example, the embedding vector of user U123=>[0.15, 0.22, 0.35], and the embedding vector of product product A=>[0.10, 0.30, 0.20].

[0192] Then cluster the user's static features and behavior data to form a user interest graph. For example, the high-order features of user U123 may show a high interest in electronic products and household appliances. Finally, the user's high-order features are output, such as the high-order features of user U123: interest vector => [electronic products: 0.7, household appliances: 0.5, others: 0.2].

[0193] When training a customized model, first select seed users. For example, based on a specific marketing campaign, select seed users with a history of purchasing electronic products and home appliances, such as users U123, U456, and U789.

[0194] The training is performed in combination with the behavior and feature data of the seed users. For example, the model parameters are adjusted through the behavior data of the seed users (such as user 123's click on product A) to obtain a customized recommendation model that is more suitable for specific marketing activities.

[0195] In the recall process, internal knowledge recall is performed first: a general model or a customized model is used to predict the scores of user U123 for all products. For example, product A=>0.8, product B=>0.7, product C=>0.5, and the top K, for example, the top 3 products, namely product A, product B, and product C, are selected.

[0196] Then, external knowledge integration is performed: the user query (for example, "recommend electronic products suitable for young men") is rewritten into keywords ("which electronic products are suitable for young people") using a large language model. Then, the external search is called to cause the API to recall relevant content, such as technical reviews, user reviews, etc., and information is extracted from the recalled content to obtain knowledge items related to the candidate products (such as the latest reviews of product A). These knowledge items are integrated into the description of the candidate products to enrich the details of the candidate products and obtain the final recall results. For example, the details of product A are combined with external knowledge items: "Product A (brand X, 1,000 yuan, mobile phone) has a very good evaluation and is suitable for young men."

[0197] Marriage certificate constructs query-product pairs, which pairs user queries with product details that integrate external knowledge to form model input data. For example:

[0198] Sample 1: query text feature vector [0.2, 0.5, 0.1, 0.8]; recall content feature vector [0.1, 0.3, 0.6, 0.9]; label: 1 (match);

[0199] Sample 2: query text feature vector [0.7, 0.3, 0.6, 0.2]; recall content feature vector [0.4, 0.2, 0.8, 0.1]; label: -1 (no match);

[0200] Each sample pair contains a query text feature vector, a recall content feature vector, and a label, where the label indicates whether the query text and the recall content are correctly matched.

[0201] The obtained sample pairs are used to fine-tune the pre-trained model, and the training data consists of query-knowledge item pairs and their relevance labels.

[0202] Finally, score the knowledge item pairs, that is, use the fine-tuned evaluation model to score the recalled segmented knowledge item pairs, for example, the knowledge item pair "Product A has a good evaluation" scores 0.8, and the knowledge item pair "Product B has a moderate price" scores 0.6. Filter and integrate high-scoring knowledge item pairs, filter the knowledge item pairs that need to be retained according to the scoring threshold, and integrate them in order to form the final recommendation content. For example, retain the two knowledge item pairs "Product A has a good evaluation" and "Product B has a moderate price" to generate the final recommendation results.

[0203] It is understandable that in the embodiments of the present application, user information, such as user basic data, behavioral data and other related data, is involved. When the embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards.

[0204] Next, the structure of the electronic device according to the embodiment of the present application is described.

[0205] See also Figure 7 , Figure 7 It is an electronic device provided in an embodiment of the present application ( Figure 1 The server 110 is a structural diagram of an implementation of an electronic device, Figure 7 The electronic device 400 shown includes: at least one processor 410, a memory 450, and at least one network interface 420. The various components in the electronic device 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, the bus system 440 is not described in detail. Figure 7 Various buses are labeled as bus system 440 .

[0206] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0207] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices that are physically remote from the processor 410.

[0208] The memory 450 includes a volatile memory or a nonvolatile memory, and may also include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 450 described in the embodiments of the present application is intended to include any suitable type of memory.

[0209] In some embodiments, memory 450 can store data to support various operations, examples of which include programs, modules, and data structures, or a subset or superset thereof, as exemplarily described below.

[0210] Operating system 451, including system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic services and processing hardware-based tasks;

[0211] A network communication module 452, used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 include: Bluetooth, Wireless Compatibility Certification (WiFi) and Universal Serial Bus (USB), etc.;

[0212] In some embodiments, the recommended information determination device provided in the embodiments of the present application may be implemented in software. Figure 7 The recommended information determination device 455 stored in the memory 450 is shown, which can be software in the form of a program and a plug-in, etc., including the following software modules: a feature determination module 4551, a content recall module 4552, an information determination module 4553 and a model training module 4554. These modules are logical, so they can be arbitrarily combined or further split according to the functions implemented. The functions of each module will be explained below.

[0213] In other embodiments, the recommendation information determination device provided in the embodiments of the present application can be implemented in hardware. As an example, the recommendation information determination device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the recommendation information determination method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application specific integrated circuits (Application Specific Integrated Circuit, ASIC), digital signal processors (Digital Signal Processor, DSP), programmable logic devices (Programmable Logic Device, PLD), complex programmable logic devices (Complex Programmable Logic Device, CPLD), field programmable gate arrays (Field-Programmable Gate Array, FPGA) or other electronic components.

[0214] In some embodiments, the electronic device can implement the recommended information determination method provided by the embodiment of the present application by running various computer executable instructions or computer programs. For example, the computer executable instructions can be commands, machine instructions or software instructions at the microprogram level. The computer program can be a native program or software module in the operating system; it can be a native (Native) application (APPlication, APP), that is, a program that needs to be installed in the operating system to run, such as a shopping APP; it can also be a small program that can be embedded in any APP, that is, a program that can be run only by downloading it to a browser environment. In short, the above-mentioned computer executable instructions can be instructions in any form, and the above-mentioned computer program can be an application, module or plug-in in any form.

[0215] The following is a description of an exemplary structure of the recommended information determination device 455 provided in the embodiment of the present application implemented as a software module. In some embodiments, Figure 2 As shown, the software modules stored in the recommended information determining device 455 of the memory 450 may include:

[0216] The feature determination module 4551 is used to determine the object attribute features of the recommended object from the attribute data of the recommended object; determine the interest map of the recommended object based on the behavior data of the recommended object and the first recommended content, the recommendation time of the first recommended content is earlier than the current time; determine the behavior pattern features of the recommended target object based on the behavior data;

[0217] A content recall module 4552, configured to determine recalled content from the content to be recommended based on the basic object features, the interest graph, the behavior pattern features, and the content features of the content to be recommended;

[0218] The information determination module 4553 is used to determine the second recommended content recommended to the recommendation object based on the recalled content.

[0219] In the above scheme, the information determination module 4553 is also used to generate search keywords based on the query input of the recommended object; based on the search keywords, obtain the first supplementary content of the recalled content from the external content library; filter out the second supplementary content related to the recalled content from the first supplementary content; based on the recalled content and the second supplementary content, determine the second recommended content recommended to the recommended object.

[0220] In the above scheme, the information determination module 4553 is also used to generate a content pair using the recalled content and the second supplementary content, and generate a scoring feature for the content pair; extract the query feature from the query input; predict the matching value of the scoring feature and the query feature; based on the matching value, filter the content pair, and determine the filtered content pair as the second recommended content of the recommendation object.

[0221] In the above scheme, the feature determination module 4551 is also used to perform feature encoding on the attribute data of the recommended object to obtain a first encoding feature; perform feature encoding on the content data of the content to be recommended to obtain a second encoding feature; and determine the feature interaction result of the first encoding feature and the second encoding feature as the object attribute feature of the recommended object.

[0222] In the above scheme, the feature determination module 4551 is also used to determine the object cluster to which the recommended object belongs based on the behavior data of the recommended object, and determine the clustering features of the recommended object based on the object cluster; construct an object content interaction graph of the recommended object based on the behavior data and the first recommended content; perform attention optimization on the edges in the object content interaction graph to obtain an optimized interaction graph; perform neighborhood aggregation on the optimized interaction graph to obtain interaction graph features; and determine the interest graph of the recommended object based on the clustering features and the interaction graph features.

[0223] In the above solution, the feature determination module 4551 is also used to mine frequent behavior sequences from the behavior data; and determine the behavior pattern features from the frequent behavior sequences.

[0224] In the above scheme, the content recall module 4552 is also used to predict the interest of the recommendation object in the content to be recommended based on the object attribute characteristics, the interest map, the behavior pattern characteristics and the content characteristics of the content to be recommended; and determine the recalled content from the content to be recommended based on the interest.

[0225] In the above scheme, the content recall module 4552 is also used to use a general recommendation model to predict the interest of the recommended object in the content to be recommended from the object attribute characteristics, the interest graph, the behavior pattern characteristics and the content characteristics of the content to be recommended.

[0226] In the above scheme, the content recall module 4552 is also used to use a customized recommendation model to predict the interest of the recommended object in the content to be recommended from the object attribute characteristics, the interest graph, the behavior pattern characteristics and the content characteristics of the content to be recommended.

[0227] In the above scheme, the recommendation information determination device 455 also includes: a model training module 4554, which is used to select a corresponding seed training object for the specific activity; and use the data of the seed training object to fine-tune the general recommendation model to obtain the customized recommendation model for the specific activity.

[0228] In the above scheme, the model training module 4554 is also used to determine the seed selection criteria based on the business objectives of the specific activity; and select objects that meet the seed selection criteria from each object cluster as the seed training objects.

[0229] The embodiment of the present application provides a computer program product, which includes a computer program or a computer executable instruction, and the computer program or the computer executable instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer executable instruction from the computer-readable storage medium, and the processor executes the computer executable instruction, so that the electronic device executes the recommended information determination method described in the embodiment of the present application.

[0230] The present application embodiment provides a computer-readable storage medium in which computer executable instructions or computer programs are stored. When the computer executable instructions or computer programs are executed by a processor, the processor will be caused to execute the recommendation information determination method provided by the present application embodiment, for example, Figure 2 The recommended information determination method shown.

[0231] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories.

[0232] In some embodiments, computer executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0233] As an example, computer-executable instructions may, but need not, correspond to a file in a file system, may be stored as part of a file that stores other programs or data, such as in one or more scripts in a HyperText Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions).

[0234] As an example, computer executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one site, or on multiple electronic devices distributed at multiple sites and interconnected by a communication network.

[0235] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A method for determining recommended information, characterized in that: The method comprises: Determining object attribute characteristics of the recommended object from the attribute data of the recommended object; Determine an interest graph of the recommendation object based on the behavior data of the recommendation object and the first recommendation content, wherein the recommendation time of the first recommendation content is earlier than the current time; Based on the behavior data, determining the behavior pattern characteristics of the recommended object; Determining recalled content from the content to be recommended based on the object attribute characteristics, the interest graph, the behavior pattern characteristics, and the content characteristics of the content to be recommended; Based on the recalled content, second recommended content is determined to be recommended to the recommendation object.

2. The method according to claim 1, characterized in that The determining, based on the recalled content, second recommended content to be recommended to the recommendation object includes: Generating search keywords based on the query input of the recommended object; Based on the search keyword, obtaining first supplementary content of the recalled content from an external content library; Filtering the first supplementary content to obtain second supplementary content related to the recalled content; Based on the recalled content and the second supplementary content, the second recommended content recommended to the recommendation object is determined.

3. The method according to claim 2, characterized in that The determining, based on the recalled content and the second supplementary content, to recommend the second recommended content to the recommendation object includes: Generate a content pair using the recalled content and the second supplementary content, and generate a scoring feature for the content pair; Extracting query features from the query input; Predicting a matching value between the scoring feature and the query feature; Based on the matching value, the content pairs are screened, and the screened content pairs are determined as the second recommended content recommended to the recommendation object.

4. The method according to any one of claims 1 to 3, characterized in that: The step of determining the object attribute characteristics of the recommended object from the attribute data of the recommended object includes: Performing feature encoding on the attribute data of the recommended object to obtain a first encoding feature; Performing feature encoding on the content data of the content to be recommended to obtain a second encoding feature; A feature interaction result between the first coding feature and the second coding feature is determined as the object attribute feature of the recommended object.

5. The method according to any one of claims 1 to 3, characterized in that: The determining the interest graph of the recommendation object based on the behavior data of the recommendation object and the first recommendation content includes: Determining, according to the behavior data of the recommended object, the object cluster to which the recommended object belongs, and determining, based on the object cluster, a clustering feature of the recommended object; constructing an object content interaction graph of the recommended object based on the behavior data and the first recommended content; Performing attention optimization on the edges in the object content interaction graph to obtain an optimized interaction graph; Performing neighborhood aggregation on the optimized interaction graph to obtain interaction graph features; Based on the clustering features and the interaction graph features, the interest graph of the recommended object is determined.

6. The method according to any one of claims 1 to 3, characterized in that: The determining of the recalled content from the content to be recommended based on the object attribute feature, the interest graph, the behavior pattern feature and the content feature of the content to be recommended includes: Predicting the interest of the recommendation object in the content to be recommended based on the object attribute characteristics, the interest graph, the behavior pattern characteristics and the content characteristics of the content to be recommended; Based on the interest level, the recalled content is determined from the to-be-recommended content.

7. A device for determining recommended information, characterized in that: The recommended information determining device comprises: a feature determination module, configured to determine the object attribute features of the recommended object from the attribute data of the recommended object; determine the interest graph of the recommended object based on the behavior data of the recommended object and the first recommended content, wherein the recommendation time of the first recommended content is earlier than the current time; and determine the behavior pattern features of the recommended object based on the behavior data; A content recall module, configured to determine recalled content from the content to be recommended based on the object attribute characteristics, the interest graph, the behavior pattern characteristics and the content characteristics of the content to be recommended; An information determination module is used to determine second recommended content recommended to the recommendation object based on the recalled content.

8. An electronic device, characterized in that: The electronic device comprises: A memory for storing computer executable instructions or computer programs; A processor, configured to implement the method according to any one of claims 1 to 6 when executing computer executable instructions or computer programs stored in the memory.

9. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that: When the computer executable instructions or computer programs are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising computer executable instructions or a computer program, characterized in that: When the computer executable instructions or computer program are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images