Object recommendation method based on collaborative recommendation model, electronic equipment and storage medium

Abstract

One or more embodiments of the invention provide an object recommendation method based on a collaborative recommendation model, electronic equipment and a storage medium. The training method comprises the steps that sample structured data of a sample user is obtained, a sample semantic text containing semantic information is constructed based on the sample structured data, and the sample structured data comprises basic data of the sample user and / or recommendation related data generated by recommending a sample historical object to the sample user; calling a basic recommendation model to extract a first sample potential feature of the sample structured data, and calling a large language model LLM to extract a second sample potential feature of the sample semantic text; and based on the first sample potential features and the second sample potential features, performing comparative learning training on the basic recommendation model to obtain a collaborative recommendation model.

Description

Technical Field

[0001] This specification relates to one or more embodiments in the field of object recommendation, and more particularly to an object recommendation method, electronic device and storage medium based on a collaborative recommendation model. Background Technology

[0002] Personalized recommendation technologies based on user interaction feedback (such as click behavior) are widely used in advertising, marketing, and content distribution. Traditional object recommendation schemes typically utilize conventional sequence features, such as user ID, user behavior features, and object features of the recommended object (or item), through one-hot encoding, and infer user preferences for the recommended items based on the co-occurrence relationships between features.

[0003] With the rapid development of AI (Artificial Intelligence) technology, in order to improve the accuracy and efficiency of recommendations, the industry has been attempting to implement recommendation solutions based on LLM (Large Language Model). However, the online deployment and operation of large models requires significant resource costs; at the same time, their inference time is high, often making it difficult to meet the real-time requirements of object recommendation systems. For example, online recommendation scenarios generate a large amount of data every day, and LLM cannot complete the scoring of new data in a short period of time. In addition, current LLMs can usually only recognize semantic information and cannot learn the sequence features of structured data. Therefore, directly calling LLMs to perform object recommendations based on sequence features results in poor accuracy, which urgently needs improvement. Summary of the Invention

[0004] In view of the above, one or more embodiments of this specification provide the following technical solutions:

[0005] According to a first aspect of one or more embodiments of this specification, a method for training a collaborative recommendation model is proposed, comprising:

[0006] Obtain sample structured data of sample users, and construct sample semantic text containing semantic information based on the sample structured data. The sample structured data includes the basic data of the sample users and / or recommendation-related data generated by recommending sample historical objects to the sample users.

[0007] The basic recommendation model is invoked to extract the first latent features of the sample structured data, and the large language model LLM is invoked to extract the second latent features of the sample semantic text.

[0008] Based on the latent features of the first sample and the latent features of the second sample, the basic recommendation model is trained through comparative learning to obtain a collaborative recommendation model.

[0009] According to a second aspect of one or more embodiments of this specification, an object recommendation method based on a collaborative recommendation model is proposed, comprising:

[0010] Upon receiving an object recommendation request instructing the recommendation of a target object to a target user, the system obtains the structured data of the target user, the structured data including the target user's basic data and / or recommendation-related data generated from recommending historical objects to the target user;

[0011] The basic recommendation model is invoked to extract the first latent feature of the structured data, and the collaborative recommendation model is invoked to extract the second latent feature of the structured data, wherein the collaborative recommendation model is trained by the method described in the first aspect;

[0012] The recommendation evaluation result is determined based on the first latent feature and the second latent feature. The recommendation evaluation result is used to characterize the interaction of the target user with respect to the target object after the target object is recommended to the target user.

[0013] According to a third aspect of one or more embodiments of this specification, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in the first aspect by executing the executable instructions.

[0014] According to a fourth aspect of one or more embodiments of this specification, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the steps of the method as described in the first aspect.

[0015] According to a fifth aspect of one or more embodiments of this specification, a computer program product is provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the method as described in the first aspect.

[0016] As can be seen from the above embodiments, the solution in this specification is an improvement on the traditional recommendation model, that is, combining LLM and the traditional recommendation model to train a collaborative recommendation model with better recommendation performance. Specifically, the training method of the above-mentioned collaborative recommendation model constructs sample semantic text containing semantic information based on the sample structured data of sample users, so as to call LLM to accurately extract the second sample latent features; then, based on the features and the first sample latent features extracted by the basic recommendation model, the basic recommendation model is compared and learned to obtain the collaborative recommendation model.

[0017] It is understandable that because the semantically encoded text of the samples contains semantic information, the latent features of the second samples extracted by LLM contain high-dimensional features of this semantic information. Contrastive learning enables the basic recommendation model to learn the ability to extract semantic information from LLM. Therefore, the trained collaborative recommendation model can extract not only sequential features but also higher-dimensional semantic features. In other words, this scheme, through constructing semantically encoded text of samples and training with contrastive learning, successfully transfers the semantic information extraction capability of LLM, based on extensive world knowledge, to the collaborative recommendation model. This model combines the advantages of both the basic recommendation model and LLM, thus enabling accurate object recommendation based on user sequences and one-hot encoding without losing the original spatial features, effectively improving the prediction accuracy of the model in object recommendation scenarios. Furthermore, because the collaborative recommendation model is obtained through contrastive learning based on the traditional basic recommendation model, it is a lightweight model compared to the massive LLM. Therefore, the resources and costs required for deployment and operation are significantly less than those for directly calling LLM-related technical solutions, thereby greatly reducing the model's operating costs and facilitating the promotion of the solution. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the architecture of an object recommendation system provided in an exemplary embodiment.

[0019] Figure 2 This is a flowchart illustrating a training method for a collaborative recommendation model, provided in an exemplary embodiment.

[0020] Figure 3 This is a schematic diagram illustrating a decision tree for extracting structured data from head samples, provided as an exemplary embodiment.

[0021] Figure 4 This is an exemplary embodiment of a schematic diagram illustrating the construction of semantic text for samples.

[0022] Figure 5 This is a schematic diagram of a comparative learning training process provided in an exemplary embodiment.

[0023] Figure 6 This is a flowchart illustrating an object recommendation method based on a collaborative recommendation model, provided in an exemplary embodiment.

[0024] Figure 7 This is a schematic diagram of an object recommendation method based on a collaborative recommendation model, provided in an exemplary embodiment.

[0025] Figure 8 This is a schematic diagram of the structure of a device provided in an exemplary embodiment.

[0026] Figure 9 This is a block diagram of a training apparatus for a collaborative recommendation model provided in an exemplary embodiment.

[0027] Figure 10 This is a block diagram of an object recommendation device based on a collaborative recommendation model, provided in an exemplary embodiment. Detailed Implementation

[0028] Personalized recommendation technologies based on user interaction feedback (such as click behavior) are widely used in advertising, marketing, and content distribution. Traditional object recommendation schemes typically utilize conventional sequence features, such as user ID, user behavior features, and object features of the recommended object (or item), through one-hot encoding, and infer user preferences for the recommended items based on the co-occurrence relationships between features.

[0029] With the rapid development of AI technology, the industry is attempting to implement recommendation solutions based on LLM (Limited Learning Model) to improve the accuracy and efficiency of recommendations. However, the online deployment and operation of large models requires significant resource costs; simultaneously, their inference time is high, often failing to meet the real-time requirements of object recommendation systems. For example, online recommendation scenarios generate a large amount of data daily, and LLM cannot score the newly added data in a short time. Furthermore, current LLM models typically only recognize semantic information and cannot learn the sequence features of structured data. Therefore, directly calling LLM to perform object recommendations based on sequence features results in poor accuracy, which urgently needs improvement.

[0030] To address the aforementioned technical problems in related technologies, this specification proposes a novel collaborative recommendation model and an object recommendation scheme based on this model. Specifically, the collaborative recommendation model is obtained by comparative learning and training a conventional basic recommendation model using LLM. The collaborative recommendation model and the basic recommendation model are then used together to predict the objects to be recommended, aiming to reduce the model's operating cost and improve the accuracy of object recommendations, thereby achieving a higher click-through rate (CTR) at a lower cost.

[0031] The novel solution proposed in this specification will be described in detail below with reference to the accompanying drawings and related embodiments.

[0032] Figure 1 This is a schematic diagram of the architecture of an object recommendation system provided in an exemplary embodiment. For example... Figure 1 As shown, the system may include a server 11 and several electronic devices, such as mobile phones 14, 15, and 16.

[0033] During operation, server 11 can run server-side programs for a specific application to implement its related functions. For example, when server 11 runs an object recommendation service program, it can act as the server for that service (hereinafter referred to as the object recommendation server). In addition to server 11 and several electronic devices, the object recommendation system can also interface with or include servers that implement specific functions, such as server 12 and / or server 13. Server 12 can run an object management service program to act as the corresponding object management server, providing the object recommendation server with object description information of the target objects to be recommended or recommendation-related data corresponding to historical objects that have already been recommended. Server 13 can run a model management service program to act as the model server, such as deploying and running a trained collaborative recommendation model locally, and providing the object recommendation server with corresponding inference services (i.e., interaction prediction services for target users and target objects) through the model's calling interface. Of course, the aforementioned collaborative recommendation model can also run locally on server 11 (in which case the object recommendation system may not include the aforementioned server 13) so that the object recommendation server can quickly call it, which will not be elaborated further. Among them, any one of the servers 11, 12 and 13 can be a physical server containing an independent host, or server 11 can be a virtual server hosted by a host cluster.

[0034] Any electronic device can run a client-side program for an application to implement its functions. For example, when the electronic device runs an object recommendation service program, it can act as an object recommendation client for that service (e.g., mobile phones 14, 15, and 16 each implement different object recommendation clients). The application for any of these clients can be launched and run on the corresponding electronic device. For instance, this program can be a native application installed on the electronic device, or it can be a mini-program, quick app, or other similar form. When using web technologies such as HTML5, the relevant functions can be implemented through a page displayed by a browser. This browser can be a standalone browser application or a browser module embedded in some applications. Of course, the aforementioned object recommendation client can also be integrated into the clients of other applications (such as online shopping applications, social applications, travel applications, logistics applications, etc.) as a functional plugin for those applications, so that they can recommend objects to users while providing corresponding services. Further details are omitted. Additionally, mobile phones are only one type of electronic device that users can use. In practice, users can obviously also use electronic devices such as PCs (Personal Computers), tablets, laptops, PDAs (Personal Digital Assistants), wearable devices (such as smart glasses, smartwatches, etc.), etc., and one or more embodiments of this specification do not limit this.

[0035] It is understandable that, from a hardware perspective, the object recommendation system includes terminal devices (i.e., the aforementioned electronic devices, such as mobile phones 14 to 16) and servers (such as the aforementioned server 11); while from a software perspective, the system includes object recommendation clients (such as the client-side program of the object recommendation service running in each mobile phone) and object recommendation servers (such as the server-side program of the object recommendation service running in server 11).

[0036] The users mentioned in this specification (such as sample users and target users as described below) can be either legitimate users who have completed registration in the object recommendation system or unregistered temporary users (or "guests" of the system). This specification does not impose any limitations on these categories. The object recommendation system can provide object recommendation services to any user through their object recommendation client, i.e., recommend relevant objects to them. Correspondingly, any user can trigger the recommended objects (i.e., perform corresponding triggering operations on the objects), such as clicking on a recommended product to view details, or adding it to favorites, sharing it, or purchasing it; or clicking on a recommended coupon to redeem it or give it to other users, etc., which will not be elaborated further.

[0037] For the network 10 used for interaction between electronic devices such as mobile phones 14 and server 11, communication can be achieved using either wired or wireless networks, depending on the communication methods supported by the respective electronic devices. This specification does not impose any restrictions on this. For example, mobile phones 14 typically only support wireless communication, so they can use wireless networks for communication; while PCs can communicate with server 11 using either wired or wireless methods, which will not be elaborated further.

[0038] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this manual, such as the basic data and recommended related data mentioned below, are all information and data authorized by the relevant users or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0039] The object recommendation scheme described in this specification needs to be implemented based on a collaborative recommendation model trained in a completely new way. The specific training process of this model will be explained below with reference to an example.

[0040] Figure 2 This is a flowchart illustrating a training method for a collaborative recommendation model, as provided in an exemplary embodiment. Figure 2 As shown, the method includes the following steps 202-206.

[0041] Step 202: Obtain sample structured data of sample users, and construct sample semantic text containing semantic information based on the sample structured data. The sample structured data includes the basic data of the sample users and / or recommendation-related data generated by recommending sample historical objects to the sample users.

[0042] Prior to this training, the TT or other recommendation platforms may have served a large number of users, providing object recommendation services to these users. However, this specification does not limit the historical objects recommended; they can include online-sold goods (such as footwear, apparel, pharmaceuticals, consumer electronics, etc.) or services (on-site installation services, online communication services, etc.), usable coupons (such as discount coupons, buy-one-get-one-free coupons, trial coupons, redemption coupons, free coupons, points, cash red envelopes, etc.), and viewable content (such as articles, pictures, videos, live streams, etc.).

[0043] When providing object recommendation services to these users, the system or platform generates and maintains corresponding data, such as the user's basic data (e.g., age, gender, occupation, linked bank card and its opening bank) and recommendation-related data generated from previous object recommendations to these users (e.g., object description information of the recommended object). For example, when the recommended object is a product, its object description information includes basic product information (name, style, model, color, size, etc.), product price (e.g., single item price, bulk purchase price, discounted price, etc.), and / or seller information (e.g., store name, store rating, store qualifications, etc.); while when the recommended object is a coupon, its object description information may include the name of the issuing institution, coupon type, discount amount, scope of application, and / or redemption validity period, etc., which will not be elaborated further. The data mentioned above is usually stored in a structured manner (such as following a fixed format, adhering to preset rules, and being logically expressed using a two-dimensional table structure), such as using tables, CSV (Comma-Separated Values), JSON (JavaScript Object Notation), XML (eXtensible Markup Language), etc. Therefore, this data is called structured data.

[0044] Before training begins, sample users can be selected from the aforementioned users who have been served. This selection can be based on factors such as user account level, historical service frequency, and / or user activity. Once the sample users are determined, their corresponding data becomes the corresponding structured sample data. Of course, to improve the effectiveness of the samples, further selection can be made from all the data of the sample users to obtain structured sample data, and / or preprocessing such as cleaning can be performed on the obtained structured sample data; these details will not be elaborated further.

[0045] Object recommendation schemes in related technologies typically employ one-hot encoding to convert raw tabular data into one-hot vectors for model training and inference. However, this method neglects the semantic information between feature fields and their corresponding values, resulting in a lack of semantic information in the object recommendation system. Consequently, the system can only rely on the co-occurrence relationships of features as signals for prediction. When these relationships are weak in certain specific scenarios (such as cold start or low-frequency long-tail scenarios), the prediction performance relying solely on co-occurrence relationships is often poor. To address this, this solution introduces LLM to capture necessary semantic information, effectively bridging the information gap and thus improving the overall performance of the object recommendation system.

[0046] In one embodiment, for any sample user, the corresponding sample semantic text can be generated using all the sample structured data of the data, so that the model can learn more comprehensive semantic information and improve the generalization characteristics of the model.

[0047] However, given the potentially large types and volume of structured sample data, and the fact that some types of data are not closely related to the object recommendation scenario, generating semantic text for all such data could lead to an overabundance of semantic information in the sample semantic text. This would not only significantly increase the resources consumed by LLM in extracting latent features of the second sample, resulting in lower efficiency, but also potentially lead to overfitting and a low signal-to-noise ratio in subsequent collaborative recommendation models. To address this, this solution proposes a data simplification approach, selecting highly relevant data for generating sample semantic text. For example, head sample structured data can be determined based on statistical results for various sample structured data, or a decision tree algorithm can be used to select at least one head sample structured data that is strongly related to the historical objects of the sample from the sample structured data. Then, semantic text containing semantic information can be constructed based on the selected head sample structured data. In this way, the system can select at least one head sample structured data that is strongly related to the sample historical object to construct sample semantic text, thereby greatly reducing the amount of sample structured data and the amount of data of the generated sample semantic text. It can simplify the construction of sample semantic text as much as possible without losing important information, which helps LLM to quickly understand the structural space information behind the semantics, reduce unnecessary consumption of computing resources, and improve training speed.

[0048] For example, in a credit card coupon recommendation scenario, the system may serve hundreds of millions of users, and the structured data of each user's sample may contain hundreds or even thousands of entries. Using the method described above, the top 20 data points can be selected, including user age, the largest monthly successful credit card transaction amount in the last 3 months, the instant discount amount issued in the last 7 days, the number of credit cards linked historically, the amount redeemed in the last 90 days, the number of redemptions in the last 30 days, the number of prizes issued in the last 30 days, and the total transaction amount in the last 30 days, to generate semantic text for the sample.

[0049] The decision tree algorithm can employ the XGBoost ensemble learning algorithm, which combines multiple weak classifiers into a strong classifier to better evaluate the importance and gain of each feature (i.e., the structured data of each user's samples) to the target (i.e., the accuracy of object recommendation), thereby effectively filtering out the head data that has a greater impact on model performance. Figure 3As shown, the importance of various data to the final goal can be calculated using the XGboost ensemble learning algorithm, and then the top few head data with the highest importance can be determined as the head sample structured data.

[0050] In one embodiment, given that the collaborative recommendation model trained by this scheme is ultimately used to predict user interactions with recommended objects (i.e., to provide users with interaction prediction services, such as predicting interactions with user A...), S After recommending a coupon, does the user actually use the coupon? (This is followed by a seemingly unrelated question about predicting the coupon's use on user B.) S (Whether a user will click to purchase a product after it is recommended, etc.) Therefore, to improve the accuracy of subsequent inferences, the aforementioned head sample structured data can be selectively filtered based on the subsequent service targets. For example, the filtering conditions of the decision tree algorithm can be set according to the object description information of the preset object corresponding to the collaborative recommendation model. For example, when the preset object is a coupon, its object description information includes at least one of the following: the name of the issuing institution, coupon type, discount amount, scope of use, and redemption validity period; when the preset object is a product, its object description information includes at least one of the following: basic product information, product price, and seller information, which will not be elaborated further.

[0051] It is understood that the semantic text of the samples described in this solution is the prompt word generated for LLM. This text is used to specify the aforementioned sample structured data and corresponding semantic information to LLM so that LLM can accurately identify these data and extract high-dimensional semantic features (such as potential features of subsequent second samples).

[0052] In one embodiment, when constructing sample semantic text, a pre-created semantic text template can be obtained first, and then the sample structured data can be inserted into the corresponding position in the semantic text template to obtain the sample semantic text. This method allows for the automatic and rapid generation of standardized sample semantic text after obtaining the sample structured data (or the selected header sample structured data), which helps improve overall training efficiency. The semantic text template can be pre-created based on formatted semantic information to ensure that the sample semantic text generated based on this template can ultimately be accurately recognized by LLM and have its corresponding high-dimensional features extracted.

[0053] like Figure 5As shown to the left of the dotted line, the obtained sample structured data is in tabular form, containing multiple entries. For example, the first column "age" represents the sample user's age, and "cc_max_succ_month_amt_3m" represents the maximum monthly successful credit card transaction amount in the last 3 months. Further details are omitted. Filling the data from this table into the corresponding semantic text template yields the sample semantic text: "My age is 26 years old. My maximum monthly successful credit card transaction amount in the last 3 months is 2503.30 yuan. The instant discount amount issued in the last 7 days is 0.24 yuan. Historically, I have 2 credit cards linked to my account. The amount redeemed in the last 90 days is 23.58 yuan. The number of prizes issued in the last 30 days is 10…". The formatted semantic information in this text includes… Figure 5 As shown to the right of the dotted line, "My age" represents the feature region, "is" represents the feature connection, "26 years old" represents the feature value, and commas are feature separation identifiers, etc., which will not be elaborated further. This text describes user-related features such as age and behavior, and also provides features related to the distribution of prizes / benefits / coupons; in addition, it also involves user-related behavioral features such as recent payments. Each feature is separated by commas, which comprehensively constructs a series of semantic information about the user in installment projects. This is beneficial for large models to comprehensively capture feature relationships in high-dimensional space, make up for the information gap in recommendation systems, and further effectively improve recommendation accuracy.

[0054] Understandably, generating semantic text for samples aims to add semantic information to the structured sample data, enabling the LLM to accurately identify and extract the corresponding high-order features. The semantic text generated in this way can then be used as cue words input into the LLM.

[0055] Step 204: Call the basic recommendation model to extract the first latent features of the sample structured data, and call the large language model LLM to extract the second latent features of the sample semantic text.

[0056] In one embodiment, the basic recommendation model, as an improvement to this solution, can be any conventional recommendation model in the industry, and this specification does not limit this. For example, it can be DeepFM (Deep Factorization Machine, a model combining factorization machines and deep neural networks, capable of simultaneously capturing low-order and high-order feature interactions, suitable for sparse data scenarios), Wide & Deep Learning (a model that improves recommendation performance by combining wide and deep networks, with the width part used for memorizing features and the depth part for generalization ability), NFM (Neural Factorization Machine, a model combining factorization machines and neural networks, utilizing neural networks to capture high-order interactions between features), AutoRec / Item-KNN (the former is an autoencoder recommendation model, a recommendation model based on autoencoders used to learn representations of users and items; the latter is an item-based K-nearest neighbor algorithm recommendation model, an item-based recommendation method using the K-nearest neighbor algorithm), etc. It is evident that this solution is compatible with and adaptable to existing mainstream object recommendation models, exhibiting good adaptability and low implementation difficulty.

[0057] In one embodiment, the LLM described in this specification can also employ any conventional large language model in the industry. Such models typically possess extensive world knowledge and the ability to extract high-dimensional semantic information, thus enabling them to extract higher-dimensional semantic information from sample semantic text as potential features of the second sample. Furthermore, the LLM can be an open-source or closed-source model; this specification does not limit the embodiments thereto.

[0058] It should be noted that the first and second sample latent features are the encoding results of the corresponding model's neural network in the latent space, such as features extracted by the hidden layer (which are essentially vectors or matrices), and not the final result output by the network's final fully connected layer.

[0059] Step 206: Based on the latent features of the first sample and the latent features of the second sample, perform comparative learning training on the basic recommendation model to obtain a collaborative recommendation model.

[0060] After obtaining the first sample latent features extracted by the base recommendation model and the second sample latent features extracted by the LLM, the base recommendation model can be trained using contrastive learning to obtain the final collaborative recommendation model. It can be understood that this method of contrastive learning training of the base recommendation model is intended to enable the trained collaborative recommendation model to learn the LLM's ability to extract high-dimensional semantic information.

[0061] In one embodiment, if the first and second latent features have the same length, they can be directly used for comparative learning training. However, if the first and second latent features have different sizes, they can first undergo linear projection to map them to the same dimension; then, the basic recommendation model can be trained using the result of the linear projection. This method ensures that comparative learning training can be successfully completed even with different sizes of first and second latent features. It is understood that the processing result is the two intermediate features obtained by projecting the first and second latent features respectively (of course, in some cases, either intermediate feature may be the smaller of the first and second latent features), which will not be elaborated further.

[0062] The sample structured data and the sample semantic text are two different modalities of data. This scheme uses contrastive learning to perform cross-modal fine-grained knowledge alignment between structured spatial features (such as tabular spatial features) and semantic spatial features, thereby enabling the basic recommendation model (i.e., the small model) to fully learn the high-dimensional spatial feature extraction capability of LLM (i.e., the large model) for semantics, and realize the reconstruction of the basic recommendation model.

[0063] This scheme utilizes a contrastive learning algorithm to achieve cross-modal knowledge alignment. For example... Figure 5 As shown, the sample structured data and sample semantic text are processed by the basic recommendation model and LLM to extract corresponding high-dimensional space codes (i.e., the first sample latent features and the second sample latent features), respectively. These two codes of different lengths are then linearly projected to map to the same dimension. Next, a contrastive learning method is used to compare the representations of the two in different latent spaces. By evaluating the distance between sample pairs, similar codes are brought as close as possible in the feature space, while dissimilar codes are kept as far apart as possible. This further learns the similarity and differences between the latent space codes. This approach helps to assist in the fine-grained alignment of the two modalities, while simultaneously enabling the training and reconstruction of the small model's feature extraction capabilities.

[0064] The core objective of contrastive learning algorithms is to make the representations of similar samples as close as possible, while making the representations of different samples as far apart as possible. In one embodiment, InfoNCE can be used to align representations in different spaces. Representations of the same instance in text and table modalities form positive pairs, while corresponding modalities of different samples form negative pairs, which can be obtained through batch sampling.

[0065] In the comparative learning process, it is assumed that... and Let the k-th instance be represented in text (i.e., semantic text of the sample) and table (i.e. structured data of the sample) modalities, respectively. Then, the contrastive loss from text to table can be found in Equation (1), and the contrastive loss from table to text can be found in Equation (2):

[0066]

[0067]

[0068] Where τ is the temperature coefficient, N is the number of instances in the batch (i.e., batch size), and the function sim() is used to measure the similarity between two vectors, which can be cosine similarity. To reduce the spatial bias of the collaborative recommendation model, the above contrastive loss function can be designed using JS (Jensen-Shannon) divergence to achieve uniform alignment across modal spaces.

[0069] Finally, the comparative loss of the base recommendation model and LLM can be defined as the average of the two losses mentioned above, as shown in Equation (3):

[0070]

[0071] After comparative learning and training, LLM successfully transferred its semantic information extraction capabilities to the basic recommendation model based on extensive world knowledge, enabling the basic recommendation model to produce semantics similar to the high-dimensional space of large models.

[0072] As can be seen from the above embodiments, the solution in this specification is an improvement on the traditional recommendation model, that is, combining LLM and the traditional recommendation model to train a collaborative recommendation model with better recommendation performance. Specifically, the training method of the above-mentioned collaborative recommendation model constructs sample semantic text containing semantic information based on the sample structured data of sample users, so as to call LLM to accurately extract the second sample latent features; then, based on the features and the first sample latent features extracted by the basic recommendation model, the basic recommendation model is compared and learned to obtain the collaborative recommendation model.

[0073] It is understandable that because the semantically encoded text of the samples contains semantic information, the latent features of the second samples extracted by LLM contain high-dimensional features of this semantic information. Contrastive learning enables the basic recommendation model to learn the ability to extract semantic information from LLM. Therefore, the trained collaborative recommendation model can extract not only sequential features but also higher-dimensional semantic features. In other words, this scheme, through constructing semantically encoded text of samples and training with contrastive learning, successfully transfers the semantic information extraction capability of LLM, based on extensive world knowledge, to the collaborative recommendation model. This model combines the advantages of both the basic recommendation model and LLM, thus enabling accurate object recommendation based on user sequences and one-hot encoding without losing the original spatial features, effectively improving the prediction accuracy of the model in object recommendation scenarios. Furthermore, because the collaborative recommendation model is obtained through contrastive learning based on the traditional basic recommendation model, it is a lightweight model compared to the massive LLM. Therefore, the resources and costs required for deployment and operation are significantly less than those for directly calling LLM-related technical solutions, thereby greatly reducing the model's operating costs and facilitating the promotion of the solution.

[0074] After training the collaborative recommendation model through the aforementioned embodiments, object recommendations can be performed based on the model. Figure 6 This is a flowchart illustrating an object recommendation method based on a collaborative recommendation model, as provided in an exemplary embodiment. Figure 6 As shown, the method includes the following steps 602-606.

[0075] Step 602: Receive an object recommendation request instructing the target user to recommend target objects, and obtain the structured data of the target user, the structured data including the target user's basic data and / or recommendation-related data generated from recommending historical objects to the target user.

[0076] The object recommendation request can be initiated by the object recommendation client used by the target user or by the object recommendation server. By initiating this request, the collaborative recommendation model and the basic recommendation model can be invoked to predict the interaction behavior of the recommendation event of "recommending a target object to the target user." That is, the two models work together to predict the user's interaction with the target object after it has been recommended to them. For example, predicting whether user A will redeem and use coupon S1 within its validity period after recommending it to user A; or predicting whether user B will click to purchase product S2 after recommending it to user B, etc.

[0077] Of course, the embodiments in this specification do not limit the interaction methods of the target user with respect to the target object. For example, it can be clicking on the page to view details, redeeming coupons, liking, collecting, and / or forwarding the target object, etc., which will not be elaborated further. In addition, the target object can be any form of available resource. The coupon described in the embodiments of this specification is only one form of resource. Of course, it can also be other types of available resources, which will not be elaborated further.

[0078] For the target user and target object specified in the object recommendation request, the structured data of the target user can be obtained in response to the request. Similar to the method of obtaining the sample structured data mentioned above, the structured data stored in a pre-structured form can be obtained from the server or the corresponding database, which will not be described in detail here.

[0079] Step 604: Invoke the basic recommendation model to extract the first latent feature of the structured data, and invoke the collaborative recommendation model to extract the second latent feature of the structured data. The collaborative recommendation model is trained by the method described in any of the preceding embodiments.

[0080] After obtaining the aforementioned structured data, the basic recommendation model and the collaborative recommendation model can be invoked to extract the corresponding features, such as using the basic recommendation model to extract the first latent feature and using the collaborative recommendation model to extract the second latent feature. Similar to the first and second latent features of the aforementioned samples, the first latent feature is the co-occurrence feature extracted by the basic recommendation model, while the second latent feature is the semantic feature extracted by the collaborative recommendation model. Both are latent space features extracted by the corresponding models in their respective latent spaces, which can be matrices or vectors extracted by the hidden layers of the neural network, and will not be elaborated further.

[0081] In one embodiment, all acquired structured data can be input into the base recommendation model to extract more comprehensive co-occurrence features. Alternatively, corresponding to the training process of the aforementioned collaborative recommendation model, only the highly relevant data from all structured data can be used to extract the second latent features. This reduces the extraction workload while ensuring inference quality, thereby reducing the resources required for inference and improving inference speed. For example, at least one head structured data point with strong correlation to the recommended historical object (specifically, the historical object recommended to the target user) can be determined from the structured data, and then the collaborative recommendation model can be invoked to extract the second latent features of the at least one head structured data point.

[0082] When determining at least one header structured data point from all the target user's structured data, this can be done according to the header data identifiers pre-recorded during the training phase. That is, after determining the header sample structured data in the aforementioned manner during the training phase, the corresponding header data identifiers (such as...) can be recorded. Figure 4 (At least some of the identifiers shown in the first row of the table), so that the corresponding header structured data can be directly determined from the structured data of the target user obtained during the inference stage, thereby simplifying the logic for determining the header structured data and improving the accuracy and speed of data.

[0083] Of course, the header structured data can also be determined temporarily. For example, the filtering conditions of the decision tree algorithm can be set according to the object description information of the target object, and the decision tree algorithm can be used to filter out at least one header structured data from the structured data. The decision tree algorithm can be the same as the algorithm in the training phase, and will not be described in detail here.

[0084] Step 606: Determine a recommendation evaluation result based on the first latent feature and the second latent feature. The recommendation evaluation result is used to characterize the interaction of the target user with respect to the target object after the target object is recommended to the target user.

[0085] In one embodiment, since the first and second latent features extracted by the base recommendation model and the collaborative recommendation model are the latent space features of their respective models, a fully connected layer can be pre-constructed to obtain the final recommendation evaluation result. The first and second latent features are then concatenated and input into this fully connected layer, and the recommendation evaluation result output by the fully connected layer is received. It is understood that the fully connected layer is a special layer in a neural network, where each neuron is connected to all neurons in the previous layer. The main function of this layer is to integrate the features learned from previous layers, map the features to the output space, and thus complete tasks such as classification and regression. This scheme utilizes this layer to output the final recommendation evaluation result.

[0086] like Figure 7 As shown, after the structured data is input into the basic recommendation model and the collaborative recommendation model respectively, the first latent features and the second latent features extracted by the two models are respectively input into the fully connected layer, and the layer finally outputs the recommendation evaluation result.

[0087] In one embodiment, the recommendation evaluation result can be a binary result, such as "1" (representing "yes") or "0" (representing "no"). In this case, the result directly indicates whether the target user will trigger the target object, such as whether they click to purchase a product or use a coupon. Alternatively, the recommendation evaluation result can also be a probability value, the magnitude of which characterizes the likelihood of the target user triggering the target object.

[0088] Based on this, if the recommendation evaluation result indicates that the target user will trigger the target object, or if the recommendation evaluation result indicates that the probability value of the target user triggering the target object is not lower than a probability threshold, then it means that after recommending the target object to the target user, the user is highly likely to trigger the object. In this case, the target object can be recommended to the target user to obtain the highest possible click-through rate. Taking the prediction before recommending a coupon to a user as an example, if the recommendation evaluation result is "1", it means that the model predicts that the target user will redeem the coupon, and the coupon can be recommended to the user; or, if the probability value of the recommendation evaluation result is not lower than a probability threshold (such as 60%, 80%, etc.), the coupon can be recommended to the user. This method uses the recommendation evaluation result to guide subsequent object recommendation activities, which helps to improve the interaction rate of the recommended objects.

[0089] Understandably, this solution combines a traditional basic recommendation model with a collaborative recommendation model trained in the aforementioned manner for object recommendation. On one hand, with the assistance of the collaborative recommendation model, it can achieve feature fusion across different dimensions, combining semantic features with co-occurrence relationships to improve the accuracy of object recommendations in multiple scenarios and assist in performance improvement in cold start and low-frequency long-tail scenarios. On the other hand, by using a relatively small collaborative recommendation model and a basic recommendation model for inference, rather than directly responding to business requests by calling the LLM, it significantly reduces the amount of resources and operating costs required in the model inference stage, making the solution highly practical.

[0090] Figure 8 This is a schematic structural diagram of a device provided in an exemplary embodiment. Please refer to... Figure 8At the hardware level, the device includes a processor 802, an internal bus 804, a network interface 806, memory 808, and non-volatile memory 810, and may also include other hardware required for its functions. One or more embodiments of this specification can be implemented in software, for example, the processor 802 reads the corresponding computer program from the non-volatile memory 810 into memory 808 and then runs it. Of course, besides software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0091] Please refer to Figure 9 The training apparatus for collaborative recommendation models can be applied to, for example... Figure 8 The device shown is used to implement the technical solution of this specification. The device may include:

[0092] The semantic construction unit 901 is used to acquire sample structured data of sample users and construct sample semantic text containing semantic information based on the sample structured data. The sample structured data includes the basic data of the sample users and / or recommendation-related data generated by recommending sample historical objects to the sample users.

[0093] The feature extraction unit 902 is used to call the basic recommendation model to extract the first latent features of the sample structured data, and to call the large language model LLM to extract the second latent features of the sample semantic text.

[0094] The contrastive learning unit 903 is used to perform contrastive learning training on the basic recommendation model based on the latent features of the first sample and the latent features of the second sample to obtain a collaborative recommendation model.

[0095] Optionally, the semantic construction unit 901 is specifically used for:

[0096] At least one head sample structured data is selected from the sample structured data using a decision tree algorithm, and the head sample structured data has a strong correlation with the sample historical objects.

[0097] Based on the structured data of at least one selected head sample, construct sample semantic text containing semantic information.

[0098] Optionally, a filter condition generation unit 904 is also included, for:

[0099] The selection criteria of the decision tree algorithm are set according to the object description information of the preset object corresponding to the collaborative recommendation model;

[0100] Wherein, when the preset object is a coupon, its object description information includes at least one of the following: the name of the issuing institution, the coupon type, the discount amount, the scope of use, and the redemption validity period;

[0101] When the preset object is a commodity, its object description information includes at least one of the following: basic commodity information, commodity price, and seller information.

[0102] Optionally, the semantic construction unit 901 is specifically used for:

[0103] Obtain a semantic text template containing formatted semantic information, and insert the sample structured data into the corresponding position in the semantic text template to obtain the sample semantic text.

[0104] Optional,

[0105] It also includes a feature projection unit 905, which is used to perform linear projection processing on the first sample latent feature and the second sample latent feature when the sizes of the first sample latent feature and the second sample latent feature are different, so as to map the two to the same dimension;

[0106] The contrastive learning unit 903 is specifically used to perform contrastive learning training on the basic recommendation model based on the processing results of the linear projection processing.

[0107] Please refer to Figure 10 Object recommendation devices based on collaborative recommendation models can be applied to, for example, Figure 8 The device shown is used to implement the technical solution of this specification. The device may include:

[0108] The data acquisition unit 1001 is configured to receive an object recommendation request that instructs the recommendation of a target object to a target user, and acquire the structured data of the target user, wherein the structured data includes the basic data of the target user and / or recommendation-related data generated from recommending historical objects to the target user;

[0109] The feature extraction unit 1002 is used to call the basic recommendation model to extract the first latent feature of the structured data, and to call the collaborative recommendation model to extract the second latent feature of the structured data. The collaborative recommendation model is trained by the device described in any of the preceding embodiments.

[0110] The result determination unit 1003 is used to determine a recommendation evaluation result based on the first latent feature and the second latent feature. The recommendation evaluation result is used to characterize the interaction of the target user with respect to the target object after the target object is recommended to the target user.

[0111] Optionally, the feature extraction unit 1002 is specifically used for:

[0112] At least one header structured data is determined from the structured data, and the header structured data is strongly correlated with the recommended historical object;

[0113] The collaborative recommendation model is invoked to extract the second latent feature of the at least one head structured data.

[0114] Optionally, the feature extraction unit 1002 is specifically used for:

[0115] The decision tree algorithm is set according to the object description information of the target object, and at least one header structured data is selected from the structured data using the decision tree algorithm.

[0116] Optionally, the result determination unit 1003 is specifically used for:

[0117] The first latent feature and the second latent feature are concatenated and input into a pre-built fully connected layer, and the recommended evaluation result output by the fully connected layer is received.

[0118] Optionally, it also includes an object recommendation unit 904, used for:

[0119] If the recommendation evaluation result indicates that the target user will trigger the target object, or if the recommendation evaluation result indicates that the probability value of the target user triggering the target object is not lower than the probability threshold, then the target object is recommended to the target user.

[0120] Based on the same concept as the methods described above, this specification also provides an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein the processor performs the steps of the method as described in any of the above embodiments by executing the executable instructions.

[0121] Based on the same concept as the methods described above, this specification also provides a computer-readable storage medium having computer instructions stored thereon that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

[0122] Based on the same concept as the methods described above, this specification also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods as described in any of the above embodiments.

Claims

1. A training method for a collaborative recommendation model, comprising: Obtain sample structured data of sample users, and construct sample semantic text containing semantic information based on the sample structured data. The sample structured data includes the basic data of the sample users and / or recommendation-related data generated by recommending sample historical objects to the sample users. The basic recommendation model is invoked to extract the first latent features of the sample structured data, and the large language model LLM is invoked to extract the second latent features of the sample semantic text. Based on the latent features of the first sample and the latent features of the second sample, the basic recommendation model is trained through comparative learning to obtain a collaborative recommendation model.

2. The method according to claim 1, wherein constructing sample semantic text containing semantic information based on the sample structured data comprises: At least one head sample structured data is selected from the sample structured data using a decision tree algorithm, and the head sample structured data has a strong correlation with the sample historical objects. Based on the structured data of at least one selected head sample, construct sample semantic text containing semantic information.

3. The method according to claim 2, further comprising: The selection criteria of the decision tree algorithm are set according to the object description information of the preset object corresponding to the collaborative recommendation model; Wherein, when the preset object is a coupon, its object description information includes at least one of the following: the name of the issuing institution, the coupon type, the discount amount, the scope of use, and the redemption validity period; When the preset object is a commodity, its object description information includes at least one of the following: basic commodity information, commodity price, and seller information.

4. The method according to claim 1, wherein constructing sample semantic text containing semantic information based on the sample structured data comprises: Obtain a semantic text template containing formatted semantic information, and insert the sample structured data into the corresponding position in the semantic text template to obtain the sample semantic text.

5. The method according to claim 1, Also includes: When the dimensions of the first sample latent feature and the second sample latent feature are different, linear projection processing is performed on the first sample latent feature and the second sample latent feature to map them to the same dimension; The step of performing comparative learning training on the basic recommendation model based on the latent features of the first sample and the latent features of the second sample includes: performing comparative learning training on the basic recommendation model based on the processing result of the linear projection processing.

6. An object recommendation method based on a collaborative recommendation model, comprising: Upon receiving an object recommendation request instructing the recommendation of a target object to a target user, the system obtains the structured data of the target user, the structured data including the target user's basic data and / or recommendation-related data generated from recommending historical objects to the target user; The basic recommendation model is invoked to extract the first latent feature of the structured data, and the collaborative recommendation model is invoked to extract the second latent feature of the structured data, wherein the collaborative recommendation model is trained by the method of any one of claims 1 to 5; The recommendation evaluation result is determined based on the first latent feature and the second latent feature. The recommendation evaluation result is used to characterize the interaction of the target user with respect to the target object after the target object is recommended to the target user.

7. The method according to claim 6, wherein invoking the collaborative recommendation model to extract the second latent feature of the structured data comprises: At least one header structured data is determined from the structured data, and the header structured data is strongly correlated with the recommended historical object; The collaborative recommendation model is invoked to extract the second latent feature of the at least one head structured data.

8. The method of claim 7, wherein determining at least one header structured data from the structured data comprises: The decision tree algorithm is set according to the object description information of the target object, and at least one header structured data is selected from the structured data using the decision tree algorithm.

9. The method according to claim 6, wherein determining the recommendation evaluation result based on the first latent feature and the second latent feature comprises: The first latent feature and the second latent feature are concatenated and input into a pre-built fully connected layer, and the recommended evaluation result output by the fully connected layer is received.

10. The method of claim 6, further comprising: If the recommendation evaluation result indicates that the target user will trigger the target object, or if the recommendation evaluation result indicates that the probability value of the target user triggering the target object is not lower than the probability threshold, then the target object is recommended to the target user.

11. An electronic device, characterized in that, include: processor; A memory for storing processor-executable instructions; wherein the processor implements the steps of the method as described in any one of claims 1-10 by executing the executable instructions.

12. A computer-readable storage medium, characterized in that, It stores computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-10.

13. A computer program product, characterized in that, Includes a computer program / instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1-10.

Images