Content recommendation method, device, apparatus, medium, and computer program product

Abstract

The application discloses a content recommendation method and device, equipment, medium and computer program product, and relates to the technical field of computers. The method comprises the following steps: obtaining target data of a target account; training parameters of a weight search model based on a state transition condition corresponding to the target data to obtain candidate model parameters; searching for weights corresponding to at least two evaluation indexes through the target weight search model based on current state information of the target account to obtain a target weight relationship between the at least two evaluation indexes; and fusing the at least two evaluation indexes through the target weight relationship to obtain recommended content to be pushed to the target account. The optimization target is to maximize the long-term satisfaction of users, so that the weight search model used to determine the evaluation indexes can provide more accurate weight relationships.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, medium, and computer program product for the present invention. Background Technology

[0002] Recommendation systems are an essential component in the process of users obtaining information (such as products, articles, images, videos, etc.). Recommendation systems determine what to recommend to users by ranking the content to be recommended. Typically, the ranking criteria are based on the user's prediction metrics for the recommended content.

[0003] In related technologies, when there are multiple prediction metrics, it is necessary to determine the weight relationship of each metric when ranking the recommended content. Methods for determining weight parameters generally include manually setting parameters and web-based hyperparameter search. Manually setting parameters involves selecting several sets of values ​​as candidate solution vectors for the fusion weights based on subjective experience, and then selecting the optimal solution vector through A / B experiments. In contrast, the grid search method exhaustively iterates through every possible combination of the optional hyperparameter set, and then selects the optimal hyperparameter combination as the fusion weights through cross-validation.

[0004] However, manually setting parameters and grid search involve large hyperparameter spaces, high costs of obtaining user feedback, and heavy reliance on expert knowledge, resulting in low efficiency in parameter determination. Furthermore, the corresponding recommendation models are limited to the user's preference for the current recommendation, leading to low accuracy in recommending content. Summary of the Invention

[0005] This application provides a content recommendation method, apparatus, device, medium, and computer program product, which can improve the accuracy of content recommendations to users. The technical solution is as follows:

[0006] On the one hand, a content recommendation method is provided, the method comprising:

[0007] Obtain target data for the target account;

[0008] Based on the state transition information corresponding to the target data, the parameters of the weighted search model are trained to obtain candidate model parameters. The state transition information is used to indicate the account state changes of the target account under historical weight relationships.

[0009] Based on the current status information of the target account, the weights corresponding to at least two evaluation indicators are searched through the target weight search model to obtain the target weight relationship between the at least two evaluation indicators. The model parameters of the target weight search model are the candidate model parameters, and the evaluation indicators are indicators that indicate the prediction of the recommendation situation of the recommended content.

[0010] By fusing the at least two evaluation indicators through the target weight relationship, recommended content is obtained to be pushed to the target account.

[0011] On the other hand, a content recommendation device is provided, the device comprising:

[0012] The acquisition module is used to acquire target data for the target account;

[0013] The training module is used to train the parameters of the weighted search model based on the state transition information corresponding to the target data, and obtain candidate model parameters. The state transition information is used to indicate the account state changes of the target account under the historical weight relationship.

[0014] The determination module is used to search for the weights corresponding to at least two evaluation indicators based on the current status information of the target account, and obtain the target weight relationship between the at least two evaluation indicators by using a target weight search model. The model parameters of the target weight search model are the candidate model parameters, and the evaluation indicators are indicators that indicate the prediction of the recommendation situation of the recommended content.

[0015] The recommendation module is used to fuse the at least two evaluation indicators through the target weight relationship to obtain recommended content to be pushed to the target account.

[0016] On the other hand, a computer device is provided, the terminal including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the recommended method described in any of the embodiments of this application.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one piece of program code is stored therein, the program code being loaded and executed by a processor to implement the recommended method described in any of the embodiments of this application.

[0018] On the other hand, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the recommended method described in any of the above embodiments.

[0019] The technical solution provided in this application includes at least the following beneficial effects:

[0020] Based on the state transitions corresponding to the target account's target data, a target weight search model is trained. When content recommendation is needed for the target account, the target weight relationship between evaluation metrics is determined based on the target account's current state information and the target weight search model. This target weight relationship is then used to fuse at least two evaluation metrics to determine the recommended content, which can then be pushed to the terminal corresponding to the target account. In other words, training the weight search model based on the changes in the target account's state under historical weight relationships, as indicated by the target data, allows for optimization with the goal of maximizing long-term user satisfaction, enabling the weight search model used to determine evaluation metrics to provide more accurate weight relationships. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of this application;

[0023] Figure 2 This is a flowchart illustrating a recommended method for providing content in an exemplary embodiment of this application;

[0024] Figure 3 This is a flowchart illustrating a recommended method for providing content in another exemplary embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a batch reinforcement learning model framework provided in an exemplary embodiment of this application;

[0026] Figure 5 This is a flowchart illustrating a recommended method for providing content in another exemplary embodiment of this application;

[0027] Figure 6 This is a flowchart of a security inspection module provided in an exemplary embodiment of this application;

[0028] Figure 7 This is a schematic diagram of the structure of a recommendation system provided in an exemplary embodiment of this application;

[0029] Figure 8 This is a recommended device structure block diagram of the content provided in an exemplary embodiment of this application;

[0030] Figure 9 This is a recommended device structure block diagram of the content provided in another exemplary embodiment of this application;

[0031] Figure 10 This is a schematic diagram of the structure of a server provided in an exemplary embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0033] First, a brief introduction to the terms used in the embodiments of this application:

[0034] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0035] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, as well as machine learning / deep learning, autonomous driving, and intelligent transportation.

[0036] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0037] In this embodiment, reinforcement learning is applied to the content recommendation method, which solves the problem that traditional recommendation methods cannot model long-term user satisfaction. It can improve the accuracy of content recommendation by maximizing long-term user satisfaction as the optimization goal.

[0038] Based on the above definitions, the application scenarios of the embodiments of this application are illustrated below. The recommended methods provided in this application can be applied to the following scenarios.

[0039] First, this content recommendation method can be applied to short video platforms. These platforms can push short video content to users based on their preferences. Illustratively, the platform server of a short video platform includes an offline model training module and an online recommendation module. The offline model training module trains the model parameters of the weighted search model based on the target data of the user account and publishes the trained model parameters to the online recommendation module. The target data includes the user account's historical viewing time, likes, and comments on short videos. The weighted search model in the online recommendation module updates the model parameters to obtain a target weighted search model. When a user terminal requests recommended short videos from the platform server, the online recommendation module inputs the current status information of the user account into the weighted search model, outputs the weight relationships corresponding to each evaluation indicator, predicts the estimated values ​​corresponding to each evaluation indicator, generates corresponding proxy functions, and sorts the short video content in the set of recommended short videos using these proxy functions to obtain a target sorting queue. This target sorting queue is then pushed to the user terminal, which pushes short video content according to the target sorting queue.

[0040] Secondly, this content recommendation method can be applied to e-commerce platforms. These platforms can push product content to users. Illustratively, the e-commerce platform's server includes an offline model training module and an online recommendation module. The offline model training module trains the model parameters of the weighted search model based on the target data of the user's account and publishes the trained model parameters to the online recommendation module. The target data includes the user's historical product purchase records, historical product browsing records, historical product favorites records, and shopping cart item records. The model parameters from the offline model training module are applied to the weighted search model of the online recommendation module and to determining the recommended product content. The platform server then feeds back the determined recommended product content to the user's terminal, where the recommended products are displayed.

[0041] Third, this content recommendation method can be applied to social media platforms. These platforms include interactive social media platforms for sharing short, real-time information. Illustratively, the platform server of a social media platform includes an offline model training module and an online recommendation module. The offline model training module trains the model parameters of the weighted search model based on the target data of the user account and publishes the trained model parameters to the online recommendation module. The target data includes the user account's historical information posting records, historical likes, historical reposts, and historical comments. The model parameters from the offline model training module are applied to the weighted search model of the online recommendation module and used to determine the recommended information content. The platform server then feeds back the determined recommended information content to the user terminal, where the user terminal displays the recommended information.

[0042] The recommendation methods described above can also be applied to other scenarios such as the random recommendation function of music platforms and the preference recommendation function of article platforms. Here, we only use the three scenarios mentioned above as examples and do not limit the specific application scenarios.

[0043] Please refer to Figure 1 The diagram illustrates an implementation environment provided by an exemplary embodiment of this application. This implementation environment includes: a terminal 110, a server 120, and a communication network 130.

[0044] Terminal 110 includes various types of terminal devices such as mobile phones, tablets, desktop computers, laptops, and vehicle terminals. The target application in terminal 110 requests content recommendation services from server 120. This target application, illustratively speaking, includes applications capable of providing content recommendations, such as short video applications, e-commerce applications, social applications, and article applications. The target application can be a standalone application, a web application, or a mini-program within a host application; no specific limitation is made here. Terminal 110 is also used to record user behavior data related to the target application and upload it to server 120.

[0045] Server 120 is used to provide content recommendation services. Server 120 receives behavioral data uploaded by terminal 110, stores it in the offline model training module corresponding to the user account, and trains the weight search model based on the target data of the stored target account. The trained model parameters are applied to the weight search model of the online recommendation module. After server 120 receives the content recommendation request from terminal 110, it determines the weight relationship of each evaluation index through the weight search model, determines the proxy function through the above weight relationship, sorts the content to be recommended through the proxy function, and sends the final target sorting queue to terminal 110. Terminal 110 performs content recommendation based on the target sorting queue.

[0046] It is worth noting that the aforementioned server 120 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0047] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0048] In some embodiments, the server 120 can also be implemented as a node in a blockchain system. Blockchain is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and cryptographic algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0049] A blockchain underlying platform can include processing modules such as user management, basic services, smart contracts, and operation monitoring. The user management module is responsible for managing the identity information of all blockchain participants, including maintaining public and private key generation (account management), key management, and maintaining the correspondence between user real identities and blockchain addresses (access management). Under authorization, it also monitors and audits transactions of certain real identities and provides risk control rule configuration (risk control audit). The basic service module is deployed on all blockchain node devices to verify the validity of business requests. After consensus is reached on valid requests, they are recorded in storage. For a new business request, the basic service first performs interface adaptation parsing and authentication (interface adaptation), then encrypts the business information using a consensus algorithm (consensus management), and transmits it completely and consistently to the shared ledger (network communication) for recording and storage. The smart contract module is responsible for contract registration, issuance, triggering, and execution. Developers can define contract logic using a programming language and publish it to the blockchain (contract registration). Based on the contract terms, it calls keys or other events to trigger execution, completing the contract logic. It also provides functions for contract upgrades and cancellations. The operation monitoring module is mainly responsible for deployment, configuration modification, contract settings, cloud adaptation, and real-time status visualization during product launch.

[0050] To illustrate, terminal 110 and server 120 are connected via communication network 130.

[0051] Please refer to Figure 2 This illustrates a recommended method for the content shown in one embodiment of this application, in which the method is applied to, for example, Figure 1 In the server shown, the method includes:

[0052] Step 201: Obtain the target data for the target account.

[0053] The content recommendation method in this application embodiment is a personalized recommendation method for a target account, where the target account is a user account.

[0054] For illustrative purposes, the aforementioned target data refers to historical user behavior data, which is data recorded by the user terminal based on the account behavior of the target account with the user's authorization. Optionally, taking the application of the method of this application to a short video platform as an example, the aforementioned account behavior includes, but is not limited to, at least one of the following behaviors: liking, collecting, video posting, and viewing time.

[0055] The server stores the account behavior uploaded by the terminal and the corresponding target account in the user database. In some embodiments, the server is also used to preprocess the account behavior, which transforms the account behavior into sample data for model training, such as user profile data, viewing trajectory data, etc.

[0056] Step 202: Based on the state transition situation corresponding to the target data, train the parameters of the weighted search model to obtain the candidate model parameters.

[0057] The state transition information indicates changes in the target account's state under historical weighting relationships. These changes refer to variations in user behavior under different recommended content, which correspond to different weighting relationships. Illustratively, processing the target data yields historical state information and historical weighting relationships for the target account, and these historical time periods correspond to each other.

[0058] For illustrative purposes, the aforementioned status information includes the target account's static information and historical behavior information. The static information includes the target account's user age, gender, and geographical location, while the historical behavior information includes the target account's liking, saving, posting, and viewing duration within the target time period.

[0059] The aforementioned historical weight relationship refers to the weight relationship between evaluation indicators obtained through the historical weight search model when the target account recommends content within a historical period. These evaluation indicators are used to predict the recommendation performance of the recommended content.

[0060] Indicatively, the target data can generate at least one set of corresponding data <historical state information, historical weight relationship> based on different historical time periods.

[0061] In some embodiments, the aforementioned historical state information and historical weight relationship also correspond to reward information. This reward information is determined by account behavior obtained after content recommendation based on the aforementioned historical weight relationship. That is, after the server determines the historical weight relationship based on the historical state information and determines the recommended content based on the historical weight relationship, the terminal pushes the recommended content, records the user behavior, and sends the feedback operation information corresponding to the recommended content to the server. This feedback operation information includes, but is not limited to, viewing time, liking behavior, commenting behavior, and empty behavior. The server can determine the reward information corresponding to the aforementioned historical state information and historical weight relationship based on this feedback operation information. In some embodiments, the aforementioned reward information r can be expressed by Formula 1, where s is the aforementioned historical state information, a is the aforementioned historical weight relationship, and m iLet g represent the i-th type of feedback operation information, where i is a positive integer. The calculation method g between the various feedback operation information can be a weighted average method, a normalized summation method, or other calculation methods, without specific limitations here.

[0062] Formula 1: r = R(s,a) = g(m1,m2,…)

[0063] In some embodiments, the aforementioned historical state information and historical weight relationship also correspond to next state information, wherein the next state information is used to indicate the state information of the target account after content recommendation by the historical weight relationship.

[0064] In some embodiments, the difference between the historical state information and the next state information is used to indicate the state transition of the target account under the historical weight relationship, and the weight search model is trained based on the state transition. Optionally, the model parameters of the weight search model being trained can be randomly initialized model parameters or weight parameters corresponding to the historical weight search model, which is not limited here.

[0065] Step 203: Based on the current status information of the target account, search for the weights corresponding to at least two evaluation indicators through the target weight search model to obtain the target weight relationship between at least two evaluation indicators.

[0066] In some embodiments, the model parameters of the target weight search model are the trained candidate model parameters.

[0067] To illustrate, the server obtains the current status information of the target account, inputs the current status information into the target weight search model, and outputs the target weight relationship between the above evaluation indicators.

[0068] Optionally, the aforementioned evaluation metrics can be preset by the system or customized according to the target account. These evaluation metrics include at least two of the following: predicted viewing time of the recommended content by the target account, predicted completion rate, predicted likes, and predicted comments.

[0069] Step 204: Integrate at least two evaluation metrics through target weight relationships to obtain recommended content to be pushed to the target account.

[0070] In this embodiment of the application, the recommended content can be determined in the following ways:

[0071] (i) By obtaining the evaluation scores corresponding to the content to be recommended and at least two evaluation indicators, and calculating the fusion score corresponding to the content to be recommended based on the evaluation scores and the target weight relationship between the evaluation indicators, the content to be recommended is pushed to the terminal as recommended content when the fusion score reaches the recommendation score threshold. In some embodiments, each evaluation indicator corresponds to a score prediction model, which can determine the evaluation score of the content to be recommended under the corresponding evaluation indicator based on the current status information of the target account.

[0072] (II) By obtaining the predicted scores corresponding to at least two of the above evaluation metrics, and based on the target weight relationship and predicted scores corresponding to at least two evaluation metrics, the candidate recommendation content set is sorted to obtain a target ranking list. The target ranking queue is then sent to the terminal corresponding to the target account, and the terminal retrieves the recommended content from the server according to the target ranking queue. The predicted scores are determined by a metric prediction model that can obtain the predicted score corresponding to each evaluation metric based on the current state information of the target account. Illustratively, the metric prediction model can be a multi-task learning (MTL) model, or other models, which are not limited here.

[0073] In some embodiments, the predicted scores of the above-mentioned evaluation indicators can be combined with a multi-objective ranking fusion method to perform weighted fusion of the predicted scores and obtain the final ranking score used for ranking. Illustratively, the above-mentioned objective ranking fusion method is implemented through a surrogate function, which can employ linear weighted fusion or weighted multiplicative fusion, without limitation. In some embodiments, the form of the surrogate function is shown in Formula 2, where score is the ranking score, α = (α1, α2, ...) are the model parameters to be searched, and score... i Let i be the estimated value of the i-th objective output in the MTL model, where i is a positive integer.

[0074] Formula 2: score=f α (score i )

[0075] Optionally, the above-mentioned recommended content includes, but is not limited to, content in the form of text, video, and audio.

[0076] In some embodiments, after receiving recommended content, the terminal displays the recommended content and simultaneously records the target account's feedback operation information regarding the recommended content, and sends the feedback operation information to the server. After receiving the target account's feedback operation information regarding the recommended content, the server generates corresponding target data based on the feedback operation information and stores the target data and the target account in the database.

[0077] In summary, the content recommendation method provided in this application trains a target weight search model based on the state transitions corresponding to the target account's target data. When content recommendation is needed for a target account, the target weight relationship between evaluation indicators is determined based on the target account's current state information and the target weight search model. This target weight relationship is then used to fuse at least two evaluation indicators to determine the recommended content, which is then pushed to the terminal corresponding to the target account. That is, by training the weight search model based on the changes in the target account's state under historical weight relationships indicated by the target data, the optimization objective is to maximize long-term user satisfaction, enabling the weight search model used to determine evaluation indicators to provide more accurate weight relationships.

[0078] Please refer to Figure 3 This document illustrates a recommended method for the content shown in one embodiment of this application. In this embodiment, the training process of a weighted search model is illustratively described, and the method includes:

[0079] Step 301: Obtain the target data for the target account.

[0080] For illustrative purposes, the aforementioned target data refers to data recorded by the user terminal based on the account behavior of the target account. Optionally, taking the application of the method of this application to a short video platform as an example, the aforementioned account behavior includes, but is not limited to, at least one of the following behaviors: liking, collecting, video posting, and viewing time.

[0081] The server stores the account behavior uploaded by the terminal and the corresponding target account in the user database.

[0082] Step 302: Generate state transition data based on the target data.

[0083] The aforementioned state transition data is used to indicate the changes in the historical account status of the target account. These changes in historical account status are determined by content recommendation based on historical state information under historical weight relationships, and there is a historical time period correspondence between historical state information and historical weight relationships.

[0084] In this embodiment of the application, the state transition data consists of a quadruple (s, a, r, s'), that is, the state transition data includes historical state information (state), historical weight relationship (action), reward information (reward) and next state information.

[0085] Status information includes static information and historical behavior information of the target account. The static information includes the user's age, gender, and geographical location information, while the historical behavior information includes the target account's liking, collection, posting, and viewing time during the target time period.

[0086] The reward information is determined by the changes in account status obtained after content recommendation based on the aforementioned historical weight relationship. That is, the server determines the historical weight relationship based on the historical status information, and then determines the recommended content based on the historical weight relationship. After the terminal pushes the recommended content, it records the user's behavior and sends the feedback operation information corresponding to the recommended content to the server. This feedback operation information includes, but is not limited to, viewing time, liking behavior, commenting behavior, and empty behavior. Based on this feedback operation information, the server can determine the reward information corresponding to the aforementioned historical status information and historical weight relationship.

[0087] Step 303: Train the parameters of the weighted search model based on the state transition data to obtain candidate model parameters.

[0088] As an illustration, depending on the specific application scenario, the above weight search model can be implemented by different deep learning models, such as reinforcement learning models, transfer learning models, inductive learning models, and teaching learning models.

[0089] In this embodiment, the weight search model employs a batch constrained Q-learning (BCQ) model to reduce the impact of overestimation and extrapolation errors during reinforcement learning training. Illustratively, the overall framework of the BCQ model is as follows: Figure 4 As shown, the BCQ model 400 includes a Variational Auto-Encoder (VAE) network 410, a perturbation network 420, and an evaluation network 430. The VAE network 410 includes an encoder 411 and a decoder 412. This VAE network 410 generates n weight relationships that conform to the training data distribution, where n is a positive integer. The perturbation network 420 determines the weight relationship with the maximum cumulative reward information, and the evaluation network 430 calculates the cumulative reward corresponding to the weight relationship, reducing the temporal difference error of the weight relationships output by the weight search model.

[0090] To illustrate, the training process of the BCQ model requires completing a target number of training iterations. This target number of iterations can be preset by the system or determined based on the model's loss function. For example, after a single training iteration, the model's loss value is determined based on the loss function. When the loss value is less than a preset threshold, the model is considered to have converged, thus completing the overall training process of the model.

[0091] In a single iteration of the BCQ model training, the VAE network is trained first. Specifically, the parameters of the VAE network are trained based on historical state information and historical weight relationships to obtain the first network parameters. These first network parameters are the network parameters obtained by the VAE network during the current iteration. Illustratively, historical state information is input into the variational autoencoder network, and the output is the first training weight relationship. Based on the difference between the first training weight relationship and the historical weight relationship, the parameters of the variational autoencoder network are iteratively trained to obtain the first network parameters. In some embodiments, the VAE network parameters ω are updated according to Formula 3, where s and a come from the state transition quadruple, and a' is the predicted weight relationship output by the VAE network, i.e., the first training weight relationship, which is the historical state information (s) and historical weight relationship (a) in the state transition data. This indicates that z follows a Gaussian distribution. Representing a 0-1 distribution, argmin represents minimizing the formula, and D KL This indicates the calculation of relative entropy.

[0092] Formula 3:

[0093]

[0094] in, For encoders in VAE networks, This is the decoder in the VAE network.

[0095] After determining the first network parameters, a variational autoencoder network based on the first network parameters is used to generate n first weight relationships corresponding to the historical state information. The parameters of the weight search model are trained based on the n first weight relationships and the historical state information to obtain the final candidate model parameters. The above first weight relationships conform to the data distribution of the state transition data, and n is a positive integer.

[0096] Indicatively, after training the variational autoencoder network, the variational autoencoder generates the first weight parameters that conform to the data distribution of the state transition data. That is, based on the variational autoencoder network with the first network parameters, n first weight relations corresponding to the historical state information are generated, where n is a positive integer. Based on the n first weight relations and the historical state information, the parameters of the weight search model are trained to obtain the candidate model parameters.

[0097] In some embodiments, the subsequent perturbation network and evaluation network are trained using the aforementioned first weight relationships and historical state information. Illustratively, perturbation weight relationships corresponding to n first weight relationships are generated based on the initial perturbation parameters; the parameters of the evaluation network are trained based on the perturbation weight relationships and state transition data to obtain second network parameters; the third network parameters corresponding to the perturbation network are determined based on the second network parameters and historical state information; and the parameters of the weight search model are trained based on the second and third network parameters to obtain candidate model parameters.

[0098] In some embodiments, the second network parameters are determined by perturbation weight relationships, reward information, and the next state information s' in the state transition quadruple. That is, the perturbation weight relationships, reward information, and next state information are substituted into the evaluation optimization function to obtain the optimized parameters of the evaluation network; the cumulative reward information corresponding to the historical weight relationships and the second weight relationships is obtained; the parameters of the evaluation network are trained based on the difference between the optimized parameters and the cumulative reward information to obtain the second network parameters. Specifically, the parameters of the evaluation network are trained based on minimizing the temporal-difference (TD) error. In one example, the evaluation optimization function is expressed by Equation 4, where r is the reward information in the state transition quadruple, γ is the future reward discount decay parameter, 0 < γ < 1, and a′ i，p The above perturbation weight relationship is given, where λ is the balance weight of the two networks. For the target evaluation network, and the current evaluation network and They have the same network structure.

[0099] Formula 4:

[0100]

[0101] The update formula for the network parameters θ of the current evaluation network is shown in Formula 5, where s represents historical state information, a represents historical weight relationships, y is obtained from Formula 4, and Q... θ For evaluating the network, B is the current training sample.

[0102] Formula 5: θ←argmin θ ∑ (s，a)∈B (yQ θ (s, a)) 2

[0103] In some embodiments, the third network parameters are determined by the second network parameters, historical state information s, and the weight relationships output under the first network parameters. Specifically, the historical state information is input into the variational autoencoder network under the first network parameters, and a second weight relationship is output. Based on the second network parameters, historical state information, and the second weight relationship, the initial perturbation parameters of the perturbation network are trained to obtain the third network parameters. Wherein, if the current training cycle is the first training cycle, the initial perturbation parameters are randomly initialized parameters of the perturbation network; if the current training cycle is not the first training cycle, the initial perturbation parameters are the parameters of the perturbation network obtained in the previous training cycle.

[0104] In one example, the update formula for the network parameter φ of the perturbed network is shown in Formula 6, where, G ω (s) refers to the above, that is This refers to the second weight relationship output by the variational autoencoder under the first network parameters described above. For perturbation coefficient networks, To evaluate the network, B represents the current training sample, s represents historical state information, and a represents historical weight information.

[0105] Formula Six:

[0106] In some embodiments, the BCQ model uses Equations 7 and 8 to perform delayed updates on the target network, where τ is the update rate of the target network, φ is the network parameter of the perturbed network, and θ is the network parameter of the evaluation network.

[0107] Formula 7: θ′←τθ+(1-τ)θ′

[0108] Formula 8: φ′←τφ+(1-τ)φ′

[0109] In summary, the update and training algorithm for the BCQ model is as follows:

[0110] (a) Determine the input data: training samples The training samples are defined as follows: number of iterations T, target network update rate τ, mini-batch size N, maximum perturbation ρ, number of action samples n, and dual-network balancing weights λ. These training samples are generated from target data obtained through the target account.

[0111] (b) Parameter initialization: Use random parameters θ1, θ2, etc. and ω-initialized evaluation network Disturbance network ξ φ and VAE model And initialize the target network and ξφ′ The parameters are θ′1←θ1, θ′2←θ2, and φ′←φ.

[0112] (c) Iterative training: For t=1…T

[0113] (c1) from Sample N state transition quadruples (s, a, r, s′);

[0114] (c2)

[0115] (c3) Update the VAE network using Formula 3;

[0116] (c4) Generate n weight relationships for each sample based on the VAE network:

[0117] (c5) Generate perturbation weight relationship:

[0118] (c6) Calculate y using Formula 4;

[0119] (c7) Update the evaluation network using Formula 5:

[0120] (c8) Update the perturbation network using Equation 6:

[0121] (c9) Update the target network using Equations 7 and 8;

[0122] (c10)end

[0123] Step 304: Update the parameters of the (i-1)th weight search model using the candidate model parameters to obtain the target weight search model.

[0124] Here, the candidate model parameters are the parameters of the i-th weight search model, where i is a positive integer. After the candidate model parameters are determined, the parameters of the preceding weight search model are updated. This preceding weight search model is the (i-1)-th weight search model, ultimately yielding the target weight search model.

[0125] Step 305: Based on the current status information of the target account, search for the weights corresponding to at least two evaluation indicators through the target weight search model to obtain the target weight relationship between at least two evaluation indicators.

[0126] To illustrate, the server obtains the current status information of the target account, inputs the current status information into the target weight search model, and outputs the target weight relationship between the above evaluation indicators.

[0127] Optionally, the aforementioned evaluation metrics can be preset by the system or customized according to the target account. These evaluation metrics include at least two of the following: predicted viewing time of the recommended content by the target account, predicted completion rate, predicted likes, and predicted comments.

[0128] Step 306: At least two evaluation indicators are fused through the target weight relationship to obtain the recommended content to be pushed to the target account.

[0129] In this embodiment, the predicted scores corresponding to at least two of the aforementioned evaluation metrics are obtained. Based on the target weight relationship and predicted scores corresponding to the at least two evaluation metrics, the candidate recommendation content set is sorted to obtain a target ranking list. The target ranking queue is then sent to the terminal corresponding to the target account, and the terminal retrieves recommendation content from the server according to the target ranking queue. The predicted scores are determined by a metric prediction model, which can obtain the predicted score corresponding to each evaluation metric based on the current state information of the target account. Illustratively, the metric prediction model can be an MTL model or other models, which are not limited here.

[0130] In summary, the content recommendation method provided in this application trains a target weight search model based on the state transitions corresponding to the target account's target data. When content recommendation is needed for a target account, the target weight relationship between evaluation indicators is determined based on the target account's current state information and the target weight search model. This target weight relationship is then used to fuse at least two evaluation indicators to determine the recommended content, which is then pushed to the terminal corresponding to the target account. That is, by training the weight search model based on the changes in the target account's state under historical weight relationships indicated by the target data, the optimization objective is to maximize long-term user satisfaction, enabling the weight search model used to determine evaluation indicators to provide more accurate weight relationships.

[0131] In this embodiment, an offline reinforcement learning method is used to perform weight search for multi-objective ranking fusion, and training samples are generated through users' historical viewing behavior, which reduces the cost of obtaining user feedback.

[0132] Please refer to Figure 5 This document illustrates a method for recommending content according to an embodiment of this application. In this embodiment, the recommendation system in the server includes an offline model training subsystem and an online recommendation subsystem. The offline model training subsystem is used to train the parameters of a weighted search model based on target data to obtain candidate model parameters. The online recommendation subsystem is used to obtain recommendation requests from terminals in real time and recommend content based on these requests. The method includes:

[0133] Step 501: Obtain the target data for the target account.

[0134] In this embodiment, the offline model training subsystem includes three main modules: a data preprocessing and sample generation module, a model training module, and a security check module. The data preprocessing and sample generation module aims to transform data from the user database into suitable input for the model, while the data in the user database comes from feedback operation information from target accounts received by the online recommendation subsystem.

[0135] Step 502: Based on the state transition situation corresponding to the target data, train the parameters of the weighted search model to obtain the candidate model parameters.

[0136] In this embodiment of the application, the parameter training of the weight search model is completed in the above-mentioned model training module. The weight search model is implemented using the BCQ model. The specific training process is shown in steps 302 to 303, which will not be described in detail here.

[0137] Step 503: Obtain the test weight distribution corresponding to the candidate weight search model.

[0138] The above candidate weight search model is a weight search model composed of candidate model parameters.

[0139] In this embodiment of the application, the above-mentioned test weight distribution refers to the distribution of the weight relationships output after the test samples are input into the candidate weight search model.

[0140] In this embodiment of the application, the candidate weight search model trained by the offline model training subsystem needs to pass through the security check module. The security check module aims to constrain the differences between the new model and the previous original model, and prevent drastic changes in the model from affecting online business.

[0141] Step 504: Obtain the historical weight distribution.

[0142] The above historical weight distribution corresponds to the weight search model of the (i-1)th iteration. That is, the historical weight distribution is the distribution of the weight relationships obtained by the test samples through the (i-1)th weight search model.

[0143] Step 505: In response to the constraint requirement that the difference between the test weight distribution and the historical weight distribution meets the requirement, the parameters of the (i-1)th weight search model are updated using the candidate model parameters to obtain the target weight search model.

[0144] In some embodiments, the security check module operates after each model training iteration, checking the difference between the historical distribution of weight relationships and the new model to ensure that the changes in each dimension do not exceed a pre-set threshold ∈. After a total of R model training iterations, if the weight relationship distribution meets the requirements, a new model is released; otherwise, the parameters of the original model remain unchanged.

[0145] Step 506: In response to the failure to match the constraint requirements due to the difference between the test weight distribution and the historical weight distribution, the (i-1)th weight search model is taken as the target weight search model.

[0146] This is illustrative; please refer to it. Figure 6 The flowchart of the security check module is shown, which includes: model training 601; statistical weight distribution 602; determining whether the stability check is satisfied 603, if yes, proceed to 604, if no, proceed to 606; exporting the model 604 and updating the historical weight distribution 605; determining whether the current number of retries is less than R 606, if yes, proceed to 601, if no, end; and publishing the model 607.

[0147] Step 507: Based on the current status information of the target account, search for the weights corresponding to at least two evaluation indicators through the target weight search model to obtain the target weight relationship between at least two evaluation indicators.

[0148] In this embodiment, the online recommendation subsystem includes a proxy function calculation module, a recommendation item sorting and distribution module, and a user behavior log recording module. After the terminal sends a recommendation request, the online request server sends the current status information corresponding to the target account to the target weight search model, and outputs a weight relationship vector, i.e., the target weight relationship.

[0149] Schematic illustration: The target weight search model described above is a BCQ model, which includes a VAE network that generates weight relationships (actions). Disturbance network ξ φ and dual evaluation network in Includes encoding module and decoding module Given the current state information, generate n values ​​that conform to the training data distribution. Disturbance network ξ φ The inputs are the current state information and the a generated by the VAE network. i The output is the action a after adding the perturbation. i，p =a i +ξ φ (s, a) i The goal of this network is to select a suitable a). p This makes the current cumulative reward Q θ (s, a) is maximized, where θ is the evaluation network parameter. This process can be formally expressed by Equation 9.

[0150] Formula Nine:

[0151] Step 508: Obtain the predicted scores corresponding to at least two evaluation indicators.

[0152] In this embodiment of the application, the online request server inputs the current status information of the target account into the MTL ranking model, which can output the predicted score corresponding to each evaluation indicator.

[0153] Step 509: Determine the function parameters corresponding to the target surrogate function based on the target weight relationship and predicted score corresponding to at least two evaluation indicators.

[0154] The surrogate function is used to determine the content recommendation score corresponding to the candidate recommended content.

[0155] In this embodiment, the online request server inputs the target weight relationship output by the target weight search model and the predicted score corresponding to the evaluation index output by the MTL ranking model into the surrogate function calculation module to determine the function parameters corresponding to the surrogate function. In one example, the surrogate function is shown in Formula 10, where α i For the target weight relationship, score i β is the predicted value output for the i-th evaluation metric in the MTL ranking model. i The bias constant is specified based on prior knowledge.

[0156] Formula 10: score = α i log(score i +β i )

[0157] Step 510: Determine the recommendation score of the candidate recommended content in the candidate recommended content set based on the target surrogate function.

[0158] In this embodiment of the application, the recommendation score of the candidate recommended content in the candidate recommended content set can be determined by the above-mentioned proxy function.

[0159] Step 511: Generate a target ranking list based on the recommendation scores of the candidate recommended content.

[0160] As an illustration, the recommendation item sorting and distribution module generates a target sorting list based on the recommendation scores obtained from the aforementioned proxy function. The content to be recommended in this target sorting list is arranged according to the recommendation scores. Optionally, the target sorting list may include all content in the set of content to be recommended, or it may include a portion of the content in the set of content to be recommended; this is not limited here.

[0161] Step 512: Send the target sorting queue to the terminal corresponding to the target account.

[0162] As an illustration, the recommendation sorting and distribution module distributes the generated target sorting queue to the terminal corresponding to the target account. In some embodiments, the target sorting queue includes content identifiers corresponding to the recommended content, and the terminal retrieves content from the server according to the recommended content order indicated by the target sorting queue based on the content identifiers in the target sorting queue.

[0163] In some embodiments, after receiving the recommended content, the terminal displays the recommended content and simultaneously records the target account's feedback operation information regarding the recommended content, then sends the feedback operation information to the server. After receiving the target account's feedback operation information regarding the recommended content, the user operation recording module in the online recommendation subsystem generates corresponding target data based on the feedback operation information and stores the target data in the user database of the offline model training subsystem.

[0164] The structure of the recommendation system 700 provided in this application embodiment is illustrative as follows: Figure 7 As shown, it includes an offline model training subsystem 710 and an online recommendation subsystem 720. The offline model training subsystem 710 includes a user database 711, a data preprocessing module 712, a sample database 713, a model training module 714, and a security check module 715. The online recommendation subsystem 720 includes a prediction model module 721, a request server 722, a proxy function calculation module 723, a recommendation item ranking and distribution module 724, and an operation record module 725. The prediction model module 721 includes a reinforcement learning model and an MTL model.

[0165] The feedback operation information corresponding to the target account in terminal 730 is sent to operation record module 725. Operation record module 725 sends the above data to user database 711. User database 711 stores the above feedback operation information as target data of target account and stores it in correspondence with target account. Data preprocessing module 712 obtains target data from user database 711 for preprocessing to obtain training sample data and stores it in sample database 713. Model training module 714 reads training sample data from sample database 713 and trains reinforcement learning model to obtain candidate model parameters. The candidate model parameters are sent to security check module 715. If the candidate model parameters meet the constraint requirements, security check module 715 updates the candidate model parameters to the reinforcement learning model of prediction model module 721 in online recommendation subsystem 720. After receiving the recommendation request from the terminal 730, the request server 722 sends the current status information of the target account to the prediction model module 721. The reinforcement learning model in the prediction model module 721 outputs the target weight relationship corresponding to the evaluation index, and the MTL model outputs the predicted score corresponding to the evaluation index. The request server 722 sends the above target weight relationship and predicted score to the surrogate function calculation module 723. The surrogate function calculation module 723 determines the function parameters corresponding to the surrogate function. The recommendation item sorting and distribution module 724 determines the recommendation score corresponding to the recommended content in the set of recommended content based on the above function parameters, generates a target sorting queue, and sends it to the terminal 730. The terminal 730 then sends the feedback operation information of the recommended content corresponding to the target sorting queue to the operation record module 725.

[0166] In some embodiments, the training frequency of the offline model training module can be preset, for example, the offline model training module trains model parameters once a day; or it can be determined based on the activity level of the target account, that is, in response to the target account's activity level meeting the preset activity requirements, parameter training is performed at a first preset training frequency; in response to the target account's activity level not meeting the preset activity requirements, parameter training is performed at a second preset training frequency, wherein the first preset training frequency is higher than the second preset training frequency. Optionally, the aforementioned activity level can be determined based on information such as the target account's online application duration and application usage frequency.

[0167] In summary, the content recommendation method provided in this application trains a target weight search model based on the state transitions corresponding to the target account's target data. When content recommendation is needed for a target account, the target weight relationship between evaluation indicators is determined based on the target account's current state information and the target weight search model. This target weight relationship is then used to fuse at least two evaluation indicators to determine the recommended content, which is then pushed to the terminal corresponding to the target account. That is, by training the weight search model based on the changes in the target account's state under historical weight relationships indicated by the target data, the optimization objective is to maximize long-term user satisfaction, enabling the weight search model used to determine evaluation indicators to provide more accurate weight relationships.

[0168] In this embodiment, training samples are generated based on users' historical viewing behavior, reducing the cost of obtaining user feedback. Weight search based on the reinforcement learning model can increase the application's usage time and the number of account interactions. Simultaneously, the closed-loop model of offline training and online real-time recommendation and data acquisition allows for model optimization based on users' latest preferences, ensuring the real-time performance of the recommendation system.

[0169] Please refer to Figure 8 This illustrates a recommended device structure block diagram of an exemplary embodiment of the content provided in this application, the device comprising the following modules:

[0170] Module 810 is used to obtain target data for the target account;

[0171] The training module 820 is used to train the parameters of the weighted search model based on the state transition situation corresponding to the target data, and obtain candidate model parameters. The state transition situation is used to indicate the account state change of the target account under the historical weight relationship.

[0172] The determination module 830 is used to search for the weights corresponding to at least two evaluation indicators based on the current status information of the target account through a target weight search model to obtain the target weight relationship between the at least two evaluation indicators. The model parameters of the target weight search model are the candidate model parameters, and the evaluation indicators are indicators that indicate the prediction of the recommendation situation of the recommended content.

[0173] The recommendation module 840 is used to fuse the at least two evaluation indicators through the target weight relationship to obtain recommended content to be pushed to the target account.

[0174] In some alternative embodiments, such as Figure 9 As shown, the training module 820 further includes:

[0175] The generation unit 821 is used to generate state transition data based on the target data. The state transition data is used to indicate the changes in the historical account status of the target account. The changes in the historical account status are determined by the historical status information after content recommendation under the historical weight relationship. The historical time period corresponds between the historical status information and the historical weight relationship.

[0176] Training unit 822 is used to train the parameters of the weight search model based on the state transition data to obtain the candidate model parameters.

[0177] In some optional embodiments, the weight search model includes a variational autoencoder network, and the state transition data includes the historical state information and the historical weight relationships;

[0178] The training unit 822 is further configured to train the parameters of the variational autoencoder network based on the historical state information and the historical weight relationship to obtain the first network parameters;

[0179] The generation unit 821 is further configured to generate n first weight relationships corresponding to the historical state information based on the variational autoencoder network under the first network parameters, wherein the first weight relationships conform to the data distribution of the state transition data, and n is a positive integer;

[0180] The training unit 822 is also used to train the parameters of the weight search model based on the n first weight relationships and the historical state information to obtain the candidate model parameters.

[0181] In some optional embodiments, the generation unit 821 is further configured to input the historical state information into the variational autoencoder network and output a first training weight relationship;

[0182] The training unit 822 is further configured to iteratively train the parameters of the variational autoencoder network based on the difference between the first training weight relationship and the historical weight relationship, so as to obtain the first network parameters.

[0183] In some optional embodiments, the weight search model further includes a perturbation network and an evaluation network. The perturbation network is used to determine the weight relationship with the largest cumulative reward information, and the evaluation network is used to reduce the time difference error of the weight relationship output by the weight search model. The perturbation network has perturbation initial parameters.

[0184] The generation unit 821 is further configured to generate perturbation weight relationships corresponding to the n first weight relationships based on the perturbation initial parameters;

[0185] The training unit 822 is also used to train the parameters of the evaluation network based on the perturbation weight relationship and the state transition data to obtain the second network parameters;

[0186] The generation unit 821 is further configured to determine the third network parameters corresponding to the disturbance network based on the second network parameters and the historical state information;

[0187] The training unit 822 is further configured to train the parameters of the weight search model based on the second network parameters and the third network parameters to obtain the candidate model parameters.

[0188] In some optional embodiments, the generation unit 821 is further configured to input the historical state information into the variational autoencoder network under the first network parameters and output a second weight relationship;

[0189] The training unit 822 is further configured to train the initial perturbation parameters of the perturbation network based on the second network parameters, the historical state information, and the second weight relationship to obtain the third network parameters.

[0190] In some optional embodiments, the state transition data further includes reward information and next state information, wherein the reward information is determined by the account state change obtained by content recommendation based on the historical weight relationship, and the next state information is used to indicate the state information corresponding to the target account after content recommendation based on the historical weight relationship;

[0191] The generation unit 821 is further configured to substitute the perturbation weight relationship, the reward information and the next state information into the evaluation optimization function to obtain the optimization parameters of the evaluation network;

[0192] The generation unit 821 is further configured to obtain the cumulative reward information corresponding to the historical weight relationship and the second weight relationship;

[0193] The training unit 822 is further configured to train the parameters of the evaluation network based on the difference between the optimized parameters and the cumulative reward information, thereby obtaining the second network parameters.

[0194] In some optional embodiments, the candidate model parameters are the parameters of the candidate i-th weighted search model, where i is a positive integer;

[0195] The device further includes an update module 850, used to update the parameters of the (i-1)th weight search model using the candidate model parameters, to obtain the target weight search model.

[0196] In some optional embodiments, the update module 850 further includes:

[0197] The first acquisition unit 851 is used to acquire the test weight distribution corresponding to the candidate weight search model, wherein the candidate weight search model is a weight search model composed of the candidate model parameters.

[0198] The first acquisition unit 851 is further configured to acquire historical weight distribution, wherein the historical weight distribution corresponds to the (i-1)th weight search model;

[0199] The determining unit 852 is used to update the parameters of the (i-1)th weight search model through the candidate model parameters in response to the constraint requirement that the difference between the test weight distribution and the historical weight distribution meets the constraint requirement, so as to obtain the target weight search model.

[0200] In some optional embodiments, the determining unit 852 is further configured to, in response to the failure of the difference between the test weight distribution and the historical weight distribution to match the constraint requirement, use the (i-1)th weight search model as the target weight search model.

[0201] In some optional embodiments, the recommendation module 840 further includes:

[0202] The second acquisition unit 841 is used to acquire the predicted scores corresponding to the at least two evaluation indicators;

[0203] The sorting unit 842 is used to sort the candidate recommendation content set based on the target weight relationship corresponding to the at least two evaluation indicators and the predicted score, so as to obtain a target sorting list;

[0204] The push unit 843 is used to send the target sorting queue to the terminal corresponding to the target account.

[0205] In some optional embodiments, the sorting unit 842 is further configured to determine the function parameters corresponding to the target proxy function based on the target weight relationship corresponding to the at least two evaluation indicators and the predicted score, wherein the proxy function is used to determine the content recommendation score corresponding to the candidate recommended content;

[0206] The sorting unit 842 is further configured to determine the recommendation score of the candidate recommendation content in the candidate recommendation content set based on the target proxy function;

[0207] The sorting unit 842 is also used to generate the target sorted list based on the recommendation scores of the candidate recommended content.

[0208] In some alternative embodiments, the apparatus further includes:

[0209] Feedback module 860 is used to receive feedback operation information from the target account regarding the recommended content;

[0210] The feedback module 860 is also used to generate the target data based on the feedback operation information;

[0211] The feedback module 860 is also used to store the target data and the target account in the database.

[0212] In summary, the content recommendation device provided in this application training method obtains a target weight search model based on the state transitions corresponding to the target account's target data. When content recommendation is needed for the target account, the target weight relationship between evaluation indicators is determined based on the target account's current state information and the target weight search model. This target weight relationship is then used to fuse at least two evaluation indicators to determine the recommended content, which is then pushed to the terminal corresponding to the target account. That is, by training the weight search model based on the changes in the target account's state under historical weight relationships indicated by the target data, the optimization objective is to maximize long-term user satisfaction, enabling the weight search model used to determine the evaluation indicators to provide more accurate weight relationships.

[0213] It should be noted that the content recommendation device provided in the above embodiments is only an example of the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the content recommendation device and the content recommendation method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0214] Figure 10 This illustration shows a schematic diagram of the structure of a server provided in an exemplary embodiment of this application. Specifically, it includes the following structure.

[0215] Server 1000 includes a Central Processing Unit (CPU) 1001, a system memory 1004 including Random Access Memory (RAM) 1002 and Read Only Memory (ROM) 1003, and a system bus 1005 connecting the system memory 1004 and the CPU 1001. Server 1000 also includes a mass storage device 1006 for storing the operating system 1013, application programs 1014, and other program modules 1015.

[0216] Mass storage device 1006 is connected to central processing unit 1001 via a mass storage controller (not shown) connected to system bus 1005. Mass storage device 1006 and its associated computer-readable media provide non-volatile storage for server 1000. That is, mass storage device 1006 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drive.

[0217] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile disc (DVD) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1004 and mass storage device 1006 described above can be collectively referred to as memory.

[0218] According to various embodiments of this application, server 1000 can also be connected to a remote computer on a network, such as the Internet. That is, server 1000 can be connected to network 1012 via network interface unit 1011 connected to system bus 1005, or it can use network interface unit 1011 to connect to other types of networks or remote computer systems (not shown).

[0219] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0220] Embodiments of this application also provide a computer device including a processor and a memory. The memory stores at least one instruction, at least one program, a code set, or an instruction set. The processor loads and executes the at least one instruction, at least one program, a code set, or an instruction set to implement the biometric identification method provided in the above-described method embodiments. Optionally, the computer device may be a terminal or a server.

[0221] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the biometric identification method provided in the above-described method embodiments.

[0222] Embodiments of this application also provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform any of the biometric methods described in the above embodiments.

[0223] Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0224] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0225] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A content recommendation method, characterized in that, The method includes: Obtain target data for the target account; Based on the state transition information corresponding to the target data, the parameters of the weighted search model are trained to obtain candidate model parameters. The state transition information is used to indicate the account state changes of the target account under historical weight relationships. Based on the current status information of the target account, the weights corresponding to at least two evaluation indicators are searched through the target weight search model to obtain the target weight relationship between the at least two evaluation indicators. The model parameters of the target weight search model are the candidate model parameters, and the evaluation indicators are indicators that indicate the prediction of the recommendation situation of the recommended content. By fusing the at least two evaluation indicators through the target weight relationship, recommended content is obtained to be pushed to the target account.

2. The method according to claim 1, characterized in that, The parameters of the weighted search model are trained based on the state transition information corresponding to the target data to obtain candidate model parameters, including: State transition data is generated based on the target data. The state transition data is used to indicate the changes in the historical account status of the target account. The changes in the historical account status are determined by content recommendation based on the historical status information under the historical weight relationship. The historical time periods correspond between the historical status information and the historical weight relationship. The parameters of the weighted search model are trained based on the state transition data to obtain the candidate model parameters.

3. The method according to claim 2, characterized in that, The weight search model includes a variational autoencoder network, and the state transition data includes the historical state information and the historical weight relationships. The parameters of the weighted search model are trained based on the target number of state transition data to obtain the candidate model parameters, including: The parameters of the variational autoencoder network are trained based on the historical state information and the historical weight relationship to obtain the first network parameters; Based on the variational autoencoder network under the first network parameters, n first weight relationships are generated corresponding to the historical state information. The first weight relationships conform to the data distribution of the state transition data, and n is a positive integer. The parameters of the weight search model are trained based on the n first weight relationships and the historical state information to obtain the candidate model parameters.

4. The method according to claim 3, characterized in that, The first network parameters are obtained by training the parameters of the variational autoencoder network based on the historical state information and the historical weight relationship, including: The historical state information is input into the variational autoencoder network, and the first training weight relationship is output. Based on the difference between the first training weight relationship and the historical weight relationship, the parameters of the variational autoencoder network are iteratively trained to obtain the first network parameters.

5. The method according to claim 3, characterized in that, The weight search model further includes a perturbation network and an evaluation network. The perturbation network is used to determine the weight relationship with the largest cumulative reward information, and the evaluation network is used to reduce the time difference error of the weight relationship output by the weight search model. The perturbation network has perturbation initial parameters. The process of training the parameters of the weight search model based on the n first weight relationships and the historical state information to obtain the candidate model parameters includes: Based on the initial perturbation parameters, a perturbation weight relationship corresponding to the n first weight relationships is generated; The parameters of the evaluation network are trained based on the perturbation weight relationship and the state transition data to obtain the second network parameters; Based on the second network parameters and the historical state information, the third network parameters corresponding to the disturbance network are determined; The parameters of the weight search model are trained based on the second network parameters and the third network parameters to obtain the candidate model parameters.

6. The method according to claim 5, characterized in that, The step of determining the third network parameters corresponding to the perturbation network based on the second network parameters and the historical state information includes: The historical state information is input into the variational autoencoder network under the first network parameters, and the second weight relationship is output. The third network parameters are obtained by training the initial perturbation parameters of the perturbation network based on the second network parameters, the historical state information, and the second weight relationship.

7. The method according to claim 6, characterized in that, The state transition data also includes reward information and next state information. The reward information is determined by the account state change obtained by content recommendation based on the historical weight relationship. The next state information is used to indicate the state information of the target account after content recommendation based on the historical weight relationship. The second network parameters are obtained by training the parameters of the evaluation network based on the perturbation weight relationship and the state transition data, including: Substituting the perturbation weight relationship, the reward information, and the next state information into the evaluation optimization function, the optimization parameters of the evaluation network are obtained; Obtain the cumulative reward information corresponding to the historical weight relationship and the second weight relationship; The parameters of the evaluation network are trained based on the difference between the optimized parameters and the cumulative reward information to obtain the second network parameters.

8. The method according to any one of claims 1 to 7, characterized in that, The candidate model parameters are the parameters of the i-th weighted search model, where i is a positive integer; Before obtaining the target weight relationship between at least two evaluation indicators by searching the weights corresponding to the at least two evaluation indicators based on the current status information of the target account using a target weight search model, the method further includes: The parameters of the (i-1)th weight search model are updated using the candidate model parameters to obtain the target weight search model.

9. The method according to claim 8, characterized in that, The step of updating the parameters of the (i-1)th weight search model using the candidate model parameters to obtain the target weight search model includes: Obtain the test weight distribution corresponding to the candidate weight search model, wherein the candidate weight search model is a weight search model composed of the candidate model parameters; Obtain the historical weight distribution, which corresponds to the (i-1)th weight search model; In response to the constraint requirement that the difference between the test weight distribution and the historical weight distribution meets the requirement, the parameters of the (i-1)th weight search model are updated using the candidate model parameters to obtain the target weight search model.

10. The method according to claim 9, characterized in that, The method further includes: In response to the failure of the difference between the test weight distribution and the historical weight distribution to match the constraint requirement, the (i-1)th weight search model is taken as the target weight search model.

11. The method according to any one of claims 1 to 7, characterized in that, The step of fusing the at least two evaluation indicators through the target weight relationship to obtain recommended content to be pushed to the target account includes: Obtain the predicted scores corresponding to the at least two evaluation indicators; Based on the target weight relationship corresponding to the at least two evaluation indicators and the predicted score, the candidate recommendation content set is sorted to obtain a target ranking list; The target sorting queue is sent to the terminal corresponding to the target account.

12. The method according to claim 11, characterized in that, The process of sorting the candidate recommendation content set based on the target weight relationship corresponding to the at least two evaluation indicators and the predicted score to obtain a target ranking list includes: Based on the target weight relationship and the predicted score corresponding to the at least two evaluation indicators, the function parameters corresponding to the target proxy function are determined, and the proxy function is used to determine the content recommendation score corresponding to the candidate recommended content. The recommendation score of the candidate recommended content in the candidate recommended content set is determined based on the target proxy function; The target ranking list is generated based on the recommendation scores of the candidate recommended content.

13. The method according to any one of claims 1 to 7, characterized in that, The method further includes: Receive feedback information from the target account regarding the recommended content; The target data is generated based on the feedback operation information; The target data and the target account are stored in the database.

14. A content recommendation device, characterized in that, The device includes: The acquisition module is used to acquire target data for the target account; The training module is used to train the parameters of the weighted search model based on the state transition information corresponding to the target data, and obtain candidate model parameters. The state transition information is used to indicate the account state changes of the target account under the historical weight relationship. The determination module is used to search for the weights corresponding to at least two evaluation indicators based on the current status information of the target account, and obtain the target weight relationship between the at least two evaluation indicators by using a target weight search model. The model parameters of the target weight search model are the candidate model parameters, and the evaluation indicators are indicators that indicate the prediction of the recommendation situation of the recommended content. The recommendation module is used to fuse the at least two evaluation indicators through the target weight relationship to obtain recommended content to be pushed to the target account.

15. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, a code set, or an instruction set, the at least one instruction, the at least one program, the code set, or the instruction set being loaded and executed by the processor to implement the recommended method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one piece of program code, which is loaded and executed by a processor to implement the recommended method as described in any one of claims 1 to 13.

17. A computer program product, the computer program product comprising a computer program / instructions, characterized in that, The computer program / instructions are stored in a computer-readable storage medium, the processor of the computer device reads the computer program / instructions from the computer-readable storage medium, and the processor executes the computer program / instructions to cause the computer device to perform the recommended method to implement the content as described in any one of claims 1 to 13.

Images