Recommendation method and apparatus based on artificial intelligence, electronic device and storage medium

HK40025261BActive Publication Date: 2026-08-28TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
HK42020014815
Authority / Receiving Office
HK · HK
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-08-21
Publication Date
2026-08-28

AI Technical Summary

Technical Problem

In existing personalized news recommendation systems, popular articles are prone to overfitting, resulting in an overemphasis on popular articles in the recommendation results. This makes it difficult to accurately capture the balance between personalization and popularity for users, thus affecting the user experience.

Method used

We employ a reinforcement learning-based recommendation model that balances personalization and popularity. Through a predictor network and a reinforcement learning model, we learn the degree of user demand for popularity and personalization, and adjust the proportion of each category of information in the recommendation results in real time to meet the diverse needs of users.

Benefits of technology

This achieves a balanced distribution of information across different categories in the recommendation results, improves user experience, meets users' balanced needs for personalization and popularity, and enhances the accuracy and diversity of recommendations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 00000000_0000_ABST
    Figure 00000000_0000_ABST
Patent Text Reader

Abstract

This invention provides a recommendation method, apparatus, electronic device, and storage medium based on artificial intelligence, relating to artificial intelligence technology. The method includes: acquiring user characteristics and environmental characteristics of a user to be recommended, and combining them into state characteristics of the user to be recommended; predicting the arrangement of recommendation information corresponding to the user to be recommended based on the state characteristics, wherein the arrangement includes multiple recommendation categories and the number of recommendation information items in each category; acquiring the recommendation information items corresponding to the number of recommendation information items in each category, and combining them into a set of recommendation information items corresponding to the user to be recommended; and performing a recommendation operation corresponding to the user to be recommended based on the set of recommendation information items. This invention achieves intelligent balance in multi-category information recommendation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to artificial intelligence technology, and more particularly to a recommendation method, apparatus, electronic device, and storage medium based on artificial intelligence. Background Technology

[0002] Artificial intelligence (AI) is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0003] Personalized news recommendation recommends news to users based on their interests and reading behavior. Related technologies often use ranking models to achieve this. User-side features and article-side features are input into a conventional neural network model for training, predicting the probability of an article being clicked. Due to the characteristics of popular articles, they are more likely to be sampled during the model's sampling phase. This makes the model more prone to overfitting on popular articles, resulting in generally higher prediction scores for popular articles during the ranking phase. Summary of the Invention

[0004] This invention provides an artificial intelligence-based recommendation method, apparatus, electronic device, and storage medium that can achieve intelligent balance in recommending multiple categories of information.

[0005] The technical solution of this invention is implemented as follows:

[0006] This invention provides an artificial intelligence-based recommendation method, comprising:

[0007] Obtain the user characteristics and environmental characteristics of the user to be recommended, and combine them into the state characteristics of the user to be recommended;

[0008] Based on the state features, the recommended information arrangement for the user to be recommended is predicted, wherein the recommended arrangement includes multiple recommendation categories and the number of recommended information for each recommendation category;

[0009] Obtain the number of recommended information corresponding to the quantity of each of the recommended categories, and combine them into a set of recommended information for the corresponding recommended user;

[0010] Based on the set of recommendation information, a recommendation operation is performed for the user to be recommended.

[0011] This invention provides an artificial intelligence-based recommendation device, comprising:

[0012] The feature acquisition module is used to acquire the user features and environmental features of the user to be recommended, and combine them into the status features of the user to be recommended.

[0013] The prediction module is used to predict the arrangement of recommendation information for the user to be recommended based on the state features, wherein the recommendation arrangement includes multiple recommendation categories and the number of recommendation information to be recommended for each recommendation category;

[0014] The module for obtaining the set of information to be recommended is used to obtain the information to be recommended corresponding to the quantity of each of the recommended categories, and combine them into a set of information to be recommended for the corresponding recommended user;

[0015] The recommendation module is used to perform recommendation operations for the user to be recommended based on the set of recommendation information.

[0016] In the above scheme, the feature acquisition module is further configured to:

[0017] Obtain at least one of the following user characteristics corresponding to the user to be recommended:

[0018] Basic attribute features used to characterize the basic information of the users to be recommended; social relationship features used to characterize the social relationships of users; interactive behavior features used to characterize the interactive behavior of users; and reading psychological features used to characterize the reading preferences of users.

[0019] Obtain at least one of the following environmental characteristics corresponding to the user to be recommended:

[0020] The time characteristics used to characterize the recommendation time when a recommendation is made to the user to be recommended; the user location characteristics used to characterize the location of the user to be recommended; the device characteristics used to characterize the device used by the user to be recommended; the network characteristics used to characterize the network environment of the device used by the user to be recommended; and the information characteristics used to characterize the recommendation information displayed on the previous screen when the recommendation is made to the user to be recommended.

[0021] In the above scheme, the prediction module is further configured to:

[0022] Determine the recommendation categories included in the arrangement of recommendation information for the user to be recommended;

[0023] Invoke the predictor corresponding to the recommendation category, and multiply the dense matrix in the predictor corresponding to the recommendation category with the state features of the user to be recommended to obtain a dense representation of the state features;

[0024] The dense representation corresponding to the state feature is mapped to a score of the number of recommended information items corresponding to the recommendation category;

[0025] The number of recommended information items corresponding to the highest rating is determined as the number of recommended information items corresponding to the recommended category.

[0026] In the above scheme, the prediction module is further configured to:

[0027] The at least two information categories that the page to be recommended to the user is adapted to are determined as the recommendation categories included in the recommendation information layout method;

[0028] The user to be recommended is given at least two preferred information categories that are pre-defined by the user. These categories are then used as the recommendation categories in the information arrangement.

[0029] At least two information categories that are similar to the user characteristics of the user to be recommended are identified as the recommendation categories included in the recommendation information arrangement.

[0030] In the above scheme, the prediction module is further configured to:

[0031] The multiple categories configured on the page displayed to the user to be recommended are determined as multiple categories included in the arrangement of recommendation information for the user to be recommended; wherein, the recommendation categories include popular recommendation categories and personalized recommendation categories;

[0032] Based on the state characteristics, predict the number of recommended information items for each category to be presented on the page.

[0033] In the above scheme, the recommendation module is further used for:

[0034] Before obtaining the recommendation information corresponding to the quantity of each of the recommended categories, the method further includes:

[0035] Obtain the recall information for the users to be recommended;

[0036] Based on the user characteristics and environmental characteristics of the users to be recommended, and the information characteristics corresponding to the recall information, the scores of the recall information are sorted in descending order;

[0037] The module for obtaining the set of information to be recommended is also used for:

[0038] The recall information that ranks first in the descending sort and belongs to the recommendation category is selected as the recommendation information to be recommended for the corresponding recommendation category.

[0039] In the above scheme, the recommendation module is further used for:

[0040] Obtain at least one of the following types of information as recall information for the corresponding user to be recommended:

[0041] Information that is similar to the historical browsing information of the user to be recommended, and whose content similarity is not less than the content similarity threshold;

[0042] Information whose behavior is similar to the historical behavior information of the user to be recommended, and whose behavior similarity is not less than the behavior similarity threshold.

[0043] In the above scheme, the recommendation module is further used for:

[0044] The features of the historical browsing information of the user to be recommended are merged with the features corresponding to each recall information to obtain the information features corresponding to the recall information;

[0045] The user characteristics and environmental characteristics of the user to be recommended, as well as the characteristics corresponding to the recall information, are subjected to feature combination processing based on feature correlation to obtain the indicator characteristics corresponding to the recall information.

[0046] The information features and indicator features corresponding to each recall information are fused to obtain a score for each recall information.

[0047] The recall information is sorted in descending order based on its score.

[0048] In the above scheme, the module for obtaining the set of information to be recommended is further used for:

[0049] Based on the predicted number of recommended information corresponding to each of the multiple recommendation categories, identify recommended information whose number exceeds the number of information in the corresponding recommendation category, and delete information until the number of recommended information in the corresponding recommendation category does not exceed the number of information in the corresponding recommendation category.

[0050] Recall the same number of recommended information items that were deleted but were not retained in descending order;

[0051] The information to be recommended, obtained through deletion and recall, is combined into a set of information to be recommended for the corresponding user.

[0052] In the above scheme, the device further includes:

[0053] The training module is used for:

[0054] Obtain the behavior logs of the users to be recommended in order to construct a training sample set for training the predictor;

[0055] The predictor is used to predict the arrangement of the recommendation information based on a single recommendation category.

[0056] The predictor is trained using reinforcement learning based on the training sample set.

[0057] In the above scheme, the training module is further used for:

[0058] Extract log sample data corresponding to the single recommendation category from the behavior logs of the user to be recommended;

[0059] Extract multiple quantities of behavioral sample data corresponding to the single recommendation category of the information to be recommended from the log sample data, as well as state feature samples corresponding to the behavioral sample data;

[0060] The behavioral sample data includes: browsing behavior, liking behavior, forwarding behavior, saving behavior, and commenting behavior;

[0061] The behavioral sample data is statistically processed to obtain indicator data representing multiple quantities of real scores for the information to be recommended.

[0062] The state feature samples, the number of information to be recommended, and the corresponding index data are combined to form a training sample set for training the predictor.

[0063] In the above scheme, the training module is further used for:

[0064] Initialize the parameters of the predictor and the parameters of the target training network corresponding to the predictor;

[0065] In the initialization, the parameters of the predictor are the same as the parameters of the target training network of the predictor, and the target training network is used to train the parameters of the predictor.

[0066] The following processing is performed during each iteration of the predictor's training:

[0067] The predictor performs prediction processing on the state feature samples and the number of recommended information in the recommendation category to obtain multiple training scores and multiple new state feature samples that correspond one-to-one with the multiple numbers of recommended information in the recommendation category. The predicted number of recommended information is determined from the multiple numbers of recommended information using a greedy mechanism.

[0068] Obtain new state feature samples and corresponding rewards for the state feature samples, and store the state feature samples, the new state feature samples, the rewards, the number of predicted recommendation information, and the progress status of the state feature samples as an array into the experience replay set;

[0069] Multiple sample groups are randomly sampled from the experience replay set, and the target training score corresponding to each sample group is determined;

[0070] The target training score and the training score corresponding to the number of recommended information to be predicted are substituted into the loss function to update the parameters of the predictor through gradient backpropagation until the progress state of the state feature sample is the termination state.

[0071] This invention provides an electronic device, comprising:

[0072] Memory, used to store executable instructions;

[0073] The processor, when executing executable instructions stored in the memory, implements the AI-based recommendation method provided in this embodiment of the invention.

[0074] This invention provides a storage medium storing executable instructions that, when executed by a processor, implement the AI-based recommendation method provided in this invention.

[0075] The embodiments of the present invention have the following beneficial effects:

[0076] By predicting the recommendation categories to which multiple pieces of information to be recommended belong and the number of pieces of information to be recommended in each category, an ideal balance is achieved in the number of pieces of information to be recommended in each category, thereby meeting the diverse needs of users for recommendation information in multiple categories. Attached Figure Description

[0077] Figure 1 This is an optional structural diagram of the recommendation system architecture based on artificial intelligence provided in an embodiment of the present invention;

[0078] Figure 2 This is a flowchart illustrating the recommendation system 100 in the artificial intelligence-based recommendation method provided in this embodiment of the invention.

[0079] Figure 3 This is a schematic diagram of the predictor network structure used in the AI-based recommendation method provided in this embodiment of the invention.

[0080] Figure 4 This is a schematic diagram of the structure of a server 200 that applies an artificial intelligence-based recommendation method according to an embodiment of the present invention;

[0081] Figure 5 This is an optional flowchart illustrating an artificial intelligence-based recommendation method provided in an embodiment of the present invention.

[0082] Figures 6A-6B This is an optional flowchart illustrating an artificial intelligence-based recommendation method provided in an embodiment of the present invention.

[0083] Figure 7This is a schematic diagram of the sorting model provided in an embodiment of the present invention;

[0084] Figure 8 This is an architecture diagram of the reinforcement learning training model provided in an embodiment of the present invention. Detailed Implementation

[0085] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on the present invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0087] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein is for the purpose of describing embodiments of the invention only and is not intended to limit the invention.

[0088] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0089] 1) Recommendation system: A recommendation system is a tool that automatically connects users and items. It can help users discover information that interests them in an information overload environment, and can also push information to users who are interested in it.

[0090] 2) Personalized news recommendations: Recommend news that users are interested in based on their interests and reading behavior.

[0091] 3) Recall: Select a batch of products or information to be recommended to users, which is equivalent to coarse sorting.

[0092] 4) Click-through rate prediction module: Calculates the click-through rate of candidate items or information based on the feature list and ranking model.

[0093] 5) Factorization Machine (FM) model, a sorting model, mainly solves the problem of feature combination in the case of sparse data.

[0094] 6) Reinforcement Learning (RL): Also known as reinforcement learning, evaluation learning, or enhancement learning, it is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent learns strategies to maximize rewards or achieve specific goals in the process of interacting with the environment.

[0095] Machine learning (ML) is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and instructional learning.

[0096] The solutions provided in this invention involve technologies such as reinforcement learning in artificial intelligence, and are specifically illustrated through the following embodiments.

[0097] Personalized news recommendation is based on users' interests and reading behavior. News is just one type of recommended information; other information can also be recommended, such as recommendations displayed on social media or shopping apps. This example illustrates how a personalized news recommendation system recommends news that users are interested in. A typical personalized news recommendation system includes four modules: a user profiling module, a recall module, a click-through rate (CTR) prediction module, and a re-ranking module. The recall module filters and sorts articles, then outputs them to the CTR prediction module. The CTR prediction module refines these recalled articles, and after passing through the re-ranking module, they are finally recommended to the user. The CTR prediction module plays a crucial role in the entire recommendation process. It relies on a ranking model (which can also be based on other metrics such as number of visits or page views). The quality of the ranking model determines the accuracy of the recommendation results. Related technologies... The ranking model used in the procedure employs a combination of factorization machine and neural network. It inputs user-side features (such as user preference categories, user preference tags, user age, gender, device model, etc.), article-side features (such as article category, article tags, article click count, article exposure, article author, article tone, etc.), and contextual features (such as access time, content displayed on the previous screen, etc.) into the model for training. The prediction term is the probability of an article being clicked. The inventors discovered that due to the characteristics of popular articles, they are more likely to be sampled during the model's sampling phase. This makes the model more prone to overfitting to popular articles, resulting in generally higher prediction scores for popular articles during the ranking phase. This leads to several problems: First, popular articles are easily prioritized in the recommendation results. For example, if a celebrity dies, due to high popularity, it is highly likely to be recommended to users. However, some users do not have a strong preference for popular articles and may even dislike them.Second, it fails to accurately capture the balance between personalization and popularity for users. While satisfying the needs of some users who want to track trending topics, it also harms the user experience of others who are overly averse to trending topics. This is not limited to the balance conflict between personalization and popularity, but also includes conflicts between other recommendation categories. This example illustrates the balance conflict between personalization and popularity. Based on this, this invention provides an AI-based recommendation method, which employs a reinforcement learning-based model to balance personalization and popularity. In this reinforcement learning model, the action is the number of popular articles displayed per screen, the reward is the resulting pageviews / plays, and the old state represents the environment and user characteristics before the action was taken. `new_state` represents the environment and user characteristics after taking the action. Whether it terminates (`is_terminal`) specifically determines if it's the last time browsing news. The predictor network uses a multilayer perceptron model, with the last fully connected layer outputting the action and action score. Through reinforcement learning, it learns the user's demand for popularity (a specific category of recommended information), capturing the balance between personalization and popularity, and accurately reflecting this in the recommendation model in real time. This aims to satisfy the needs of all users as much as possible, striving for a better user experience. The predictor model that balances personalization and popularity provided in this embodiment modifies the ranking models in related technologies, better meeting users' balanced needs for personalization and popularity, maximizing the satisfaction of more users' needs, and achieving a better user experience.

[0098] This invention provides an artificial intelligence-based recommendation method, apparatus, electronic device, and storage medium, capable of meeting users' needs for balanced recommendation information across multiple categories. The following describes exemplary applications of the electronic device provided in this invention. The electronic device can be implemented as various types of user terminals such as laptops, tablets, desktop computers, set-top boxes, and mobile devices (e.g., mobile phones, portable music players, personal digital assistants, dedicated messaging devices, portable gaming devices), or as a server. The following describes exemplary applications when the device is implemented as a server.

[0099] See Figure 1 , Figure 1This is an optional architecture diagram of an AI-based application scenario 100 provided in this embodiment of the invention. It includes a recommendation system 800, comprising a recall server 500, a ranking server 600, a prediction server 200, and a re-ranking server 700. A terminal 400 connects to the recommendation system 800 via a network 300. The recall server 500 connects to the ranking server 600. The network 300 can be a wide area network (WAN), a local area network (LAN), or a combination of both. The recommendation system 800 typically includes four modules: a user profiling module, a recall module (recall server), a click-through rate prediction module (ranking server), and a re-ranking module (re-ranking server). The user's terminal 400 sends requests to the recommendation system 800. These requests can be sent directly by the user or can be sent when the user's SDK detects information on the page during use. See [link to relevant documentation]. Figure 2 , Figure 2 This is a flowchart illustrating the recommendation system 800 in the AI-based recommendation method provided in this embodiment of the invention. After receiving a request from the client access layer, the recommendation system first calls the user profile service module to obtain the user's profile, and then passes the user's profile to the recall part (see...). Figure 1 The recall server (500) filters articles from the article pool and outputs them to the click-through rate (CTR) prediction module (see ranking server 600). The CTR prediction module then performs fine ranking on these recalled articles and sends the top K articles from the fine ranking to the re-ranking module (see re-ranking server 700) for final filtering and diversity ranking. Finally, the CTR prediction is displayed on the user's terminal (400). In the entire recommendation process, the prediction server (200) plays a crucial role. CTR prediction depends on the ranking model, and the quality of the ranking model determines the accuracy of the recommendation results. After the ranking model sorts the articles, the prediction server (200) calls the reinforcement learning-based predictor model for balancing multiple recommendation categories in the AI-based recommendation method provided in this embodiment of the invention. This model adjusts the proportion or number of recommended information for each recommendation category in the displayed results. Based on the user's equilibrium point for each recommendation category learned by the model, the prediction server (200) determines the proportion or number of recommended information for each recommendation category that will appear on the current display page and feeds it back to the recommendation results in real time, striving to obtain the maximum benefit. This benefit can be metrics such as page views, visits, and play counts.

[0100] See Figure 3 , Figure 3This is a schematic diagram of the predictor network structure used in the AI-based recommendation method provided in this embodiment of the invention. The predictor network adopts a multi-layer perceptual structure. The output of the last fully connected layer is the action space and the scores of each corresponding work area. The input to the predictor network is the state features. The dense matrix in the embedding lookup is multiplied with the state features of the user to be recommended to obtain a dense representation of the corresponding state features. The dense representation of the corresponding state features is mapped to a score for the number of recommended information corresponding to the recommendation category. There can be multiple recommendation categories. When the recommendation category is a popular category, it is mapped to a score for the number of recommended information corresponding to the popular category. That is, the number is the number of recommended information belonging to the popular category, such as 1, 2, 3, 4, and 5. The score is the score obtained when the number of recommended information belonging to the popular category is 1. The score is represented by indicators such as page views and play counts, or it can be obtained by converting page views and play counts.

[0101] See Figure 4 , Figure 4 This is a schematic diagram of the structure of a prediction server 200 that applies an artificial intelligence-based recommendation method according to an embodiment of the present invention. Figure 4 The server 200 shown includes at least one processor 210, memory 250, and at least one network interface 220. Various components in the terminal 400 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 4 The general labeled all buses as Bus System 240.

[0102] Processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0103] The memory 250 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 250 may optionally include one or more storage devices physically located away from the processor 210.

[0104] The memory 250 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 250 described in this embodiment is intended to include any suitable type of memory.

[0105] In some embodiments, memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0106] Operating system 251 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0107] The network communication module 252 is used to reach other computing devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, WiFi, and Universal Serial Bus (USB).

[0108] In some embodiments, the apparatus provided in this invention can be implemented in software. Figure 4 An AI-based recommendation device 255 stored in memory 250 is shown. It can be software in the form of programs and plug-ins, including the following software modules: feature acquisition module 2551, prediction module 2552, information set acquisition module 2553, recommendation module 2554, and training module 2555. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0109] In other embodiments, the AI-based recommendation device provided in this invention can be implemented in hardware. For example, the AI-based recommendation device provided in this invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the AI-based recommendation method provided in this invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0110] The following will describe the AI-based recommendation method provided by the embodiments of the present invention, using the exemplary application and implementation of the server provided in the embodiments of the present invention. Specifically, it will be described in two stages: the first stage involves the training stage of the model, which mainly involves the predictor model, and the second stage involves the application stage of the model.

[0111] See Figure 5 , Figure 5 This is an optional flowchart illustrating an artificial intelligence-based recommendation method provided in an embodiment of the present invention, which will be combined with... Figure 5 Steps 101-102 shown are explained.

[0112] In step 101, the server obtains the behavior logs of the users to be recommended in order to construct a training sample set for training the predictor, wherein the predictor is used to predict the arrangement of recommendation information based on a single recommendation category.

[0113] In some embodiments, the server described in step 101 may be a training server for training the prediction server. The step 101, which involves obtaining the behavior logs of the user to be recommended to construct a training sample set for training the predictor, can be achieved through the following technical solutions: extracting log sample data corresponding to a single recommendation category from the behavior logs of the user to be recommended; extracting multiple quantities of behavior sample data corresponding to the single recommendation category of the information to be recommended from the log sample data, as well as state feature samples of the corresponding behavior sample data; wherein, the behavior sample data includes: browsing behavior, liking behavior, forwarding behavior, collection behavior, and commenting behavior; performing statistical processing on the behavior sample data to obtain indicator data representing multiple quantities of real ratings of the information to be recommended; and combining the state feature samples, the quantity of information to be recommended, and the corresponding quantity of indicator data to form a training sample set for training the predictor.

[0114] In some embodiments, the behavior log of the user to be recommended records all the behavior data of the user. In order to train the predictor in a targeted manner, it is necessary to extract log sample data corresponding to a single recommendation category from the log. Here, the single recommendation category corresponds one-to-one with the predictor. Different recommendation categories correspond to different predictors. Taking the popular category as an example, the log sample data obtained here is at least related to the popular category recommendation. For example, the display page must contain at least the popular category recommendation. From the log sample data of the corresponding popular category, extract multiple quantities of behavior sample data of the information to be recommended for the single recommendation category, as well as the state feature samples of the corresponding behavior sample data. The behavior sample data here includes browsing behavior, liking behavior, forwarding behavior, collection behavior, and commenting behavior. The behavior sample data here is actually the user behavior feedback set by the user for a certain quantity of the information to be recommended under the recommendation category, which serves as the rating of the corresponding action. That is, statistical processing of the behavior sample data can obtain indicator data that respectively represent the true ratings of multiple quantities of the information to be recommended. The state feature samples, the quantity of the information to be recommended, and the corresponding quantity of indicator data are combined to form a training sample set for training the predictor.

[0115] In step 102, the server trains the predictor using reinforcement learning based on the training sample set.

[0116] In some embodiments, the server described in step 102 may be a training server for training the prediction server. The training of the predictor in step 102 using reinforcement learning based on the training sample set can be achieved through the following technical solution: initializing the parameters of the predictor and the parameters of the target training network corresponding to the predictor; wherein, during initialization, the parameters of the predictor are the same as the parameters of the target training network, and the target training network is used to train the parameters of the predictor; and performing the following processing during each iteration of the predictor's training: predicting the number of state feature samples and the number of recommended information for each recommendation category using the predictor, obtaining multiple quantities of recommended information corresponding one-to-one with the recommended categories. Multiple training scores and multiple new state feature samples are used, and a greedy mechanism is combined to determine the number of predicted information to be recommended from multiple quantities of information to be recommended. The rewards for the new state feature samples and the corresponding state feature samples are obtained, and the state feature samples, new state feature samples, rewards, the number of predicted information to be recommended, and the progress state of the state feature samples are stored as an array in the experience replay set. Multiple sample groups are randomly sampled from the experience replay set, and the target training score corresponding to each sample group is determined. The target training score and the training score corresponding to the number of predicted information to be recommended are substituted into the loss function, and the parameters of the predictor are updated through gradient backpropagation until the progress state of the state feature samples is the termination state.

[0117] In some embodiments, the predictor can be trained using the training process of a base training model in reinforcement learning. The inputs during the forward pass include the number of iterations T, the state feature dimension n, the action set A, the step size α, the decay factor γ, and the exploration rate. The predictor's network structure (Q-network), the number of samples for batch gradient descent m, and the output are the Q-network parameters. The intermediate steps are as follows: randomly initialize all parameters w of the Q-network, initialize the values ​​Q corresponding to all states and actions based on w, clear the experience replay set D, and start iterating from iteration number 1. The iteration process is as follows: initialize the first state of the current state sequence, and obtain the feature vector corresponding to the state. (S), used in Q networks (S) is used as input to obtain the Q-value output corresponding to all actions (each action in the action space) of the Q-network, and the result is obtained using the exploration rate-based method. The greedy algorithm selects the corresponding action A in the current Q-value output, executes the current action A in state S, and obtains a new state S′, corresponding to the feature vector of the new state. (S′) and the reward R of action A, and whether the current state is terminated, will { (S), A, R, (S′), whether it is a termination state} This quintuple is stored in the experience replay set D. The new state from the above steps becomes the current state. m samples are then sampled from the experience replay set D. (S j ), A j R j , (S′ j Is it in a terminated state? j}, j=1-m for any value, refer to the following formula (1) to calculate the current target Q value y j :

[0118] (1)

[0119] The parameters w of the Q-network are updated using the mean squared error loss function and backpropagation of the neural network's gradient. If S′ is the termination state, meaning this is the last refresh of recommendation information, then the current iteration is complete; otherwise, the forward propagation process is repeated. In practical applications, to ensure better convergence of the algorithm, the exploration rate is... The value needs to decrease as the iteration progresses. Here, "action" refers to the number of recommended information items of a certain category displayed per screen, which can be the number of popular articles displayed per screen. "Action space" is the range of selectable actions, i.e., the candidate values ​​of the number. "Reward" is the score obtained due to the action and state features, which can be represented by views, play counts, etc. "Current state" (old_state) is the representation of the environment and user features before taking the action, "new state" (new_state) is the representation of the environment and user features after taking the action, and "is_end" indicates whether the current is the last refresh of recommended information.

[0120] In some embodiments, the predictor can also be trained through a training process that improves the training model in reinforcement learning. The inputs during the forward pass include the number of iterations T, the state feature dimension n, the action set A, the step size α, the decay factor γ, and the exploration rate. The predictor's network structure (Q-network), the corresponding target network Q′, the number of samples for batch gradient descent m, the target network Q′ parameter update frequency C, and the output are the Q-network parameters. The intermediate steps are as follows: randomly initialize all parameters w of the Q-network, initialize the values ​​Q corresponding to all states and actions based on w, initialize the parameters w′=w of the target network Q′, clear the experience replay set D, and start iterating from iteration round number 1. The iteration process is as follows: initialize the first state of the current state sequence, and obtain the feature vector corresponding to the state. (S), used in Q networks (S) is used as input to obtain the Q-value output corresponding to all actions (each action in the action space) of the Q-network, and the result is obtained using the exploration rate-based method. The greedy algorithm selects the corresponding action A in the current Q-value output, executes the current action A in state S, and obtains a new state S′, corresponding to the feature vector of the new state. (S′) and the reward R of action A, and whether the current state is terminated, will { (S), A, R, (S′), whether it is a termination state} This quintuple is stored in the experience replay set D. The new state from the above steps becomes the current state. m samples are then sampled from the experience replay set D. (S j ), A j R j , (S′ j Is it in a terminated state? j}, j=1-m for any value, refer to the following formula (2) to calculate the current target Q value y j :

[0121] (2)

[0122] Using the mean squared error loss function, all parameters w of the predictor network are updated through backpropagation of the neural network's gradient. If the magnitudes of T and C are 1, the parameters w' of the target network Q' are updated to w, i.e., the parameters of the target network Q' are updated periodically. The Q network is updated in each iteration, but the current parameters of the Q network are only assigned to the target network Q' when the magnitudes of T and C are 1. If S' is the termination state, i.e., this is the last refresh of recommendation information, then the current iteration is complete; otherwise, the above forward propagation process is repeated. In practical applications, to ensure better convergence of the algorithm, the exploration rate is... The value needs to decrease as the iteration progresses. Here, "action" refers to the number of recommended information items of a certain category displayed per screen, which can be the number of popular articles displayed per screen. "Action space" is the range of selectable actions, i.e., the candidate values ​​of the number. "Reward" is the score obtained due to the action and state features, which can be represented by views, play counts, etc. "Current state" (old_state) is the representation of the environment and user features before taking the action, "new state" (new_state) is the representation of the environment and user features after taking the action, and "is_end" indicates whether the current is the last refresh of recommended information.

[0123] In some embodiments, the predictor can also be trained through a training process that improves the training model in reinforcement learning. The inputs during the forward pass include the number of iterations T, the state feature dimension n, the action set A, the step size α, the decay factor γ, and the exploration rate. The predictor's network structure (Q-network), the corresponding target network Q′, the number of samples for batch gradient descent m, the target network Q′ parameter update frequency C, and the output are the Q-network parameters. The intermediate steps are as follows: randomly initialize all parameters w of the Q-network, initialize the values ​​Q corresponding to all states and actions based on w, initialize the parameters w′=w of the target network Q′, clear the experience replay set D, and start iterating from iteration round number 1. The iteration process is as follows: initialize the first state of the current state sequence, and obtain the feature vector corresponding to the state. (S), used in Q networks (S) is used as input to obtain the Q-value output corresponding to all actions (each action in the action space) of the Q-network, and the result is obtained using the exploration rate-based method. The greedy algorithm selects the corresponding action A in the current Q-value output, executes the current action A in state S, and obtains a new state S′, corresponding to the feature vector of the new state. (S′) and the reward R of action A, and whether the current state is terminated, will { (S), A, R, (S′), whether it is a termination state} This quintuple is stored in the experience replay set D. The new state from the above steps becomes the current state. m samples are then sampled from the experience replay set D. (S j ), A j R j , (S′ j Is it in a terminated state? j}, j=1-m for any value, refer to the following formula (2) to calculate the current target Q value y j :

[0124] (3)

[0125] Using the mean squared error loss function, all parameters w of the predictor network are updated through backpropagation of the gradient of the neural network. If the magnitude of T and C is 1, the parameters w′=w of the target network Q′ are updated, that is, the parameters of the target network Q′ are updated periodically. The Q network will be updated in each iteration, but the parameters of the current Q network will only be assigned to the target network Q′ when the magnitude of T and C is 1. The mean squared error loss function is shown in formula (4).

[0126] (4)

[0127] If S′ is the terminating state, meaning this is the last refresh of recommendation information, then the current iteration is complete; otherwise, repeat the forward propagation process described above. In practical applications, to ensure better convergence of the algorithm, the exploration rate... The value needs to decrease as the iteration progresses. Here, "action" refers to the number of recommended information items of a certain category displayed per screen, which can be the number of popular articles displayed per screen. "Action space" is the range of selectable actions, i.e., the candidate values ​​of the number. "Reward" is the score obtained due to the action and state features, which can be represented by views, play counts, etc. "Current state" (old_state) is the representation of the environment and user features before taking the action, "new state" (new_state) is the representation of the environment and user features after taking the action, and "is_end" indicates whether the current is the last refresh of recommended information.

[0128] In some embodiments, instead of directly finding the largest Q value among all actions in the target Q network, the action corresponding to the largest Q value is first found in the current predictor network, i.e., see formula (5).

[0129] (5)

[0130] Then use this selected action To calculate the target Q-value within the target network, refer to formula (6):

[0131] (6)

[0132] Combining formulas (5) and (6) gives formula (3).

[0133] See Figure 6A , Figure 6A This is an optional flowchart illustrating an artificial intelligence-based recommendation method provided in an embodiment of the present invention, which will be combined with... Figure 6A Steps 103-106 shown will be explained.

[0134] In step 103, the server obtains the user characteristics and environmental characteristics of the user to be recommended, and combines them into the status characteristics of the user to be recommended.

[0135] In some embodiments, the server described in step 103 may be Figure 1In the prediction server, step 103, obtaining the user characteristics and environmental characteristics of the user to be recommended, can be achieved through the following technical solutions: obtaining at least one of the following user characteristics corresponding to the user to be recommended: basic attribute characteristics representing the basic information of the user to be recommended; social relationship characteristics representing the user's social relationships; interactive behavior characteristics representing the user's interactive behavior; reading psychological characteristics representing the user's reading preferences; obtaining at least one of the following environmental characteristics corresponding to the user to be recommended: time characteristics representing the recommendation time to the user to be recommended; user location characteristics representing the location of the user to be recommended; device characteristics representing the device used by the user to be recommended; network characteristics representing the network environment of the device used by the user to be recommended; and information characteristics representing the recommendation information displayed on the previous screen to the user to be recommended.

[0136] In some embodiments, basic information can be basic attributes such as the user's gender, age, and long-term residence; social relationships can be social attributes such as marital status or job position; user interaction behavior can be actions such as liking, forwarding, or collecting; reading preferences can be reading interests; points of interest can be entertainment gossip or international news; the category of information to be recommended can be the category of information display carrier, such as video information, image information, or text information; and the content of information to be recommended can be the content theme, such as educational topics or entertainment topics.

[0137] In some embodiments, environmental features also influence the prediction of the arrangement of recommended information. The timing of pushing information to the user to be recommended affects whether the user has free time to watch the recommended information, thus affecting the final score. That is, the baseline values ​​of each score predicted under the current environmental features will be relatively low. The location of the user to be recommended represents the current life scenario of the user to be recommended. Different life scenarios have an impact on the prediction of indicators. For example, when the location feature indicates that the user to be recommended is in a movie theater, the prediction results for duration and click-through rate will be significantly different from the prediction results when the location feature indicates that the user to be recommended is in a study room. The network of the device used by the user to be recommended will affect whether the user to be recommended wants to receive recommended information such as videos that require a lot of network resources.

[0138] In step 104, the server predicts the arrangement of recommended information for the corresponding user based on state features. The arrangement includes multiple recommendation categories and the number of recommended information items for each category.

[0139] Here, the server described in step 104 can be Figure 1The number of prediction servers can be the specific numerical value of the information to be recommended for each recommendation category, or the ratio between the numerical value of the information to be recommended for each recommendation category and the numerical value of the information to be recommended for all categories.

[0140] See Figure 6B , Figure 6B This is an optional flowchart of the AI-based recommendation method provided in this embodiment of the invention. In step 104, predicting the arrangement of recommendation information for the corresponding user based on state features can be achieved through steps 1041-1044, combining... Figure 6B Steps 1041-1044 are explained below.

[0141] In step 1041, the recommendation categories included in the arrangement of recommendation information for the user to be recommended are determined.

[0142] In some embodiments, determining the recommendation categories included in the recommendation information arrangement for the corresponding user to be recommended in step 1041 can be achieved through the following technical solutions: determining at least two information categories adapted to the page viewed by the user to be recommended as the recommendation categories included in the recommendation information arrangement; determining at least two information categories preferred by the user to be recommended as the recommendation categories included in the recommendation information arrangement; and determining at least two information categories similar to the user characteristics of the user to be recommended as the recommendation categories included in the recommendation information arrangement.

[0143] In some embodiments, when no selection operation for a recommendation category is received within the waiting time threshold, the behavior log of the user to be recommended is queried to obtain the categories of the recommended information where the user's behavior activity data exceeds the behavior activity threshold. Here, the behavior activity data exceeding the behavior activity threshold means that the user usually prefers this type of recommended information. That is, the preferred recommendation type is used as the recommendation type in the recommendation information arrangement, which is the main type to be predicted, thereby saving model prediction resources.

[0144] In step 1042, the predictor for the corresponding recommendation category is invoked, and the dense matrix in the predictor for the corresponding recommendation category is multiplied with the state features of the user to be recommended to obtain a dense representation of the corresponding state features.

[0145] In step 1043, the dense representation of the corresponding state feature is mapped to a score of the number of recommended information items corresponding to the recommendation category.

[0146] In step 1044, the number of recommended information items corresponding to the highest rating is determined as the number of recommended information items corresponding to the recommendation category.

[0147] In some embodiments, the predictor network adopts a multilayer perceptron structure. The output of the last fully connected layer is the action space and the scores of each task. The input to the predictor network is the state features. The dense matrix in the embedding lookup is multiplied with the state features of the user to be recommended to obtain a dense representation of the corresponding state features. The dense representation of the corresponding state features is mapped to a score of the number of recommended information corresponding to the recommendation category. There are many recommendation categories, such as personalized recommendation categories, popular recommendation categories, advertising recommendation categories, social recommendation categories, etc. When the recommendation category is a popular category, it is mapped to a score of the number of recommended information corresponding to the popular category. That is, the number here is the number of recommended information belonging to the popular category, such as 1, 2, 3, 4, and 5. The score is the score obtained when the number of recommended information belonging to the popular category is 1. The score here is represented by indicators such as page views and play counts, or it can be obtained by converting page views and play counts.

[0148] In some embodiments, the prediction of the arrangement of recommended information for the corresponding user based on state features in step 104 can also be achieved through the following technical solution: determining the multiple categories configured in the page displayed for the corresponding user as multiple categories included in the arrangement of recommended information for the corresponding user; wherein, the recommended categories include popular recommended categories and personalized recommended categories; and predicting the number of recommended information items in each category presented on the page based on state features.

[0149] In some embodiments, the predicted recommended information arrangement includes the predicted recommended information category and the number of recommended information belonging to each category. Here, the recommended information arrangement is predicted based on a displayed page, and "multiple" here means at least two.

[0150] In some embodiments, the predicted arrangement of recommended information is not based on a single displayed page; that is, the predicted arrangement of recommended information is independent of the displayed page. The recommended information arrangement may include multiple recommendation categories, which are not constrained by the page displayed on the client. The number of recommended information items under each recommendation category is also not constrained by the page displayed on the client. In other words, in the subsequent implementation process, the client needs to decide how to display the set of recommended information items for the corresponding recommended user during the recommendation process.

[0151] In step 105, the server obtains the information to be recommended corresponding to the quantity of each recommendation category, and combines them into a set of information to be recommended for the corresponding recommended user.

[0152] Here, the server described in step 105 can be Figure 1 The prediction server in the middle.

[0153] Before obtaining the recommendation information corresponding to the quantity of each recommendation category in step 105, the AI-based recommendation method provided in this embodiment of the invention further includes the following technical solutions: Figure 1 The ranking server in the middle implements the acquisition of recall information of the corresponding users to be recommended; based on the user characteristics and environmental characteristics of the users to be recommended, as well as the information characteristics corresponding to the recall information, the scores of the recall information are sorted in descending order. In step 105, the information to be recommended corresponding to the number of each recommendation category can be acquired by the following technical solution: select the recall information that is ranked first in the descending order and belongs to the recommendation category, and use it as the information to be recommended for the corresponding recommendation category.

[0154] In some embodiments, the recall information is obtained from the recall module, or it can be obtained through coarse sorting. A model combining factorization machine and neural network is used to train the model by inputting user-side features (such as user preference categories, user preference tags, user age, gender, device model, etc.), article-side features (such as article category, article tags, article click count, article exposure, article author, article tone, etc.), and contextual features (such as access time, content displayed on the previous screen, etc.). The prediction term can be the probability of the article being clicked.

[0155] In some embodiments, the acquisition of recall information for the corresponding user to be recommended can be achieved through the following technical solutions: Figure 1 The recall server in the system obtains at least one of the following types of information as recall information for the corresponding user to be recommended: information that is similar to the historical browsing information of the corresponding user to be recommended, and the content similarity is not less than the content similarity threshold; information that is similar to the historical behavior information of the corresponding user to be recommended, and the behavior similarity is not less than the behavior similarity threshold.

[0156] In some embodiments, multiple pieces of information to be recommended can be obtained through the recall server in the recommendation system. Obtaining multiple pieces of information to be recommended is in response to user requests from users who want to be recommended. These user requests can be query requests with specific goals or requests to initialize the application. Here, behavioral similarity refers to the similarity between user's historical behavior information and the information to be recommended, and content similarity refers to the similarity between user's historical browsing information and the information to be recommended.

[0157] In some embodiments, the step of sorting the scores of recall information in descending order based on the user characteristics and environmental characteristics of the user to be recommended, and the information characteristics corresponding to the recall information, can be achieved through the following technical solution: merging the features of the historical browsing information of the user to be recommended with the features corresponding to each recall information to obtain the information features of the corresponding recall information; performing feature combination processing based on feature correlation on the user characteristics and environmental characteristics of the user to be recommended, and the features corresponding to the recall information, to obtain the indicator features of the corresponding recall information; fusing the information features and indicator features corresponding to each recall information to obtain the score of each recall information; and sorting the scores of each recall information in descending order.

[0158] In some embodiments, the features of the user's historical browsing information and the features corresponding to each recall information are merged to obtain the information features of the corresponding recall information. Here, the historical browsing information can be the 10 click sequences closest to the current time point in the historical browsing records. The recall information is the information that needs to be predicted at present. The information features include article-side features (such as article category, article tags, article click count, article exposure, article author, article tone, etc.). The features of each click sequence (current and historical click sequences) are mapped to a low-dimensional space through an embedded query layer. Then, the feature vectors in the low-dimensional space are connected through a connection function. Finally, the information features of the corresponding recall information are obtained through a fully connected layer. This process combines the user features and environmental features of the user to be recommended, as well as the features corresponding to the recall information. The features are processed by combining features based on feature correlation to obtain the corresponding index features of recall information. This feature combination processing based on feature correlation is implemented based on factorization machine. Common linear models, such as linear regression and logistic regression, only consider the individual impact of each feature on the result, without considering the impact of the combination of features on the result. However, in some cases, the combination or association of several features has a significant impact on the result. The index features here are the predictive features obtained by combining the above features based on their correlation. The information features and index features corresponding to each recall information are fused to obtain the score of each recall information. Finally, the result of the factorization machine model is summed with the output of the neural network, and the sum is activated by an activation function to obtain the final predicted value.

[0159] In some embodiments, the step 105 of obtaining the number of recommended information corresponding to the quantity of each recommendation category and combining them into a set of recommended information for the corresponding recommended user can be achieved through the following technical solution: based on the predicted number of recommended information corresponding to each of the multiple recommendation categories, determine the number of recommended information exceeding the quantity of the corresponding recommendation category, and delete it down to the quantity of recommended information not exceeding the quantity of the corresponding recommendation category; recall the recommended information that is not retained in descending order and has the same number as the deleted recommended information; combine the recommended information obtained after deletion and recall into a set of recommended information for the corresponding recommended user.

[0160] In some embodiments, the deletion and recall operations are adjustments based on the arrangement of the recommended information. The number of recommended information items in each category of the recommended information obtained after sorting can be different. For example, for the recommended information items in the popular category, the predicted number is 3 out of every 10, but there may be 8 or more recommended information items in the popular category among the recommended information obtained after sorting. Therefore, it is necessary to delete the recommended information items belonging to the popular category. The deletion order can be based on the sorting score or random deletion, until there are no more than 3 recommended information items belonging to the popular category. Since the recommended information items belonging to the popular category are deleted, the number of recommended information items is less than 10. Therefore, it is necessary to recall the recommended information items in other recommended categories. The number of items recalled can be 5 (8-3=5). The result after recall needs to satisfy that the number of recommended information items in each category does not exceed the number of recommended information items in each category in the predicted arrangement of recommended information.

[0161] In step 106, the server performs a recommendation operation for the corresponding user to be recommended based on the set of recommendation information.

[0162] In some embodiments, the server described in step 106 may be Figure 1In the prediction server, step 106, based on the recommendation information set, performs the recommendation operation for the corresponding user to be recommended. This can be achieved through the following technical solution: when the number of empty display information slots per screen of the user's client is not less than the number of recommended information slots in the recommendation information set, all recommended information is extracted from the recommendation information set through the user's client; when the number of empty display information slots per screen of the user's client is less than the number of recommended information slots in the recommendation information set, the same number of recommended information slots as the number of empty display information slots per screen is extracted from the recommended information set through the user's client; in response to receiving a refresh operation for the client, the same number of recommended information slots as the current empty display information slots are extracted from the remaining recommended information in the recommendation information set through the client, until all recommended information in the recommendation information set is pushed to the client.

[0163] In some embodiments, the above-mentioned extraction of the same number of recommended information as the number of empty display information slots per screen from the recommended information through the client of the user to be recommended can be achieved by the following technical solution: obtaining the number of recommended information for each recommendation category through the client, and determining the proportion of different types of recommended information in the total number of recommended information in the recommended information set based on the number of recommended information for each recommendation category; determining the proportion of different types of recommended information in the total number of recommended information as the display proportion of different types of recommended information displayed in the client; and extracting different types of recommended information from all recommended information in the recommended information set according to the display proportion.

[0164] In some embodiments, for the front-end page displayed to the user to be recommended, the total number of recommended information predicted in the recommendation information layout method can be the same as or greater than the number that can be displayed in one display on the page. Thus, it can be displayed through page refresh. Here, refresh can be a periodic refresh or a refresh operation generated when leaving the current page and returning to the page. That is, the page corresponding to the recommendation information layout method can be just the current page, or it can be a continuous display process, that is, multiple pages generated by multiple screens, or multiple page displays triggered by the client in a single-screen device through a refresh operation.

[0165] The following will describe an exemplary application of the AI-based recommendation method provided in this embodiment of the invention in a real-world application scenario.

[0166] The AI-based recommendation method provided in this invention can be applied to news recommendation systems. It employs a reinforcement learning model, where the action is the number of popular articles displayed per screen, the reward is the resulting pageviews / plays, the old state (previous environment and user feature representation) and the new state (after taking the action), and whether it terminates (is_terminal) specifically determines whether it's the last time browsing news. The predictor network uses a multilayer perceptron model, with the last fully connected layer outputting the action and action score. Through the reinforcement learning model, it learns the user's demand for popularity (a specific category of recommended information), capturing the balance between personalization and popularity, and accurately reflecting this in the recommendation model in real time. This aims to satisfy the needs of all users as much as possible, striving for a better user experience. The predictor model that balances personalization and popularity provided in this invention modifies the ranking models in related technologies, better meeting users' balanced needs for personalization and popularity, maximizing the satisfaction of more users' needs, and achieving a better user experience.

[0167] News recommendation systems typically consist of four modules: a user profiling module, a recall module, a click-through rate (CTR) prediction module, and a re-ranking module. After receiving a request from the access layer, the recommendation system first calls the user profiling service module to obtain the user's profile. This profile is then passed to the recall module, which filters articles from the article pool and outputs them to the CTR prediction module. The CTR prediction module performs a fine-grained ranking on these recalled articles, and finally, the top-k articles from this ranking are sent to the re-ranking module for final filtering and diversity sorting before being displayed to the user. Throughout the entire recommendation process, the CTR prediction module plays a crucial role. It relies on the ranking model, and the quality of the ranking model determines the accuracy of the recommendation results.

[0168] The ranking model provided in this invention combines factorization machines and conventional neural network models. It inputs user-side features (such as user preference categories, user preference tags, user age, gender, device model, etc.), article-side features (such as article category, article tags, article click count, article exposure, article author, article tone, etc.), and contextual features (such as access time, content displayed on the previous screen, etc.) into the model for training. The prediction term is the probability of an article being clicked. See [link / reference]. Figure 7 , Figure 7This is a schematic diagram of the ranking model provided in this embodiment of the invention. The loss function used is the cross-entropy loss function, the optimization algorithm is the stochastic gradient descent algorithm, the number of samples selected for one training is 256, the samples in the training set are trained twice, and the update is performed in hourly increments. The graphics processor is used in a single-machine, single-card manner.

[0169] Here, the features of the historical browsing information (historical click sequence) of the user to be recommended are merged with the features corresponding to each recall information (current sequence) to obtain the information features of the corresponding recall information; the user features and environmental features of the user to be recommended, as well as the features corresponding to the recall information, are combined based on feature correlation to obtain the indicator features of the corresponding recall information; the information features and indicator features corresponding to each recall information are fused to obtain the score of each recall information; and the recall information is sorted in descending order according to the score of each recall information.

[0170] Specifically, the features of the user's historical browsing information are merged with the features corresponding to each recall information to obtain the information features of the corresponding recall information. The historical browsing information can be the 10 click sequences closest to the current time point in the historical browsing records. The recall information is the information that needs to be predicted now. These information features include article-side features (such as article category, article tags, article click count, article exposure, article author, article tone, etc.). An embedded query layer maps the features of each click sequence (current and historical click sequences) to a low-dimensional space. Then, a join function connects the feature vectors in the low-dimensional space. Finally, a fully connected layer obtains the information features of the corresponding recall information. The user features and environmental features of the user to be recommended, as well as the features corresponding to the recall information, are combined based on feature correlation to obtain the index features of the corresponding recall information. This feature correlation-based feature combination processing is implemented using a factorization machine. Figure 7 The characteristic combination formula (7) of the factorization machine is as follows:

[0171] (7)

[0172] Among them, parameters It represents the dot product of two vectors. This represents the i-th dimension vector of the coefficient matrix V. The representation needs to be combined features, where w is the weight coefficient of each feature and n is the dimension of the sample. In the Factorization Machine (FM) model, the first two parts are traditional linear models, and the last part takes into account the relationship between two distinct feature components. The Factorization Machine (FM) can also be generalized to a higher-order form, that is, to take into account the relationship between more distinct feature components.

[0173] Common linear models, such as linear regression and logistic regression, only consider the individual impact of each feature on the result, without considering the impact of feature combinations. However, in some cases, the combination or association of several features can have a significant impact on the result. The indicator feature here is the feature used for prediction obtained by combining the above features with related features. The information features corresponding to each recall information and the indicator feature are fused to obtain the score of each recall information. Finally, the result of the factorization machine model is summed with the output of the neural network, and the sum is activated by an activation function to obtain the final predicted value.

[0174] This invention proposes a reinforcement learning-based predictor model for balancing personalization and popularity. Employing a reinforcement learning model, it learns the user's demand for popularity, captures the balance point between personalization and popularity, and accurately reflects this in the recommendation model in real time, striving to meet the needs of all users and achieve a better user experience. The `action` parameter represents the number of popular articles displayed per screen; the `action space` represents the range of selectable actions; the `reward` represents the number of views and plays obtained as a result; `old_state` represents the environment and user features before taking the action; `new_state` represents the environment and user features after taking the action; and `is_end` indicates whether this is the last time the news is refreshed. The predictor (Q-network) uses a multi-layer perceptron structure, with the last fully connected layer outputting the action space and the scores of each action within that action space.

[0175] The training methods for reinforcement learning models are described below. (See also...) Figure 8 , Figure 8 This is an architecture diagram of the reinforcement learning training model provided in this embodiment of the invention. The inputs during the forward propagation process include the number of iteration rounds T, the state feature dimension n, the action set A, the step size α, the decay factor γ, and the exploration rate. The predictor's network structure (Q-network), the target network Q′ corresponding to the Q-network, the number of samples m for batch gradient descent, the parameter update frequency C of the target network Q′, and the output are the Q-network parameters. The intermediate steps are as follows: randomly initialize the values ​​Q corresponding to all states and actions, randomly initialize all parameters w of the current Q-network, initialize the parameters w′=w of the target network Q′, clear the experience replay set D, and start iterating from iteration round number 1. The iteration process is as follows: initialize the first state of the current state sequence, and obtain the feature vector corresponding to the state. (S), used in Q networks (S) is used as input to obtain the Q-value output corresponding to all actions (each action in the action space) of the Q-network, and the result is obtained using the exploration rate-based method. The greedy algorithm selects the corresponding action A in the current Q-value output, executes the current action A in state S, and obtains a new state S′, corresponding to the feature vector of the new state. (S′) and the reward R of action A, and whether the current state is terminated (is_end), will { (S), A, R, The quintuple (S′), indicating whether it is a termination state (is_end), is stored in the experience replay set D. The new state from the previous steps becomes the current state. Then, m samples are sampled from the experience replay set D. (S j ), A j R j , (S′ j Is it in a terminated state? j}, j=1-m for any value, refer to the following formula (3) to calculate the current target Q value y j :

[0176] (3)

[0177] Using the mean squared error loss function, all parameters w of the predictor network are updated through backpropagation of the gradient of the neural network. If the magnitude of T and C is 1, the parameters w′=w of the target network Q′ are updated, that is, the parameters of the target network Q′ are updated periodically. The Q network will be updated in each iteration, but the parameters of the current Q network will only be assigned to the target network Q′ when the magnitude of T and C is 1. The mean squared error loss function is shown in formula (4).

[0178] (4)

[0179] If S′ is the terminating state, meaning this is the last refresh of recommendation information, then the current iteration is complete; otherwise, repeat the forward propagation process described above. In practical applications, to ensure better convergence of the algorithm, the exploration rate... The value needs to decrease as the iteration progresses. Instead of directly finding the largest Q value among all actions in the target Q network, we first find the action corresponding to the largest Q value in the current predictor network, i.e., see formula (5).

[0180] (5)

[0181] Then use this selected action To calculate the target Q-value within the target network, refer to formula (6):

[0182] (6)

[0183] Combining formulas (5) and (6) gives formula (3).

[0184] After the ranking model sorts the data, a reinforcement learning-based predictor model that balances personalization and popularity is invoked to adjust the proportion of popular articles in the displayed results. Based on the balance point between user preferences for personalization and popularity learned by the model, the number of popular articles appearing on the current screen is determined and fed back to the recommendation results in real time, aiming to maximize benefits. The personalized and popular recommendation model provided in this embodiment of the invention corrects the ranking model by learning the user's demand for popularity and capturing the user's balance point between personalization and popularity, reflecting it accurately in real time in the recommendation model. This better meets the user's balanced needs for personalization and popularity, maximizing the satisfaction of more users' needs and achieving a better user experience. The main innovation of this embodiment of the invention lies in using reinforcement learning to train the predictor model to learn the user's demand for popularity and capture the user's balance point between personalization and popularity. The predictor (Q-network) uses a multilayer perceptron structure, but other network forms can also be used.

[0185] The following description continues to illustrate the exemplary structure of the AI-based recommendation device 255 provided in the embodiments of the present invention as a software module. In some embodiments, such as... Figure 3 As shown, the software modules stored in the AI-based recommendation device 255 in the memory 250 may include: a feature acquisition module 2551, used to acquire user features and environmental features of the user to be recommended, and combine them into state features of the user to be recommended; a prediction module 2552, used to predict the arrangement of recommendation information for the corresponding user to be recommended based on the state features, wherein the recommendation arrangement includes multiple recommendation categories and the number of recommendation information for each recommendation category; a set of recommendation information acquisition module 2553, used to acquire the recommendation information corresponding to the number of each recommendation category, and combine them into a set of recommendation information for the corresponding user; and a recommendation module 2554, used to perform recommendation operations for the corresponding user to be recommended based on the set of recommendation information.

[0186] In some embodiments, the feature acquisition module 2551 is further configured to: acquire at least one of the following user features corresponding to the user to be recommended: basic attribute features representing the basic information of the user to be recommended; social relationship features representing the user's social relationships; interactive behavior features representing the user's interactive behavior; reading psychological features representing the user's reading preferences; and acquire at least one of the following environmental features corresponding to the user to be recommended: time features representing the recommendation time of the user to be recommended; user location features representing the location of the user to be recommended; device features representing the device used by the user to be recommended; network features representing the network environment of the device used by the user to be recommended; and information features representing the recommendation information displayed on the previous screen recommended to the user to be recommended.

[0187] In some embodiments, the prediction module 2552 is further configured to: determine the recommendation categories included in the arrangement of recommendation information for the user to be recommended; call the predictor of the corresponding recommendation category, multiply the dense matrix in the predictor of the corresponding recommendation category with the state features of the user to be recommended, and obtain a dense representation of the corresponding state features; map the dense representation of the corresponding state features to a score of the number of recommendation information to be recommended corresponding to the recommendation category; and determine the number of recommendation information to be recommended corresponding to the highest score as the number of recommendation information to be recommended corresponding to the recommendation category.

[0188] In some embodiments, the prediction module 2552 is further configured to: determine at least two information categories adapted to the page viewed by the user to be recommended as recommendation categories included in the recommendation information layout; determine at least two information categories preferred by the user to be recommended as recommendation categories included in the recommendation information layout; and determine at least two information categories similar to the user characteristics of the user to be recommended as recommendation categories included in the recommendation information layout.

[0189] In some embodiments, the prediction module 2552 is further configured to: determine the multiple categories configured in the page displayed for the user to be recommended as multiple categories included in the arrangement of recommendation information for the user to be recommended; wherein the recommendation categories include popular recommendation categories and personalized recommendation categories; and predict the number of recommendation information items for each category to be presented on the page based on state features.

[0190] In some embodiments, the recommendation module 2554 is further configured to: before obtaining the number of recommended information corresponding to each recommendation category, the method further includes: obtaining recall information of the corresponding user to be recommended; sorting the scores of the recall information in descending order based on the user characteristics and environmental characteristics of the user to be recommended, and the information characteristics corresponding to the recall information; the recommended information set acquisition module 2553 is further configured to: select the recall information that is ranked first in the descending order and belongs to the recommendation category, as the recommended information of the corresponding recommendation category.

[0191] In some embodiments, the recommendation module 2554 is further configured to: obtain at least one of the following types of information as recall information for the corresponding user to be recommended: information that is similar to the historical browsing information of the corresponding user to be recommended, and the content similarity is not less than the content similarity threshold; information that is similar to the historical behavior information of the corresponding user to be recommended, and the behavior similarity is not less than the behavior similarity threshold.

[0192] In some embodiments, the recommendation module 2554 is further configured to: merge the features of the historical browsing information of the user to be recommended with the features corresponding to each recall information to obtain the information features of the corresponding recall information; perform feature combination processing based on feature correlation on the user features and environmental features of the user to be recommended, as well as the features corresponding to the recall information, to obtain the index features of the corresponding recall information; perform fusion processing on the information features and index features corresponding to each recall information to obtain the score of each recall information; and sort the recall information in descending order according to the score of each recall information.

[0193] In some embodiments, the module 2553 for obtaining the set of recommended information is further configured to: determine the number of recommended information that exceeds the number of the corresponding recommendation category based on the predicted number of recommended information that corresponds one-to-one with multiple recommendation categories, and delete the number of recommended information that does not exceed the number of the corresponding recommendation category; recall the recommended information that is not retained in descending order and has the same number as the deleted recommended information; and combine the recommended information obtained after deletion and recall into a set of recommended information for the corresponding recommended user.

[0194] In some embodiments, the AI-based recommendation device 255 further includes: a training module 2555, configured to: acquire the behavior logs of the user to be recommended, in order to construct a training sample set for training a predictor; wherein the predictor is used to predict the arrangement of recommendation information based on a single recommendation category; and train the predictor using reinforcement learning based on the training sample set.

[0195] In some embodiments, the training module 2555 is further configured to: extract log sample data corresponding to a single recommendation category from the behavior logs of the user to be recommended; extract multiple quantities of behavior sample data of the information to be recommended corresponding to a single recommendation category, and state feature samples of the corresponding behavior sample data from the log sample data; wherein, the behavior sample data includes: browsing behavior, liking behavior, forwarding behavior, collection behavior, and commenting behavior; perform statistical processing on the behavior sample data to obtain indicator data that respectively represent multiple quantities of real ratings of the information to be recommended; and combine the state feature samples, the quantity of the information to be recommended, and the corresponding quantity of indicator data into a training sample set for training the predictor.

[0196] In some embodiments, the training module 2555 is further configured to: initialize the parameters of the predictor and the parameters of the target training network corresponding to the predictor; wherein, during initialization, the parameters of the predictor are the same as the parameters of the target training network of the predictor, and the target training network is used to train the parameters of the predictor; and perform the following processing in each iteration of the predictor's training: perform prediction processing on the state feature samples and the number of recommended information for each recommendation category by the predictor to obtain multiple training scores and multiple new state feature samples that correspond one-to-one with the multiple numbers of recommended information for each recommendation category, and determine the predicted number of recommended information from the multiple numbers of recommended information using a greedy mechanism; obtain the rewards for the new state feature samples and the corresponding state feature samples, and store the state feature samples, new state feature samples, rewards, the predicted number of recommended information, and the progress state of the state feature samples as an array into the experience replay set; randomly sample multiple sample groups from the experience replay set and determine the target training score corresponding to each sample group; substitute the target training score and the training score corresponding to the predicted number of recommended information into the loss function to update the parameters of the predictor through gradient backpropagation until the progress state of the state feature samples is the termination state.

[0197] This invention provides a storage medium storing executable instructions. When these executable instructions are executed by a processor, they cause the processor to execute the AI-based recommendation method provided in this invention, for example... Figure 5 , Figures 6A-6B The AI-based recommendation method is shown.

[0198] In some embodiments, the storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0199] In some embodiments, executable instructions may take the form of a program, software, software module, script, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

[0200] As an example, executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple collaborating files (e.g., a file that stores one or more modules, subroutines, or code sections).

[0201] As an example, executable instructions can be deployed to execute on a single computing device, or on multiple computing devices located in one location, or on multiple computing devices distributed across multiple locations and interconnected via a communication network.

[0202] In summary, by predicting the recommendation categories to which multiple pieces of information to be recommended belong and the number of pieces of information to be recommended in each category, the recommended information in each category can be adjusted to meet users' balanced needs for recommendations across multiple categories.

[0203] The above description is merely an embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of the present invention are included within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based recommendation method, characterized in that, The method includes: Obtain the user characteristics and environmental characteristics of the user to be recommended, and combine them into the state characteristics of the user to be recommended; A pre-trained predictor is invoked to predict the arrangement of recommended information for the user to be recommended based on the state features. The arrangement includes multiple recommendation categories and the number of recommended information items for each category. The multiple recommendation categories are classifications based on recommendation strategies and include at least two of personalized recommendation categories and popular recommendation categories. Invoking the pre-trained predictor to predict the arrangement of recommended information for the user to be recommended based on the state features includes: multiplying a dense matrix in the predictor with the state features of the user to be recommended to obtain a dense representation corresponding to the state features; mapping the dense representation to multiple ratings of different numbers of recommended information items corresponding to the recommendation categories; and determining the number of items corresponding to the highest rating as the number of recommended information items corresponding to the recommendation category. Obtain the number of recommended information corresponding to the quantity of each of the recommended categories, and combine them into a set of recommended information for the corresponding recommended user; Based on the set of information to be recommended, perform a recommendation operation for the user to be recommended. The predictor is trained using reinforcement learning, the action space of which includes different combinations of the multiple recommendation categories, and the reward is a preset user engagement metric obtained from the set of information to be recommended.

2. The method according to claim 1, characterized in that, The process of obtaining the user characteristics and environmental characteristics of the users to be recommended includes: Obtain at least one of the following user characteristics corresponding to the user to be recommended: Basic attribute features used to characterize the basic information of the users to be recommended; social relationship features used to characterize the social relationships of users; interactive behavior features used to characterize the interactive behavior of users; and reading psychological features used to characterize the reading preferences of users. Obtain at least one of the following environmental characteristics corresponding to the user to be recommended: The time characteristics used to characterize the recommendation time when a recommendation is made to the user to be recommended; the user location characteristics used to characterize the location of the user to be recommended; the device characteristics used to characterize the device used by the user to be recommended; the network characteristics used to characterize the network environment of the device used by the user to be recommended; and the information characteristics used to characterize the recommendation information displayed on the previous screen when the recommendation is made to the user to be recommended.

3. The method according to claim 1, characterized in that, The step of determining the recommendation categories included in the arrangement of recommendation information corresponding to the user to be recommended includes: The at least two information categories that the page to be recommended to the user is adapted to are determined as the recommendation categories included in the recommendation information arrangement method; The user to be recommended is given at least two information categories that they prefer, which are determined in advance. These categories are then included in the recommended information arrangement. At least two information categories that are similar to the user characteristics of the user to be recommended are identified as the recommendation categories included in the recommendation information arrangement method.

4. The method according to claim 1, characterized in that, Based on the state features, predict the arrangement of recommendation information for the user to be recommended, including: The popular recommendation categories and personalized recommendation categories configured in the page displayed for the user to be recommended are determined as multiple recommendation categories included in the arrangement of recommendation information for the user to be recommended; Based on the state characteristics, predict the amount of information to be recommended when the popular recommendation category and the personalized recommendation category are presented on the page.

5. The method according to claim 1, characterized in that, Before obtaining the recommendation information corresponding to the quantity of each of the recommended categories, the method further includes: Obtain the recall information for the users to be recommended; Based on the user characteristics and environmental characteristics of the users to be recommended, and the information characteristics corresponding to the recall information, the scores of the recall information are sorted in descending order; The step of obtaining the information to be recommended corresponding to the quantity of each of the recommended categories includes: The recall information that is ranked first in the descending order and belongs to the recommendation category is selected as the information to be recommended for the corresponding recommendation category.

6. The method according to claim 5, characterized in that, The step of obtaining the recall information corresponding to the user to be recommended includes: Obtain at least one of the following types of information as recall information for the corresponding user to be recommended: Information that is similar to the historical browsing information of the user to be recommended, and whose content similarity is not less than the content similarity threshold; Information whose behavior is similar to the historical behavior information of the user to be recommended, and whose behavior similarity is not less than the behavior similarity threshold.

7. The method according to claim 5, characterized in that, The step of sorting the scores of the recall information in descending order based on the user characteristics and environmental characteristics of the user to be recommended, and the information characteristics corresponding to the recall information, includes: The features of the historical browsing information of the user to be recommended are merged with the features corresponding to each recall information to obtain the information features corresponding to the recall information; The user characteristics and environmental characteristics of the user to be recommended, as well as the characteristics corresponding to the recall information, are subjected to feature combination processing based on feature correlation to obtain the indicator characteristics corresponding to the recall information. The information features and indicator features corresponding to each recall information are fused to obtain a score for each recall information. The recall information is sorted in descending order based on its score.

8. The method according to claim 7, characterized in that, The step of obtaining the number of recommended information corresponding to the quantity of each of the recommended categories and combining them into a set of recommended information for the recommended user includes: Multiple pieces of information to be recommended are obtained for each of the recommendation categories, and the number of pieces of information to be recommended for each recommendation category is greater than the number predicted for the corresponding recommendation category; In the information to be recommended for each recommendation category, some information to be recommended is deleted so that the number of remaining information to be recommended is consistent with the number predicted for the corresponding recommendation category. Recall the same number of recommended information items that were deleted and were not retained in the descending sort; The filtered information to be recommended and the recalled information to be recommended are combined into a set of information to be recommended for the user.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the behavior logs of the users to be recommended in order to construct a training sample set for training the predictor; The predictor is used to predict the arrangement of the recommendation information based on a single recommendation category. The predictor is trained using reinforcement learning based on the training sample set.

10. The method according to claim 9, characterized in that, The step of obtaining the behavior logs of the users to be recommended in order to construct a training sample set for training the predictor includes: Extract log sample data corresponding to the single recommendation category from the behavior logs of the user to be recommended; Extract multiple behavioral sample data corresponding to the single recommendation category and state feature samples corresponding to the behavioral sample data from the log sample data; The behavioral sample data includes at least one of the following: browsing behavior; liking behavior; forwarding behavior; saving behavior; commenting behavior; The behavioral sample data is statistically processed to obtain index data representing multiple real scores of the information to be recommended. The state feature samples, the number of information to be recommended, and the corresponding index data are combined to form a training sample set for training the predictor.

11. The method according to claim 9, characterized in that, The step of training the predictor using reinforcement learning based on the training sample set includes: Initialize the parameters of the predictor and the parameters of the target training network corresponding to the predictor; In the initialization, the parameters of the predictor are the same as the parameters of the target training network of the predictor, and the target training network is used to train the parameters of the predictor. The following processing is performed during each iteration of the predictor's training: The predictor performs prediction processing on the state feature samples and the number of recommended information in the recommendation category to obtain multiple training scores and multiple new state feature samples that correspond one-to-one with the multiple numbers of recommended information in the recommendation category, and determines the predicted number of recommended information from the multiple numbers of recommended information. Determine the new state feature sample and the corresponding reward for the state feature sample, and store the state feature sample, the new state feature sample, the reward, the number of predicted recommendation information and the progress status of the state feature sample as an array into the experience replay set; Multiple sample groups are randomly sampled from the experience replay set, and the target training score corresponding to each sample group is determined; The target training score and the corresponding training score for the number of recommended information items predicted are substituted into the loss function to update the parameters of the predictor through gradient backpropagation until the progress state of the state feature sample is the termination state.

12. A recommendation device based on artificial intelligence, characterized in that, include: The feature acquisition module is used to acquire the user features and environmental features of the user to be recommended, and combine them into the status features of the user to be recommended. The prediction module is used to invoke a pre-trained predictor to predict the arrangement of recommendation information for the user to be recommended based on the state features. The arrangement of recommendation information includes multiple recommendation categories and the number of recommendation items for each category. The multiple recommendation categories are classifications based on recommendation strategies and include at least two of personalized recommendation categories and popular recommendation categories. The predictor is trained using reinforcement learning, and the action space of the reinforcement learning includes different combinations of the number of recommendation categories. The reward is a preset user engagement metric obtained based on the set of recommendation items. The module for obtaining the set of information to be recommended is used to obtain the information to be recommended corresponding to the quantity of each of the recommended categories, and combine them into a set of information to be recommended for the recommended user. The recommendation module is used to perform a recommendation operation for the user to be recommended based on the set of information to be recommended. The prediction module is further configured to: multiply the dense matrix in the predictor with the state features of the user to be recommended to obtain a dense representation corresponding to the state features; map the dense representation to multiple ratings of different numbers of information to be recommended corresponding to the recommendation category; and determine the number corresponding to the highest rating as the number of information to be recommended corresponding to the recommendation category.

13. The apparatus according to claim 12, characterized in that, The feature acquisition module is further configured to: acquire at least one of the following user features corresponding to the user to be recommended: Basic attribute features used to characterize the basic information of the user to be recommended; social relationship features used to characterize the user's social relationships; Interactive behavior features used to characterize user interaction behavior; Reading psychology characteristics used to characterize user reading preferences; Obtain at least one of the following environmental characteristics corresponding to the user to be recommended: The time characteristics used to characterize the recommendation time when a recommendation is made to the user to be recommended; the user location characteristics used to characterize the location of the user to be recommended; the device characteristics used to characterize the device used by the user to be recommended; and the network characteristics used to characterize the network environment of the device used by the user to be recommended. Information features used to characterize the recommendation information displayed on the previous screen when recommended to the user to be recommended.

14. The apparatus according to claim 12, characterized in that, The prediction module is further configured to: determine at least two information categories that the page to be recommended to the user to view as the recommendation categories included in the recommendation information arrangement method; The user to be recommended is given at least two information categories that they prefer, which are determined in advance. These categories are then included in the recommended information arrangement. At least two information categories that are similar to the user characteristics of the user to be recommended are identified as the recommendation categories included in the recommendation information arrangement method.

15. The apparatus according to claim 12, characterized in that, The prediction module is further configured to: determine the popular recommendation categories and personalized recommendation categories configured in the page displayed for the user to be recommended as multiple recommendation categories included in the arrangement of recommendation information for the user to be recommended; Based on the state characteristics, predict the amount of information to be recommended when the popular recommendation category and the personalized recommendation category are presented on the page.

16. The apparatus according to claim 12, characterized in that, The recommendation module is further configured to: before obtaining the number of recommended information corresponding to each recommendation category, the method further includes: obtaining recall information corresponding to the user to be recommended; and sorting the scores of the recall information in descending order based on the user characteristics and environmental characteristics of the user to be recommended, and the information characteristics corresponding to the recall information. The module for obtaining the set of information to be recommended is further configured to: select the recall information that is ranked first in the descending order and belongs to the recommendation category, as the information to be recommended for the corresponding recommendation category.

17. The apparatus according to claim 16, characterized in that, The recommendation module is further configured to: obtain at least one of the following types of information as recall information for the user to be recommended: Information that is similar to the historical browsing information of the user to be recommended, and whose content similarity is not less than the content similarity threshold; Information whose behavior is similar to the historical behavior information of the user to be recommended, and whose behavior similarity is not less than the behavior similarity threshold.

18. The apparatus according to claim 16, characterized in that, The recommendation module is further configured to: merge the features of the historical browsing information of the user to be recommended with the features corresponding to each recall information to obtain the information features corresponding to the recall information; The user characteristics and environmental characteristics of the user to be recommended, as well as the characteristics corresponding to the recall information, are subjected to feature combination processing based on feature correlation to obtain the indicator characteristics corresponding to the recall information. The information features and indicator features corresponding to each recall information are fused to obtain a score for each recall information. The recall information is sorted in descending order based on its score.

19. The apparatus according to claim 18, characterized in that, The module for obtaining the set of information to be recommended is further configured to: obtain multiple pieces of information to be recommended for each of the recommendation categories, wherein the number of pieces of information to be recommended for each recommendation category is greater than the number predicted for the corresponding recommendation category; In the information to be recommended for each recommendation category, some information to be recommended is deleted so that the number of remaining information to be recommended is consistent with the number predicted for the corresponding recommendation category. Recall the same number of recommended information items that were deleted and were not retained in the descending sort; The filtered information to be recommended and the recalled information to be recommended are combined into a set of information to be recommended for the user.

20. The apparatus according to claim 12, characterized in that, The device further includes a training module, configured to: acquire the behavior logs of the user to be recommended in order to construct a training sample set for training the predictor; The predictor is used to predict the arrangement of the recommendation information based on a single recommendation category. The predictor is trained using reinforcement learning based on the training sample set.

21. The apparatus according to claim 20, characterized in that, The training module is further configured to: extract log sample data corresponding to the single recommendation category from the behavior logs of the user to be recommended; Extract multiple behavioral sample data corresponding to the single recommendation category and state feature samples corresponding to the behavioral sample data from the log sample data; The behavioral sample data includes at least one of the following: browsing behavior; liking behavior; forwarding behavior; saving behavior; commenting behavior; The behavioral sample data is statistically processed to obtain index data representing multiple real scores of the information to be recommended. The state feature samples, the number of information to be recommended, and the corresponding index data are combined to form a training sample set for training the predictor.

22. The apparatus according to claim 20, characterized in that, The training module is further configured to: initialize the parameters of the predictor and the parameters of the target training network corresponding to the predictor; In the initialization, the parameters of the predictor are the same as the parameters of the target training network of the predictor, and the target training network is used to train the parameters of the predictor. The following processing is performed during each iteration of the predictor's training: The predictor performs prediction processing on the state feature samples and the number of recommended information in the recommendation category to obtain multiple training scores and multiple new state feature samples that correspond one-to-one with the multiple numbers of recommended information in the recommendation category, and determines the predicted number of recommended information from the multiple numbers of recommended information. Determine the new state feature sample and the corresponding reward for the state feature sample, and store the state feature sample, the new state feature sample, the reward, the number of predicted recommendation information and the progress status of the state feature sample as an array into the experience replay set; Multiple sample groups are randomly sampled from the experience replay set, and the target training score corresponding to each sample group is determined; The target training score and the corresponding training score for the number of recommended information items predicted are substituted into the loss function to update the parameters of the predictor through gradient backpropagation until the progress state of the state feature sample is the termination state.

23. An electronic device, characterized in that, include: Memory, used to store executable instructions; A processor, when executing executable instructions stored in the memory, implements the AI-based recommended method as described in any one of claims 1 to 11.

24. A storage medium, characterized in that, It stores executable instructions for causing a processor to execute, thereby implementing the AI-based recommendation method as described in any one of claims 1 to 11.

25. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by an electronic device, it implements the AI-based recommendation method according to any one of claims 1 to 11.