Data processing method and related apparatus

Abstract

A data processing method, which can be applied to the field of artificial intelligence. The method comprises: acquiring a prompt and a first feature representation, wherein the prompt is obtained according to information related to a recommendation scenario, and the first feature representation is obtained by performing feature extraction on attribute information of a user and an item; according to a fusion result of the prompt and the first feature representation, predicting operation information of the user with respect to the item by means of a recommendation model; and recommending the item to the user when the operation information meets a preset condition. In the present application, a prompt capable of carrying information related to a recommendation scenario is used as an input of a recommendation model, and the recommendation model can be guided to predict operation information with a bias towards characteristics of the scenario which is indicated by the prompt, thereby improving the prediction accuracy of a multi-recommendation scenario without increasing the scale of the recommendation model.

Description

A data processing method and related device

[0001] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office on June 16, 2023, with application number 202310729226.4 and invention name “A data processing method and related device”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence, and in particular to a data processing method and related devices. Background Art

[0003] Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a branch of computer science that seeks to understand the essence of intelligence and develop new intelligent machines that can respond in a manner similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0004] Selection rate prediction (also known as click-through rate prediction) refers to predicting the probability that a user will select an item under specific circumstances. For example, selection rate prediction plays a key role in recommendation systems for applications such as app stores and online advertising. This can maximize a company's revenue and improve user satisfaction. Recommendation systems must consider both the user's selection rate and the item's bid price. The selection rate is predicted by the recommendation system based on historical user behavior, while the item's bid price represents the system's revenue after the item is selected / downloaded. For example, a function can be constructed that calculates a function value based on the predicted user selection rate and item bid price. The recommendation system then sorts items in descending order based on this function value.

[0005] To meet the personalized needs of users, the recommendation system includes a variety of recommendation scenarios: browsers, negative one screens, video streams, etc. Users have different behaviors in different scenarios based on their preferences, and each scenario has user-specific behavioral characteristics. Under normal circumstances, a separate model is created for each scenario. Independent modeling for a single scenario cannot effectively capture the user's behavioral characteristics in different scenarios, as the same user may have different behaviors in different scenarios. In addition, when there are many scenarios, independent modeling and maintenance for each scenario will result in a large consumption of manpower and resources. Furthermore, if a feature extraction network is used to extract features from multiple scenarios, due to the differences in features between different scenarios, the network cannot learn the common behavioral characteristics, resulting in poor prediction accuracy of operation information.

[0006] Summary of the Invention

[0007] This application provides a data processing method that can improve the prediction accuracy of multiple recommendation scenarios without increasing the scale of the recommendation model.

[0008] In a first aspect, the present application provides a data processing method, comprising: obtaining a prompt and a first feature representation; the prompt is obtained based on information related to a recommendation scenario; the first feature representation is obtained by extracting attribute information of a user and an item; based on a fusion result of the prompt and the first feature representation, predicting the user's operation information on the item through a recommendation model; and recommending the item to the user when the operation information meets a preset condition.

[0009] In existing implementations, feature representations corresponding to user and item attribute information are used as input to the recommendation model to predict user operation information on items. However, in order to obtain highly accurate prediction results in multiple recommendation scenarios, one implementation is to build a more complex recommendation model. The recommendation model includes multiple branches (that is, multiple towers), and each branch can be responsible for processing the prediction of a recommendation scenario. However, as the number of recommendation scenarios continues to increase, the scale of the recommendation model will become larger, which is unacceptable in some scenarios.

[0010] In an embodiment of the present application, in addition to using the feature representation corresponding to the attribute information of the user and the item as the input of the recommendation model, a prompt that can carry information related to the recommendation scenario (that is, the prompt is obtained based on the information related to the recommendation scenario) is also used as the input of the recommendation model. This can guide the recommendation model to make predictions based on the characteristics of the scenario indicated by the prompt when predicting the operation information, thereby improving the prediction accuracy of multiple recommendation scenarios without increasing the scale of the recommendation model.

[0011] In a possible implementation, the information related to the recommendation scenario includes: indication information of the recommendation scenario; or attribute information of the user; or feature representation of the attribute information of the user.

[0012] In one possible implementation, obtaining the prompt includes: performing one-hot encoding on the information related to the recommendation scenario to obtain the prompt; or processing the information related to the recommendation scenario through a neural network to obtain the prompt; or processing the information related to the recommendation scenario through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and fusing the multiple feature representations according to the weights to obtain the prompt.

[0013] In one possible implementation, the recommendation scenario can be an application that serves specific needs, such as Huawei Browser and Huawei Video, or it can refer to a specific channel, such as the entertainment channel, news channel, technology channel, etc. in the browser information flow.

[0014] In one possible implementation, different recommendation scenarios are different applications, or different recommendation scenarios are different types of applications (for example, video applications and browser applications are different applications), or different recommendation scenarios are different functions of the same application (for example, different channels of the same application, such as news channels, technology channels, etc.). The above different functions can be divided according to recommendation categories.

[0015] In a possible implementation, the different recommendation scenarios may be different physical scenarios where the recommendations are made, such as different cities, different provinces, etc.

[0016] In a possible implementation, different recommendation scenarios may be application programs developed by different developers.

[0017] In a possible implementation, different recommendation scenarios may be different or different types of e-commerce products.

[0018] In a possible implementation, the attribute information includes user attributes of the user, and the user attributes include at least one of the following: gender, age, occupation, income, hobbies, and education level.

[0019] In a possible implementation, the attribute information includes item attributes of the item, and the item attributes include at least one of the following: item name, developer, installation package size, category, and praise level.

[0020] The user's attribute information may be an attribute related to the user's preference characteristics, at least one of gender, age, occupation, income, hobbies, and education level. Gender may be male or female, age may be a number between 0 and 100, occupation may be teacher, programmer, chef, etc., hobbies may be basketball, tennis, running, etc., and education level may be elementary school, junior high school, high school, college, etc. This application does not limit the specific type of the user's attribute information.

[0021] Among them, the items can be physical items or virtual items, such as APPs, audio and video, web pages, and news information. The attribute information of the items can be at least one of the item name, developer, installation package size, category, and praise. Among them, taking the item as an application as an example, the category of the item can be chat, parkour games, office, etc., and the praise degree can be a score, comment, etc. for the item; this application does not limit the specific type of attribute information of the item.

[0022] In a second aspect, the present application provides a data processing method, comprising: obtaining a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting feature information of the user and the item; based on the fusion result of the prompt and the first feature representation, the user's operation information on the item is predicted through a recommendation model; and the recommendation model is updated based on the operation information and the corresponding label.

[0023] In existing implementations, when fine-tuning a recommendation model for multiple recommendation scenarios, it is necessary to update the entire embedding layer and the recommendation model. This can lead to negative transfer and a seesaw effect, where improving the recommendation accuracy of a certain scenario comes at the expense of performance in other scenarios, resulting in poor recommendation accuracy in multiple scenarios.

[0024] In an embodiment of the present application, a prompt that can indicate a recommendation scenario is introduced as the input of the recommendation model. When fine-tuning the model (optional, and can also be applied to pre-training), the common parts between different scenarios can be fixed, and the parts of the model that are strongly related to the scenario (such as the neural network used to generate the prompt or the linear layer in the recommendation model, etc.) can be updated. This can improve the recommendation accuracy of the model for a recommendation scenario when training the model for the scenario, without compromising the performance of the model in other recommendation scenarios.

[0025] In one possible implementation, obtaining the prompt and the first feature representation includes: processing the information related to the recommendation scenario through a neural network to obtain the prompt; or processing the information related to the recommendation scenario through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and fusing the multiple feature representations according to the weights to obtain the prompt; and extracting features of the attribute information of the user and the item through an embedding layer to obtain the first feature representation.

[0026] In one possible implementation, the method further includes: updating the neural network according to the operation information and the corresponding label while keeping the parameters of the embedding layer fixed.

[0027] In a possible implementation, updating the recommendation model includes: updating only part of the networks in the recommendation model.

[0028] In a possible implementation, the partial network is a linear layer.

[0029] In one possible implementation, the information related to the recommendation scenario includes:

[0030] Indicative information of the recommended scenario; or

[0031] Attribute information of the user; or

[0032] A feature representation of the attribute information of the user.

[0033] In one possible implementation,

[0034] Different recommendation scenarios are different applications; or,

[0035] Different recommendation scenarios are different types of applications; or,

[0036] Different recommendation scenarios are different functions of the same application; or,

[0037] The different recommendation scenes are different physical scenes where the recommendation is made; or,

[0038] Different recommendation scenarios are applications developed by different developers; or

[0039] Different recommendation scenarios are different or different types of e-commerce products.

[0040] In a third aspect, the present application provides a data processing device, comprising:

[0041] a processing module configured to obtain a first feature representation and a second feature representation; the first feature representation is obtained by extracting features from a prompt, and the second feature representation is obtained by extracting features from attribute information of the user and the item; the prompt carries information related to the recommendation scenario; and based on the fusion result of the first and second feature representations, predict the user's operation information on the item using a recommendation model;

[0042] The recommendation model is used to recommend the item to the user when the operation information meets a preset condition.

[0043] In one possible implementation, the information related to the recommendation scenario includes:

[0044] Indicative information of the recommended scenario; or

[0045] Attribute information of the user; or

[0046] A feature representation of the attribute information of the user.

[0047] In a possible implementation, the processing module is specifically configured to:

[0048] Performing one-hot encoding on the information related to the recommendation scenario to obtain the prompt; or,

[0049] Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or

[0050] The information related to the recommendation scenario is processed through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and the multiple feature representations are fused according to the weights to obtain the prompt.

[0051] In one possible implementation,

[0052] Different recommendation scenarios are different applications; or,

[0053] Different recommendation scenarios are different types of applications; or,

[0054] Different recommendation scenarios are different functions of the same application; or,

[0055] The different recommendation scenes are different physical scenes where the recommendation is made; or,

[0056] Different recommendation scenarios are applications developed by different developers; or

[0057] Different recommendation scenarios are different or different types of e-commerce products.

[0058] In a fourth aspect, the present application provides a data processing device, comprising:

[0059] A processing module, configured to obtain a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting attribute information of the user and the item;

[0060] Predicting the user's operation information on the item through a recommendation model based on a fusion result of the prompt and the first feature representation;

[0061] An updating module is used to update the recommendation model according to the operation information and the corresponding label.

[0062] In a possible implementation, the processing module is specifically configured to:

[0063] Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or processing the information related to the recommendation scenario through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and fusing multiple feature representations according to the weights to obtain the prompt;

[0064] The attribute information of the user and the item is subjected to feature extraction through the embedding layer to obtain the first feature representation.

[0065] In a possible implementation, the update module is further configured to:

[0066] The neural network is updated according to the operation information and the corresponding label while keeping the parameters of the embedding layer fixed.

[0067] In a possible implementation, the update module is specifically configured to:

[0068] Only part of the network in the recommendation model is updated.

[0069] In a possible implementation, the partial network is a linear layer.

[0070] In one possible implementation, the information related to the recommendation scenario includes:

[0071] Indicative information of the recommended scenario; or

[0072] Attribute information of the user; or

[0073] A feature representation of the attribute information of the user.

[0074] In one possible implementation,

[0075] Different recommendation scenarios are different applications; or,

[0076] Different recommendation scenarios are different types of applications; or,

[0077] Different recommendation scenarios are different functions of the same application; or,

[0078] The different recommendation scenes are different physical scenes where the recommendation is made; or,

[0079] Different recommendation scenarios are applications developed by different developers; or

[0080] Different recommendation scenarios are different or different types of e-commerce products.

[0081] In a fifth aspect, an embodiment of the present application provides a data processing device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to execute the programs in the memory to perform any optional method as described in the first aspect above.

[0082] In a sixth aspect, an embodiment of the present application provides a data processing device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to execute the programs in the memory to perform any optional method as described in the second aspect above.

[0083] In the seventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer-readable storage medium is run on a computer, it enables the computer to execute the above-mentioned first aspect and any optional method, and any optional method of the above-mentioned third aspect.

[0084] In an eighth aspect, an embodiment of the present application provides a computer program product, comprising code, which, when executed, is used to implement the above-mentioned first aspect and any optional method, and any optional method of the above-mentioned second aspect.

[0085] In a ninth aspect, the present application provides a chip system comprising a processor for supporting an execution device or a training device in implementing the functions described in the aforementioned aspects, such as transmitting or processing data or information involved in the aforementioned methods. In one possible design, the chip system further comprises a memory for storing program instructions and data necessary for the execution device or the training device. The chip system may be comprised of a single chip or may include a chip and other discrete components. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] Figure 1 is a schematic diagram of the structure of the artificial intelligence main framework;

[0087] FIG2 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0088] FIG3 is a schematic diagram of a system architecture provided in an embodiment of the present application;

[0089] FIG4 is a schematic diagram of a recommendation scenario provided in an embodiment of the present application;

[0090] FIG5 is a flow chart of a data processing method provided in an embodiment of the present application;

[0091] FIG6 is a flow chart of a data processing method provided in an embodiment of the present application;

[0092] FIG7 is a flow chart of a data processing method provided in an embodiment of the present application;

[0093] FIG8 is a flow chart of a data processing method provided in an embodiment of the present application;

[0094] FIG9 is a flow chart of a data processing method provided in an embodiment of the present application;

[0095] FIG10 is a flow chart of a data processing method provided in an embodiment of the present application;

[0096] FIG11 is a schematic structural diagram of a data processing device provided in an embodiment of the present application;

[0097] FIG12 is a schematic diagram of an execution device provided in an embodiment of the present application;

[0098] FIG13 is a schematic diagram of a training device provided in an embodiment of the present application;

[0099] FIG14 is a schematic diagram of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION

[0100] The following describes the embodiments of the present invention in conjunction with the accompanying drawings. The terms used in the embodiments of the present invention are only used to explain the specific embodiments of the present invention, and are not intended to limit the present invention.

[0101] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0102] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0103] First, let's describe the overall workflow of an AI system. See Figure 1, which shows a schematic diagram of the main AI framework. This framework will be explained from two perspectives: the "intelligent information chain" (horizontal axis) and the "IT value chain" (vertical axis). The "intelligent information chain" reflects the entire process from data acquisition to processing. For example, it could be the general process of intelligent information perception, intelligent information representation and formation, intelligent reasoning, intelligent decision-making, and intelligent execution and output. Throughout this process, data undergoes a condensed journey from "data-information-knowledge-wisdom." The "IT value chain," spanning the underlying infrastructure of human intelligence, information (provided and processed by technology), and the system's industrial ecosystem, reflects the value that AI brings to the information technology industry.

[0104] (1) Infrastructure

[0105] Infrastructure provides computing power for AI systems, enabling communication with the outside world and supporting this through a foundational platform. External communication occurs through sensors; computing power is provided by intelligent chips (CPUs, NPUs, GPUs, ASICs, FPGAs, and other hardware accelerators). The foundational platform includes a distributed computing framework and network-related platform guarantees and support, including cloud storage and computing, and interconnected networks. For example, sensors communicate with the outside world to acquire data, which is then fed into the intelligent chips within the distributed computing system provided by the foundational platform for computation.

[0106] (2) Data

[0107] Data above the infrastructure layer represents data sources for AI. This data includes graphics, images, voice, and text, as well as IoT data from traditional devices. This includes business data from existing systems and sensor data such as force, displacement, liquid level, temperature, and humidity.

[0108] (3) Data processing

[0109] Data processing generally includes data training, machine learning, deep learning, search, reasoning, decision-making, etc.

[0110] Among them, machine learning and deep learning can symbolize and formalize data for intelligent information modeling, extraction, preprocessing, and training.

[0111] Reasoning refers to the process of simulating human intelligent reasoning in computers or intelligent systems, using formalized information to perform machine thinking and solve problems based on reasoning control strategies. Typical functions are search and matching.

[0112] Decision-making refers to the process of making decisions after intelligent information is reasoned, and usually provides functions such as classification, sorting, and prediction.

[0113] (4) General ability

[0114] After the data has undergone the data processing mentioned above, some general capabilities can be further formed based on the results of the data processing, such as algorithms or a general system, for example, translation, text analysis, computer vision processing, speech recognition, image recognition, etc.

[0115] (5) Smart products and industry applications

[0116] Smart products and industry applications refer to the products and applications of artificial intelligence systems in various fields. They are the encapsulation of the overall artificial intelligence solution, which productizes intelligent information decision-making and realizes practical application. Its application areas mainly include: smart terminals, smart transportation, smart medical care, autonomous driving, smart cities, etc.

[0117] The embodiments of the present application can be applied to the field of information recommendation, which includes but is not limited to scenarios involving e-commerce product recommendations, search engine result recommendations, application market recommendations, music recommendations, video recommendations, etc. The recommended items in various application scenarios can also be called "objects" to facilitate subsequent descriptions, that is, in different recommendation scenarios, the recommended object can be an APP, or a video, or music, or a certain product (such as the presentation interface of an online shopping platform, which will display different products for presentation according to different users, which can actually be presented through the recommendation results of a recommendation model). These recommendation scenarios usually involve user behavior log collection, log data preprocessing (for example, quantization, sampling, etc.), sample set training to obtain a recommendation model, and analysis and processing of the objects involved in the scenario corresponding to the training sample items (such as APP, music, etc.) according to the recommendation model. For example, the samples selected in the recommendation model training link come from the user's operation behavior on the recommended APP in the mobile application market. The recommendation model trained thereby is applicable to the above-mentioned mobile APP application market, or can be used for the APP application market of other types of terminals to recommend terminal APPs. The recommendation model will ultimately calculate the recommendation probability or score of each object to be recommended. The recommendation system selects the recommendation results based on certain selection rules, such as sorting them by recommendation probability or score, and presents them to users through corresponding applications or terminal devices. Users operate on the objects in the recommendation results to generate user behavior logs, etc.

[0118] Referring to Figure 4, during the recommendation process, when a user interacts with the recommendation system, a recommendation request is triggered. The recommendation system inputs this request and its associated feature information into the deployed recommendation model, which then predicts the user's click-through rate for all candidate items. The candidate items are then sorted in descending order based on the predicted click-through rates and displayed in sequential order at different locations as the recommended results for the user. Users browse the displayed items and perform user actions, such as browsing, clicking, and downloading. These user actions are logged as training data, and the offline training module periodically updates the recommendation model parameters to improve the model's recommendation effectiveness.

[0119] For example, when a user opens a mobile app market, the app market's recommendation module is triggered. This module predicts the user's likelihood of downloading each candidate app based on historical download and click history, app features, and contextual information such as time and location. Based on these predictions, the app market displays apps in descending order of likelihood, increasing the probability of download. Specifically, apps more likely to be downloaded are ranked higher, while apps less likely to be downloaded are ranked lower. User behavior is also logged, and the prediction model parameters are trained and updated through an offline training module.

[0120] For example, in applications related to lifelong companions, based on the user's historical data in video, music, news and other domains, various models and algorithms can be used to imitate the human brain mechanism to build a cognitive brain and establish a user lifelong learning system framework. Lifelong companions can record past events of users based on system data and application data, understand users' current intentions, predict users' future actions or behaviors, and ultimately realize intelligent services. In the current first phase, based on music apps, video apps, browser apps, etc., user behavior data (including end-side text messages, photos, email events, etc.) is obtained. On the one hand, a user portrait system is built, and on the other hand, learning and memory modules based on user information filtering, association analysis, cross-domain recommendations, causal reasoning, etc. are implemented to build a user's personal knowledge graph.

[0121] Next, the application architecture of the embodiment of the present application is introduced.

[0122] Referring to FIG2 , an embodiment of the present invention provides a recommendation system architecture 200. The data acquisition device 260 is used to collect samples. A training sample can be composed of multiple feature information (or described as attribute information, such as user attributes and item attributes). Feature information can be of multiple types, specifically including user feature information, object feature information, and label features. User feature information is used to characterize user characteristics, such as gender, age, occupation, hobbies, etc. Object feature information is used to characterize the characteristics of the object pushed to the user. Different recommendation systems correspond to different objects, and the types of features that need to be extracted from different objects are also different. For example, the object features extracted from the training samples of the APP market can be the name (identification), type, size, etc. of the APP. ; The object features mentioned in the training samples of e-commerce apps can be the name of the product, the category it belongs to, the price range, etc.; the label feature is used to indicate whether the sample is a positive example or a negative example. Usually, the label feature of the sample can be obtained through the user's operation information on the recommended object. The sample in which the user operates the recommended object is a positive example, and the sample in which the user does not operate the recommended object or only browses the recommended object is a negative example. For example, when the user clicks, downloads, or purchases the recommended object, the label feature is 1, indicating that the sample is a positive example. If the user does not perform any operation on the recommended object, the label feature is 0, indicating that the sample is a negative example. After collection, the sample can be stored in the database 230. Some or all of the feature information in the sample in the database 230 can also be directly obtained from the client device 240, such as user feature information, user operation information on the object (used to determine the type identification), object feature information (such as object identification), etc. The training device 220 obtains a model parameter matrix based on the sample training in the database 230 for generating a recommendation model 201 (such as the feature extraction network and neural network in the embodiment of the present application). The following will describe in more detail how the training device 220 trains to obtain the model parameter matrix used to generate the recommendation model 201. The recommendation model 201 can be used to evaluate a large number of objects to obtain the scores of each object to be recommended. Furthermore, a specified or preset number of objects can be recommended from the evaluation results of a large number of objects. The calculation module 211 obtains the recommendation results based on the evaluation results of the recommendation model 201 and recommends them to the client device through the I / O interface 212.

[0123] In an embodiment of the present application, the training device 220 can select positive and negative samples from the sample set in the database 230 and add them to the training set, and then use the recommendation model to train the samples in the training set to obtain a trained recommendation model; the implementation details of the calculation module 211 can refer to the detailed description of the method embodiment shown in Figure 5.

[0124] After the training device 220 obtains the model parameter matrix based on sample training and uses it to construct the recommendation model 201, the recommendation model 201 is sent to the execution device 210, or the model parameter matrix is ​​directly sent to the execution device 210, and a recommendation model is constructed in the execution device 210 for making recommendations for the corresponding system. For example, the recommendation model obtained based on video-related sample training can be used to recommend videos to users on video websites or APPs, and the recommendation model obtained based on APP-related sample training can be used to recommend APPs to users in the application market.

[0125] Execution device 210 is equipped with an I / O interface 212 for data exchange with external devices. Execution device 210 can obtain user characteristic information, such as user ID, user identity, gender, occupation, and hobbies, from client device 240 via I / O interface 212. This information can also be obtained from a system database. Recommendation model 201 recommends target objects to the user based on the user characteristic information and the characteristic information of the recommended objects. Execution device 210 can be located in a cloud server or in a user client.

[0126] The execution device 210 can call data, code, etc. in the data storage system 250, and can also store output data in the data storage system 250. The data storage system 250 can be set in the execution device 210, can be set independently, or can be set in other network entities, and can be one or more.

[0127] The calculation module 211 uses the recommendation model 201 to process the user characteristic information and the characteristic information of the object to be recommended. For example, the calculation module 211 uses the recommendation model 201 to analyze and process the user characteristic information and the characteristic information of the object to be recommended, so as to obtain the score of the object to be recommended, and sort the objects to be recommended according to the score, among which the objects with higher rankings will be recommended to the client device 240.

[0128] Finally, the I / O interface 212 returns the recommendation result to the client device 240 and presents it to the user.

[0129] More deeply, the training device 220 can generate corresponding recommendation models 201 based on different sample feature information for different goals to provide users with better results.

[0130] It is worth noting that Figure 2 is only a schematic diagram of a system architecture provided by an embodiment of the present invention. The positional relationship between the devices, components, modules, etc. shown in the figure does not constitute any limitation. For example, in Figure 2, the data storage system 250 is an external memory relative to the execution device 210. In other cases, the data storage system 250 can also be placed in the execution device 210.

[0131] In an embodiment of the present application, the training device 220, the execution device 210, and the client device 240 can be three different physical devices respectively. It is also possible that the training device 220 and the execution device 210 are on the same physical device or a cluster. It is also possible that the execution device 210 and the client device 240 are on the same physical device or a cluster.

[0132] Refer to Figure 3, which is a system architecture 300 proposed in an embodiment of the present invention. In this architecture, the execution device 210 is implemented by one or more servers, and optionally cooperates with other computing devices, such as data storage, routers, load balancers and other devices; the execution device 210 can be arranged at one physical site, or distributed at multiple physical sites. The execution device 210 can use the data in the data storage system 250, or call the program code in the data storage system 250 to implement the object recommendation function. Specifically, the information of the object to be recommended is input into the recommendation model, and the recommendation model generates an estimated score for each object to be recommended, and then sorts them in order from high to low according to the estimated score, and recommends the object to be recommended to the user according to the sorting result. For example, the first 10 objects in the sorting result are recommended to the user.

[0133] Among them, the data storage system 250 is used to receive and store the parameters of the recommendation model sent by the training device, as well as the data for storing the recommendation results obtained by the recommendation model. Of course, it may also include the program code (or instructions) required for the normal operation of the storage system 250. The data storage system 250 can be a distributed storage cluster composed of one device or multiple devices deployed outside the execution device 210. In this case, when the execution device 210 needs to use the data on the storage system 250, the storage system 250 can send the data required by the execution device to the execution device 210. Accordingly, the execution device 210 receives and stores (or caches) the data. Of course, the data storage system 250 can also be deployed in the execution device 210. When deployed in the execution device 210, the distributed storage system can include one or more memories. Optionally, when there are multiple memories, different memories are used to store different types of data. For example, the model parameters of the recommendation model generated by the training device and the data of the recommendation results obtained by the recommendation model can be stored in two different memories respectively.

[0134] Users can operate their respective user devices (e.g., local device 301 and local device 302) to interact with execution device 210. Each local device can represent any computing device, such as a personal computer, a computer workstation, a smartphone, a tablet computer, a smart camera, a smart car or other type of cellular phone, a media consumption device, a wearable device, a set-top box, a game console, etc.

[0135] Each user's local device can interact with the execution device 210 through a communication network of any communication mechanism / communication standard. The communication network can be a wide area network, a local area network, a point-to-point connection, etc., or any combination thereof.

[0136] In another implementation, the execution device 210 can be implemented by a local device. For example, the local device 301 can implement the recommendation function of the execution device 210 based on the recommendation model to obtain user feature information and feedback the recommendation results to the user, or provide services to the user of the local device 302.

[0137] Since the embodiments of the present application involve the application of a large number of neural networks, in order to facilitate understanding, the relevant terms and related concepts such as neural networks involved in the embodiments of the present application are first introduced below.

[0138] 1. Click-throughrate (CTR)

[0139] Click probability, also known as click-through rate, refers to the ratio of the number of clicks and exposures of recommended information (for example, recommended items) on a website or application. Click-through rate is usually an important indicator for measuring recommendation systems in recommendation systems.

[0140] 2. Personalized recommendation system

[0141] A personalized recommendation system refers to a system that uses machine learning algorithms to analyze users' historical data (such as the operation information in the embodiments of the present application), predicts new requests based on this data, and provides personalized recommendation results.

[0142] 3. Offline training

[0143] Offline training refers to a module in a personalized recommendation system that iteratively updates the recommendation model parameters according to the machine learning algorithm based on the user's historical data (such as the operation information in the embodiment of the present application) until the set requirements are met.

[0144] 4. Online Inference

[0145] Online prediction refers to predicting the user's preference for recommended items in the current context based on the characteristics of the user, item, and context based on the offline trained model, and predicting the probability of the user selecting the recommended item.

[0146] For example, Figure 4 is a schematic diagram of the recommendation system provided by an embodiment of the present application. As shown in Figure 4, when a user enters the system, a recommendation request will be triggered. The recommendation system will input the request and related information (such as the operation information in the embodiment of the present application) into the recommendation model, and then predict the user's selection rate of items in the system. Furthermore, the items are sorted in descending order according to the predicted selection rate or a function based on the selection rate, that is, the recommendation system can display the items in different positions in sequence as recommendation results for the user. The user browses items in different positions and performs user behaviors, such as browsing, selecting, and downloading. At the same time, the user's actual behavior will be stored in the log as training data, and the parameters of the recommendation model will be continuously updated through the offline training module to improve the prediction effect of the model.

[0147] For example, a user opening an app store on a smart device (e.g., a mobile phone) triggers the app store's recommendation system. The app store's recommendation system predicts the probability of the user downloading each recommended app based on the user's historical behavior logs, such as their download history and selection history, as well as the app store's own characteristics, such as time, location, and other environmental characteristics. Based on the calculated results, the app store's recommendation system can display the candidate apps in descending order of predicted probability, thereby increasing the download probability of the candidate apps.

[0148] For example, an APP with a higher predicted user selection rate may be displayed in a front recommendation position, and an APP with a lower predicted user selection rate may be displayed in a back recommendation position.

[0149] The above-mentioned recommendation model can be a neural network model. The following introduces the relevant terms and concepts of neural networks that may be involved in the embodiments of the present application.

[0150] (1) Neural Network

[0151] A neural network can be composed of neural units. A neural unit can refer to an operation unit that takes xs (i.e., input data) and intercept 1 as input. The output of the operation unit can be:

[0152] Where s = 1, 2, ... n, n is a natural number greater than 1, Ws is the weight of xs, and b is the bias of the neural unit. f is the activation function of the neural unit, which is used to introduce nonlinear characteristics into the neural network to convert the input signal of the neural unit into the output signal. The output signal of the activation function can be used as the input of the next convolutional layer, and the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple single neural units mentioned above, that is, the output of one neural unit can be the input of another neural unit. The input of each neural unit can be connected to the local receptive field of the previous layer to extract the features of the local receptive field. The local receptive field can be an area composed of several neural units.

[0153] (2) Deep Neural Networks

[0154] Deep Neural Network (DNN), also known as multi-layer neural network, can be understood as a neural network with many hidden layers. There is no special metric for "many" here. Based on the position of different layers in DNN, the neural network inside DNN can be divided into three categories: input layer, hidden layer, and output layer. Generally speaking, the first layer is the input layer, the last layer is the output layer, and the layers in between are all hidden layers. The layers are fully connected, that is, any neuron in the i-th layer must be connected to any neuron in the i+1-th layer. Although DNN looks complicated, the work of each layer is actually not complicated. Simply put, it is the following linear relationship expression: in, is the input vector, is the output vector, is the offset vector, W is the weight matrix (also called coefficient), and α() is the activation function. Each layer is just an input vector After such a simple operation, the output vector Since there are many DNN layers, the coefficient W and the offset vector The definition of these parameters in DNN is as follows: Take the coefficient W as an example: Assume that in a three-layer DNN, the linear coefficient from the 4th neuron in the second layer to the 2nd neuron in the third layer is defined as The superscript 3 represents the layer number of the coefficient W, while the subscript corresponds to the output of the third layer index 2 and the input of the second layer index 4. In summary, the coefficient from the kth neuron in the L-1th layer to the jth neuron in the Lth layer is defined as It's important to note that the input layer has no W parameter. In deep neural networks, more hidden layers allow the network to better capture complex real-world situations. Theoretically, a model with more parameters has higher complexity and greater "capacity," meaning it can handle more complex learning tasks. Training a deep neural network is essentially the process of learning the weight matrix, with the ultimate goal of obtaining the weight matrices for all layers of a trained deep neural network (a weight matrix formed by the vectors W across many layers).

[0155] (3) Loss function

[0156] During the training of a deep neural network, because we want the output of the deep neural network to be as close as possible to the desired predicted value, we can compare the current network's predicted value with the desired target value and then update the weight vector of each layer of the neural network based on the difference between the two. (Of course, before the first update, there is usually an initialization process, which pre-configures the parameters for each layer in the deep neural network.) For example, if the network's predicted value is too high, the weight vector is adjusted to make it predict a lower value. This adjustment is continued until the deep neural network can predict the desired target value or a value very close to the desired target value. Therefore, it is necessary to predefine "how to compare the difference between the predicted value and the target value." This is the loss function (or objective function), which is an important equation used to measure the difference between the predicted value and the target value. For example, the loss function output value (loss) indicates a greater difference, so training a deep neural network becomes a process of minimizing this loss.

[0157] (4) Backpropagation algorithm

[0158] The back propagation (BP) algorithm can be used to correct the size of the initial model parameters during training, reducing the model's error loss. Specifically, forward propagation of the input signal to the output generates error loss. This error loss information is then backpropagated to update the parameters in the initial model, thereby converging the error loss. The BP algorithm is a backward propagation movement driven by error loss, aiming to obtain optimal model parameters, such as the weight matrix.

[0159] (5) Machine Learning Systems

[0160] Based on the input data and labels, the parameters of the machine learning model are trained through optimization methods such as gradient descent, and finally the trained model is used to complete the prediction of unknown data.

[0161] (6) Personalized recommendation system

[0162] A system that uses machine learning algorithms to analyze and model users' historical data, predict new user requests, and provide personalized recommendation results.

[0163] (7) Recommended scenarios

[0164] The recommendation scenario can be an application (APP) that serves specific needs, such as Huawei Browser and Huawei Video, or it can refer to a specific channel, such as the entertainment channel, news channel, technology channel, etc. in the browser information flow.

[0165] (8) Multi-scenario modeling

[0166] Integrate data from multiple scenarios, train and generate a model to serve multiple scenarios.

[0167] (9) Prompt

[0168] A natural language term that can be categorized into hard templates and soft templates. Hard templates are generally natural language words or sentences with a specific meaning; soft templates are generally parameterized representation vectors without any meaning.

[0169] Machine learning systems, including personalized recommendation systems, train the parameters of machine learning models based on input data and labels using optimization methods such as gradient descent. Once the model parameters converge, the model can be used to predict unknown data. For example, in a personalized recommendation system, click-through rate prediction involves input data including user and product attributes. Predicting personalized recommendations based on user preferences plays a crucial role in improving the accuracy of the recommendation system.

[0170] To meet personalized user needs, recommendation systems include multiple recommendation scenarios: browser, negative one screen, video streaming, and more. Users behave differently in different scenarios based on their preferences, and each scenario has both user-specific and shared behavioral characteristics. Typically, a separate model is created for each scenario.

[0171] However, independent modeling of a single scene cannot effectively capture the behavioral characteristics of users in different scenarios, since the same user may have different behaviors in different scenarios, thereby failing to fully learn user preferences. In addition, when there are many scenarios, independent modeling and maintenance for each scenario will result in a large consumption of manpower and resources.

[0172] In existing implementations of the STAR (Star Topology Adaptive Recommender) model, a multi-scenario model is used to capture the common behavioral characteristics of users in different scenarios. For example, in the STAR model, a common feature extraction network is trained to adapt to various scenarios. However, the common feature extraction network in existing technologies cannot extract an embedded representation that can accurately characterize the common behavioral characteristics of users in different scenarios, resulting in poor generalization of the recommendation model in various scenarios.

[0173] In order to solve the above problems, the present application provides a data processing method, which can be an inference process of a model.

[0174] 5 , which is a schematic diagram of an embodiment of a data processing method provided in an embodiment of the present application, as shown in FIG5 , a data processing method provided in an embodiment of the present application includes:

[0175] 501. Obtain a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting attribute information of the user and the item;

[0176] In existing implementations, feature representations corresponding to user and item attribute information are used as input to the recommendation model to predict user operation information on items. However, in order to obtain highly accurate prediction results in multiple recommendation scenarios, one implementation is to build a more complex recommendation model. The recommendation model includes multiple branches (that is, multiple towers), and each branch can be responsible for processing the prediction of a recommendation scenario. However, as the number of recommendation scenarios continues to increase, the scale of the recommendation model will become larger, which is unacceptable in some scenarios.

[0177] In an embodiment of the present application, in addition to using the feature representation corresponding to the attribute information of the user and the item as the input of the recommendation model, a prompt that can carry information related to the recommendation scenario (that is, the prompt is obtained based on the information related to the recommendation scenario) is also used as the input of the recommendation model. This can guide the recommendation model to make predictions based on the characteristics of the scenario indicated by the prompt when predicting the operation information, thereby improving the prediction accuracy of multiple recommendation scenarios without increasing the scale of the recommendation model.

[0178] In one possible implementation, the recommendation scenario can be an application that serves specific needs, such as Huawei Browser and Huawei Video, or it can refer to a specific channel, such as the entertainment channel, news channel, technology channel, etc. in the browser information flow.

[0179] In one possible implementation, different recommendation scenarios are different applications, or different recommendation scenarios are different types of applications (for example, video applications and browser applications are different applications), or different recommendation scenarios are different functions of the same application (for example, different channels of the same application, such as news channels, technology channels, etc.). The above different functions can be divided according to recommendation categories.

[0180] In a possible implementation, the different recommendation scenarios may be different physical scenarios where the recommendations are made, such as different cities, different provinces, etc.

[0181] In a possible implementation, different recommendation scenarios may be application programs developed by different developers.

[0182] In a possible implementation, different recommendation scenarios may be different or different types of e-commerce products.

[0183] Next, we introduce the prompt in the embodiment of this application:

[0184] In a possible implementation, prompt is obtained based on information related to the recommendation scenario, wherein the information related to the recommendation scenario may include indication information of the recommendation scenario. Furthermore, prompt is obtained by extracting features of the indication information of the recommendation scenario.

[0185] For example, the indication information of the recommended scene may be an identifier used to indicate the recommended scene.

[0186] In an embodiment of the present application, the recommendation system can obtain the current scenario of information recommendation, and then determine the indication information of the recommendation scenario based on the current scenario of information recommendation.

[0187] The prompt obtained according to the indication information of the recommended scene can also be called a domain prompt, and the scene characteristics are used to generate unique prompt parameters for each scene.

[0188] In a possible implementation, the information related to the recommendation scenario may be the user's attribute information (or may be referred to as user portrait features). Furthermore, the prompt is obtained by extracting features from the user's attribute information.

[0189] The prompts obtained based on the user's attribute information can also be called user prompts. User portrait features are used to generate user personalized improvement parameters for personalized modeling.

[0190] Among them, the user's attribute information can be attributes related to the user's preference characteristics, at least one of gender, age, occupation, income, hobbies and education level, among which gender can be male or female, age can be a number between 0-100, occupation can be teacher, programmer, chef, etc., hobbies can be basketball, tennis, running, etc., and education level can be elementary school, junior high school, high school, university, etc.; this application does not limit the specific type of user attribute information.

[0191] For example, prompt may include the first feature representation in the embodiment of the present application (or it may be obtained by extracting features from the user's attribute information through a different feature extraction network (e.g., an embedding layer) than that used to obtain the first feature representation, which is not limited in the present application).

[0192] In a possible implementation, prompt may be a low-dimensional representation corresponding to information related to the recommendation scenario.

[0193] In one possible implementation, the prompt can be obtained by performing one-hot encoding on the information related to the recommendation scene. This method can directly generate a prompt vector based on the low-dimensional representation of the information related to the recommendation scene (such as the ID feature), and is suitable for situations with a small number of categories, such as generating prompts for scene-specific information.

[0194] In a possible implementation, information related to the recommendation scenario may be processed through a neural network to obtain the prompt.

[0195] As shown in the left image of Figure 6, a fully connected neural network can be used to directly generate prompts by inputting information related to the recommendation scenario. This method is suitable for situations with a large number of categories, such as generating prompts based on user attribute information. User profile features can be used as input to the fully connected neural network to directly generate corresponding user prompts, thus avoiding the huge parameter overhead associated with mapping-based methods.

[0196] In one possible implementation, information related to the recommendation scenario can be processed through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and multiple feature representations are fused according to the weight to obtain the prompt.

[0197] As shown in the right figure of Figure 6, this method uses the method of the attention mechanism, takes the feature as the query, maintains a pool of meta-prompts (that is, the feature representation in the embodiment of the present application) as the value, and performs a weighted operation on the meta-prompt (that is, the feature representation in the embodiment of the present application) through the attention network (for example, normalized with the softmax function, and the temperature coefficient as a hyperparameter). This method is also applicable to situations where there are a large number of categories, such as user prompt generation. User portrait features can be used as queries.

[0198] For example, the user portrait features can be calculated using the following formula:

[0199] Where R is the number of meta-prompts, MP r is the rth meta-hint, w r is the output prediction weight of the attention network.

[0200] In one possible implementation, feature extraction can be performed on user and item attribute information to obtain a first feature representation. Optionally, the first feature representation can be features trained during model training and stored in a storage space, so the first feature representation can be directly obtained from the corresponding storage space.

[0201] Among them, the items can be physical items or virtual items, such as applications (APP), audio and video, web pages, and news information. The attribute information of the items can be at least one of the item name, developer, installation package size, category, and praise. Among them, taking the item as an application as an example, the category of the item can be chat, parkour games, office, etc., and the praise degree can be a score, comment, etc. for the item; this application does not limit the specific type of attribute information of the item.

[0202] It should be understood that if the prompt includes both information obtained based on the information indicating the recommendation scenario and information obtained based on the user's attribute information, the prompt may be a fusion result (eg, a splicing result) of multiple types of information.

[0203] 502. Predict the user's operation information on the item through a recommendation model based on the fusion result of the prompt and the first feature representation.

[0204] In one possible implementation, the prompt and the first feature representation can be fused (e.g., concatenated), and the fusion result obtained can be input into the recommendation model. The recommendation model can obtain the recommendation prediction (i.e., the user's operation information on the item) through feature interaction and output layer.

[0205] Optionally, the recommendation model can be any existing deep recommendation model, such as DeepFM, DCN, AutoInt, etc.

[0206] For example, the feature interaction layer needs to understand the relationship between different feature domains. Generally, a linear layer is used to model linear (low-order) feature interactions, while a deep neural network is used to model high-order feature interactions. The output layer takes the latent vector h generated by the previous layer as input and outputs the final prediction result. For example, the output layer can be calculated using the following formula.

[0207] In one possible implementation, the predicted operation information can indicate whether the user has performed a target operation. The target operation can be a type of user behavior operation. On online platforms and applications, users often have a variety of interaction forms with items (that is, there are multiple types of operations), such as browsing, clicking, adding to shopping carts, purchasing, and other operation types in users' behavior on e-commerce platforms.

[0208] In a possible implementation, the operation information may be the probability that the user will perform a target operation on the item.

[0209] For example, the operation information may be whether the user clicks or the probability of clicking.

[0210] 503. When the operation information meets a preset condition, recommend the item to the user.

[0211] In a possible implementation, during the inference process of the model, when the operation information meets a preset condition, it is determined to recommend the item to the user.

[0212] Through the above method, the probability of the user performing an operation on the item can be obtained, and information recommendation can be performed based on the above probability. Specifically, when the recommended information meets the preset conditions, it can be determined to recommend the item to the user.

[0213] When recommending information, the recommended information can be recommended to users in the form of a list page in the hope that users will take action.

[0214] An embodiment of the present application provides a data processing method. In addition to using the feature representation corresponding to the attribute information of users and items as the input of the recommendation model, a prompt that can carry information related to the recommendation scenario (that is, the prompt is obtained based on information related to the recommendation scenario) is also used as the input of the recommendation model. This can guide the recommendation model to make predictions based on the characteristics of the scenario indicated by the prompt when predicting operation information, thereby improving the prediction accuracy of multiple recommendation scenarios without increasing the scale of the recommendation model.

[0215] The present application provides a data processing method, which can be a model training process, for example, a model pre-training process or a fine-tuning process.

[0216] 7 , which is a schematic diagram of an embodiment of a data processing method provided in an embodiment of the present application. As shown in FIG7 , a data processing method provided in an embodiment of the present application includes:

[0217] 701. Obtain a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting attribute information of the user and the item;

[0218] In a possible implementation, the information related to the recommendation scenario includes: indication information of the recommendation scenario; or attribute information of the user; or feature representation of the attribute information of the user.

[0219] In one possible implementation, the information related to the recommendation scenario can be processed through a neural network to obtain the prompt; or, the information related to the recommendation scenario can be processed through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and based on the weights, the multiple feature representations are fused to obtain the prompt; and the attribute information of the user and the item is feature extracted through an embedding layer to obtain the first feature representation.

[0220] In one possible implementation, different recommendation scenarios are different applications; or, different recommendation scenarios are different types of applications; or, different recommendation scenarios are different functions of the same application; or, different recommendation scenarios are different physical scenarios where the recommendation is made; or, different recommendation scenarios are applications developed by different developers; or, different recommendation scenarios are different or different types of e-commerce products.

[0221] For a detailed description of step 701 , reference may be made to the introduction of step 501 in the above embodiment, and similarities will not be repeated here.

[0222] 702. Predicting the user's operation information on the item using a recommendation model based on the fusion result of the prompt and the first feature representation;

[0223] For a detailed description of step 702 , reference may be made to the introduction of step 502 in the above embodiment, and similarities will not be repeated here.

[0224] 703. Update the recommendation model according to the operation information and the corresponding label.

[0225] The label corresponding to the operation information may be the user's operation data on the item (the operation data may be real operation data or obtained through a certain data enhancement method, which is not limited in this application). The user's operation data may be obtained based on the interaction records between the user and the item (such as the user's behavior log), and the operation data may include the user's real operation records on each item.

[0226] The operation information is the feedforward output during model training. The label corresponding to the operation information can be used as the true value and the operation information to construct the loss, and the recommendation model is updated based on the loss.

[0227] For example, the overall optimization loss is as follows (taking the binary classification task as an example).

[0228] In existing implementations, when fine-tuning a recommendation model for multiple recommendation scenarios, it is necessary to update the entire embedding layer and the recommendation model. This can lead to negative transfer and a seesaw effect, where improving the recommendation accuracy of a certain scenario comes at the expense of performance in other scenarios, resulting in poor recommendation accuracy in multiple scenarios.

[0229] In an embodiment of the present application, a prompt that can indicate a recommendation scenario is introduced as the input of the recommendation model. When fine-tuning the model (optional, and can also be applied to pre-training), the common parts between different scenarios can be fixed, and the parts of the model that are strongly related to the scenario (such as the neural network used to generate the prompt or the linear layer in the recommendation model, etc.) can be updated. This can improve the recommendation accuracy of the model for a recommendation scenario when training the model for the scenario, without compromising the performance of the model in other recommendation scenarios.

[0230] Specifically, in a possible implementation, only a portion of the network in the recommendation model may be updated, where the portion of the network may be a linear layer.

[0231] Specifically, in a possible implementation, the neural network (the neural network used to generate the prompt) can be updated based on the operation information and the corresponding label while keeping the parameters of the embedding layer fixed.

[0232] For example, reference may be made to FIG8 , which is a flowchart of the embodiment corresponding to FIG7 applied to the model pre-training process. FIG9 is a flowchart of the embodiment corresponding to FIG7 applied to the model fine-tuning process. As shown in FIG8 and FIG9 , during the model training process, after the scene prompts and user prompts are calculated, they are concatenated with the original feature representation [P domain ||P user||e], which is fed into the feature interaction part of the original single-scene recommendation module. The model training process adopts a two-stage optimization process, including two processes: (1) pre-training and (2) prompt tuning. The updated parameters of the two stages are as follows: the pre-training stage updates all parameters, and the prompt tuning stage updates some parameters, such as only the parameters of scene prompts and user prompts, as well as the parameters of linear feature interactions.

[0233] Refer to Figure 10, which is a flowchart of an algorithm corresponding to an embodiment of the present application.

[0234] The multi-scenario modeling in the prior art uses different parameter sharing methods to transfer useful knowledge between scenarios, but there are negative transfer and seesaw effects. At the same time, the training is difficult and the pressure on parameter storage is increased. The embodiment of the present application has made two improvements to the above problems. First, the global information about commonality learned in the pre-training stage is fixed in the fine-tuning stage. Second, the cross-scenario characteristics and personalization are clearly modeled, showing the role of prompt components and two-stage optimization. It is more in line with the deployment method of the industry and achieves better prediction results at the same time.

[0235] Taking the click prediction task as an example, offline experiments were conducted on three public datasets: Douban, Amazon-5core, and Ali-CCP. Each dataset has three scenarios, and the evaluation metrics used are Area Under Curve (AUC) and Logistic Loss (Logloss), with higher and lower values ​​representing better recommendation performance. Table 1 shows the dataset statistical metrics, including users, items, number of interactions, and click sparsity.

[0236] Table 1 Statistical indicators of experimental data sets

[0237] Table 2 shows the experimental results of the present embodiment on three public datasets, using DeepFM (Deep Factorization Machine) as the original single-scene recommendation module. In addition, the compatibility and effectiveness of the present embodiment as a multi-scene modeling paradigm were verified using four other original single-scene recommendation modules, as shown in Table 3.

[0238] Table 2

[0239] Table 3

[0240] It can be seen that the embodiments of the present application have achieved significantly better performance than other comparison baseline methods on the three public data sets, and have exceeded pre-training + fine-tuning on different original single-scene recommendation modules. At the same time, the embodiments of the present application also have high storage and computing efficiency. Specifically, considering the number of parameters (storage efficiency), taking DeepFM as an example as the original single-scene recommendation module, pre-training + fine-tuning requires updating all parameters, while in the experiments on the above three public data sets, PLATE only needs to update 6.36% (0.54%), 5.95% (0.10%), and 4.45% (0.17%) parameters, and can achieve better recommendation performance. The parameters in brackets represent the proportion of the newly introduced prompt parameters, and the linear feature interaction part is the main updated parameter. On the other hand, considering the training time (computational efficiency), especially when the commercial recommendation system faces more scenarios, it becomes unrealistic to maintain a tower for each scenario (such as thousands of scenarios). In the original Amazon dataset with 24 scenes, the GPU training time of the above comparison models is as follows: Single (24 hours), Pre-train & Fine-tune (9 hours), Shared Bottom (6 hours), MMoE (8 hours), PLE (12 hours), STAR (6 hours), while the training time of the model in the embodiment of the present application is also 6 hours. It can be seen that the training time of the present invention is the shortest and more efficient.

[0241] Next, a data processing device provided by an embodiment of the present application will be described from the perspective of a device. Referring to FIG. 11 , FIG. 11 is a schematic diagram of the structure of a data processing device provided by an embodiment of the present application. As shown in FIG. 11 , a data processing device 1100 provided by an embodiment of the present application includes:

[0242] Acquisition module 1101 is configured to acquire a first feature representation and a second feature representation; the first feature representation is obtained by extracting features from a prompt, and the second feature representation is obtained by extracting features from user and item attributes; the prompt carries information related to the recommendation scenario; and based on the fusion result of the first and second feature representations, a recommendation model is used to predict the user's action on the item.

[0243] For a detailed introduction to the acquisition module 1101 , reference may be made to the description of steps 501 and 502 in the above embodiment, which will not be repeated here.

[0244] The recommendation module 1102 is configured to recommend the item to the user when the operation information meets a preset condition.

[0245] The detailed introduction of the recommendation module 1102 may refer to the description of step 503 in the above embodiment, which will not be repeated here.

[0246] In one possible implementation, the information related to the recommendation scenario includes:

[0247] Indicative information of the recommended scenario; or

[0248] Attribute information of the user; or

[0249] A feature representation of the attribute information of the user.

[0250] In a possible implementation, the processing module is specifically configured to:

[0251] Performing one-hot encoding on the information related to the recommendation scenario to obtain the prompt; or,

[0252] Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or

[0253] The information related to the recommendation scenario is processed through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and the multiple feature representations are fused according to the weights to obtain the prompt.

[0254] In one possible implementation,

[0255] Different recommendation scenarios are different applications; or,

[0256] Different recommendation scenarios are different types of applications; or,

[0257] Different recommendation scenarios are different functions of the same application; or,

[0258] The different recommendation scenes are different physical scenes where the recommendation is made; or,

[0259] Different recommendation scenarios are applications developed by different developers; or

[0260] Different recommendation scenarios are different or different types of e-commerce products.

[0261] The present application also provides a data processing device, comprising:

[0262] a processing module configured to obtain a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting attribute information of the user and the item; and based on the fusion result of the prompt and the first feature representation, predict the user's operation information on the item through a recommendation model;

[0263] An updating module is used to update the recommendation model according to the operation information and the corresponding label.

[0264] In one possible implementation, the processing module is specifically used to: process the information related to the recommendation scenario through a neural network to obtain the prompt; or, process the information related to the recommendation scenario through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and fuse the multiple feature representations according to the weights to obtain the prompt; and extract features from the attribute information of the user and item through an embedding layer to obtain the first feature representation.

[0265] In a possible implementation, the update module is further used to update the neural network according to the operation information and the corresponding label while keeping the parameters of the embedding layer fixed.

[0266] In a possible implementation, the updating module is specifically configured to update only a portion of the network in the recommendation model.

[0267] In a possible implementation, the partial network is a linear layer.

[0268] In a possible implementation, the information related to the recommendation scenario includes: indication information of the recommendation scenario; or attribute information of the user; or feature representation of the attribute information of the user.

[0269] In one possible implementation, different recommendation scenarios are different applications; or, different recommendation scenarios are different types of applications; or, different recommendation scenarios are different functions of the same application; or, different recommendation scenarios are different physical scenarios where the recommendation is made; or, different recommendation scenarios are applications developed by different developers; or, different recommendation scenarios are different or different types of e-commerce products.

[0270] Next, a terminal device provided in an embodiment of the present application is introduced. Please refer to Figure 12. Figure 12 is a structural schematic diagram of a terminal device provided in an embodiment of the present application. The terminal device 1200 can be specifically manifested as a mobile phone, a tablet, a laptop computer, a smart wearable device, etc., which is not limited here. Among them, the terminal device 1200 implements the function of the data processing method in the embodiment corresponding to Figure 5. Specifically, the terminal device 1200 includes: a receiver 1201, a transmitter 1202, a processor 1203 and a memory 1204 (wherein the number of processors 1203 in the terminal device 1200 can be one or more), wherein the processor 1203 may include an application processor 12031 and a communication processor 12032. In some embodiments of the present application, the receiver 1201, the transmitter 1202, the processor 1203 and the memory 1204 may be connected via a bus or other means.

[0271] The memory 1204 may include a read-only memory and a random access memory, and provides instructions and data to the processor 1203. A portion of the memory 1204 may also include non-volatile random access memory (NVRAM). The memory 1204 stores processor and operation instructions, executable modules, or data structures, or subsets or extended sets thereof. The operation instructions may include various operation instructions for implementing various operations.

[0272] Processor 1203 controls the operation of the terminal device. In specific applications, the various components of the terminal device are coupled together via a bus system. In addition to a data bus, the bus system may also include a power bus, a control bus, and a status signal bus. However, for clarity, all bus systems are referred to as a bus system in the figure.

[0273] The method disclosed in the above embodiment of the present application can be applied to the processor 1203, or implemented by the processor 1203. The processor 1203 can be an integrated circuit chip with signal processing capabilities. During the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 1203 or the instruction in the form of software. The above-mentioned processor 1203 can be a general-purpose processor, a digital signal processor (digital signal processing, DSP), a microprocessor or a microcontroller, and a vision processor (vision processing unit, VPU), a tensor processor (tensor processing unit, TPU) and other processors suitable for AI computing, and can further include an application specific integrated circuit (ASIC), a field programmable gate array (field-programmable gate array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The processor 1203 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in conjunction with the embodiments of the present application can be directly implemented as being executed by a hardware decoding processor, or can be implemented by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in memory 1204, and processor 1203 reads the information in memory 1204 and, in conjunction with its hardware, completes steps 501 to 503, or steps 701 to 703, in the above embodiments.

[0274] Receiver 1201 can be used to receive input digital or character information and generate signal input related to the relevant settings and function control of the terminal device. Transmitter 1202 can be used to output digital or character information through the first interface. Transmitter 1202 can also be used to send instructions to the disk group through the first interface to modify the data in the disk group. Transmitter 1202 can also include a display device such as a display screen.

[0275] The embodiment of the present application also provides a server. Please refer to Figure 13, which is a schematic diagram of the structure of a server provided by the embodiment of the present application. Specifically, the server 1300 is implemented by one or more servers. The server 1300 may have relatively large differences due to different configurations or performance. It may include one or more central processing units (CPUs) 1313 (for example, one or more processors) and memory 1332, and one or more storage media 1330 (for example, one or more mass storage devices) for storing application programs 1342 or data 1344. Among them, the memory 1332 and the storage medium 1330 can be temporary storage or permanent storage. The program stored in the storage medium 1330 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the server. Furthermore, the central processing unit 1313 can be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the server 1300.

[0276] The server 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358; or one or more operating systems 1341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0277] Specifically, the server may perform steps 501 to 503 or steps 701 to 703 in the above embodiment.

[0278] An embodiment of the present application also provides a computer program product, which, when running on a computer, enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0279] A computer-readable storage medium is also provided in an embodiment of the present application, which stores a program for signal processing. When the computer-readable storage medium is run on a computer, it enables the computer to execute the steps executed by the aforementioned execution device, or enables the computer to execute the steps executed by the aforementioned training device.

[0280] The execution device, training device or terminal device provided in the embodiments of the present application can specifically be a chip, and the chip includes: a processing unit and a communication unit, the processing unit can be, for example, a processor, and the communication unit can be, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute the computer execution instructions stored in the storage unit, so that the chip in the execution device executes the data processing method described in the above embodiment, or so that the chip in the training device executes the data processing method described in the above embodiment. Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip in the wireless access device, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc.

[0281] Specifically, see Figure 14 , which is a schematic diagram of the structure of a chip provided in an embodiment of the present application. The chip can be represented as a neural network processor NPU 1400. NPU 1400 is mounted on the host CPU (host CPU) as a coprocessor and is assigned tasks by the host CPU. The core of the NPU is arithmetic circuit 1403, which is controlled by controller 1404 to extract matrix data from memory and perform multiplication operations.

[0282] The NPU 1400 can implement the data processing method provided in the embodiment described in FIG. 5 or FIG. 7 through the mutual cooperation between various internal components.

[0283] More specifically, in some implementations, the arithmetic circuit 1403 in the NPU 1400 includes multiple processing units (PEs). In some implementations, the arithmetic circuit 1403 is a two-dimensional systolic array. The arithmetic circuit 1403 can also be a one-dimensional systolic array or other electronic circuitry capable of performing mathematical operations such as multiplication and addition. In some implementations, the arithmetic circuit 1403 is a general-purpose matrix processor.

[0284] For example, assume there are input matrix A, weight matrix B, and output matrix C. The computation circuit retrieves the corresponding data for matrix B from weight memory 1402 and caches it on each PE in the computation circuit. The computation circuit then retrieves the data for matrix A from input memory 1401 and performs a matrix operation on it with matrix B. The partial or final matrix result is stored in accumulator 1408.

[0285] Unified memory 1406 is used to store input and output data. Weight data is directly transferred to weight memory 1402 through the Direct Memory Access Controller (DMAC) 1405. Input data is also transferred to unified memory 1406 through the DMAC.

[0286] BIU stands for Bus Interface Unit, i.e., bus interface unit 1410 , which is used for interaction between the AXI bus, DMAC, and instruction fetch buffer (IFB) 1409 .

[0287] The bus interface unit 1410 (BIU) is used for the instruction fetch memory 1409 to obtain instructions from the external memory, and is also used for the storage unit access controller 1405 to obtain the original data of the input matrix A or the weight matrix B from the external memory.

[0288] DMAC is mainly used to move input data in the external memory DDR to the unified memory 1406 or move weight data to the weight memory 1402 or move input data to the input memory 1401.

[0289] The vector calculation unit 1407 includes multiple operation processing units. When necessary, it further processes the output of the operation circuit 1403, such as vector multiplication, vector addition, exponential operation, logarithmic operation, size comparison, etc. It is mainly used for non-convolutional / fully connected layer network calculations in neural networks, such as batch normalization, pixel-level summation, and upsampling of feature planes.

[0290] In some implementations, the vector calculation unit 1407 can store the processed output vector in the unified memory 1406. For example, the vector calculation unit 1407 can apply a linear function or a nonlinear function to the output of the operation circuit 1403, such as linear interpolation of the feature plane extracted by the convolution layer, or accumulate a vector of values ​​to generate an activation value. In some implementations, the vector calculation unit 1407 generates a normalized value, a pixel-level summed value, or both. In some implementations, the processed output vector can be used as an activation input to the operation circuit 1403, for example, for use in subsequent layers in a neural network.

[0291] An instruction fetch buffer 1409 connected to the controller 1404 is used to store instructions used by the controller 1404;

[0292] Unified memory 1406, input memory 1401, weight memory 1402, and instruction fetch memory 1409 are all on-chip memories. External memories are private to the NPU hardware architecture.

[0293] The processor mentioned in any of the above places can be a general-purpose central processing unit, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the above program.

[0294] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0295] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits, or dedicated circuits. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0296] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0297] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive (SSD)).

Claims

1. A data processing method, characterized in that: The method comprises: Obtaining a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting the attribute information of the user and the item; According to the fusion result of the prompt and the first feature representation, predicting the operation information of the user on the item through a recommendation model; When the operation information meets a preset condition, the item is recommended to the user.

2. The method according to claim 1, characterized in that The information related to the recommended scenario includes: Indicative information of the recommended scenario; or, Attribute information of the user; or A feature representation of the attribute information of the user.

3. The method according to claim 1 or 2, characterized in that: The obtaining prompt includes: Performing one-hot encoding on the information related to the recommendation scenario to obtain the prompt; or, Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or, The information related to the recommendation scenario is processed through a neural network to obtain multiple weights, each weight corresponds to a feature representation, and based on the weight, multiple feature representations are fused to obtain the prompt.

4. The method according to any one of claims 1 to 3, characterized in that: Different recommendation scenarios are different applications; or, Different recommendation scenarios are different types of applications; or, Different recommendation scenarios are different functions of the same application; or, The different recommendation scenes are different physical scenes where the recommendations are made; or, Different recommendation scenarios are applications developed by different developers; or, The different recommendation scenarios are different or different types of e-commerce products.

5. A data processing method, characterized in that: The method comprises: Obtaining a prompt and a first feature representation; the prompt is obtained based on information related to the recommendation scenario; the first feature representation is obtained by extracting the attribute information of the user and the item; According to the fusion result of the prompt and the first feature representation, predicting the operation information of the user on the item through a recommendation model; The recommendation model is updated according to the operation information and the corresponding label.

6. The method according to claim 5, characterized in that The obtaining prompt includes: Performing one-hot encoding on the information related to the recommendation scenario to obtain the prompt; or, Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or, The information related to the recommendation scenario is processed through a neural network to obtain multiple weights, each weight corresponds to a feature representation, and based on the weight, multiple feature representations are fused to obtain the prompt.

7. The method according to claim 6, characterized in that The obtaining of the first feature representation comprises: Extracting features of the attribute information of the user and the item through the embedding layer to obtain the first feature representation; The method also includes: updating the neural network according to the operation information and the corresponding label while keeping the parameters of the embedding layer fixed.

8. The method according to any one of claims 5 to 7, characterized in that: The updating of the recommendation model comprises: Only part of the network in the recommendation model is updated.

9. The method according to claim 8, characterized in that The partial network is a linear layer.

10. The method according to any one of claims 5 to 9, characterized in that: The information related to the recommended scenario includes: Indicative information of the recommended scenario; or, Attribute information of the user; or A feature representation of the attribute information of the user.

11. A data processing device, characterized in that: The device comprises: A processing module, used to obtain a prompt and a first feature representation; the prompt is obtained according to information related to the recommendation scenario; the first feature representation is obtained by extracting the attribute information of the user and the item; and according to the fusion result of the prompt and the first feature representation, predicting the operation information of the user on the item through the recommendation model; The recommendation model is used to recommend the item to the user when the operation information meets a preset condition.

12. The device according to claim 11, characterized in that The information related to the recommended scenario includes: Indicative information of the recommended scenario; or, Attribute information of the user; or A feature representation of the attribute information of the user.

13. The device according to claim 11 or 12, characterized in that The processing module is specifically used for: Performing one-hot encoding on the information related to the recommendation scenario to obtain the prompt; or, Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or, The information related to the recommendation scenario is processed through a neural network to obtain multiple weights, each weight corresponds to a feature representation, and based on the weight, multiple feature representations are fused to obtain the prompt.

14. The device according to any one of claims 11 to 13, characterized in that: Different recommendation scenarios are different applications; or, Different recommendation scenarios are different types of applications; or, Different recommendation scenarios are different functions of the same application; or, The different recommendation scenes are different physical scenes where the recommendations are made; or, Different recommendation scenarios are applications developed by different developers; or, The different recommendation scenarios are different or different types of e-commerce products.

15. A data processing device, characterized in that: The device comprises: A processing module, used to obtain a prompt and a first feature representation; the prompt is obtained according to information related to the recommendation scenario; the first feature representation is obtained by extracting the attribute information of the user and the item; and according to the fusion result of the prompt and the first feature representation, predicting the operation information of the user on the item through the recommendation model; An updating module is used to update the recommendation model according to the operation information and the corresponding label.

16. The device according to claim 15, characterized in that The processing module is specifically used for: Processing the information related to the recommendation scenario through a neural network to obtain the prompt; or, processing the information related to the recommendation scenario through a neural network to obtain multiple weights, each weight corresponding to a feature representation, and fusing multiple feature representations according to the weights to obtain the prompt; The attribute information of the user and the item is subjected to feature extraction through the embedding layer to obtain the first feature representation.

17. The device according to claim 16, characterized in that The update module is further used for: The neural network is updated according to the operation information and the corresponding labels while keeping the parameters of the embedding layer fixed.

18. The device according to any one of claims 15 to 17, characterized in that The update module is specifically used for: Only part of the network in the recommendation model is updated.

19. The device according to claim 18, characterized in that The partial network is a linear layer.

20. The device according to any one of claims 15 to 19, characterized in that The information related to the recommended scenario includes: Indicative information of the recommended scenario; or, Attribute information of the user; or A feature representation of the attribute information of the user.

21. A computing device, characterized in that The computing device includes a memory and a processor; the memory stores codes, and the processor is configured to obtain the codes and execute the method according to any one of claims 1 to 10.

22. A computer storage medium, characterized in that The computer storage medium stores one or more instructions, which, when executed by one or more computers, enable the one or more computers to implement the method of any one of claims 1 to 10.

23. A computer program product comprising code, characterized in that When the code is executed, it is used to implement the method according to any one of claims 1 to 10.

Images