Information prediction method, device, electronic device and medium based on multi-task model

Abstract

The embodiments of the present application provide an information prediction method, device, electronic device and medium based on a multi-task model, which belongs to the field of financial technology. The method includes: obtaining first historical behavior data of users in a first group and second historical behavior data of users in a second group; inputting the multi-task model for weight calculation to obtain a first attention vector and a second attention vector; and inputting the shared layer for vector sharing, outputting the shared vector and feature sharing information; performing differential learning on the shared vector and feature sharing information to obtain an embedded vector; performing feature learning on the multi-task model to obtain a trained multi-task model; inputting the obtained first click information and second click information into the trained multi-task model for click prediction to obtain first prediction information and second prediction information. The embodiments of the present application can increase the flexibility of information prediction and improve the correlation between users and items.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to an information prediction method, device, electronic device and medium based on a multi-task model. Background Art

[0002] In information recommendation scenarios, there's a need to recommend information to users in a targeted manner in order to improve the revenue of the information recommendation platform, the revenue of the information distributor, and the user experience. For example, when recommending financial products to users, a financial recommendation platform needs to first understand the user's click preferences and then recommend targeted ads or links related to these preferences. Commonly used multi-user modeling methods in the industry include: clustered recommendations based on basic user attributes, clustered recommendations based on clustering algorithms, and clustered recommendations based on business rules.

[0003] However, clustering recommendations based on basic user attributes rely on clustering based on basic attributes, resulting in a single recommendation and a significant Matthew effect. For example, the platform may recommend financial products that users frequently browse while ignoring financial products that users have not clicked on. Clustering recommendations based on clustering algorithms, due to the sparse user behavior, may produce unreliable results and poor results. For example, when a user browses for auto insurance, the platform may recommend pension insurance. Clustering recommendations based on business rules consume a large amount of manual operating costs, are inflexible, and time-consuming. Therefore, most of the user clustering methods commonly used in the industry can only achieve clustering recommendations based on data statistics or unsupervised algorithms. However, these methods cannot measure the similarity between users and items. Financial markets change rapidly, and investors need to make decisions quickly. However, the related technologies lack personalization, resulting in poor online performance. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to propose an information prediction method, device, electronic device and medium based on a multi-task model to increase the flexibility of information prediction and improve the correlation between users and items.

[0005] To achieve the above-mentioned object, a first aspect of an embodiment of the present application proposes an information prediction method based on a multi-task model, wherein the multi-task model includes a shared layer; the method includes:

[0006] Acquire first historical behavior data of users in a first group and second historical behavior data of users in a second group, wherein the user types of the first group and the second group are different;

[0007] Inputting the first historical behavior data and the second historical behavior data into the multi-task model for weight calculation to obtain a first attention vector and a second attention vector;

[0008] Inputting the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into the sharing layer for vector sharing, and outputting a shared vector and feature sharing information;

[0009] Performing differential learning on the shared vector and the feature shared information based on a preset gate mechanism to obtain an embedding vector;

[0010] Performing feature learning on the multi-task model according to the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model;

[0011] Obtaining first click information of users in the first group and second click information of users in the second group;

[0012] The first click information and the second click information are input into a trained multi-task model for click prediction to obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information.

[0013] In some embodiments, the multi-task model includes an attention structure; inputting the first historical behavior data and the second historical behavior data into the multi-task model for weight calculation to obtain the first attention vector and the second attention vector includes:

[0014] Obtaining a first preference feature of the first historical behavior data and a second preference feature of the second historical behavior data;

[0015] Performing feature conversion on the first preference feature and the second preference feature to obtain a first feature vector and a second feature vector;

[0016] The first eigenvector and the second eigenvector are input into the attention structure for weight calculation to obtain a first attention vector and a second attention vector.

[0017] In some embodiments, inputting the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into the sharing layer for vector sharing, and outputting the shared vector and feature sharing information includes:

[0018] Inputting the first attention vector and the second attention vector into the shared layer for vector splicing to obtain the shared vector;

[0019] The first historical behavior data and the second historical behavior data are input into the sharing layer for feature sharing to obtain the shared information.

[0020] In some embodiments, the sharing layer includes a position bias network; and inputting the first historical behavior data and the second historical behavior data into the sharing layer for feature sharing to obtain the shared information includes:

[0021] determining first bias information corresponding to the first historical behavior data and second bias information corresponding to the second historical behavior data;

[0022] Inputting the first bias information and the second bias information into the position bias network to perform position learning to obtain a first position vector and a second position vector;

[0023] Performing feature splicing on the first position vector and the first historical behavior data to obtain first shared information;

[0024] Performing feature splicing on the second position vector and the second historical behavior data to obtain second shared information;

[0025] Shared information is generated according to the first shared information and the second shared information.

[0026] In some embodiments, the multi-task model includes a ranking model; and performing feature learning on the multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model includes:

[0027] Inputting the embedding vector, the first historical behavior data, and the second historical behavior data into the ranking model for back propagation to obtain a first error value and a second error value;

[0028] The embedding vector is reversely optimized according to the first error value and the second error value to obtain a trained multi-task model.

[0029] In some embodiments, inputting the embedding vector, the first historical behavior data, and the second historical behavior data into a preset ranking model for back propagation to obtain a first error value and a second error value includes:

[0030] The embedding vector, the first historical behavior data, and the second historical behavior data are input into a preset sorting model, so that the sorting model performs nonlinear learning on the first historical behavior data and the embedding vector to obtain a first error value, and performs nonlinear learning on the second historical behavior data and the embedding vector to obtain a second error value.

[0031] In some embodiments, inputting the first click information and the second click information into a trained multi-task model for click prediction to obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information includes:

[0032] Inputting the first click information and the second click information into a trained multi-task model, so that the multi-task model scores the first click information and the second click information to obtain a first prediction probability and a second prediction probability;

[0033] First prediction information corresponding to the first click information is determined based on the first prediction probability, and second prediction information corresponding to the second click information is determined based on the second prediction probability.

[0034] To achieve the above-mentioned object, a second aspect of an embodiment of the present application provides an information prediction device based on a multi-task model, wherein the multi-task model includes a shared layer; the device includes:

[0035] a historical behavior acquisition module, configured to acquire first historical behavior data of users in a first group and second historical behavior data of users in a second group, wherein the user types of the first group and the second group are different;

[0036] a weight calculation module, configured to input the first historical behavior data and the second historical behavior data into the multi-task model for weight calculation to obtain a first attention vector and a second attention vector;

[0037] a vector sharing module, configured to input the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into the sharing layer for vector sharing, and output a shared vector and feature sharing information;

[0038] a difference learning module, configured to perform difference learning on the shared vector and the feature shared information based on a preset gate mechanism to obtain an embedding vector;

[0039] a feature learning module, configured to perform feature learning on the multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model;

[0040] A click information module, configured to obtain first click information of users in the first group and second click information of users in the second group;

[0041] A probability prediction module is used to input the first click information and the second click information into a trained multi-task model for click prediction, and obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information.

[0042] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, an electronic device, the electronic device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the information prediction method based on the multi-task model as described in the first aspect when executing the computer program.

[0043] To achieve the above-mentioned purpose, the fourth aspect of an embodiment of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the information prediction method based on the multi-task model as described in the first aspect.

[0044] The information prediction method, device, electronic device, and storage medium based on a multi-task model proposed in this application first obtain first historical behavior data of users in a first group and second historical behavior data of users in a second group, thereby obtaining data information of different user types. The first historical behavior data and the second historical behavior data are input into a sharing layer for vector sharing, and the shared vector and feature sharing information are output, thereby focusing on user interests and avoiding large information sparsity between groups. Then, based on a preset gate mechanism, the shared vector and feature sharing information are differentially learned to obtain an embedding vector, thereby learning the differences between different groups and improving the effect between different groups. Then, feature learning is performed on a multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model, thereby improving the generalization and association performance of the multi-task model. Finally, the first click information of users in the first group and the second click information of users in the second group are obtained, and the first click information and the second click information are input into the trained multi-task model for click prediction. The first prediction information corresponding to the first click information and the second prediction information corresponding to the second click information are obtained, thereby realizing the prediction of user click behavior, increasing the flexibility of information prediction, and improving the association between users and items. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 is a flowchart of an information prediction method based on a multi-task model provided in an embodiment of the present application;

[0046] Figure 2 yes Figure 1 Flowchart of step S102 in FIG.

[0047] Figure 3yes Figure 1 Flowchart of step S103 in FIG.

[0048] Figure 4 yes Figure 3 Flowchart of step S302 in FIG.

[0049] Figure 5 yes Figure 1 Flowchart of step S105 in FIG.

[0050] Figure 6 yes Figure 5 Flowchart of step S501 in FIG.

[0051] Figure 7 yes Figure 1 Flowchart of step S107 in FIG.

[0052] Figure 8 Schematic diagram of the structure of the information prediction device based on the multi-task model provided in an embodiment of the present application;

[0053] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0055] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0057] First, let’s analyze some of the terms used in this application:

[0058] Natural language processing (NLP): NLP uses computers to process, understand, and apply human languages ​​(such as Chinese and English). A branch of artificial intelligence, NLP is an interdisciplinary field between computer science and linguistics, often referred to as computational linguistics. Natural language processing encompasses grammatical analysis, semantic analysis, and discourse comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It encompasses data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research related to language processing, and linguistics research related to language computing.

[0059] Multi-task Modeling (MMOE): MMOE models relationships between tasks, balancing task-specificity with shared representations. It automatically allocates parameters to learn information shared between tasks or unique to each task. MMOE is easy to train and converges to a good loss in just a few rounds.

[0060] Multilayer Perceptron (MLP): A multilayer perceptron is a feedforward artificial neural network model that maps multiple input data sets to a single output data set. Also called an artificial neural network (ANN), a multilayer perceptron can have multiple hidden layers in addition to the input and output layers. The simplest MLP has only one hidden layer, meaning it has a three-layer structure.

[0061] Sigmoid function: A common S-shaped function in biology, also known as an S-shaped growth curve. In information science, due to its monotonic increasing properties and the monotonic increasing properties of its inverse function, the sigmoid function is often used as an activation function in neural networks, mapping variables to a value between 0 and 1.

[0062] Based on this, the embodiments of the present application provide an information prediction method and device, an electronic device, and a storage medium based on a multi-task model, which can increase the flexibility of information prediction and improve the correlation between users and items.

[0063] The information prediction method and device based on the multi-task model, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the information prediction method based on the multi-task model in the embodiments of the present application is described.

[0064] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0065] Fundamental AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, module management for online customer service systems, natural language processing, and machine learning / deep learning.

[0066] The information prediction method based on the multi-task model provided in the embodiment of the present application relates to the field of data processing technology. The information prediction method based on the multi-task model provided in the embodiment of the present application can be applied to the terminal, can also be applied to the server side, and can also be software running in the terminal or the server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the information prediction method based on the multi-task model, etc., but is not limited to the above forms.

[0067] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0068] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with the relevant laws, regulations, and standards of the relevant countries and regions. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0069] Changes in the financial market are very complex, and different factors will affect the prices and yields of financial products. Therefore, financial institutions need to effectively manage risks so that investors can make decisions quickly according to their needs. Therefore, financial institutions usually use information recommendations to provide risk warnings and product recommendations.

[0070] In information recommendation scenarios, there's a need to recommend information to users in a targeted manner in order to improve the revenue of the information recommendation platform, the revenue of the information distributor, and the user experience. For example, when recommending financial products to users, a financial recommendation platform needs to first understand the user's click preferences and then recommend targeted ads or links related to these preferences. Commonly used multi-user modeling methods in the industry include: clustered recommendations based on basic user attributes, clustered recommendations based on clustering algorithms, and clustered recommendations based on business rules.

[0071] However, cluster recommendations based on basic user attributes need to rely on clustering of basic attributes, resulting in the singleness of recommendations and a more obvious Matthew effect. For example, the platform recommends financial products that users frequently browse and ignores financial products that users have not clicked on. User cluster recommendations based on clustering algorithms will produce unreliable results to a certain extent and have poor effects due to the sparse user behavior itself. For example, a user browses car insurance, but the platform recommends pension insurance to the user. Cluster recommendations based on business rules consume a lot of manual operating costs, are not flexible enough, and are time-consuming and labor-intensive. Therefore, most of the user clusters commonly used in the industry can only be based on data statistics or unsupervised algorithms for cluster recommendations. However, in fact, these methods cannot measure the similarity between users and items, lack personalization, and lead to poor online effects.

[0072] In order to solve the above technical problems, the present application provides an information prediction method based on a multi-task model, which includes the following steps:

[0073] Figure 1 This is an optional flowchart of the information prediction method based on the multi-task model provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0074] It should be noted that the multi-task model includes a shared layer, and the multi-task model is an MMOE model.

[0075] Step S101, obtaining first historical behavior data of users in a first group and second historical behavior data of users in a second group;

[0076] It should be noted that the user types of the first group and the second group are different.

[0077] In step S101 of some embodiments, first historical behavior data of users in a first group and second historical behavior data of users in a second group are obtained, thereby obtaining data information of different user types.

[0078] It should be noted that the first group may be financial agents, engineers, teachers, doctors, financial analysts, etc., and the second group may be non-agents, experience users, students, patients, etc. This embodiment does not impose specific restrictions.

[0079] It is worth noting that the first historical behavior data and the second historical behavior data include the portrait features, click-forwarding behaviors and other preference features of the first and second groups, for example, the financial products and financial science knowledge frequently browsed by the first and second groups, and also include information such as the content dimension of the click behavior. Among them, the content dimension includes the subject information and labels of the consultation, for example, the name, amount, policy term, etc. of the relevant financial products.

[0080] Step S102: Input the first historical behavior data and the second historical behavior data into the multi-task model for weight calculation to obtain a first attention vector and a second attention vector;

[0081] In step S102 of some embodiments, the first historical behavior data and the second historical behavior data are input into a multi-task model for weight calculation to obtain a first attention vector and a second attention vector, thereby avoiding the impact of user interest drift and focusing on the user's interest.

[0082] Step S103: Input the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into a sharing layer for vector sharing, and output a shared vector and feature sharing information;

[0083] In step S103 of some embodiments, the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data are input into the sharing layer for vector sharing, and the shared vector and feature sharing information are output, thereby avoiding the problem of large differences in feature sparsity between different groups and solving the problem of model learning bias.

[0084] It should be noted that sharing can indirectly supplement the characteristics of groups with rich behaviors and reduce the impact of sparse group characteristics.

[0085] Step S104: performing differential learning on the shared vector and feature shared information based on a preset gate mechanism to obtain an embedding vector;

[0086] It should be noted that the multi-task model includes multiple expert networks, where the number of multiple expert networks corresponds to the number of groups, and a gate mechanism is pre-set in each expert network.

[0087] In step S104 of some embodiments, the shared vector and feature sharing information are input into the multi-expert network, so that the multi-expert network performs differential learning on the shared vector and feature sharing information based on a preset gate mechanism to obtain an embedded vector, thereby being able to differentially learn the differences between different groups and improve the effects between different groups.

[0088] Step S105, performing feature learning on the multi-task model according to the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model;

[0089] In step S105 of some embodiments, feature learning is performed on the multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model, thereby improving the generalization performance of the multi-task model and the accuracy of the prediction probability.

[0090] Step S106, obtaining first click information of users in the first group and second click information of users in the second group;

[0091] In step S106 of some embodiments, first click information of users in the first group and second click information of users in the second group are obtained to facilitate the subsequent multi-task model to predict the click information of users in different groups and further explore the association between users and items.

[0092] Step S107: Input the first click information and the second click information into the trained multi-task model to perform click prediction, and obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information.

[0093] In step S107 of some embodiments, the first click information and the second click information are input into a trained multi-task model for click prediction, and first prediction information corresponding to the first click information and second prediction information corresponding to the second click information are obtained, thereby realizing the prediction of user click behavior, increasing the flexibility of information prediction, and improving the correlation between users and items.

[0094] In steps S101 to S107 illustrated in the embodiment of the present application, first, first historical behavior data of users in the first group and second historical behavior data of users in the second group are obtained, so as to obtain data information of different user types. The first historical behavior data and the second historical behavior data are input into the sharing layer for vector sharing, and the shared vector and feature sharing information are output, thereby focusing on the user's interests and avoiding a large degree of information sparsity between groups. Then, based on the preset gate mechanism, the shared vector and the feature sharing information are differentially learned to obtain an embedding vector, so as to learn the differences between different groups and improve the effect between different groups. Then, feature learning is performed on the multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model, thereby improving the generalization performance and association performance of the multi-task model. Finally, the first click information of users in the first group and the second click information of users in the second group are obtained, and the first click information and the second click information are input into the trained multi-task model for click prediction, thereby obtaining first prediction information corresponding to the first click information and second prediction information corresponding to the second click information, thereby realizing the prediction of user click behavior, increasing the flexibility of information prediction, and improving the association between users and items.

[0095] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S203:

[0096] It should be noted that the multi-task model includes an attention structure.

[0097] Step S201, obtaining a first preference feature of the first historical behavior data and a second preference feature of the second historical behavior data;

[0098] In step S201 of some embodiments, a first preference feature of the first historical behavior data and a second preference feature of the second historical behavior data are obtained, thereby obtaining interest features of different groups of users to facilitate subsequent operations such as feature sharing.

[0099] It should be noted that the first preference characteristics include the portrait characteristics, click-forwarding behavior and other characteristics of users in the first group, and the second preference characteristics include the portrait characteristics, click-forwarding behavior and other characteristics of users in the second group. This embodiment does not make specific restrictions.

[0100] It is understandable that the user's portrait features include but are not limited to basic portraits such as the user's age and gender, and the click-forwarding behavior includes but is not limited to the user's number of clicks, forwardings, collections, etc. For example, clicking on pension insurance four times, forwarding child insurance twice, collecting car insurance three times, etc.

[0101] Step S202: performing feature conversion on the first preference feature and the second preference feature to obtain a first feature vector and a second feature vector;

[0102] In step S202 of some embodiments, feature conversion is performed on the first preference feature and the second preference feature. In the process of feature conversion, the first preference feature and the second preference feature are first scored to obtain a first eigenvalue corresponding to the first preference feature and a second eigenvalue corresponding to the second preference feature. Then, the first eigenvalue and the second eigenvalue are represented by vectors. For example, the first eigenvalue is 1 and the second eigenvalue is 0. The first eigenvalue is converted into a vector of [0.1, 0, 3, 0, 4] and the second eigenvalue is converted into a vector of [0.3, 0.4, 0.5] to obtain a first eigenvector and a second eigenvector. In this way, many eigenvalues ​​can be represented in a vector manner, thereby improving the convergence of the multi-task model.

[0103] Step S203: Input the first eigenvector and the second eigenvector into the attention structure for weight calculation to obtain the first attention vector and the second attention vector.

[0104] In step S203 of some embodiments, the first eigenvector and the second eigenvector are input into the attention structure for weight calculation to obtain the first attention vector and the second attention vector, thereby avoiding the influence of user interest drift and focusing the user's interest.

[0105] It is understandable that the weights learned for users whose historical behavioral data has been obtained are also applicable to new users of the same type. For example, if the historical behavioral data of agent A has been obtained before, the weights learned from agent A can be applied to agent B of the same type. For example, if agent A is a financial analyst and the relevant historical behavioral data is "how to understand the financial market", then for agent B, who is also a financial analyst, the weights of "how to understand the financial market" can be directly applied to agent B.

[0106] See also Figure 3 In some embodiments, step S103 may include but is not limited to steps S301 to S302:

[0107] Step S301: input the first attention vector and the second attention vector into the shared layer for vector splicing to obtain a shared vector;

[0108] In step S301 of some embodiments, the first attention vector and the second attention vector are input into a shared layer for vector splicing to obtain a shared vector, thereby avoiding the problem of large differences in feature sparsity between different groups and solving the problem of model learning bias.

[0109] It should be noted that vector concatenation is to concatenate the first attention vector and the second attention vector to obtain a large shared vector, which is convenient for subsequent model training.

[0110] Step S302: Input the first historical behavior data and the second historical behavior data into a sharing layer for feature sharing to obtain shared information.

[0111] In step S302 of some embodiments, the first historical behavior data and the second historical behavior data are input into the sharing layer for feature sharing to obtain shared information. Through sharing, the characteristics of the group with rich behaviors can be indirectly used as a certain supplement to reduce the impact of sparse group characteristics.

[0112] See also Figure 4 In some embodiments, step S302 may include but is not limited to steps S401 to S405:

[0113] It should be noted that the shared layer includes a position bias network, also known as a position bias network.

[0114] Step S401, determining first bias information corresponding to first historical behavior data and second bias information corresponding to second historical behavior data;

[0115] In step S401 of some embodiments, first bias information corresponding to the first historical behavior data and second bias information corresponding to the second historical behavior data are determined to avoid the influence of bias in the first historical behavior data and the second historical behavior data.

[0116] Step S402: Input the first bias information and the second bias information into a position bias network for position learning to obtain a first position vector and a second position vector;

[0117] In step S402 of some embodiments, the first bias information and the second bias information are input into a position bias network for position learning to obtain a first position vector and a second position vector, thereby avoiding bias effects, improving the overall recommendation effect, and allowing more other products to have the opportunity to be exposed.

[0118] It is understandable that users often browse related recommended links such as "pension insurance", "car insurance", and "currency", but rarely browse related recommended links such as "childcare insurance", "income tax", and "funds". In order to avoid the Matthew effect, it is necessary to determine the first bias information corresponding to the first historical behavior data of the first few items and the second bias information corresponding to the second historical behavior data of the last few items, so as to improve the accuracy and fairness of the information and enhance the credibility and credibility of the information.

[0119] It should be noted that there is certain bias information in both the first historical behavior data and the second historical behavior data. Among them, the bias information is mostly concentrated on position bias, that is, different display positions have different effects on click-through rates. For example, the click-through rate of products displayed in the front will be higher, and the click-through rate of products displayed in the back will be lower. Therefore, it is necessary to determine the first bias information and the second bias information to avoid the influence of bias.

[0120] Step S403: performing feature splicing on the first position vector and the first historical behavior data to obtain first shared information;

[0121] Step S404: performing feature splicing on the second position vector and the second historical behavior data to obtain second shared information;

[0122] Step S405: Generate shared information according to the first shared information and the second shared information.

[0123] In steps S403 to S405 of some embodiments, the first position vector and the second position vector output by the position bias network are feature-concatenated with the first historical behavior data and the second historical behavior data, respectively, to obtain first shared information and second shared information. The first shared information and the second shared information are then integrated to obtain shared information, thereby improving the overall recommendation effect, avoiding the influence of position bias, avoiding the Matthew effect, and avoiding recommending only financial products or financial articles that users frequently browse or click on.

[0124] It should be noted that, in the process of feature splicing, it is necessary to splice the first position vector with multiple features in the first historical behavior data, and splice the second position vector with multiple features in the second historical behavior data, so as to improve the accuracy of the prediction.

[0125] See also Figure 5 In some embodiments, step S105 may also include but is not limited to steps S501 to S502:

[0126] It should be noted that the multi-task model includes a sorting model, wherein the sorting model includes a multi-layer perceptron.

[0127] Step S501: Input the embedding vector, the first historical behavior data, and the second historical behavior data into the sorting model for back propagation to obtain a first error value and a second error value;

[0128] It should be noted that the backpropagation mechanism is a learning mechanism that comes with the neural network. The sorting model includes the output layer, hidden layer, and input layer.

[0129] In step S501 of some embodiments, the embedding vector, the first historical behavior data, and the second historical behavior data are input into the sorting model for back propagation to obtain a first error value and a second error value, so as to calculate the error between the estimated value of the embedding vector and the actual value of the first historical behavior data and the actual value of the second historical behavior data, thereby facilitating subsequent reverse optimization.

[0130] Step S502: reversely optimize the embedding vector according to the first error value and the second error value to obtain a trained multi-task model.

[0131] In step S502 of some embodiments, the first error value and the second error value are back-propagated from the output layer to the hidden layer until they are propagated to the input layer, thereby completing the reverse optimization process and obtaining a trained multi-task model, thereby improving the generalization performance of the multi-task model and the accuracy of the prediction probability.

[0132] See also Figure 6In some embodiments, step S501 includes but is not limited to step S601:

[0133] Step S601: Input the embedding vector, the first historical behavior data, and the second historical behavior data into a preset sorting model, so that the sorting model performs nonlinear learning on the first historical behavior data and the embedding vector to obtain a first error value, and performs nonlinear learning on the second historical behavior data and the embedding vector to obtain a second error value.

[0134] In step S601 of some embodiments, after the embedding vector, the first historical behavior data, and the second historical behavior data are input into a preset sorting model, the sorting model performs nonlinear learning on the first historical behavior data and the embedding vector, thereby learning the relationship between the embedding vector and the feature information in the first historical behavior data, and obtaining a first error value. Similarly, the sorting model performs nonlinear learning on the second historical behavior data and the embedding vector, thereby learning the relationship between the embedding vector and the feature information in the second historical behavior data, and obtaining a second error value, which facilitates subsequent reverse optimization and improves the generalization performance of the multi-task model.

[0135] See also Figure 7 In some embodiments, step S107 includes but is not limited to steps S701 to S702:

[0136] It should be noted that the multi-task model includes a sigmiod function.

[0137] Step S701: Input the first click information and the second click information into a trained multi-task model, so that the multi-task model scores the first click information and the second click information to obtain a first prediction probability and a second prediction probability;

[0138] Step S702: determining first prediction information corresponding to the first click information according to the first prediction probability, and determining second prediction information corresponding to the second click information according to the second prediction probability.

[0139] In steps S701 to S702 of some embodiments, the first click information and the second click information are input into a trained multi-task model, so that the sigmiod function in the multi-task model scores the first click information and the second click information to obtain the first prediction probability and the second prediction probability, thereby improving the accuracy of the prediction of the user's click information. Then, the first prediction information corresponding to the first click information is determined based on the first prediction probability, and the second prediction information corresponding to the second click information is determined based on the second prediction probability, thereby realizing the prediction of the user's click behavior, increasing the flexibility of information prediction, and improving the correlation between the user and the item.

[0140] See also Figure 8The embodiment of the present application further provides an information prediction device based on a multi-task model, wherein the multi-task model includes a shared layer; the device includes:

[0141] A historical behavior acquisition module 801 is configured to acquire first historical behavior data of users in a first group and second historical behavior data of users in a second group, wherein the user types of the first group and the second group are different;

[0142] A weight calculation module 802 is configured to input the first historical behavior data and the second historical behavior data into a multi-task model for weight calculation to obtain a first attention vector and a second attention vector;

[0143] A vector sharing module 803 is configured to input the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into a sharing layer for vector sharing, and output a shared vector and feature sharing information;

[0144] A difference learning module 804 is configured to perform difference learning on the shared vector and feature shared information based on a preset gate mechanism to obtain an embedding vector;

[0145] A feature learning module 805 is configured to perform feature learning on the multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model;

[0146] A click information module 806 is used to obtain first click information of users in the first group and second click information of users in the second group;

[0147] The probability prediction module 807 is used to input the first click information and the second click information into the trained multi-task model for click prediction, and obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information.

[0148] The specific implementation of the information prediction device based on the multi-task model is basically the same as the specific embodiment of the information prediction method based on the multi-task model described above, and will not be repeated here.

[0149] The present application also provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned multi-task model-based information prediction method is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0150] See also Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0151] The processor 901 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0152] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 902 and is called by the processor 901 to execute the information prediction method based on the multi-task model in the embodiments of this application.

[0153] Input / output interface 903, used to implement information input and output;

[0154] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0155] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0156] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0157] An embodiment of the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned information prediction method based on the multi-task model.

[0158] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0159] The information prediction method, device, electronic device and storage medium based on the multi-task model provided by the embodiment of the present application first obtain the first historical behavior data of users in the first group and the second historical behavior data of users in the second group, so as to obtain data information of different user types, and input the first historical behavior data and the second historical behavior data into the sharing layer for vector sharing, output the shared vector and feature sharing information, so as to focus on the user's interest and avoid the large degree of information sparsity between groups. Afterwards, the shared vector and feature sharing information are differentially learned based on the preset gate mechanism to obtain the embedded vector, so as to learn the differences between different groups and improve The effect between different groups is obtained, and then the feature learning of the multi-task model is performed according to the embedding vector, the first historical behavior data and the second historical behavior data to obtain a trained multi-task model, thereby improving the generalization performance and association performance of the multi-task model. Finally, the first click information of users in the first group and the second click information of users in the second group are obtained, and the first click information and the second click information are input into the trained multi-task model for click prediction, thereby obtaining the first prediction information corresponding to the first click information and the second prediction information corresponding to the second click information, thereby realizing the prediction of user click behavior, increasing the flexibility of information prediction, and improving the correlation between users and items.

[0160] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0161] It will be understood by those skilled in the art that Figure 1-7 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0162] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0163] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0164] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0165] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0166] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0167] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0168] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0169] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0170] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. An information prediction method based on a multi-task model, characterized in that: The multi-task model includes a shared layer; the method includes: Acquire first historical behavior data of users in a first group and second historical behavior data of users in a second group, wherein the user types of the first group and the second group are different; Inputting the first historical behavior data and the second historical behavior data into the multi-task model for weight calculation to obtain a first attention vector and a second attention vector; Inputting the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into the sharing layer for vector sharing, and outputting a shared vector and feature sharing information; Performing differential learning on the shared vector and the feature shared information based on a preset gate mechanism to obtain an embedding vector; Performing feature learning on the multi-task model according to the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model; Obtaining first click information of users in the first group and second click information of users in the second group; The first click information and the second click information are input into a trained multi-task model for click prediction to obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information.

2. The information prediction method based on the multi-task model according to claim 1, characterized in that: The multi-task model includes an attention structure; the first historical behavior data and the second historical behavior data are input into the multi-task model for weight calculation to obtain a first attention vector and a second attention vector, including: Obtaining a first preference feature of the first historical behavior data and a second preference feature of the second historical behavior data; Performing feature conversion on the first preference feature and the second preference feature to obtain a first feature vector and a second feature vector; The first eigenvector and the second eigenvector are input into the attention structure for weight calculation to obtain a first attention vector and a second attention vector.

3. The information prediction method based on the multi-task model according to claim 1, characterized in that: The step of inputting the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into the sharing layer for vector sharing, and outputting the shared vector and feature sharing information includes: Inputting the first attention vector and the second attention vector into the shared layer for vector splicing to obtain the shared vector; The first historical behavior data and the second historical behavior data are input into the sharing layer for feature sharing to obtain the shared information.

4. The information prediction method based on the multi-task model according to claim 3, characterized in that: The sharing layer includes a position bias network; the first historical behavior data and the second historical behavior data are input into the sharing layer for feature sharing to obtain the shared information, including: determining first bias information corresponding to the first historical behavior data and second bias information corresponding to the second historical behavior data; Inputting the first bias information and the second bias information into the position bias network to perform position learning to obtain a first position vector and a second position vector; Performing feature splicing on the first position vector and the first historical behavior data to obtain first shared information; Performing feature splicing on the second position vector and the second historical behavior data to obtain second shared information; Shared information is generated according to the first shared information and the second shared information.

5. The information prediction method based on the multi-task model according to claim 1, characterized in that: The multi-task model includes a sorting model; and performing feature learning on the multi-task model according to the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model includes: Inputting the embedding vector, the first historical behavior data, and the second historical behavior data into the ranking model for back propagation to obtain a first error value and a second error value; The embedding vector is reversely optimized according to the first error value and the second error value to obtain a trained multi-task model.

6. The information prediction method based on the multi-task model according to claim 5, characterized in that: Inputting the embedding vector, the first historical behavior data, and the second historical behavior data into a preset sorting model for back propagation to obtain a first error value and a second error value includes: The embedding vector, the first historical behavior data, and the second historical behavior data are input into a preset sorting model, so that the sorting model performs nonlinear learning on the first historical behavior data and the embedding vector to obtain a first error value, and performs nonlinear learning on the second historical behavior data and the embedding vector to obtain a second error value.

7. The information prediction method based on the multi-task model according to claim 1, characterized in that: Inputting the first click information and the second click information into a trained multi-task model to perform click prediction, and obtaining first prediction information corresponding to the first click information and second prediction information corresponding to the second click information, includes: Inputting the first click information and the second click information into a trained multi-task model, so that the multi-task model scores the first click information and the second click information to obtain a first prediction probability and a second prediction probability; First prediction information corresponding to the first click information is determined based on the first prediction probability, and second prediction information corresponding to the second click information is determined based on the second prediction probability.

8. An information prediction device based on a multi-task model, characterized in that: The multi-task model includes a shared layer; the apparatus includes: a historical behavior acquisition module, configured to acquire first historical behavior data of users in a first group and second historical behavior data of users in a second group, wherein the user types of the first group and the second group are different; a weight calculation module, configured to input the first historical behavior data and the second historical behavior data into the multi-task model for weight calculation to obtain a first attention vector and a second attention vector; a vector sharing module, configured to input the first attention vector, the second attention vector, the first historical behavior data, and the second historical behavior data into the sharing layer for vector sharing, and output a shared vector and feature sharing information; a difference learning module, configured to perform difference learning on the shared vector and the feature shared information based on a preset gate mechanism to obtain an embedding vector; a feature learning module, configured to perform feature learning on the multi-task model based on the embedding vector, the first historical behavior data, and the second historical behavior data to obtain a trained multi-task model; A click information module, configured to obtain first click information of users in the first group and second click information of users in the second group; A probability prediction module is used to input the first click information and the second click information into a trained multi-task model for click prediction, and obtain first prediction information corresponding to the first click information and second prediction information corresponding to the second click information.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the information prediction method based on the multi-task model according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the information prediction method based on a multi-task model according to any one of claims 1 to 7 is implemented.

Images