Multi-target prediction method and device for content, equipment and storage medium
By using attention mechanism to update features in the multi-objective prediction model, the problem of uneven optimization of expert networks in the MoE model is solved, and the prediction accuracy and efficiency are improved.
Patent Information
- Application Number
- CN202310645985.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-02
- Publication Date
- 2024-12-03
AI Technical Summary
During the training process of the existing hybrid expert network (MoE) model, the expert network with better performance on the first few training samples was over-optimized, resulting in the inability to fully train the remaining expert networks, resulting in the inability to learn global features of the multi-objective prediction results. Reduced prediction accuracy.
By obtaining the initial features of the content to be predicted, using the attention mechanism to update the features, extracting local and global reference information, and combining with the multi-objective prediction model for training, ensuring effective linkage between each target prediction task.
The prediction accuracy of multi-objective prediction tasks is improved, the modeling complexity and operation complexity are reduced, and the prediction efficiency is improved.
Smart Images

Figure CN119066258A_ABST
Abstract
Description
Background Art
[0002] With the development of Internet technology, content in the form of short videos, concise graphics and texts has gradually become a new way for people to obtain information. Content recommendation has also received widespread attention in the field of information technology research. Reasonably and efficiently recommending favorite content to the target is the key to the recommendation system.
[0003] At present, the recommendation system under related technologies is usually based on the Mixture of Experts (MoE) model to complete the multi-objective prediction tasks of content (such as click-through rate prediction, play rate prediction, like rate prediction, etc.), and recommend and distribute the content based on the prediction results.
[0004] The MoE model can be viewed as a multi-branch network with gates, where each branch is called an expert network, which is responsible for learning a target prediction task. However, during the training of the MoE model, the expert network that performs well on the first few training samples will be assigned a larger gate function output, which will cause its corresponding expert network to be selected more frequently and be more fully optimized, which makes it impossible for the remaining expert networks in the MoE model to be fully trained. In extreme cases, the MoE model will only use the output results of a single expert network, making it impossible for the final multi-target prediction results to learn the global features corresponding to the input, thereby reducing the prediction accuracy of the multi-target prediction task. Summary of the invention
[0005] Embodiments of the present application provide a multi-objective prediction method, device, electronic device, and storage medium for content to improve the prediction accuracy of multi-objective prediction tasks.
[0006] In a first aspect, an embodiment of the present application provides a multi-objective prediction method for content, the method comprising:
[0007] Acquire initial features of the content to be predicted, where the initial features at least characterize the content category and multimodal characteristics of the content to be predicted;
[0008] For each target prediction task in the multiple target prediction tasks, the following operations are performed respectively: based on the element weights of each initial feature element contained in the initial feature under the target prediction task, the initial feature is updated to obtain an intermediate feature of the target prediction task, where the intermediate feature contains local reference information required to perform the target prediction task;
[0009] For each of the multiple intermediate features obtained, the following operations are performed respectively: using an attention mechanism, based on the importance of each intermediate feature relative to the intermediate feature, the intermediate feature is updated to obtain a target feature of the intermediate feature, where the target feature contains global reference information required to perform the corresponding target prediction task;
[0010] Based on the obtained multiple target features, corresponding multi-target prediction results are obtained respectively.
[0011] In a second aspect, an embodiment of the present application further provides a multi-objective prediction device for content, the device comprising:
[0012] An acquisition module, used to acquire initial features of the content to be predicted, where the initial features at least represent the content category and multimodal characteristics of the content to be predicted;
[0013] The first updating module is used to perform the following operations for each target prediction task in the multiple target prediction tasks: based on the element weights of each initial feature element contained in the initial feature under the target prediction task, the initial feature is updated to obtain the intermediate feature of the target prediction task, where the intermediate feature contains the local reference information required to perform the target prediction task;
[0014] The second updating module is used to perform the following operations for each of the obtained multiple intermediate features: using an attention mechanism, based on the importance of each intermediate feature relative to the intermediate feature, updating the intermediate feature to obtain a target feature of the intermediate feature, where the target feature contains global reference information required to perform a corresponding target prediction task;
[0015] The prediction module is used to obtain corresponding multi-target prediction results based on the obtained multiple target features.
[0016] In a possible implementation, an attention mechanism is used to update the intermediate features based on the importance of each intermediate feature relative to the intermediate features. When the target feature of the intermediate feature is obtained, the second updating module is used to:
[0017] Each of the multiple attention networks is used to perform the following operations on the intermediate features:
[0018] Based on the attention weight matrix corresponding to each intermediate feature in the attention network, the importance of each intermediate feature relative to the intermediate feature is obtained respectively;
[0019] Based on the obtained importance levels, the intermediate features are updated to obtain attention features;
[0020] The attention features obtained through multiple attention networks are concatenated to obtain the target features of the intermediate features.
[0021] In a possible implementation, the attention weight matrix includes at least a query weight matrix and a key weight matrix; based on the attention weight matrix corresponding to each intermediate feature in the attention network, when obtaining the importance of each intermediate feature relative to the intermediate feature, the second updating module is used to:
[0022] Based on the query weight matrix and key weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding query matrix and key matrix, wherein each element in the query weight matrix represents: the importance of the corresponding intermediate feature element, and each element in the key weight matrix represents: the importance of at least one other intermediate feature element related to the corresponding intermediate feature element;
[0023] Based on the query matrix of the intermediate features and the key matrix of each intermediate feature, a plurality of association matrices are obtained, wherein the plurality of association matrices respectively represent the importance of each intermediate feature relative to the intermediate features.
[0024] In a possible implementation, the attention weight matrix also includes a value weight matrix; based on the obtained importance levels, the intermediate features are updated, and when the attention features are obtained, the second updating module is used to:
[0025] Based on the value weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding value matrix, wherein each element in the value matrix represents: the importance of the correlation between the corresponding intermediate feature element and at least one other related intermediate feature element;
[0026] Attention features are obtained based on multiple correlation matrices of intermediate features and value matrices corresponding to each intermediate feature.
[0027] In a possible implementation, when corresponding multi-objective prediction results are obtained based on the obtained multiple object features, the prediction module is used to:
[0028] For each of the multiple target features, perform the following operations:
[0029] Based on the intermediate feature corresponding to the target feature, the target feature is modified to obtain a modified target feature;
[0030] Based on the modified target feature and its corresponding prediction weight vector under the current target prediction task, the corresponding target prediction result is obtained, wherein each element in the prediction weight vector represents: the importance of each target feature element in the modified target feature.
[0031] In a possible implementation, the multi-objective prediction result is obtained by inputting the initial features into the multi-objective prediction model, wherein the multi-objective prediction model is obtained by training the acquisition module, the first update module, the second update module and the prediction module in the following manner:
[0032] A training sample set is constructed based on the distributed historical content, and the sample initial features corresponding to each sample content in the training sample set are extracted, wherein a sample label corresponding to a plurality of target prediction tasks is set for each sample content;
[0033] Based on the initial features of each sample and multiple sample labels corresponding to the content of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain a multi-objective prediction model.
[0034] In a possible implementation, based on the initial features of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain the multi-objective prediction model, wherein one iteration process includes:
[0035] For each target prediction task in the multiple target prediction tasks, the following operations are performed respectively: based on the element weights of each sample initial feature element contained in a sample initial feature under the target prediction task, the sample initial feature is updated to obtain the sample intermediate feature of the target prediction task;
[0036] For each of the obtained multiple sample intermediate features, the following operations are performed respectively: using an attention mechanism, based on the importance of each sample intermediate feature relative to the sample intermediate feature, the sample intermediate feature is updated to obtain a sample target feature of the sample intermediate feature;
[0037] Based on the obtained multiple sample target features, corresponding multi-target prediction results are obtained respectively;
[0038] Based on the loss value between the multi-objective prediction results and the sample labels corresponding to the sample contents, the parameters of the multi-objective prediction model to be trained are adjusted.
[0039] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements a method as described in any one of the first aspects when executing the computer program.
[0040] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any method described in the first aspect.
[0041] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when called by a computer, enables the computer to execute the method described in the first aspect.
[0042] The beneficial effects of this application are as follows:
[0043] In the embodiment of the present application, a method, device, equipment and storage medium for content multi-target prediction are proposed. The server obtains the initial features of the content to be predicted, and updates the initial features based on the element weights of each initial feature element contained in the initial features under a target prediction task to obtain intermediate features; the attention mechanism is used to update the intermediate features to obtain target features; and corresponding multi-target prediction results are obtained based on the obtained multiple target features. In this way, the dependency relationship between multi-target semantics is deeply extracted by using the attention mechanism, so that each target feature contains the global reference information required to perform the corresponding target prediction task, so that the feature processing of a single target prediction task and other target prediction tasks are effectively linked, avoiding the closed learning of a single target prediction task, resulting in low prediction accuracy. At the same time, multi-target modeling is regarded as sequence modeling, and the temporal order of each target prediction task can also be used, so that this scheme improves the accuracy of each target prediction result while realizing multi-target prediction. In addition, the parallel calculation method of multi-head attention reduces the computational complexity of the attention network, thereby reducing the modeling complexity and improving the prediction efficiency of multi-target prediction.
[0044] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A schematic diagram of an application scenario of an embodiment of the present application;
[0046] Figure 2 A flow chart of a training method for a multi-objective prediction model provided in an embodiment of the present application;
[0047] Figure 3 This is a flow chart of an iterative training process in an embodiment of the present application;
[0048] Figure 4 A schematic diagram of the multi-objective prediction model training in the embodiment of the present application;
[0049] Figure 5 This is a schematic diagram of the internal structure of the Transformer structure in the embodiment of the present application;
[0050] Figure 6 This is a flow chart of a method for performing multi-objective prediction on content to be predicted in an embodiment of the present application;
[0051] Figure 7 A schematic diagram of a scenario for obtaining initial features in an embodiment of the present application;
[0052] Figure 8 A schematic diagram of a scenario in which a linear transformation of initial features is performed in an embodiment of the present application;
[0053] Fig. 9 A flow chart of a method for obtaining target features provided in an embodiment of the present application;
[0054] Fig.10 A schematic diagram of a process for obtaining an important degree in an embodiment of the present application;
[0055] Fig.11 A schematic diagram of a single attention head updating an intermediate feature in an embodiment of the present application;
[0056] Fig.12 A schematic diagram of a process for obtaining attention features in an embodiment of the present application;
[0057] Fig.13 A schematic diagram of obtaining target features in an embodiment of the present application;
[0058] Fig.14 A flowchart of obtaining target prediction results in an embodiment of the present application;
[0059] Fig.15 A schematic diagram of a residual connection provided in an embodiment of the present application;
[0060] Fig.16 A schematic diagram of the structure of a multi-objective prediction device according to an embodiment of the present application;
[0061] Fig.17 A schematic diagram of a hardware structure of a computing device in an embodiment of the present application;
[0062] Fig.18 A schematic diagram of the hardware composition structure of another computing device in an embodiment of the present application. DETAILED DESCRIPTION
[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.
[0064] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the invention described herein can be implemented in sequences other than those illustrated or described herein.
[0065] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.
[0066] Transformer: It is a deep learning model based on the self-attention mechanism. It can directly obtain global information and more easily capture the long-term dependencies before and after long sequences. At the same time, it can perform parallel calculations of multiple attention heads to alleviate the high complexity of attention calculations.
[0067] Multi-head self-attention: is a self-attention mechanism widely used in deep learning, and is often used to process sequence data, such as text, speech, and time series data. In multi-head self-attention, the input sequence is first converted into a sequence of three vectors: query, key, and value. Then, by calculating the similarity between the query vector and the key vector, the weight distribution of each query vector for all key vectors is obtained, and then the weight distribution is weighted averaged with the value vector to obtain the output representation of each query vector. Multi-head attention repeats this process multiple times, each time using different query, key, and value projection matrices to generate different attention subspaces, and finally splicing the outputs of all subspaces together to form the final output. Through multi-head self-attention, the model can learn the relationship between each position in the input sequence, so as to better capture its intrinsic structure and semantic information. It is widely used in fields such as natural language processing and speech recognition.
[0068] Residual connection: refers to the sum of the output of the shallow layer and the output of the deep layer as the input of the next stage. It allows the network to transmit information in a cross-layer manner and is used to alleviate the gradient vanishing problem caused by increasing the depth of deep neural networks.
[0069] Feedforward neural network (FNN), also known as feedforward network, is a type of artificial neural network. Feedforward neural network adopts a unidirectional multi-layer structure, in which each layer contains a number of neurons, each neuron can receive signals from the neurons in the previous layer and generate outputs to the next layer.
[0070] The following is a brief introduction to the design concept of the embodiment of the present application:
[0071] When people consume content such as videos or pictures, they usually click to browse first, and then perform operations such as likes and comments. In other words, the user's click behavior on the content and other types of interactive behaviors have a time sequence. The user's click behavior can be distinguished based on whether the user watches the content in its entirety and performs subsequent operations such as likes.
[0072] Taking video content as an example, after a user clicks once, he enters the video playback page, further plays the video in full and likes it, and follows the author at the same time. The semantic features corresponding to this type of click behavior are significantly different from the semantic features corresponding to the click behavior of exiting the video immediately after playing for less than one second. When modeling multiple targets, the semantic extraction of these two types of click behaviors is deeply coupled with the semantic extraction of other targets such as likes and comments.
[0073] However, in existing multi-objective modeling methods, the target prediction task for click behavior is often only indirectly associated with other target prediction tasks through the weight allocation corresponding to the gate in the MoE model, which makes it difficult to model the coupling between multi-objective semantics.
[0074] For example, the gating of the MoE model often exhibits polarization in practical applications (i.e., the MoE model only uses the high-order output of a single expert network). Usually, the target prediction task corresponding to the click behavior has relatively more samples, and the demand for sample migration from other target prediction tasks is lower, which makes it more prone to polarization. As a result, the target prediction task corresponding to the click behavior fails to effectively link with other target prediction tasks in the existing multi-target model.
[0075] In addition, the MoE model is prone to gate polarization when the sample size is insufficient and there are significant differences between a single target prediction task and other target prediction tasks. Therefore, since it is impossible to directly and effectively model the interdependence between the various target prediction tasks, the prediction results corresponding to the existing multi-target prediction methods are often of low accuracy.
[0076] In view of this, in an embodiment of the present application, a multi-objective prediction method, apparatus, device and storage medium for content are proposed.
[0077] In an embodiment of the present application, a multi-target prediction model is provided for obtaining initial features of content to be predicted, the initial features at least characterizing the content category and multimodal characteristics to which the content to be predicted belongs; for multiple target prediction tasks, the following operations are respectively performed: based on the element weights of each initial feature element contained in the initial features under a target prediction task, the initial features are updated to obtain an intermediate feature, and an intermediate feature contains local reference information required to perform a target prediction task; for the obtained multiple intermediate features, the following operations are respectively performed: using an attention mechanism, based on the importance of each intermediate feature relative to an intermediate feature, an intermediate feature is updated to obtain a target feature, and a target feature contains global reference information required to perform a corresponding target prediction task; based on the obtained multiple target features, corresponding multi-target prediction results are respectively obtained.
[0078] In this way, multi-target modeling is regarded as sequence modeling, and the attention mechanism is used to deeply extract the dependencies between multi-target semantics, so that the feature processing of a single target prediction task and other target prediction tasks can be effectively linked. While achieving multi-target prediction, the accuracy of each target prediction result is improved, and the modeling complexity and prediction efficiency are reduced.
[0079] In the embodiments of the present application, part of the multi-objective prediction model involves artificial intelligence (AI) and machine learning technology, and is designed based on speech technology, natural language processing technology and machine learning (ML) in artificial intelligence.
[0080] Artificial intelligence is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence.
[0081] Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, so that the machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology mainly includes computer vision technology, natural language processing technology, and machine learning / deep learning. With the research and progress of artificial intelligence technology, artificial intelligence has been studied and applied in many fields, such as common smart homes, smart customer service, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, robots, smart medical care, etc. It is believed that with the development of technology, artificial intelligence will be applied in more fields and play an increasingly important role.
[0082] Machine learning is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Compared with data mining, which seeks mutual characteristics between big data, machine learning focuses more on algorithm design, allowing computers to automatically "learn" rules from data and use the rules to predict unknown data.
[0083] Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning and other technologies. Reinforcement learning (RL), also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning. It is used to describe and solve the problem of how an agent can maximize its rewards or achieve specific goals through learning strategies during its interaction with the environment.
[0084] The preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application. In addition, the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.
[0085] See also Figure 1 As shown, it is a schematic diagram of an application scenario of an embodiment of the present application.
[0086] The application scenario includes a terminal device 110 and a server 120 , and the terminal device 110 and the server 120 can communicate with each other through a communication network.
[0087] In an optional implementation, the communication network may be a wired network or a wireless network. Therefore, the terminal device 110 and the server 120 may be directly or indirectly connected via a wired or wireless communication. For example, the terminal device 110 may be indirectly connected to the server 120 via a wireless access point, or the terminal device 110 may be directly connected to the server 120 via the Internet, and this application does not limit this.
[0088] In the embodiment of the present application, the terminal device 110 includes but is not limited to mobile phones, tablet computers, laptop computers, desktop computers, e-book readers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals and other devices; various clients may be installed on the terminal device, and the client may be an application (such as a browser, game software, etc.), or a web page, a small program, etc.;
[0089] The server 120 is a background server corresponding to the client installed in the terminal device 110. The server 120 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.
[0090] It should be noted that the training method of the multi-objective prediction model in the embodiment of the present application can be deployed in a computing device, which can be a server or a terminal device, wherein the server can be Figure 1 The server 120 shown in FIG. 1 may also be Figure 1 Other servers other than the server 120 shown in FIG. 1 ; the terminal device may be Figure 1 The terminal device 110 shown in FIG. 1 may also be Figure 1 Other terminal devices other than the terminal device 110 shown in FIG. That is, the method can be executed by a server or a terminal device alone, or by a server and a terminal device together.
[0091] When the terminal device executes alone, for example, the terminal device may obtain the content to be predicted, and perform multi-objective prediction on the content to be predicted based on the trained multi-objective prediction model to obtain the multi-objective prediction result of the content to be predicted.
[0092] When the server executes alone, for example, the server may obtain the content to be predicted, and perform multi-objective prediction on the content to be predicted based on the trained multi-objective prediction model to obtain the multi-objective prediction result of the content to be predicted.
[0093] When the server and the terminal device execute together, for example, the terminal device can obtain the content to be predicted; then, the terminal device transmits the obtained content to be predicted to the server, and the server performs multi-objective prediction on the content to be predicted based on the trained multi-objective prediction model to obtain the multi-objective prediction result of the content to be predicted.
[0094] It should be noted that, in the following, the example of execution by a server alone is mainly used for illustration, and no specific limitation is made here.
[0095] Figure 1 The figure is only an example. In fact, the number of terminal devices 110 and servers 120 is not limited and is not specifically limited in the embodiments of the present application.
[0096] In the embodiment of the present application, when there are multiple servers 120, the multiple servers 120 can form a blockchain, and the server 120 is a node on the blockchain; the multi-target prediction method disclosed in the embodiment of the present application, the processing method of the multi-target prediction involved therein, etc. can be saved on the blockchain.
[0097] In the embodiment of the present application, before performing multi-objective prediction on the content to be predicted, the server first iteratively trains the multi-objective prediction model to be trained based on a training sample set constructed from distributed historical content to obtain a multi-objective prediction model.
[0098] See also Figure 2 As shown, it is a flow chart of a training method for a multi-objective prediction model provided in an embodiment of the present application. Figure 2 , and explain in detail the specific operations performed:
[0099] Step S201: construct a training sample set based on the distributed historical content, and extract sample initial features corresponding to each sample content in the training sample set.
[0100] Specifically, in the embodiment of the present application, the training sample set includes a plurality of sample contents, each of which refers to the video content or graphic content itself that has been distributed and is associated with user interaction behavior data.
[0101] When constructing a training sample set, taking video content as an example, multiple video contents are distributed to the video playback platform, and after a period of time, the interactive behavior data such as the number of clicks, playbacks, likes, comments, etc. obtained by each user for this part of the video content are counted. Based on this part of the video content, a training sample set is obtained, and the corresponding sample initial features are extracted from the original data corresponding to the video content.
[0102] Among them, the original data contains the original reference information required to perform various target prediction tasks, including at least the content category to which the sample content belongs and the multimodal characteristic data corresponding to the sample content. The content category can be, for example, technology, food, sports, etc. The multimodal characteristic data refers to the audio modal data, visual modal data and text modal data corresponding to the sample content.
[0103] Optionally, the original data may also include author information corresponding to the sample content and a timestamp corresponding to the time when the sample content was distributed, such as the current day of the week, the current time period, etc. Since different people have different browsing preferences for video content in different time periods, the distribution time of the video content will also affect the probability of the video content being clicked and played.
[0104] After the above original data is processed by Embedding and Pooling, N feature row vectors are obtained. The nth feature element is recorded as xn , dimension d n , concatenate these N feature row vectors to obtain the final input sample initial features, namely:
[0105] X=[…,x n ,…]
[0106] Among them, [...] represents vector concatenation, and the dimension of X is:
[0107]
[0108] Furthermore, in an embodiment of the present application, based on the interactive behavior data generated by each user for each sample content, a plurality of sample labels corresponding to the target prediction tasks are set for each sample content.
[0109] For example, in the training samples corresponding to the target prediction task of click behavior, if a sample content has been clicked, its sample label is 1, otherwise it is 0; in the training samples corresponding to the target prediction task of play behavior, if a sample content is played or browsed for more than 50% of the time, its sample label is 1, otherwise it is 0...
[0110] Step S202: Based on the initial features of each sample and multiple sample labels corresponding to the content of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain a multi-objective prediction model.
[0111] In an embodiment of the present application, the server selects a batch of training samples required in the training process according to actual needs, or randomly selects a training sample, and performs an iterative training on the multi-objective estimation model to be trained based on the initial sample features of the read training samples to obtain the training results of the current stage.
[0112] For details, see Figure 3 As shown, it is a schematic diagram of the process of an iterative training in the embodiment of the present application. Figure 3 , and explain the specific execution steps in detail:
[0113] Step S2021: For each target prediction task in the multiple target prediction tasks, the following operations are performed respectively: based on the element weights of each sample initial feature element contained in a sample initial feature under the target prediction task, the sample initial feature is updated to obtain the sample intermediate feature of the target prediction task.
[0114] In the embodiment of the present application, assuming that there are T target prediction tasks, T linear transformations are performed on the initial features of the samples to obtain T sample intermediate features. The vector sequence composed of the T sample intermediate features can be expressed as:
[0115] S=(…,S t ,…)
[0116] in:
[0117] S t =W t *X T +B t
[0118] Among them, the parameter matrix W t Each element in represents the element weight of the initial feature element of each sample under the tth target prediction task, and its dimension is d s *d x , the bias parameter vector B of the input layer of the t-th target prediction task t The dimension is d s , take d s <d x , the overall amount of calculation can be controlled.
[0119] Step S2022: For each of the multiple sample intermediate features obtained, perform the following operations respectively: adopt the attention mechanism to update the sample intermediate feature based on the importance of each sample intermediate feature relative to the sample intermediate feature, and obtain the sample target feature of the sample intermediate feature.
[0120] Specifically, when processing an intermediate feature of a sample, an attention mechanism is used to update the intermediate feature of all samples by utilizing the importance of the intermediate features of the sample relative to the intermediate feature of the sample, which also includes the importance of the intermediate feature of the sample relative to itself.
[0121] In the embodiment of the present application, the server takes a sequence S consisting of multiple sample intermediate features as an input sequence, processes the input sequence through a Transformer structure, and obtains multiple sample target features.
[0122] The Transformer structure includes: a multi-head attention network, a feed-forward neural network, and a normalization network. Each attention head in the multi-head attention network performs attention calculation on the intermediate features of each sample, and the calculation formula is:
[0123]
[0124] In the above formula, the multi-head attention network performs matrix multiplication based on the transpose of the query matrix Q corresponding to each sample intermediate feature and the corresponding key matrix K to obtain multiple correlation matrices, and uses the softmax function to normalize each correlation matrix. Then, the normalized result is multiplied by the corresponding value matrix V to obtain the correlation weight between any two sample intermediate features.
[0125] Specifically, assuming that there are H heads in the Transformer's multi-head attention network, the attention head h is used to predict the sample intermediate feature S corresponding to the target prediction task t t Taking the process of attention calculation as an example, we first perform three independent linear transformations on st to obtain:
[0126]
[0127] in, and The dimension is d QK ×d s The parameter matrix of The dimension is d v ×d s The parameter matrix of d v =d s / H,d QK The value of is generally the same as d v Similarly, under the attention head h, S t The correlation weight distribution between the intermediate features of the samples corresponding to the remaining target prediction tasks is calculated using the following formula:
[0128]
[0129] The key matrix of the sample intermediate features corresponding to the i-th target prediction task.
[0130] Further, Multiply and sum the value matrix corresponding to the intermediate features of each sample to obtain the output from the perspective of the attention head h, which can be specifically expressed as:
[0131]
[0132] S t The output results of H attention heads are vectorized and concatenated to obtain the S-based t Updated sample target features Specifically expressed as:
[0133]
[0134] Step S2023: Based on the obtained multiple sample target features, corresponding multi-target prediction results are obtained respectively.
[0135] Specifically, in the embodiment of the present application, the sample target feature corresponding to the target prediction task t is For example, based on the sample target feature When obtaining the prediction result of the target prediction task t, the input and output of the multi-head attention network are normalized based on the normalization network in the Transformer structure to obtain the first normalized result, and the first normalized result is nonlinearly mapped based on the forward neural network to improve the generalization ability of the multi-target prediction model. The input and output of the forward neural network are then normalized to obtain the second normalized result, and the output sequence after Transformer processing is obtained. The output sequence is dot-producted with the semantic row vector randomly initialized corresponding to each target prediction task to obtain the prediction result of the target prediction task t.
[0136] It should be noted that the multi-head attention network and forward neural network in the Transformer structure use normalized network connections to form a layer unit. In actual application, multiple layer units can be repeatedly stacked to deeply extract the dependencies between the target prediction tasks and improve the effect of the multi-target prediction model.
[0137] For example, see Figure 4 As shown, it is a schematic diagram of the multi-target prediction model training in the embodiment of the present application. The original data such as content category, multimodal characteristics, author information, distribution timestamp, etc. corresponding to the sample content are input into the EmbeddingLayer layer, and the initial features of the sample are obtained after vector concatenation through the Concat layer. Based on the number of target prediction tasks, the initial features of the sample are subjected to multiple linear transformations to obtain the intermediate features of the sample. The intermediate features of the sample are then used as the input of the Transformer structure. After the attention calculation in the Transformer structure, the corresponding sample target features are obtained. Finally, each sample target feature is dot-producted with a plurality of preset semantic row vectors to obtain the prediction results corresponding to each target prediction task. Figure 4 Only three target prediction tasks are shown as examples. In actual application, the specific number of target prediction tasks can be adjusted as needed, and this application does not impose any restrictions on this.
[0138] Among them, see Figure 5 As shown, it is a schematic diagram of the internal structure of the Transformer structure in the embodiment of the present application. Att Score represents the calculation of the correlation weight based on the query matrix Q and the key matrix K, and Att Aggregation represents the weighted summation based on the correlation weight and the value matrix. In this way, through the attention mechanism of Transformer, various features before and after the content distribution can be effectively processed, and the chronological correlation between clicks, plays and other behaviors can be explored to improve the accuracy of multi-target prediction.
[0139] Step S2024: Based on the loss value between the multi-objective prediction result and each sample label corresponding to the sample content, the multi-objective prediction model to be trained is adjusted.
[0140] In the embodiment of the present application, it is assumed that under the target prediction task t, the sample label corresponding to the sample content is q t , the prediction result obtained by the multi-objective prediction model is p t , then the loss function of the target prediction task t can be calculated as follows:
[0141] L t =-q t log(p t )-(1-q t )log(1-p t )
[0142] For T target prediction tasks, the final loss calculation of the sample content can be calculated in an accumulated manner, namely:
[0143]
[0144] Based on the accumulated loss value, the relevant parameters of the multi-objective prediction model are adjusted until the loss value between the final prediction result and the sample label meets the preset convergence condition, then the training is stopped to obtain the trained multi-objective prediction model.
[0145] The following is a description of the process of applying the trained target click rate prediction model in conjunction with the accompanying drawings:
[0146] See also Figure 6 As shown, it is a flow chart of a method for performing multi-target prediction on the content to be predicted in an embodiment of the present application. Figure 6 , and explain the specific execution steps in detail:
[0147] Step S601: Acquire initial features of the content to be predicted.
[0148] The initial features at least represent the content category and multimodal characteristics of the content to be predicted.
[0149] In an embodiment of the present application, a multi-objective prediction model is applied to a content recommendation system. Before recommending and distributing the content, the server uses the multi-objective prediction model to perform multi-objective prediction on the content to determine decisions such as the recommendation platform, recommended objects, and distribution time period of the content based on the multi-objective prediction results.
[0150] Specifically, the server obtains the content category, author information, and original data such as visual modality, audio modality, and text modality of the content to be predicted, performs Embedding and Pooling operations on it to obtain multiple feature row vectors, and concatenates the feature row vectors to obtain the final initial features.
[0151] See also Figure 7 As shown, it is a schematic diagram of the scene for obtaining the initial features in an embodiment of the present application. When obtaining the corresponding feature row vector through the original data, the existing pre-trained model can be used for feature extraction. For example, the original data of the visual modality is input into the Video Swin Transformer network to obtain the visual feature row vector, the original data of the audio modality is input into the wavlm-base-plus network to obtain the audio feature row vector, and the original data of the text modality is input into the natural language processing model (Bidirectional Encoder Representations from Transformers, BERT) to obtain the text feature row vector. The present application does not impose any restrictions on this.
[0152] Step S602: for each target prediction task in the multiple target prediction tasks, perform the following operations respectively: based on the element weights of each initial feature element contained in the initial feature under the target prediction task, update the initial feature to obtain the intermediate feature of the target prediction task.
[0153] Among them, the intermediate features contain the local reference information required to perform the target prediction task.
[0154] Specifically, in an embodiment of the present application, after obtaining the initial features, multiple linear transformations are first performed on the initial features based on the number of target prediction tasks to obtain multiple intermediate features, wherein the linear transformations corresponding to different target prediction tasks use different weight parameters, so that the multiple intermediate features obtained after the linear transformation respectively contain the core information required to perform the corresponding target prediction tasks.
[0155] For example, see Figure 8 As shown, it is a schematic diagram of a scenario in which the initial features are linearly transformed in an embodiment of the present application, which includes three target prediction tasks, namely, click-through rate, play rate and like rate. The initial feature X is subjected to the first linear transformation based on the parameter matrix W1 and the bias parameter B1 to obtain the intermediate feature S1. The specific transformation method is S1 = W1*X T +B1, the intermediate feature S1 contains the local reference information required to perform the click rate prediction task; based on the parameter matrix W2 and the bias parameter B2, the initial feature X is transformed into the second linear transformation to obtain the intermediate feature S2. The specific transformation method is S2 = W2*X T+B2, the intermediate feature S2 contains the local reference information required to perform the play rate prediction task; based on the parameter matrix W3 and the bias parameter B3, the initial feature X is transformed into the third linear transformation to obtain the intermediate feature S2. The specific transformation method is S3 = W3*X T +B3, the intermediate feature S3 contains the local reference information required to perform the like rate prediction task.
[0156] It should be noted that the target prediction tasks in actual applications are not limited to click-through rate, play rate and like rate, but can also include follow-up rate, collection rate and fast-swipe rate, etc. At the same time, according to actual needs, some target prediction tasks can also be subdivided. For example, the play rate prediction task can be divided into: the probability of play time exceeding 10%, the probability of play time exceeding 50%, the probability of play time exceeding 70%, etc.
[0157] Step S603: For each of the multiple intermediate features obtained, perform the following operations respectively: adopt the attention mechanism, update the intermediate feature based on the importance of each intermediate feature relative to the intermediate feature, and obtain the target feature of the intermediate feature.
[0158] Among them, an object feature contains the global reference information required to perform the corresponding object prediction task.
[0159] Specifically, in an embodiment of the present application, multiple intermediate features are input into the Transformer structure, and each intermediate feature is updated through the multi-head attention network in the Transformer structure to obtain multiple target features.
[0160] When processing an intermediate feature, an attention mechanism is used to update it by utilizing the importance of all intermediate features relative to the intermediate feature, which also includes the importance of the intermediate feature relative to itself.
[0161] See also Fig. 9 As shown, it is a flow chart of a method for obtaining target features provided in an embodiment of the present application. Fig. 9 , and explain the specific execution steps in detail:
[0162] Specifically, in the embodiment of the present application, when obtaining the target feature corresponding to the intermediate feature, each attention network in the multiple attention networks is used to perform the following operations on the intermediate feature:
[0163] Step S6031: Based on the attention weight matrix corresponding to each intermediate feature in the attention network, obtain the importance of each intermediate feature relative to the intermediate feature.
[0164] In an embodiment of the present application, through the training process of the multi-target prediction model, the specific parameter values of the attention weight matrix corresponding to each intermediate feature in an attention network can be obtained, and then according to each intermediate feature and its corresponding attention weight matrix, the importance of each intermediate feature relative to an intermediate feature can be obtained.
[0165] Among them, the attention weight matrix includes the query weight matrix W Q , key weight matrix W K Sum value weight matrix W V .
[0166] Each attention head in the multi-head attention network has a separate W Q , W K and W V ,In each attention head, the server first performs three linear transformations on each intermediate feature based on the ,corresponding attention weight matrix, and then obtains the importance of each intermediate feature relative to another ,intermediate feature in turn based on the results of the linear transformation.
[0167] See also Fig.10 As shown, it is a schematic diagram of the process of obtaining the importance level in the embodiment of the present application. Fig.10 , and explain the specific execution steps in detail:
[0168] Step S6031-1: Based on the query weight matrix and key weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain the corresponding query matrix and key matrix:.
[0169] Each element in a query weight matrix represents the importance of a corresponding intermediate feature element, and each element in a key weight matrix represents the importance of at least one other intermediate feature element related to the corresponding intermediate feature element.
[0170] In the embodiment of the present application, matrix multiplication is performed on each intermediate feature with the corresponding query weight matrix and key weight matrix to generate a corresponding query matrix and key matrix, which is equivalent to realizing two linear transformations on each intermediate feature.
[0171] For example, assuming there are 3 attention heads, in attention head 1, the corresponding query weight matrix is The key weight matrix is Then the query matrix corresponding to the intermediate feature S1 is The bond matrix is The processing methods for the remaining intermediate features are the same and will not be repeated here.
[0172] Step S6031 - 2: Based on the query matrix of the intermediate features and the key matrix of each intermediate feature, multiple association matrices are obtained.
[0173] Among them, multiple correlation matrices respectively represent the importance of each intermediate feature relative to the intermediate features.
[0174] Specifically, in an embodiment of the present application, a query matrix of an intermediate feature is used to perform matrix multiplication with the key matrices of all intermediate features to obtain multiple association matrices, including an association matrix obtained by matrix multiplication of the query matrix of the intermediate feature with its own key matrix.
[0175] For example, see Fig.11 As shown, it is a schematic diagram of a single attention head updating an intermediate feature in an embodiment of the present application, and the query matrix corresponding to the intermediate feature S1 is updated. and key matrix Multiply the transpose of to obtain the association matrix A1, that is, The association matrix A1 represents the importance of the intermediate feature S1 relative to itself;
[0176] The query matrix corresponding to the intermediate feature S1 The bond matrix corresponding to the intermediate feature S2 The transpose of is multiplied to obtain the association matrix A2, that is, The association matrix A2 represents the importance of the intermediate feature S2 relative to the intermediate feature S1;
[0177] The query matrix corresponding to the intermediate feature S1 The key matrix corresponding to the intermediate feature S3 The transpose of is multiplied to obtain the association matrix A3, that is, The association matrix A3 represents the importance of the intermediate feature S3 relative to the intermediate feature S1.
[0178] Step S6032: Based on the obtained importance levels, the intermediate features are updated to obtain attention features.
[0179] For details, see Fig.12 As shown in the figure, it is a schematic diagram of the process of obtaining attention features in the embodiment of the present application. Fig.12 , and explain the specific execution steps in detail:
[0180] Step S6032-1: Based on the value weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding value matrix.
[0181] Each element in a value matrix represents the importance of the correlation between the corresponding intermediate feature element and at least one other related intermediate feature element.
[0182] In the embodiment of the present application, under each attention head, each intermediate feature is also compared with the value weight matrix W included in the attention weight matrix V Performing matrix multiplication to generate the corresponding value matrix is equivalent to realizing the linear transformation of each intermediate feature.
[0183] For example, suppose in attention head 1, its corresponding value weight matrix is Then the value matrix corresponding to the intermediate feature S1 is The processing methods for the remaining intermediate features are the same and will not be repeated here.
[0184] Step S6032-2: Obtain attention features based on multiple correlation matrices of intermediate features and value matrices corresponding to each intermediate feature.
[0185] In the embodiments of the present application, the softmax function is used to obtain normalized correlation matrices, and matrix multiplication is performed on each normalized correlation matrix with the value matrix corresponding to each intermediate feature, and the multiplication results are summed to obtain an attention feature.
[0186] Among them, when the correlation matrix corresponding to an intermediate feature is multiplied by the value matrix corresponding to each intermediate feature, it also includes the matrix multiplication operation of the correlation matrix corresponding to the intermediate feature and the value matrix corresponding to itself.
[0187] For example, see Fig.11 As shown, each correlation matrix is normalized, that is, Among them, i is 1 to 3. In each normalized correlation matrix, the sum of the values of each row element is 1, which represents the attention score of each element in the corresponding value matrix. Then, the normalized correlation matrix B i The value matrices V corresponding to the intermediate features S1, S2, and S3 respectively i 1 Multiply them together and sum the multiplication results to obtain an attention feature, namely:
[0188]
[0189] Furthermore, in an embodiment of the present application, the attention features obtained through multiple attention networks are spliced to obtain the target features of the intermediate features.
[0190] For example, the attention features output by each attention head after processing the intermediate feature S1 are concatenated to obtain the target feature corresponding to the intermediate feature S1.
[0191] See also Fig.13As shown, it is a schematic diagram of obtaining the target feature in the embodiment of the present application. The attention feature obtained after the attention head 1 processes the intermediate feature S1 is The attention feature obtained after attention head 2 processes the intermediate feature S1 is The attention feature obtained after attention head 3 processes the intermediate feature S1 is Then the target feature corresponding to the updated intermediate feature S1 is
[0192] Similarly, the corresponding target features of the intermediate features S2 and S3 after updating are and
[0193] Due to the use of a multi-head attention network, the dependencies between the intermediate features are deeply extracted, so each target feature integrates the global reference information required to perform the corresponding target prediction task. For example, when consuming video content, users usually need to watch for a certain length of time before they can like it. Therefore, when performing the like rate target prediction task, the information that needs to be referenced is not only whether the content itself is attractive enough, but also whether the content's playback rate can meet certain conditions. Therefore, it can be understood that the target feature corresponding to the like rate prediction task also includes the intermediate features corresponding to the like rate prediction task and the intermediate features corresponding to the playback rate prediction task. The common reference information of the two, thereby making the final multi-target prediction result more accurate.
[0194] Step S604: Based on the obtained multiple target features, corresponding multi-target prediction results are obtained respectively.
[0195] Specifically, in an embodiment of the present application, after obtaining multiple target features based on the multi-head attention network, the layer units composed of the forward neural network and the normalization network in the Transformer structure will also perform corresponding processing on each target feature, so that the multi-target prediction model has better performance and computational efficiency when performing multi-target prediction tasks.
[0196] See also Fig.14 As shown, it is a flow chart of obtaining the target prediction result in the embodiment of the present application. Fig.14 , and explain the specific execution steps in detail:
[0197] For each target feature, the server performs the following operations:
[0198] Step S6041: based on the intermediate feature corresponding to the target feature, correct the target feature to obtain a corrected target feature.
[0199] Specifically, in an embodiment of the present application, the input and output of the multi-head attention network are summed to obtain a residual connection, and the sum result is normalized to perform a first correction on the target feature.
[0200] For example, see Fig.15 As shown, it is a residual connection schematic diagram provided in an embodiment of the present application, X represents the network input, F(X) represents the network output, and the residual connection is F(X)+X. Taking the processing of the target feature Y1 as an example, since the input of the multi-head attention network is the intermediate feature S1, its residual connection is Y1+S1.
[0201] Furthermore, the residual connection is normalized using the layer normalization (LN) method to obtain the first correction result Y1', namely:
[0202] Y1′=LN(Y1+S1)
[0203] Among them, the calculation method of layer normalization is:
[0204]
[0205] α and β are two trainable parameters, ε is a small smoothing term, μ t and σ t The mean and standard deviation calculated for all elements in x, that is,
[0206]
[0207] After obtaining the first correction result Y', Y' is sent to the feed-forward network (FFN) for processing, namely:
[0208] FFN(Y1′)=W2·[ReLU(W1*Y1′+b1)]+b2
[0209] Among them, W1 is the dimension d FF ×d S The parameter matrix of b1 is of dimension d FF The bias parameter vector of W2 is of dimension d S ×d FF The parameter matrix of b2 is of dimension d S The bias parameter vector.
[0210] Then, the input and output of the forward neural network are residually connected and layer normalized to obtain the second correction result Y1", that is:
[0211] Y1″=LN(FFN(Y1′)+Y1′)
[0212] In the embodiment of the present application, there is only one layer unit, so the second correction result Y1" is the correction target feature corresponding to Y1, and the processing method of the remaining target features is the same, which will not be repeated here.
[0213] In the above process, the purpose of residual connection is to solve the problems of gradient explosion and gradient vanishing caused by increasing network depth in multi-target prediction models. By transferring part of the previous layer information to the next layer without difference, the model performance can be effectively improved. Layer normalization normalizes the output signal to a distribution with a mean of 0 and a variance of 1 through normalization operations, which can accelerate the training process of the model, make it converge quickly, and ensure the stability of the model.
[0214] Step S6042: Based on the modified target features and their corresponding prediction weight vectors under the current target prediction task, the corresponding target prediction results are obtained.
[0215] Among them, each element in the prediction weight vector represents: the importance of each target feature element in the modified target feature.
[0216] Specifically, in the embodiment of the present application, for the click rate prediction task corresponding to the modified target feature vector Y1″, its final estimated probability value can be calculated in the following way:
[0217] p 1 =sigmoid(O1*Y1″)
[0218] Among them, O1 is the semantic row vector corresponding to the click rate prediction task, that is, the prediction weight vector in the embodiment of the present application. By doing the dot product of O1 and Y1″, the dot product result is mapped to between 0 and 1 using the sigmoid function, and the final click rate estimated probability value p is obtained. 1 .
[0219] Furthermore, after obtaining the multi-objective prediction results, the content can be sorted according to the multi-objective prediction results, thereby completing the recommendation and distribution operations, reducing the cost of manual screening and the resource consumption of the recommendation system.
[0220] Based on the same technical concept, see Fig.16 As shown, the embodiment of the present application further provides a multi-objective prediction device 1600 for content, the device comprising:
[0221] An acquisition module 1601 is used to acquire initial features of the content to be predicted, where the initial features at least represent the content category and multimodal characteristics of the content to be predicted;
[0222] The first updating module 1602 is used to perform the following operations for each target prediction task in the multiple target prediction tasks: based on the element weights of each initial feature element contained in the initial feature under the target prediction task, the initial feature is updated to obtain an intermediate feature of the target prediction task, where the intermediate feature contains local reference information required to perform the target prediction task;
[0223] The second updating module 1603 is used to perform the following operations for each of the obtained multiple intermediate features: using an attention mechanism, based on the importance of each intermediate feature relative to the intermediate feature, update the intermediate feature to obtain a target feature of the intermediate feature, where the target feature contains global reference information required to perform a corresponding target prediction task;
[0224] The prediction module 1604 is used to obtain corresponding multi-target prediction results based on the obtained multiple target features.
[0225] In a possible implementation, an attention mechanism is used to update the intermediate features based on the importance of each intermediate feature relative to the intermediate features. When the target feature of the intermediate feature is obtained, the second updating module 1603 is used to:
[0226] Each of the multiple attention networks is used to perform the following operations on the intermediate features:
[0227] Based on the attention weight matrix corresponding to each intermediate feature in the attention network, the importance of each intermediate feature relative to the intermediate feature is obtained respectively;
[0228] Based on the obtained importance levels, the intermediate features are updated to obtain attention features;
[0229] The attention features obtained through multiple attention networks are concatenated to obtain the target features of the intermediate features.
[0230] In a possible implementation, the attention weight matrix includes at least a query weight matrix and a key weight matrix; based on the attention weight matrix corresponding to each intermediate feature in the attention network, when obtaining the importance of each intermediate feature relative to the intermediate feature, the second updating module 1603 is used to:
[0231] Based on the query weight matrix and key weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding query matrix and key matrix, wherein each element in the query weight matrix represents: the importance of the corresponding intermediate feature element, and each element in the key weight matrix represents: the importance of at least one other intermediate feature element related to the corresponding intermediate feature element;
[0232] Based on the query matrix of the intermediate features and the key matrix of each intermediate feature, a plurality of association matrices are obtained, wherein the plurality of association matrices respectively represent the importance of each intermediate feature relative to the intermediate features.
[0233] In a possible implementation, the attention weight matrix also includes a value weight matrix; based on the obtained importance levels, the intermediate features are updated. When the attention features are obtained, the second updating module 1603 is used to:
[0234] Based on the value weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding value matrix, wherein each element in the value matrix represents: the importance of the correlation between the corresponding intermediate feature element and at least one other related intermediate feature element;
[0235] Attention features are obtained based on multiple correlation matrices of intermediate features and value matrices corresponding to each intermediate feature.
[0236] In a possible implementation, when corresponding multi-objective prediction results are obtained based on the obtained multiple object features, the prediction module 1604 is used to:
[0237] For each of the multiple target features, perform the following operations:
[0238] Based on the intermediate feature corresponding to the target feature, the target feature is modified to obtain a modified target feature;
[0239] Based on the modified target feature and its corresponding prediction weight vector under the current target prediction task, the corresponding target prediction result is obtained, wherein each element in the prediction weight vector represents: the importance of each target feature element in the modified target feature.
[0240] In a possible implementation, the multi-objective prediction result is obtained by inputting the initial features into the multi-objective prediction model, wherein the multi-objective prediction model is obtained by training the acquisition module 1601, the first update module 1602, the second update module 1603 and the prediction module 1604 in the following manner:
[0241] A training sample set is constructed based on the distributed historical content, and the sample initial features corresponding to each sample content in the training sample set are extracted, wherein a sample label corresponding to a plurality of target prediction tasks is set for each sample content;
[0242] Based on the initial features of each sample and multiple sample labels corresponding to the content of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain a multi-objective prediction model.
[0243] In a possible implementation, based on the initial features of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain the multi-objective prediction model, wherein one iteration process includes:
[0244] For each target prediction task in the multiple target prediction tasks, the following operations are performed respectively: based on the element weights of each sample initial feature element contained in a sample initial feature under the target prediction task, the sample initial feature is updated to obtain the sample intermediate feature of the target prediction task;
[0245] For each of the obtained multiple sample intermediate features, the following operations are performed respectively: using an attention mechanism, based on the importance of each sample intermediate feature relative to the sample intermediate feature, the sample intermediate feature is updated to obtain a sample target feature of the sample intermediate feature;
[0246] Based on the obtained multiple sample target features, corresponding multi-target prediction results are obtained respectively;
[0247] Based on the loss value between the multi-objective prediction results and the sample labels corresponding to the sample contents, the parameters of the multi-objective prediction model to be trained are adjusted.
[0248] After introducing the multi-target prediction method and apparatus according to the exemplary embodiment of the present application, next, a computing device according to another exemplary embodiment of the present application is introduced.
[0249] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.
[0250] Based on the same inventive concept as the above method embodiment, the present application embodiment further provides a computing device 1700. In one embodiment, the computing device may be a server, such as Figure 1 In this embodiment, the structure of the computing device is as follows: Fig.16 As shown, it may include at least a memory 1701 , a communication module 1703 , and at least one processor 1702 .
[0251] The memory 1701 is used to store computer programs executed by the processor 1702. The memory 1701 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and programs required for running the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc.
[0252] The memory 1701 may be a volatile memory, such as a random-access memory (RAM); the memory 1701 may also be a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1701 may be any other medium that can be used to carry or store a desired computer program in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 1701 may be a combination of the above memories.
[0253] The processor 1702 may include one or more central processing units (CPU) or a digital processing unit, etc. The processor 1702 is used to implement the multi-objective prediction method described above when calling the computer program stored in the memory 1701 .
[0254] The communication module 1703 is used to communicate with terminal devices and other servers.
[0255] The specific connection medium between the memory 1701, the communication module 1703 and the processor 1702 is not limited in the embodiment of the present application. Fig.17 In the embodiment, the memory 1701 and the processor 1702 are connected via a bus 1704. The bus 1704 is connected to the processor 1702 via a bus 1704. Fig.17 The connections between the other components are described with bold lines, which are only for illustration and are not intended to be limiting. The bus 1704 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.17 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or one type of bus.
[0256] The memory 1701 stores a computer storage medium, which stores computer executable instructions for implementing the multi-objective prediction method of the content of the embodiment of the present application. The processor 1702 is used to execute the multi-objective prediction method of the content.
[0257] In another embodiment, the computing device may also be other computing devices, such as Figure 1 In this embodiment, the structure of the computing device can be as follows: Fig.18As shown, it includes: a communication component 1810, a memory 1820, a display unit 1830, a camera 1840, a sensor 1850, an audio circuit 1860, a Bluetooth module 1870, a processor 1880 and other components.
[0258] The communication component 1810 is used to communicate with the server. In some embodiments, a wireless fidelity (WiFi) module may be included. The WiFi module belongs to a short-range wireless transmission technology. The electronic device can help the object to send and receive information through the WiFi module.
[0259] The memory 1820 can be used to store software programs and data. The processor 1880 executes various functions and data processing of the terminal device 110 by running the software programs or data stored in the memory 1820. The memory 1820 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. The memory 1820 stores an operating system that enables the terminal device 110 to run. In the present application, the memory 1820 can store an operating system and various application programs, and can also store a computer program for executing the multi-objective prediction method of the embodiment of the present application.
[0260] The display unit 1830 may also be used to display information input by the object or information provided to the object and a graphical user interface (GUI) of various menus of the terminal device 110. Specifically, the display unit 1830 may include a display screen 1832 disposed on the front of the terminal device 110. The display screen 1832 may be configured in the form of a liquid crystal display, a light emitting diode, etc. The display unit 1830 may be used to display the game simulation game interface, etc. in the embodiment of the present application.
[0261] The display unit 1830 can also be used to receive input digital or character information, and generate signal input related to the object setting and function control of the terminal device 110. Specifically, the display unit 1830 may include a touch screen 1831 set on the front of the terminal device 110, which can collect touch operations of objects on or near it, such as clicking a button, dragging a scroll box, etc.
[0262] The touch screen 1831 can be covered on the display screen 1832, or the touch screen 1831 and the display screen 1832 can be integrated to realize the input and output functions of the terminal device 110. The integrated touch screen can be referred to as a touch display screen. In this application, the display unit 1830 can display the application and the corresponding operation steps.
[0263] The camera 1840 can be used to capture static images, and the subject can publish the images taken by the camera 1840 through the application. The camera 1840 can be one or more. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the processor 1880 to convert it into a digital image signal.
[0264] The terminal device may further include at least one sensor 1850, such as an acceleration sensor 1851, a distance sensor 1852, a fingerprint sensor 1853, and a temperature sensor 1854. The terminal device may also be configured with other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, a light sensor, and a motion sensor.
[0265] The audio circuit 1860, the speaker 1861, and the microphone 1862 can provide an audio interface between the object and the terminal device 110. The audio circuit 1860 can transmit the electrical signal converted from the received audio data to the speaker 1861, which is converted into a sound signal for output. The terminal device 110 can also be configured with a volume button for adjusting the volume of the sound signal. On the other hand, the microphone 1862 converts the collected sound signal into an electrical signal, which is received by the audio circuit 1860 and converted into audio data, and then the audio data is output to the communication component 1810 to be sent to, for example, another terminal device, or the audio data is output to the memory 1820 for further processing.
[0266] The Bluetooth module 1820 is used to exchange information with other Bluetooth devices having Bluetooth modules through the Bluetooth protocol. For example, the terminal device can establish a Bluetooth connection with a wearable electronic device (such as a smart watch) that also has a Bluetooth module through the Bluetooth module 1820 to exchange data.
[0267] The processor 1880 is the control center of the terminal device. It uses various interfaces and lines to connect various parts of the entire terminal. It executes various functions of the terminal device and processes data by running or executing software programs stored in the memory 1820 and calling data stored in the memory 1820. In some embodiments, the processor 1880 may include one or more processing units; the processor 1880 may also integrate an application processor and a baseband processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the baseband processor mainly processes wireless communications. It is understandable that the above-mentioned baseband processor may not be integrated into the processor 1880. In the present application, the processor 1880 can run the operating system, application programs, user interface display and touch response, as well as the multi-target prediction method of the content of the embodiment of the present application. In addition, the processor 1880 is coupled to the display unit 1830.
[0268] In addition, it should be noted that in the specific implementation of this application, object data related to multi-objective prediction of content is involved. When the above embodiments of this application are applied to specific products or technologies, it is necessary to obtain the object's permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0269] In some possible implementations, various aspects of the multi-target prediction method for content provided by the present application may also be implemented in the form of a program product, which includes a computer program. When the program product is run on a computing device, the computer program is used to enable the computing device to execute the steps of the multi-target prediction method for content according to various exemplary implementations of the present application described above in this specification.
[0270] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0271] The program product of the embodiment of the present application may adopt a portable compact disk read-only memory (CD-ROM) and include a computer program, and can be run on an electronic device. However, the program product of the present application is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which can be used by or in combination with a command execution system, apparatus, or device.
[0272] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, wherein a readable computer program is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A readable signal medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with a command execution system, apparatus, or device.
[0273] The computer program embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
[0274] The computer program for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The computer program may be executed entirely on the user computing device, partially on the user computing device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on the remote computing device. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0275] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.
[0276] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.
[0277] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A multi-objective prediction method for content, characterized in that: include: Acquire initial features of the content to be predicted, where the initial features at least characterize the content category and multimodal characteristics to which the content to be predicted belongs; For each target prediction task in the multiple target prediction tasks, the following operations are respectively performed: based on the element weights of each initial feature element contained in the initial feature under the target prediction task, the initial feature is updated to obtain an intermediate feature of the target prediction task, wherein the intermediate feature contains local reference information required to perform the target prediction task; For each of the obtained multiple intermediate features, the following operations are performed respectively: using an attention mechanism, based on the importance of each intermediate feature relative to the intermediate feature, the intermediate feature is updated to obtain a target feature of the intermediate feature, where the target feature contains global reference information required to perform a corresponding target prediction task; Based on the obtained multiple target features, corresponding multi-target prediction results are obtained respectively.
2. The method according to claim 1, characterized in that The attention mechanism is adopted to update the intermediate feature based on the importance of each intermediate feature relative to the intermediate feature to obtain the target feature of the intermediate feature, including: Each of the multiple attention networks is used to perform the following operations on the intermediate features: Based on the attention weight matrix corresponding to each intermediate feature in the attention network, respectively obtain the importance of each intermediate feature relative to the intermediate feature; Based on the obtained importance levels, the intermediate features are updated to obtain attention features; the attention features obtained through the multiple attention networks are concatenated to obtain target features of the intermediate features.
3. The method according to claim 2, characterized in that The attention weight matrix includes at least a query weight matrix and a key weight matrix; then the importance of each intermediate feature relative to the intermediate feature is obtained based on the attention weight matrix corresponding to each intermediate feature in the attention network, including: Based on the query weight matrix and key weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding query matrix and key matrix, wherein each element in the query weight matrix represents: the importance of the corresponding intermediate feature element, and each element in the key weight matrix represents: the importance of at least one other intermediate feature element related to the corresponding intermediate feature element; Based on the query matrix of the intermediate features and the key matrix of each intermediate feature, a plurality of association matrices are obtained, wherein the plurality of association matrices respectively represent the importance of each intermediate feature relative to the intermediate feature.
4. The method according to claim 3, characterized in that The attention weight matrix also includes a value weight matrix; then the intermediate features are updated based on the obtained importance levels to obtain attention features, including: Based on the value weight matrix corresponding to each intermediate feature, each intermediate feature is linearly transformed to obtain a corresponding value matrix, wherein each element in the value matrix represents: the importance of the correlation between the corresponding intermediate feature element and at least one other related intermediate feature element; The attention feature is obtained based on multiple correlation matrices of the intermediate features and value matrices corresponding to each intermediate feature.
5. The method according to claim 1, characterized in that The method of obtaining corresponding multi-target prediction results based on the obtained multiple target features includes: For each target feature of the multiple target features, the following operations are performed respectively: Based on the intermediate feature corresponding to the target feature, the target feature is modified to obtain a modified target feature; Based on the modified target feature and its corresponding prediction weight vector under the current target prediction task, a corresponding target prediction result is obtained, wherein each element in the prediction weight vector represents: the importance of each target feature element in the modified target feature.
6. The method according to any one of claims 1 to 5, characterized in that: The multi-objective prediction result is obtained by inputting the initial features into a multi-objective prediction model, wherein the multi-objective prediction model is trained in the following manner: Building a training sample set based on the distributed historical content, and extracting sample initial features corresponding to each sample content in the training sample set, wherein a plurality of sample labels corresponding to target prediction tasks are set for each sample content; Based on the initial features of each sample and multiple sample labels corresponding to the content of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain a multi-objective prediction model.
7. The method according to claim 6, characterized in that Based on the initial features of each sample, the multi-objective prediction model to be trained is iteratively trained to obtain the multi-objective prediction model, wherein one iteration process includes: For each target prediction task in the multiple target prediction tasks, the following operations are respectively performed: based on the element weights of each sample initial feature element contained in a sample initial feature under the target prediction task, the sample initial feature is updated to obtain a sample intermediate feature of the target prediction task; For each of the obtained multiple sample intermediate features, the following operations are performed respectively: using an attention mechanism, based on the importance of each sample intermediate feature relative to the sample intermediate feature, the sample intermediate feature is updated to obtain a sample target feature of the sample intermediate feature; Based on the obtained multiple sample target features, corresponding multi-target prediction results are obtained respectively; Based on the loss value between the multi-objective prediction result and each sample label corresponding to the sample content, the multi-objective prediction model to be trained is adjusted.
8. A multi-objective prediction device for content, characterized in that: include: An acquisition module, configured to acquire initial features of the content to be predicted, wherein the initial features at least represent the content category and multimodal characteristics of the content to be predicted; A first updating module: for each target prediction task in a plurality of target prediction tasks, respectively performing the following operations: based on the element weights of each initial feature element contained in the initial feature under the target prediction task, updating the initial feature to obtain an intermediate feature of the target prediction task, wherein the intermediate feature contains local reference information required for performing the target prediction task; The second updating module is used to perform the following operations for each of the obtained multiple intermediate features: using an attention mechanism, based on the importance of each intermediate feature relative to the intermediate feature, updating the intermediate feature to obtain a target feature of the intermediate feature, wherein the target feature contains global reference information required to perform a corresponding target prediction task; The prediction module is used to obtain corresponding multi-target prediction results based on the obtained multiple target features.
9. A computing device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
11. A computer program product, characterized in that When the computer program product is called by a computer, the computer executes the method according to any one of claims 1 to 7.
