Data processing method, apparatus, computing device, system, and readable storage medium

Abstract

The application provides a data processing method and device, computing equipment, system and readable storage medium, and relates to the field of computers. The method comprises the following steps: combining statistical features, category features and attribute features of at least one source sequence into comprehensive sequence features; determining, according to the comprehensive sequence features, a first sequence feature representing expected benefits of different positions in an output sequence and a second sequence feature representing probabilities of different positions being placed with different categories of elements triggering user behaviors; and determining, under a constraint condition, the ranking of the elements in the source sequence in the output sequence by comprehensively considering the expected benefits and the probabilities of the elements triggering the user behaviors. Thus, the statistical features of the source sequence are introduced into subsequent rearrangement prediction, the prediction data can more accurately represent the differences between the source sequences of different categories of elements, the influence of the differences between different categories of elements on the expected benefits and the user behaviors is accurately learned, and the balance between the benefits and the exposure fairness of the output sequence on the platform is improved.

Description

Technical Field

[0001] This application relates to the field of computers, and more particularly to a data processing method, apparatus, computing device, system, and readable storage medium. Background Technology

[0002] Recommendation systems aim to recommend relevant items based on users' interests and are widely researched and applied in the computer, internet, and industrial sectors. Within recommendation systems, integrated ranking systems (IRS), or simply mixed ranking systems, combine sequences of items (source sequences) from multiple upstream sources such as product providers or goods suppliers. Ultimately, a platform providing information and / or services to users presents a sorted list of items from these source sequences, displaying a mixed ranking of the items.

[0003] The position of different items in the sorted list output by a mixed sorting system affects their exposure, thus impacting the revenue of the platform displaying the sorted list. Existing mixed sorting systems typically design different rule logics based on the different properties of the protected items, inserting the protected items into fixed or dynamic slots in the sorted list. However, inserting protected items based on fixed rule logic to balance the fairness of exposure among different types of items and the platform's revenue relies on subjective assumptions from domain experts. When the sequence of items to be sorted is complex, it is difficult to guarantee a balance between the fairness of exposure among different types of items and the platform's revenue, resulting in a poor balance between the platform's revenue and the fairness of exposure among different types of items. Summary of the Invention

[0004] This application provides a data processing method, apparatus, computing device, system, and readable storage medium, which can solve the problem of poor balance between the platform's revenue and the fairness of exposure of different types of items in the sorted list output by existing mixed sorting systems.

[0005] The problem.

[0006] Firstly, a data processing method is provided. This method includes: firstly, combining statistical features, category features, and attribute features of at least one source sequence into a comprehensive sequence representation; secondly, under constraints, determining, based on the comprehensive sequence representation, a first sequence representation representing the expected return at different positions in the output sequence, and a second sequence representation representing the probability of user behavior occurring corresponding to the placement of different types of elements at different positions in the output sequence; and thirdly, determining the elements placed at different positions in the output sequence based on the first and second sequence representations. Wherein, the elements in the at least one source sequence have different origins or categories, and each source sequence includes at least one element belonging to the same category. The statistical features are the statistical features of at least one source sequence. The category features and attribute features are the features of the elements contained in each source sequence. The constraints are used to indicate the weight of the expected return factor and the factor that triggers user behavior from different types of elements in influencing the decision when determining the elements placed at each position in the output sequence.

[0007] Based on the aforementioned data processing methods, the comprehensive sequence representation, along with the first and second sequence representations derived from it, incorporates at least one statistical feature of the source sequence. This allows for a more accurate representation of the differences between source sequences of different types of elements, thereby improving the accuracy of the constraints in balancing expected return factors and factors triggering user behavior. Therefore, when sorting the output sequence based on the first and second sequence representations under constraints, the impact of the differences between source sequences of different types of elements on expected returns and user behavior at different positions in the output sequence can be more accurately determined, thus improving the balance between platform revenue and exposure fairness of the output sequence.

[0008] One possible implementation involves processing category features and attribute features separately through different channels to obtain a third sequence representation representing the probability of placing elements of different categories at different positions in the output sequence, and a fourth sequence representation representing the probability of placing different elements at different positions in the output sequence. The third and fourth sequence representations are then fused to obtain a comprehensive sequence representation. Thus, by extracting the third and fourth sequence representations through different channels to determine the comprehensive sequence representation, feature interaction is achieved at both the attribute and category levels of the elements. This captures the interaction between element features at both the attribute and category levels, thereby improving the accuracy of subsequent predictions.

[0009] Optionally, when determining the third sequence representation based on category characteristics, statistical characteristics and category characteristics are introduced to jointly determine the third sequence representation, so as to improve the prediction accuracy of the probability of placing different types of elements at different positions in the output sequence.

[0010] In this embodiment, the computing device can use a neural network model to execute data processing methods to achieve the mixing of elements from different source sequences. The neural network model is a rearrangement neural network model with sorting functionality of a mixing system. The computing device uses the gated loop unit in the rearrangement neural network model to determine the third sequence representation and the fourth sequence representation.

[0011] As one possible implementation, the computing device uses a first gated loop unit to determine a third sequence representation based on statistical and categorical features, and a second gated loop unit to determine a fourth sequence representation based on attribute features.

[0012] Optionally, the structure of the second gated recurrent unit is the same as that of the traditional gated recurrent unit, with the input being attribute features and the output being the fourth sequence representation. The difference between the first and second gated recurrent units is that the first gated recurrent unit also includes an activation function. The activation function is used to determine the sequence probability of placing elements of different types of source sequences at different positions in the output sequence based on categorical and statistical features, and uses the sequence probability in the candidate hidden state calculation process.

[0013] Based on the above implementation, a gated recurrent unit is used to obtain hidden information between elements from different source sequences, which enhances the information interaction between different source sequences at the category level. This improves the representation ability of the third and fourth sequence representations of elements and improves the prediction accuracy of the subsequent element sorting of the output sequence.

[0014] As one possible implementation, after determining the third and fourth sequence representations, the computing device synthesizes the third and fourth sequence representations to obtain a comprehensive sequence representation. To ensure that the comprehensive sequence representation accurately reflects the impact of elements at different positions in the output sequence on user behavior, the computing device can also first perform a dot product on the third and fourth sequence representations to obtain the dot product result, and then fuse the text features, user features, and the dot product result to obtain the comprehensive sequence representation.

[0015] Optionally, the above merging operation is performed by the input layer in the rearranged neural network model, and the embedding operation is performed by the embedding layer in the rearranged neural network model. The input layer is connected to the hidden layer of the rearranged neural network model through the embedding layer.

[0016] As one possible implementation, the computing device utilizes a multi-layer perceptron (MLP) in the fully connected layer of a rearranged neural network model to process the comprehensive sequence representation to obtain a first sequence representation and a second sequence representation. For example, the fully connected layer includes a first MLP and a second MLP. Based on the comprehensive sequence list and the user behavior reward at each location, the first MLP is used to determine the first sequence representation, where the user behavior reward represents the reward for user behavior occurring at different locations in the output sequence. Based on the comprehensive sequence list, the second MLP is used to determine the second sequence representation.

[0017] As one possible implementation, without requiring that the order of multiple elements in each source sequence in the output sequence be the same as that in the source sequence, the computing device, under constraints, sorts the second sequence representation according to the first sequence representation to obtain a fifth sequence representation. Based on the fifth sequence representation, the elements placed at different positions in the output sequence are determined; that is, for each position in the output sequence, the element placed at that position is determined to be the element with the highest probability of being placed at that position, based on the fifth sequence representation. Here, the fifth sequence representation represents the probability of placing different elements at different positions in the output sequence after considering the factors of expected revenue and the factors corresponding to user behavior for different types of elements. Therefore, in the process of determining the element order in the output sequence, the revenue corresponding to different element sorting methods in the output sequence is introduced for decision-making, thereby improving the balance between the revenue and exposure fairness of the output sequence on the platform.

[0018] As another possible implementation, where it is necessary to ensure that the order of multiple elements contained in each source sequence in the output sequence is the same as that in the source sequence, at least one element in each source sequence is arranged in a specified order. Under constraints, the computing device sorts the second sequence representation according to the first sequence representation to obtain a fifth sequence representation. The fifth sequence representation represents the probability of different types of elements being placed at different positions in the output sequence after considering the factors of the expected return and the factors corresponding to different types of elements that trigger user behavior. Then, for the first position in the output sequence, the computing device determines, based on the fifth sequence representation, that the type of element placed at the first position has the highest probability of being that type. Then, according to the specified order, it determines the element to be placed from the unplaced elements in the source sequence corresponding to the type of element placed at the first position. The placed element is the first element among the unplaced elements arranged in the specified order. The first position can be any position in the output sequence. Therefore, after the source sequence sorting result is mixed and sorted, the elements in the output sequence still satisfy the specified order of elements in a single source sequence.

[0019] Based on the above implementation method, while ensuring the balance between the platform's revenue and exposure fairness in the output sequence, it also ensures the order preservation of each source sequence element in the output sequence. Thus, when the source sequence provider independently sorts the source sequence according to its unique needs and optimization goals, and the computing device cannot know the sorting strategy and requirements inside the source sequence, it ensures that the elements of each source sequence meet the sorting requirements of the source sequence provider in the output sequence.

[0020] In this embodiment, the computing device can use the output layer of the rearrangement neural network model to determine the elements placed at different positions in the output sequence. The rearrangement neural network model is trained based on multiple loss functions, and the constraints can be represented by the loss functions in the rearrangement neural network model. These multiple loss functions include a first loss function and a second loss function. The first loss function is used to calculate the loss corresponding to the occurrence of user behavior when different types of elements are placed at different positions in the output sequence. The second loss function is used to calculate the loss corresponding to the gain when different types of elements are placed at different positions in the output sequence.

[0021] The first loss function is obtained during the training phase of the rearranged neural network model by calculating the cross-entropy loss function between the user behavior corresponding to the elements in the output sequence and the first sequence representation. The second loss function is obtained during the training phase of the rearranged neural network model by calculating the cross-entropy loss function between the labels constructed based on the second sequence representation and the fifth sequence representation.

[0022] Optionally, the first and second loss functions in the multiple loss functions are each set with weight hyperparameters, so that the importance of different tasks in the overall task can be controlled by adjusting the weight hyperparameters, thereby improving the prediction flexibility.

[0023] Considering that the platform's revenue from the displayed output sequence is also affected by the exposure differences of different types of elements, the computing device can also use a third loss function to constrain the proportion of different types in the output sequence. For example, multiple loss functions also include a third loss function, which is used to calculate the loss between the distribution of different types of elements in the output sequence and a preset distribution, where the preset distribution includes a preset proportion of different types of elements in the output sequence.

[0024] In a second aspect, a data processing apparatus is provided, the apparatus comprising modules for performing the data processing method of the first aspect or any possible implementation thereof.

[0025] It should be noted that the data processing device described in the second aspect may be a terminal device or a network device, or it may be a chip (system) or other component or assembly that can be disposed in a terminal device or a network device, or it may be a device that includes a terminal device or a network device. This application does not limit it in this regard.

[0026] Thirdly, a computing device is provided, including a memory and a processor, the memory being used to store a set of computer instructions, which, when executed by the processor, are used to perform operational steps of a data processing method in any possible design of the first aspect.

[0027] Fourthly, a computer system is provided, including a training device and an execution device. The training device is used to perform operational steps of training a model to obtain a rearranged neural network model and training the model to obtain a three-dimensional transformation model, and to deploy the rearranged neural network model to the execution device. The execution device is used to perform operational steps of the data processing method in any possible design of the first aspect using the rearranged neural network model.

[0028] Furthermore, the technical effects of the data processing device described in the second aspect, the computing device described in the third aspect, and the computer system described in the fourth aspect can be referred to with reference to the technical effects of the data processing method described in the first aspect, and will not be repeated here.

[0029] Fifthly, a readable storage medium is provided, comprising: computer software instructions; when the computer software instructions are executed in a computer system, causing a computing device to perform the operational steps of the method as described in any possible implementation of the first aspect.

[0030] In a sixth aspect, a computer program product is provided that, when run on a computer, causes a computing device to perform the operational steps of the method as described in any possible implementation of the first aspect.

[0031] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0032] Figure 1 A schematic diagram of a neural network structure is provided in this application;

[0033] Figure 2 This is a schematic diagram of the structure of a gated loop unit in the prior art;

[0034] Figure 3 A schematic diagram illustrating the stages of a recommendation system based on existing technology;

[0035] Figure 4 A schematic diagram of the architecture of a computer system provided in this application;

[0036] Figure 5 A schematic diagram of the structure of an execution device provided in this application;

[0037] Figure 6 A flowchart illustrating a hybrid sorting scenario provided in this application;

[0038] Figure 7 A flowchart illustrating a data processing method provided in this application;

[0039] Figure 8 A schematic diagram illustrating the operation flow of a rearranged neural network model provided in this application;

[0040] Figure 9 A schematic diagram of the structure of a multi-source attention-gated recurrent unit provided in this application;

[0041] Figure 10 A comparison diagram of the effects of a mixed-layout model provided in this application;

[0042] Figure 11 A statistical chart illustrating the effect of a mixed-row model provided in this application;

[0043] Figure 12 A schematic diagram of the structure of a data processing device provided in this application;

[0044] Figure 13 A schematic diagram of the structure of a computing device provided in this application. Detailed Implementation

[0045] To facilitate understanding, the relevant terms and concepts such as neural networks involved in the embodiments of this application will be introduced below.

[0046] (1) Neural Network

[0047] Neural networks can be composed of neurons, and neurons can refer to... x s The arithmetic unit takes the intercept (i.e., bias coefficient) 1 as input. The output of this arithmetic unit satisfies the following formula:

[0048] Formula (1)

[0049] Where s = 1, 2, ..., n, n is a natural number greater than 1. W s for x s The weights are denoted by b, where b is the bias of the neuron. fActivation functions are used to introduce nonlinear characteristics into neural networks, converting input signals into output signals. The output of this activation function can serve as the input to the next layer; the activation function can be a sigmoid function. A neural network is a network formed by connecting multiple individual neurons, meaning the output of one neuron can be the input of another. The input of each neuron can be connected to the local receptive field of the previous layer to extract features from that local receptive field, which can be a region composed of several neurons. Weights represent the strength of the connections between different neurons. Weights determine the influence of the input on the output. Weights close to 0 mean that changing the input does not change the output. Negative weights mean that increasing the input decreases the output.

[0050] like Figure 1 The diagram shown is a schematic representation of a neural network structure provided in an embodiment of this application. The neural network 100 includes N processing layers, each containing one or more neurons, where N is an integer greater than or equal to 3. The first layer of the neural network 100 is an input layer 110, responsible for receiving input signals. The last layer of the neural network 100 is an output layer 130, responsible for outputting the processing results of the neural network. The layers excluding the first and last layers are intermediate layers 140, which together form a hidden layer 120. Each intermediate layer 140 in the hidden layer 120 can both receive input signals and output signals. The hidden layer 120 is responsible for processing the input signals. Each layer represents a logical level of signal processing; through multiple layers, data signals can undergo multi-level logical processing.

[0051] In some feasible embodiments, the input signal of the neural network can be various forms of signals such as video signals, voice signals, text signals, image signals, or temperature signals. Voice signals can be various sensor signals such as audio signals of a person speaking or singing recorded by a microphone (sound sensor). The input signal of the neural network also includes various other computer-processable engineering signals, which will not be listed here. Using a neural network to perform deep learning on image signals can improve the quality of image processing by the neural network.

[0052] (2) Recurrent Neural Network

[0053] Recurrent Neural Networks (RNNs) are used to process sequential data. In traditional neural network models, the layers from the input layer to the hidden layer and then to the output layer are fully connected, but the nodes within each layer are unconnected. While this type of neural network has solved many difficult problems, it remains inadequate for many others. For example, predicting the next word in a sentence requires using the preceding words because words in a sentence are not independent. RNNs are called recurrent neural networks because the current output of a sequence is related to the outputs of previous sequences. Specifically, the network memorizes previous information and applies it to the calculation of the current output. That is, the nodes within the hidden layer are no longer unconnected but connected, and the input of the hidden layer includes not only the output of the input layer but also the output of the hidden layer at the previous time step.

[0054] In theory, RNNs can process sequence data of any length. Training an RNN is similar to training a traditional Convolutional Neural Network (CNN) or Deep Neural Network (DNN). It also uses the backpropagation algorithm, but with one key difference: if an RNN is expanded, its parameters, such as W, are shared; this is not the case for traditional neural networks as illustrated above. Furthermore, in gradient descent, the output at each step depends not only on the current step's network state but also on the states of the network from previous steps. This learning algorithm is called Backpropagation Through Time (BPTT).

[0055] (3) Backpropagation algorithm

[0056] Convolutional neural networks can employ backpropagation (BP) to correct the parameters in the initial super-resolution model during training, thereby reducing the reconstruction error loss. Specifically, forward propagation of the input signal to the output generates an error loss; this error loss information is then propagated back to update the parameters in the initial super-resolution model, leading to convergence of the error loss. The backpropagation algorithm is an error-loss-driven backpropagation process aimed at obtaining the optimal parameters of the super-resolution model, such as the weight matrix.

[0057] (4) Gate control loop unit

[0058] The Gate Recurrent Unit (GRU) is a type of recurrent neural network. Like the Long Short-Term Memory (LSTM) network, it was applied to neural networks to solve the gradient problem in long-term memory and backpropagation.

[0059] like Figure 2 As shown, the gate control loop unit includes an update gate. and reset door The update gate controls the extent to which information from the previous state is incorporated into the current state. A larger update gate value indicates that more information from the previous state is incorporated. The information from the previous and current states is right-multiplied by the weight matrix, and then the sum is fed into the update gate, which is essentially multiplying by the sigmoid function. The resulting value is between 0 and 1. The reset gate controls how much historical information from the previous state is written into the current candidate set. The smaller the reset gate, the less information from the previous state is written. Similar to the data processing of the update gate, the information from the previous time step and the current time step are right-multiplied by the weight matrix, and then the sum is fed into the reset gate, which is multiplied by the sigmoid function. The resulting value is between [0, 1]. The only difference is the value and purpose of the weight matrix in the two steps.

[0060] Optionally, reset the door. As shown in formula (2) below, update the gate. As shown in formula (3) below, the candidate hidden state As shown in formula (4) below, the hidden state at the current time is As shown in formula (5) below.

[0061] Formula (2)

[0062] Formula (3)

[0063] Formula (4)

[0064] Formula (5)

[0065] in, , , Indicates weight, This represents the Sigmoid function. , , The biases and weights of the neurons are obtained during the training of the neural network.

[0066] (5) Loss function

[0067] In training a deep neural network, to ensure the output closely approximates the desired predicted value, we compare the network's prediction with the target value. Based on the difference, we update the weight vector of each layer (usually pre-configuring parameters before the initial update). For example, if the prediction is too high, the weight vector is adjusted to predict a lower value. This adjustment continues until the deep neural network predicts the target value or a value very close to it. Therefore, we need to predefine "how to compare the difference between the predicted and target values," which is the loss function or objective function. These are important equations used to measure the difference between the predicted and target values. Taking the loss function as an example, a higher output value (loss) indicates a greater difference, and training the deep neural network becomes a process of minimizing this loss.

[0068] (6) Recommendation system

[0069] In the information-saturated internet age, information producers struggle to present their information to consumers who are interested in it, while consumers find it difficult to sift through the vast sea of ​​data to discover what interests them. Recommendation systems are a crucial tool for resolving this contradiction. Their task is to connect users and information, helping users discover valuable information while ensuring that information is presented to users who are interested in it, thus achieving a win-win situation for both information consumers and producers. Unlike search systems, recommendation systems primarily utilize user behavior data, analyzing user behavior logs to provide different recommended pages, thereby improving user satisfaction and website click-through rates and conversion rates.

[0070] Common recommendation system formats include personalized recommendations, related recommendations, and popular recommendations. Common application scenarios for recommendation systems include e-commerce, personalized advertising, music and movies, and job seeking.

[0071] like Figure 3As shown, the structure of a recommendation system typically includes recall, coarse ranking, fine ranking, and re-ranking stages. The main goal of recall is to select thousands or tens of thousands of items from the recommendation pool and send them to the subsequent ranking module. Since the candidate set for recall is very large and generally requires online output, the recall module must be lightweight, fast, and have low latency. Because the subsequent ranking module provides assurance, recall does not need to be perfectly accurate, but it cannot miss any items. Coarse ranking is used because sometimes the recall results are still too many, and the fine ranking layer cannot keep up. Coarse ranking can be understood as a filtering mechanism before fine ranking, reducing the pressure on the fine ranking module. Coarse ranking is between recall and fine ranking, balancing accuracy and low latency. The task of the fine ranking stage is to obtain the results from the coarse ranking module and score and rank the candidate set. Fine ranking needs to ensure the accuracy of the scoring within the maximum allowed latency. The re-ranking stage re-sorts the sequence of Top-N items generated by fine ranking, generating a sequence of Top-K items, which serves as the final result of the ranking system and is directly presented to the user, where N is greater than or equal to K. For example, the rearrangement stage uses an Integrated Ranking System (IRS) to combine item sequences (source sequences) from multiple upstream sources, ultimately presenting the user with a sorted list that integrates the items in the item sequence.

[0072] In recommender systems, hybrid ranking systems are a crucial component in industrial applications and are attracting increasing attention driven by practical needs. Hybrid ranking systems combine items from multiple source sequences (such as applications, products, and services). These items, often from different sources or of different types, are provided by various providers, who generally believe that item exposure significantly impacts their revenue. The position of items from different sources or of different types in the ranking list output by the hybrid ranking system affects their exposure, thus influencing the provider's revenue. On one hand, existing hybrid ranking systems typically design different rule logics based on the different properties of the protected items, inserting them into fixed or dynamic slots in the ranking list. The specific position of these fixed or dynamic slots in the ranking list is determined based on subjective assumptions made by experts in the field. When there are many or complex source sequences, it is difficult to guarantee a balance between the fairness of exposure for different types of items and the platform's revenue. On the other hand, hybrid ranking systems based on deep networks model the relationships between different source sequences and select the most suitable items from the currently inferred item types to recommend to users in order to achieve the effect of hybrid ranking. Their goal is to maximize click revenue, but they do not consider the fairness of different categories of items. Similarly, there is a problem of poor balance between platform revenue and the fairness of exposure of different types of items.

[0073] This application provides a data processing method, particularly a data processing method that introduces statistical features of source sequences to enhance the balance between the output sequence's revenue on the platform and the fairness of exposure of different types of items. Specifically, a computing device combines the statistical features, category features, and attribute features of at least one source sequence into a comprehensive sequence representation. The elements of different source sequences in the at least one source sequence have different sources or categories, and each source sequence includes at least one element belonging to the same category. Then, under the constraints of factors used to constrain expected revenue and factors that trigger user behavior, a first sequence representation representing the expected revenue at different positions in the output sequence and a second sequence representation representing the probability of user behavior occurring when different types of elements are placed at different positions in the output sequence are determined based on the comprehensive sequence representation. Finally, based on the first and second sequence representations, the order of the elements contained in the at least one source sequence in the output sequence is determined. Therefore, by introducing the statistical characteristics of the source sequence into the comprehensive sequence representation, the information used in the mixed ranking decision process is more complete and can more accurately represent the differences between the source sequences of different types of elements. The first and second sequence representations obtained from the comprehensive sequence representation make the influence of the expected benefit factors and the factors that trigger user behavior on the output sequence ranking more balanced under constraints. Thus, under constraints, the sequence output by the mixed ranking based on the first and second sequence representations has a stronger balance between the platform's benefit and exposure fairness.

[0074] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0075] Figure 4 This is a schematic diagram of the architecture of a computer system provided in an embodiment of this application. Figure 4 As shown, the computer system 400 includes an execution device 410, a training device 420, a database 430, a terminal device 440, a data storage system 450, and a data acquisition device 460.

[0076] The execution device 410 can be a terminal, such as a mobile phone terminal, tablet computer, laptop computer, virtual reality (VR) device, augmented reality (AR) device, mixed reality (MR) device, extended reality (ER) device, camera or vehicle terminal, etc., or it can be an edge device (e.g., a box carrying a chip with processing capabilities).

[0077] The training device 420 can be a terminal, or other computing devices that support integer calculations, such as servers or cloud devices.

[0078] As one possible embodiment, the execution device 410 and the training device 420 are different processors deployed on different physical devices (e.g., servers in a server or cluster). For example, the execution device 410 may be a graphics processing unit (GPU), a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The training device 420 may be a graphics processing unit (GPU), a neural network processing unit (NPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits used to control the execution of the program of the present application.

[0079] In another possible embodiment, the execution device 410 and the training device 420 are deployed on the same physical device, or the execution device 410 and the training device 420 are on the same physical device.

[0080] Data acquisition device 460 is used to collect training data and store it in database 430. Data acquisition device 460, execution device 410, and training device 420 can be the same or different devices. Training data includes at least one form of data such as user history data and item table data. For example, the training data collected by data acquisition device 460 can be item table data labeled with signals that trigger user behavior from items. The item table data includes multiple source sequences, each containing at least one item of the same type (including item attributes, user profile at the time of the user behavior, time and location of the user behavior, etc.). User behavior can be actions such as clicking on an item or lingering on an item to receive item information. In this embodiment, the aforementioned items can be considered as elements in the source or output sequences.

[0081] Training device 420 is used to train the neural network using training data until the loss function in the neural network converges and the loss function value is less than a certain threshold, at which point the neural network training is complete, thereby enabling the neural network to achieve a certain level of accuracy. Alternatively, all the training data in database 430 can be used for training, thus completing the neural network training and enabling the trained neural network to perform target functions such as hybrid sorting. Then, training device 420 configures the trained neural network to execution device 410. Optionally, the neural network is a specific structural reference for rearranging the neural network model. Figure 8 The relevant descriptions will not be repeated here.

[0082] The execution device 410 is used to perform a hybrid ranking function based on a trained neural network, which is the function of rearranging items from at least one source sequence into an output sequence during the re-ranking stage of a recommendation system. For example, the execution device 410 extracts a comprehensive sequence representation from the statistical features, category features, and attribute features of at least one source sequence based on the trained neural network, and determines a first sequence representation representing the expected benefit at different positions in the output sequence, and a second sequence representation representing the probability of user behavior occurring corresponding to different types of elements placed at different positions in the output sequence, based on the comprehensive sequence representation. Then, it determines the ranking of the elements contained in at least one source sequence in the output sequence based on the first and second sequence representations. The constraints are used to indicate the weight of the expected benefit factor and the factor of user behavior occurring corresponding to different types of elements in influencing the decision when determining the element placed at each position in the output sequence.

[0083] In some embodiments, the execution device 410 and the training device 420 are the same computing device. The computing device can configure the trained neural network into itself and use the trained neural network to achieve target functions such as image recognition and speech recognition.

[0084] In other embodiments, the training device 420 can configure the trained neural network to multiple execution devices 410. Each execution device 410 utilizes the trained neural network to implement the target function of the neural network model.

[0085] It should be noted that in practical applications, the training data maintained in database 430 may not all come from data acquisition device 460; it may also be received from other devices. Furthermore, training device 420 may not necessarily train the neural network entirely based on the training data maintained in database 430; it may also obtain training data from the cloud or other sources. The above description should not be construed as limiting the embodiments of this application.

[0086] Furthermore, based on the function performed by the execution device 410, the execution device 410 can be further subdivided into, for example... Figure 5The architecture shown is as follows: Figure 5 As shown, the execution device 410 is equipped with a computing module 411, an I / O interface 412, and a preprocessing module 413.

[0087] I / O interface 412 is used for data interaction with external devices. Users can input data to I / O interface 412 through terminal device 440. Alternatively, input data can also come from database 430.

[0088] The preprocessing module 413 is used to preprocess the input data received from the I / O interface 412. In this embodiment, the preprocessing module 413 can be used to generate training data based on the input data received from the I / O interface 412.

[0089] During the preprocessing of input data by the execution device 410, or during the calculation module 411 of the execution device 410 performing calculations using the neural network 401, the execution device 410 can call data, code, etc. in the data storage system 450 for corresponding processing, or store the data and instructions obtained from the corresponding processing into the data storage system 450.

[0090] Finally, the I / O interface 412 returns the processing result to the terminal device 440, which then provides it to the user so that the user can view the processing result.

[0091] Terminal device 440 can also be used as a data acquisition end to collect data such as... Figure 5 The input data and processing results of the input I / O interface 412 shown in Figure 5 are used as new sample data and stored in the database 430. Alternatively, the data can be collected without going through the terminal device 440; instead, the I / O interface 412 can use the input data and processing results of the input I / O interface 412 shown in Figure 5 as new sample data and store them in the database 430.

[0092] Figure 4 or Figure 5 This is merely a schematic diagram of one possible architecture provided in this application. Figure 4 or Figure 5 The positional relationships between the devices, components, modules, etc. shown do not constitute any limitation.

[0093] The aforementioned computer system 400 can use data processing methods to rearrange the recommended elements into an output sequence and display it to the user in scenarios such as application market recommendations, news feed recommendations, and e-commerce item recommendations.

[0094] The following example, using an app store recommendation, illustrates the functions implemented by the computer system's 400 execution data processing method. Figure 6As shown, computer system 400 obtains multiple sequences (source sequences) of original applications (APPs) from upstream providers. In the application market scenario, original applications can include ad-supported applications, joint-operation applications, and organic applications. Correspondingly, these source sequences can be divided into three categories: ad-supported application sequences, joint-operation application sequences, and organic application sequences. Ad-supported applications refer to applications that advertisers entrust the platform to promote. If users click and download these applications, the platform will collect a certain fee from the advertisers who provided them as revenue. Joint-operation applications refer to applications that have continuous charging items after download. If users make continuous purchases in these applications, the platform will obtain a portion of the revenue. Organic applications refer to free applications, such as calendar and other utility applications, which generally meet various general user needs, thereby increasing the platform's activity. The above classification of ad-supported application sequences, joint-operation application sequences, and organic application sequences is only one example provided in this embodiment. In other embodiments, applications can also be classified according to different application providers, different target audiences, and other indicators. After acquiring the source sequences from the three types of applications—advertising, interoperability, and organic—the hybrid sorting system, i.e., computer system 400, uses data processing methods to merge these source sequences into a single hybrid sequence. This data processing method is primarily implemented in software and can be easily and quickly added to... Figure 6 In the subsequent operations of the fine ranking stage shown in Figure 3, the heterogeneous characteristics and basic information of different applications (such as application developers, application size, etc.) are taken into account during the mixed ranking process, so as to realize the mixing and display of multiple source sequences and form a complete application recommendation process.

[0095] Please refer to the following. Figure 7 The data processing methods are explained in detail here. Figure 4 The following explanation will be based on the execution device 410.

[0096] Step 710: The execution device 410 combines the statistical characteristics, categorical characteristics, and attribute characteristics of at least one source sequence into a comprehensive sequence representation.

[0097] The execution device 410 acquires at least one source sequence from the fine ranking stage output of the recommendation system, and combines the statistical features, category features, and attribute features of the at least one source sequence into a comprehensive sequence representation.

[0098] Each source sequence includes at least one element belonging to the same category. The elements in the source sequences can represent items, such as goods, products, and services, that upstream sources like suppliers or providers need to display on the platform.

[0099] The statistical characteristics of at least one source sequence are the statistical characteristics of each source sequence and / or the statistical characteristics between two or more source sequences. For example, the first source sequence in at least one source sequence includes 15 elements, the second source sequence includes 8 elements, and the third source sequence includes 6 elements. The statistical characteristics of each source sequence include the number of elements contained in 15, the number of elements contained in the second sequence is 8, and the number of elements contained in the third source sequence is 6. The statistical characteristics between two source sequences include the difference in the number of elements between any two source sequences, i.e., the difference in the number of elements between the first and second source sequences is 7, the difference in the number of elements between the first and third source sequences is 9, and the difference in the number of elements between the second and third source sequences is 2. The statistical characteristics between multiple source sequences include the variance of the number of elements, the mean, etc.

[0100] Category characteristics refer to the category classification and sequential numbering of elements in each source sequence. For example, regarding category classification, if the first source sequence contains elements belonging to advertising applications, and the second source sequence contains elements belonging to joint operation applications, then the category identifiers for advertising applications and joint operation applications represent different category characteristics. Attribute characteristics are the characteristics of the elements themselves in each source sequence. Taking an application as an example, the attribute characteristics of an application include application size, developer, etc.

[0101] Optionally, the execution device 410 determines a third sequence representation based on statistical and categorical characteristics, determines a fourth sequence representation based on attribute characteristics, and then combines the third and fourth sequence representations to determine a comprehensive sequence representation. The third sequence representation indicates the probability of placing elements of different types at different positions in the output sequence. The fourth sequence representation indicates the probability of placing different elements at different positions in the output sequence. The comprehensive sequence representation is a representation that simultaneously includes the contents of the third and fourth sequence representations.

[0102] Optionally, in addition to the third and fourth sequence representations, the execution device 410 may also introduce text features, user features, and other features to jointly determine the comprehensive sequence representation with the third and fourth sequence representations. Text features are used to represent user behavior information for an element. User behavior information includes the time, location, and / or user device model of the user behavior. User features are used to represent a user profile, which includes data such as the user's gender, occupation, and / or age.

[0103] In this embodiment, statistical features, category features, and attribute features can be low-dimensional vectors obtained through embedding operations, and are referred to as statistical embedding features, category embedding features, and attribute embedding features, respectively. The execution device 410 extracts at least one statistical embedding feature, category embedding feature, and attribute embedding feature of the source sequence using the embedding layer of the rearranged neural network model. It then uses the first gated recurrent unit of the hidden layer to determine a third sequence representation based on the statistical features and category features, and uses the second gated recurrent unit of the hidden layer to determine a fourth sequence representation based on the attribute features. For the specific structure of the embedding layer and hidden layer of the rearranged neural network model and its data processing steps, please refer to [reference needed]. Figure 8 The relevant descriptions will not be repeated here.

[0104] Step 720: The execution device 410 determines the first sequence characterization and the second sequence characterization based on the comprehensive sequence characterization.

[0105] The execution device 410 extracts a first sequence representation and a second sequence representation from the comprehensive sequence representation. The first sequence representation is used to represent the expected reward at different positions in the output sequence. The second sequence representation is used to represent the probability of user behavior occurring corresponding to different types of elements being placed at different positions in the output sequence.

[0106] In this embodiment, the execution device 410 utilizes a multilayer perceptron in the fully connected layer of the rearranged neural network model to extract the first sequence representation and the second sequence representation from the synthesized sequence representation. For the specific structure of the fully connected layer of the rearranged neural network model and its data processing steps, please refer to [reference needed]. Figure 8 The relevant descriptions will not be repeated here.

[0107] Step 730: Under constraints, the execution device 410 determines the order of elements contained in at least one source sequence in the output sequence based on the first sequence representation and the second sequence representation.

[0108] The execution device 410, based on the first sequence representation and the second sequence representation, determines the elements placed at different positions in the output sequence by considering whether the expected returns at different positions and the probability of user behavior corresponding to different types of elements placed at different positions in the output sequence satisfy the constraints. This results in an output sequence that balances platform revenue and the fairness of source sequence exposure. The constraints indicate the relative weight of expected returns and the factors triggering user behavior by different types of elements in influencing the decision when determining the elements placed at each position in the output sequence.

[0109] As one possible implementation, without requiring that the order in which multiple elements in each source sequence appear in the output sequence is the same as that in the source sequence, the execution device 410 sorts the second sequence representation according to the first sequence representation under constraints to obtain a fifth sequence representation. For each position in the output sequence, the fifth sequence representation determines the element placed at each position as the element with the highest probability of being placed at that position. The fifth sequence representation represents the probability of placing different elements at different positions in the output sequence after considering factors such as expected return and factors corresponding to different types of elements that trigger user behavior. The placed elements belong to at least one source sequence.

[0110] As another possible implementation, when it is necessary to ensure that the order of multiple elements contained in each source sequence in the output sequence is the same as that in the source sequence, at least one element in each source sequence is arranged in a specified order. Under constraints, the execution device 410 sorts the second sequence representation according to the first sequence representation to obtain a fifth sequence representation. For any position in the output sequence, such as the first position, the fifth sequence representation determines that the type of the element placed at the first position has the highest probability of being the type of the element placed at the first position. Then, the first element of the unplaced element in the source sequence corresponding to the type with the highest probability at each position is placed at that position. The order of multiple elements in the output sequence can be a fixed order determined by the upstream source of each source sequence, and the first element is the first element in the source sequence based on the specified order. The fifth sequence representation is used to represent the probability of different types of elements placed at different positions in the output sequence after considering factors of expected return and factors of user behavior corresponding to different types of elements.

[0111] For example, at least one source sequence includes a first source sequence, a second source sequence, and a third source sequence, where the first source sequence includes elements 1, 2, and 3 arranged in sequence. When determining the element to be placed at the current position of the output queue, the execution device 410 determines the type of the element with the highest probability based on the fifth sequence representation as the type of element corresponding to the first source sequence, and thus determines that the element to be placed at the current position is the head element of the first source sequence, i.e., element 1.

[0112] Optionally, the execution device 410 determines the order of the fifth sequence representation based on the weight of the factors affecting the expected return and the factors that cause user behavior corresponding to different types of elements.

[0113] Specifically, for each position in the output sequence of the fifth sequence representation, if the constraint indicates that the factors affecting the expected return have a greater influence on the decision than the factors affecting the user behavior corresponding to different types of elements, then the execution device 410 determines the probability that the position in the fifth sequence representation is the same position as the first sequence representation.

[0114] For each position in the output sequence of the fifth sequence representation, if the constraint indicates that the proportion of the factor influencing the decision based on the expected return is less than or equal to the proportion of the factor influencing the decision based on the user behavior corresponding to different types of elements, then the execution device 410 determines the probability that this position in the fifth sequence representation is the same position in the second sequence representation. Thus, the ranking of the second sequence representation based on expected return is achieved.

[0115] In this embodiment, the constraint can be a loss function. The execution device 410 trains the rearranged neural network under the constraint of the loss function. The hidden layers of the rearranged neural network obtained after training based on the loss function are constrained by the loss function when calculating sequence representations such as the first sequence representation, the second sequence representation, and the fifth sequence representation. Hidden layers trained under the constraint of the loss function can better ensure the balance between platform revenue and exposure fairness when predicting the output sequence. That is, when predicting the probability of elements placed at different positions in the output sequence, the probability of element types, and the corresponding expected revenue, the balance between platform revenue and exposure fairness can be guaranteed.

[0116] The aforementioned loss functions include a loss function representing the loss corresponding to the occurrence of user behavior when different types of elements are placed at different positions in the output sequence, and a loss function representing the loss corresponding to the gain when different types of elements are placed at different positions in the output sequence. The loss represents the deviation between the predicted result and the actual result. For the specific structure of the output layer of the rearranged neural network model and its data processing steps, please refer to [reference needed]. Figure 8 The relevant descriptions will not be repeated here.

[0117] Based on the aforementioned data processing methods, the comprehensive sequence representation, in addition to including the categorical and attribute characteristics of elements in the source sequence, also includes at least one statistical characteristic of the source sequence. That is, the comprehensive sequence representation, along with the first and second sequence representations derived from it, contain at least one statistical characteristic of the source sequence. This allows for a more accurate representation of the differences between source sequences of different types of elements, thereby improving the accuracy of constraints in balancing expected return factors and factors triggering user behavior. Therefore, when the second sequence representation is sorted according to the first sequence representation under constraints to instruct the output sequence to sort its elements, the impact of the differences between source sequences of different types of elements on the expected return and user behavior at different positions in the output sequence can be more accurately determined, thus improving the balance between the output sequence's revenue and exposure fairness on the platform. Furthermore, when the source sequence has an order preservation requirement, the element order within the source sequence can be maintained, thereby satisfying the needs and interests of the upstream provider.

[0118] The above text combined Figure 7 The data processing methods have been explained in general; the following section will combine them with... Figure 8 The specific steps of the data processing method performed by the execution device 410 using the rearranged neural network model are described in detail.

[0119] like Figure 8 The rearranged neural network model 800 shown includes an input layer 810, an embedding layer 820, a hidden layer 830, and an output layer 840 connected in sequence. The hidden layer 830 includes a first gated recurrent unit 831, a second gated recurrent unit 832, and an interaction layer 833. The first gated recurrent unit 831 and the second gated recurrent unit 832 are respectively connected to the fully connected layer 834 through the interaction layer 833. The fully connected layer 834 includes a merging layer 835, a first fully connected layer 836, a second fully connected layer 837, and a third fully connected layer 838. The input of the merging layer 835 is connected to the interaction layer 833, and the output of the merging layer 835 is connected to the first fully connected layer 836. The outputs of the first fully connected layer 836 are connected to the second fully connected layer 837 and the third fully connected layer 838, respectively. Both the second fully connected layer 837 and the third fully connected layer 838 are connected to the output layer 840. The rearrangement neural network model 800 is trained based on constraints, namely multiple loss functions. These constraints indicate the weight of factors affecting the expected return and factors that trigger user behavior of different types of elements when determining the element placed at each position in the output sequence.

[0120] Based on the above rearranged neural network model 800, the data processing method includes the following steps 801-807.

[0121] Step 801: Input layer 810 merges at least one source sequence into a mixed sequence.

[0122] The input layer 810 uses a merger to receive at least one source sequence. ,Will Merged into a mixed sequence ,in, , The total number of all elements in at least one source sequence. ,in, Indicates the number of at least one source sequence. Indicates the first The number of elements in each source sequence.

[0123] Step 802: The embedding layer 820 performs embedding operations on at least one source sequence and a mixed sequence to obtain statistical embedding features, category embedding features, and attribute embedding features.

[0124] The embedding layer 820 uses low-dimensional vectors to encode the statistical features of at least one source sequence and the features of the mixed sequence, respectively, to obtain statistical embedding features, category embedding features and attribute embedding features.

[0125] For example, the low-dimensional vector mentioned above is a one-hot encoded vector, also known as a one-bit valid encoding. The method is to use an N-bit state register to encode N states. Each state has its own independent register bit, and at any given time, only one bit is valid.

[0126] The embedding layer 820 performs one-hot encoding on the statistical features of at least one source sequence, extracts vectors from the one-hot encoding results of the statistical features, and obtains the embedding result, i.e., the statistical embedding features.

[0127] Embedding layer 820 performs one-hot encoding on the category features and attribute features of the elements contained in the mixed sequence, extracts vectors from the one-hot encoding of category features and attribute features, and obtains the embedding results, namely category embedding features and attribute embedding features.

[0128] Optionally, the mixed sequence also includes text features and user features of the elements, and the embedding layer 820 is also used to perform embedding operations on the text features and user features to obtain text embedding features and user embedding features.

[0129] Embedded features can be in the form of an embedding matrix. In this embodiment, An embedding matrix representing category embedding features. The embedding matrix represents the attribute embedding features. An embedding matrix representing text embedding features. An embedding matrix representing user-embedded features. An embedding matrix representing statistical embedding features.

[0130] by To illustrate with an example of embedded features, consider a user count of 1000. If the embedding matrix has a dimension of 32, then Let A be a matrix with a shape function representation of [1000, 32]. In practical use... When processing data, for example, if we need to read the features of the user with user identifier 10, we take the value of the 10th row in matrix A as the embedded feature of the user corresponding to that user identifier.

[0131] Since the embedding layer 810 is obtained by training the rearranged neural network model 800 based on multiple loss functions, when performing the embedding operation, the embedding layer 810 can extract the features required by the subsequent hidden layer 830 to model the embedding features and determine the features needed to represent the fifth sequence.

[0132] Step 803 and hidden layer 830 respectively model the combination of category embedding features and statistical embedding features, as well as attribute embedding features, to obtain the third sequence representation and the fourth sequence representation.

[0133] The first gated recurrent unit 831 of the hidden layer 830 embeds class features. and statistical embedding features The combination of these is the aggregate embedding matrix. Modeling is performed to obtain the third sequence representation.

[0134] Among them, the aggregated embedding matrix It can be a category-embedded feature and statistical embedding features Obtained by matrix multiplication.

[0135] The second gated recurrent unit 832 of the hidden layer 830 embeds features into attributes. Modeling was performed to obtain the fourth sequence representation.

[0136] Optionally, the second gated loop unit 832 and Figure 2 The structure and function of the conventional gated loop unit shown are the same. For example... Figure 9 As shown, the first gated recurrent unit 831 can be a multi-sourced attention-gated recurrent unit (MA-GRU). Its structure differs from a conventional gated recurrent unit in that the first gated recurrent unit 831 includes an activation function, namely a softmax function. The input of the activation function is used to receive statistical embedding features and class embedding features. The output of the activation function and the candidate hidden state The computing units are connected.

[0137] Due to the requirement to preserve the order of elements in the source sequence, the correlation and differences between different types of source sequences are crucial for mixed sorting. Therefore, category embedding features at the category level are essential. This is combined with the statistical features of all source sequences, and then passed through the first gated recurrent unit 831. The first gated recurrent unit 831 then, based on the type embedding features of the current sequence and the statistical embedding features of each source sequence, performs... The function determines the probability of different types of elements being used at the current position in the output sequence, i.e., the sequence probability. Then, based on the sequence probability, it determines the type of element that is most suitable to be placed at the current position. For example, it determines the type of element with the highest sequence probability as the type of element that is most suitable to be placed at the current position.

[0138] In this embodiment, the above-mentioned fourth sequence representation can be expressed as: The operation by which the second gated loop unit 832 obtains the representation of the fourth sequence can be represented as follows: The fourth sequence characterization described above can be represented as follows: The operation of the first gated loop unit 831 to obtain the representation of the third sequence can be represented as follows: , This represents statistical embedding features.

[0139] Step 804: Hidden layer 830 merges the third sequence representation and the fourth sequence representation into a comprehensive sequence representation.

[0140] The information at the element type level and attribute level contains information from different levels of the source sequence. The interaction layer 833 of the hidden layer 830 performs a dot product between the third sequence representation and the fourth sequence representation to obtain the comprehensive sequence representation after interaction.

[0141] The above-mentioned comprehensive sequence characterization The calculation formula is as follows:

[0142] Formula (6)

[0143] Step 805: Hidden layer 830 determines the first sequence feature and the second sequence feature based on the comprehensive sequence representation.

[0144] Hidden layer 830 uses a dual-channel multilayer perceptron to process the comprehensive sequence representation to obtain the first sequence feature and the second sequence feature.

[0145] Optionally, before inputting the comprehensive sequence representation into the multilayer perceptron, hidden layer 830 also utilizes concat layer 835 to embed text features. and user embedded features The result obtained by multiplying the third sequence representation and the fourth sequence list is concatenated to obtain the comprehensive sequence representation of the subsequent input to the first fully connected layer 836.

[0146] As one possible implementation, the function of the rearranged neural network model 800 is to predict an output sequence that achieves a good balance between platform revenue and exposure fairness. Therefore, the prediction task can be divided into a Utility-oriented Task (UOT) and a Category Prediction Task (CPT) based on the platform's revenue and exposure fairness. Figure 8 The processing steps of UOT and CPT are represented by dashed boxes. Thus, the second fully connected layer 837 forms a multilayer perceptron for extracting relevant representations (i.e., the first sequence representation) for the profit enhancement task, and the third fully connected layer 838 forms a multilayer perceptron for extracting relevant representations (i.e., the second sequence representation) for the prediction category task.

[0147] The first fully connected layer 836 outputs the processed synthesized sequence representations to the second fully connected layer 837 and the third fully connected layer 838, respectively. The second fully connected layer 837 uses a multilayer perceptron to extract the first sequence representation from the synthesized sequence representation. The third fully connected layer 838 utilizes a multilayer perceptron to extract the second sequence representation from the comprehensive sequence representation. It can be represented by the following formula:

[0148] Formula (7)

[0149] Formula (8)

[0150] in, Used to represent the probability that an element at a different position in the output sequence is clicked. The size is , indicating the sequence The probability of each element being clicked when placed in a different location, and the probability of no element being clicked when placed in any location. These click probabilities can be obtained by modeling user click behavior.

[0151] In possible implementations, if the revenue represented by the first sequence is not only click-through rate revenue, but also revenue such as profit from the element (e.g., the element is an app with a price in an app store), then the first sequence representation could also be... Price corresponding to the location application Expected gains at different positions in the output sequence obtained by multiplication For example, the following formula:

[0152] Formula (9)

[0153] Since the hidden layer 830 is obtained by training the rearranged neural network model 800 based on multiple loss functions, the hidden layer 830 can also be regarded as determining the first sequence feature and the second sequence feature based on the comprehensive sequence representation under constraints, which can ensure the balance between the platform's revenue and the fairness of exposure.

[0154] Step 806: The output layer 840 sorts the second sequence representation according to the first sequence representation under the constraints to obtain the fifth sequence representation.

[0155] The output layer 840 determines the ranking of the fifth sequence representation based on the weight of factors influencing user behavior decisions corresponding to expected returns and different types of elements. The fifth sequence representation is the final probability matrix, i.e. .

[0156] For each position in the output sequence, if the constraint indicates that the factors affecting the expected return have a greater influence on the decision than the factors affecting the user behavior corresponding to different types of elements, then the output layer 840 determines the probability that the position in the fifth sequence representation is the same position as the first sequence representation.

[0157] For each position in the output sequence, if the constraint indicates that the proportion of the factor affecting the decision-making is less than or equal to the proportion of the factor affecting the decision-making corresponding to the user behavior of different types of elements, then the execution device 410 determines the probability that the position in the fifth sequence representation is the same position in the second sequence representation.

[0158] The above constraints can be multiple loss functions. If the output layer 840 is trained based on multiple loss functions during the training phase, then when the output layer 840 sorts the second sequence representation according to the first sequence representation, it ensures that the sorting result meets the constraints, namely the balance between the platform's revenue and the fairness of exposure.

[0159] The multiple loss function includes a first loss function and a second loss function. The first loss function calculates the loss corresponding to the occurrence of user behavior when different types of elements are placed at different positions in the output sequence. The second loss function calculates the loss corresponding to the gain when different types of elements are placed at different positions in the output sequence.

[0160] For example, the first loss function is the loss function for the revenue enhancement task, i.e. The second loss function is the loss function for predicting the type of task, i.e. .

[0161] The first loss function is the signal that triggers user behavior based on elements in the output sequence during the training of the rearranged neural network model 800. and The cross-entropy loss function is obtained by calculating its formula as follows:

[0162] Formula (10)

[0163] The second loss function is the root through... One-hot encoding to construct tags Then use labels. and The cross-entropy loss function is obtained by calculating its formula as follows:

[0164] Formula (11)

[0165] Optionally, the multiple loss functions may also include a third loss function, which is used to optimize the fairness of exposure among different types of elements. For example, through... Calculate the exposure distribution of different types of elements in the current output sequence of the rearranged neural network model 800. Distribution of exposure numbers With preset distribution Calculate the KL divergence to obtain the third loss function. Its formula is expressed as follows:

[0166] Formula (12)

[0167] Among them, the preset distribution This could be the exposure ratio set by the upstream providers or platforms of different source sequences. For example, the ideal exposure ratio of ad application sequences, joint operation application sequences, and organic application sequences in the output sequence is 3:3:4, 2:4:4, etc. (Exposure quantity distribution) It can be the exposure ratio of elements from different source sequences in the output sequence.

[0168] Combining the first, second, and third loss functions in the output layer 840 yields a multi-loss function. Its formula is expressed as follows:

[0169] Formula (13)

[0170] in, It is a weighted hyperparameter used to adjust the balance between revenue-enhancing tasks and prediction-type tasks.

[0171] In practical applications of the ranking problem, the ideal balance between revenue and exposure fairness is not static but dynamic. For example, at one time, more attention may be paid to platform revenue, while at another time, more attention may be paid to the fairness of exposure. Therefore, the adjustability of the aforementioned weight hyperparameters allows the Rearrangement Neural Network 800 to dynamically adapt to different objective weight measures without using a large amount of data.

[0172] Step 807: Output layer 840 determines the types of elements placed at different positions in the output sequence based on the fifth sequence representation.

[0173] For the specific steps of predicting the types of elements placed at different positions in the output sequence based on the fifth sequence characterization in output layer 850, please refer to step 730, which will not be repeated here.

[0174] In this way, the rearrangement neural network 800 is trained by introducing a reward enhancement task and a related loss function, so that the trained rearrangement neural network 800 can perceive which type of element may bring higher reward, and arrange the elements of this type as high as possible in the position of expected reward in the output sequence, so as to guide the prediction task to optimize in the direction of higher reward, thereby improving the overall reward of the platform.

[0175] In this embodiment, the function of the rearranged neural network model 800 is to predict an output sequence with a good balance between the two indicators of platform revenue and exposure fairness. Therefore, the structure of the rearranged neural network model 800 includes a dual-channel gated recurrent unit composed of a first gated recurrent unit 831 and a second gated recurrent unit 832, as well as the UOT module and CPT module of the output layer 840. In other possible embodiments, the function of the rearranged neural network model 800 can also be to maintain the balance between two indicators other than platform revenue and exposure fairness. For example, the two indicators can be the platform's total exposure and exposure fairness, the platform's total exposure and platform revenue, and the upstream provider's revenue and exposure fairness, etc.

[0176] The aforementioned rearrangement neural network model 800, along with existing deep reinforcement learning-based advertising recommendation (Deep Reinforcement Learning for Online Advertising in Recommender, DEAR) and crossDQN (cross Deep Q-Network) models, models the information at the category level and attribute level during the rearrangement process through dual-channel feature extraction. For the category level information, MA-GRU is designed to introduce statistical features to model the relationship between elements of different source sequences, enabling the rearrangement neural network model 800 to learn more comprehensive data.

[0177] Therefore, the data processing method implemented by the aforementioned rearrangement neural network model 800 can introduce the statistical features of different source sequences, more accurately model the differences between source sequences of different types of elements through MA-GRU and GUR, and balance the platform's benefits and exposure fairness through multi-task learning. It learns a more suitable rearrangement strategy from the dynamic training process, and improves the balance between the output sequence's benefits and exposure fairness on the platform while preserving the order of the source sequences.

[0178] The above text combined Figure 8 and Figure 9 The steps of the data processing method for the rearranged neural network model 800 are described in detail. In this application, the subsequent output sequence is sorted based on the comprehensive sequence representation of at least one source sequence. Since the output sequence contains the statistical characteristics of at least one source sequence, it can more accurately represent the differences between source sequences of different types of elements. This more accurately balances the impact of expected benefit factors and factors that trigger user behavior on decision-making when sorting the output sequence, improving the balance between the output sequence and the fairness of platform benefits and exposure. The balance between the output sequence of the rearranged neural network model 800 and the fairness of platform benefits and exposure is then compared with existing technologies.

[0179] Since there is currently no publicly available dataset specifically for the merging task, this embodiment selects the three most frequently occurring categories from the Personalized Re-ranking for Recommendation (PRM) model dataset as the source sequences to be merged, and constructs new data according to the order in which the items appear in the original dataset. In the constructed PRM dataset, the characteristics of the items include their category, number, purchasing power, price, etc.

[0180] Since the goal of mixed-item ranking is to maximize the platform's revenue and balance the fairness of exposure among different types of items, two metrics, "utility" and "ratio," are used to evaluate the platform's revenue and the fairness of exposure, respectively. A higher utility indicates that the mixed-item ranking system can generate more revenue for the platform, while a ratio closer to 1 indicates better exposure fairness. Furthermore, an "alpha-revenue" mechanism is also proposed. The metric of "-utility" is used to measure whether a mixed-ranking system can achieve a balance between revenue from different categories. When a mixed-ranking system consistently exposes users to items of the same category, -utility decays exponentially, meaning that continuously recommending the same type of items will reduce subsequent benefits.

[0181] In this embodiment, common rearrangement neural network models in the prior art are selected, including the Maximum Marginal Relevance (MMR) model, the LinkedIn-Det model, and the deep hierarchical category-based recommender system (DHCRS).

[0182] Implementation examples and method comparisons of rearranged neural network model 800 on the PRM dataset are as follows: Figure 10 and Figure 11 As shown. Among them, Figure 10 Pareto fronts for different hybridization methods were plotted, representing the optimal combination of utility and ratio metrics achievable by each method. Figure 10 In the diagram, the closer the data points represented by different fusion methods are to the bottom right corner, the greater the benefits and the better the fairness of the model, which means it is more in line with the requirements of the fusion system. Figure 11 In this context, "Original" refers to the initial mixing sequence. The results. On the PRM dataset, we focus on the performance of the top 10 items in the mixed sequence. Then, in this embodiment, we constrain the value of utility@10 to be around 0.0385, 0.0355, and 0.0325, and compare the ratio@10 metric of different methods. -utility@10 metrics, displaying their results in Figure 11 middle.

[0183] The results show that, compared with other common rearranged neural network models in existing technologies, rearranged neural network model 800 achieves the best results on both utility@10 and ratio@10 evaluation metrics, and the Pareto front formed by the numerous experimental results generated by rearranged neural network model 800 is closer to... Figure 10 The bottom right corner. Furthermore, when this embodiment restricts the Utility metric to a very small range for comparison with other metrics, the Ratio of the rearranged neural network model 800 is... The utility metric also surpasses other common rearrangement models. The rearrangement neural network model 800 of this invention is primarily applied to mixed-row scenarios in application markets, but can also be applied to similar mixed-row scenarios, such as news feed recommendation systems and shopping platform item recommendation systems.

[0184] The above text combined Figures 7-11 The data processing method provided according to this embodiment and its beneficial effects are described in detail below. Figure 12 This describes the data processing apparatus provided in this embodiment.

[0185] Figure 12 This is a schematic diagram of a possible data processing apparatus provided in this embodiment. The data processing apparatus can be used to implement the functions of the execution device in the above method embodiments, and therefore can also achieve the beneficial effects of the above method embodiments. In this embodiment, the data processing apparatus can be as follows: Figure 4 The execution device 410 shown can also be a module (such as a chip) applied to a server.

[0186] The data processing device 1200 includes a feature extraction module 1210 and a task prediction module 1220. The data processing device 1200 is used to implement the functions of the execution device 410 in the method embodiment shown in Figure 7 above.

[0187] The feature extraction module 1210 is used to combine the statistical features, category features, and attribute features of at least one source sequence into a comprehensive sequence representation. Each source sequence in the at least one source sequence includes at least one element belonging to the same category. The statistical features are the statistical characteristics of the at least one source sequence, and the category features and attribute features are the features of the elements contained in each source sequence. For example, the feature extraction module 1210 is used to perform the above... Figure 7 Step 710 is shown in the figure.

[0188] The task prediction module 1220 is used to determine a first sequence representation and a second sequence representation based on the comprehensive sequence representation. The first sequence representation represents the expected reward at different positions in the output sequence, and the second sequence representation represents the probability of user behavior occurring when different types of elements are placed at different positions in the output sequence. For example, the task prediction module 1220 is used to perform the above... Figure 7 Step 720 is shown in the figure.

[0189] The task prediction module 1220 is further configured to, under constraints, determine the order of elements contained in at least one source sequence in the output sequence based on a first sequence representation and a second sequence representation. The constraints indicate the weight of factors influencing the decision-making process, including factors related to expected returns and factors corresponding to different types of elements, when determining the placement of elements at each position in the output sequence. For example, the task prediction module 1220 is used to perform the above... Figure 7 Step 730 is shown in the figure.

[0190] As one possible implementation, the feature extraction module 1210 is specifically used for:

[0191] The third sequence representation is determined based on statistical and categorical characteristics. The third sequence representation is used to represent the probability of placing different types of elements at different positions in the output sequence.

[0192] The fourth sequence representation is determined based on the attribute features. The fourth sequence representation is used to represent the probability of placing different elements at different positions in the output sequence.

[0193] Feature fusion is performed on the third sequence representation and the fourth sequence representation to obtain a comprehensive sequence representation.

[0194] As one possible implementation, the feature extraction module 1210 is specifically used for:

[0195] The third sequence representation is determined based on statistical and categorical characteristics using gated cyclic units;

[0196] The gated recurrent unit is a hidden layer in the rearranged neural network model. The gated recurrent unit includes an activation function, which is used to determine the sequence probability of placing elements of different types of source sequences at different positions in the output sequence based on the category features and statistical features. The sequence probability is used to calculate the candidate hidden state.

[0197] As one possible implementation, each source sequence also includes text features and user features for at least one element. The text features represent user behavior information for the element, including the time, location, and / or user device model of the user behavior. The user features represent a user profile, including the user's gender, occupation, and / or age. The feature extraction module 1210 is specifically used for:

[0198] Perform a dot product on the third sequence representation and the fourth sequence representation to obtain the dot product result;

[0199] Feature fusion is performed on text features, user features, third sequence representation, and fourth sequence representation to obtain a comprehensive sequence representation.

[0200] As one possible implementation, the feature extraction module 1210 is also used for:

[0201] At least one source sequence is merged to obtain a mixed sequence;

[0202] Statistical features are obtained by embedding sequence features from at least one source sequence;

[0203] Embedding operations are performed on the features of the elements contained in the mixed sequence to obtain category features and attribute features.

[0204] As one possible implementation, the merging operation is performed by the input layer in the rearranged neural network model, the embedding operation is performed by the embedding layer in the rearranged neural network model, and the input layer is connected to the hidden layer of the rearranged neural network model through the embedding layer.

[0205] As one possible implementation, the hidden layer also includes a fully connected layer, which comprises a first multilayer perceptron and a second multilayer perceptron. The feature extraction module 1210 is specifically used for:

[0206] Based on the comprehensive sequence list and the user behavior benefits at different locations, the first sequence representation is determined using the first multilayer perceptron. The user behavior benefits at different locations are used to represent the benefits of user behavior occurring at different locations in the output sequence.

[0207] The second sequence representation is determined using a second multilayer perceptron based on the integrated sequence list.

[0208] As one possible implementation, the task prediction module 1220 is specifically used for:

[0209] Under constraints, the second sequence representation is sorted according to the first sequence representation to obtain the fifth sequence representation. The fifth sequence representation is used to represent the probability of different elements being placed at different positions in the output sequence after considering the factors of comprehensive expected return and the factors of user behavior corresponding to different types of elements.

[0210] For each position in the output sequence, the element placed at each position is determined based on the fifth sequence representation, which is the element with the highest probability of being placed at that position.

[0211] As one possible implementation, the task prediction module 1220 is specifically used for:

[0212] Under constraints, the second sequence representation is sorted according to the first sequence representation to obtain the fifth sequence representation. The fifth sequence representation is used to represent the probability of different types of elements being placed at different positions in the output sequence after considering the factors of comprehensive expected return and the factors of user behavior corresponding to different types of elements.

[0213] For the first position of the output sequence, the type of the element placed at the first position is determined according to the fifth sequence representation, which is the type with the highest probability of being the type of the element placed at the first position. The first position is any position in the output sequence.

[0214] The element to be placed is determined from the unplaced elements in the source sequence corresponding to the type of element to be placed in the first position according to the specified order. The element to be placed is the first element among the unplaced elements arranged in the specified order.

[0215] As one possible implementation, the rearranged neural network model also includes an output layer. The rearranged neural network is trained based on multiple loss functions, which are used to characterize the constraints. The multiple loss functions include a first loss function and a second loss function. The first loss function is used to calculate the loss corresponding to the occurrence of user behavior when different types of elements are placed at different positions in the output sequence, and the second loss function is used to calculate the loss corresponding to the gain when different types of elements are placed at different positions in the output sequence.

[0216] As one possible implementation, the multiple loss function also includes a third loss function, which is used to calculate the loss between the distribution of different types of elements in the output sequence and a preset distribution, where the preset distribution includes a preset proportion of different types of elements in the output sequence.

[0217] As one possible implementation, the first and second loss functions in the multiple loss functions are each set with weight hyperparameters.

[0218] As one possible implementation, the first loss function is obtained by calculating the cross-entropy loss function between the user behavior corresponding to the elements in the output sequence and the first sequence representation during the training phase of the rearranged neural network model.

[0219] As one possible implementation, the second loss function is obtained by calculating the cross-entropy loss function between the labels constructed based on the second sequence representation and the fifth sequence representation during the training phase of the rearranged neural network model.

[0220] It should be understood that the data processing device 1200 in this application embodiment can be implemented by a CPU, GPU, NPU, ASIC, or a programmable logic device (PLD). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof. Furthermore, it can be implemented in software. Figure 7 In the method shown, each module of the data processing device 1200 can also be a software module.

[0221] The data processing apparatus 1200 of this application embodiment can correspond to executing the method described in the embodiments of this application, and the above and other operations and / or functions of each unit in the data processing apparatus 1200 are respectively for implementing Figure 7 For the sake of brevity, the corresponding processes of each method in the code will not be elaborated here.

[0222] This application also provides a computing device, please refer to... Figure 13 , Figure 13 This is a schematic diagram of a computing device provided in an embodiment of this application. The computing device 1300 includes a memory 1301, a processor 1302, a communication interface 1303, and a bus 1304. The memory 1301, processor 1302, and communication interface 1303 are interconnected via the bus 1304. In this embodiment, the computing device 1300 may be... Figure 7 The execution device 410 or training device 420 in the middle.

[0223] The memory 1301 can be a read-only memory, a static storage device, a dynamic storage device, or a random access memory. The memory 1301 can store computer instructions. When the computer instructions stored in the memory 1301 are executed by the processor 1002, the processor 1302 and the communication interface 1303 are used to execute the steps in the data processing method of the software system. For example, the communication interface 1303 is used to execute the above... Figure 7 The processor 1302 is used to execute the steps in the data processing method shown above. Figure 7 Steps 710, 720, and 730 in the data processing method shown, as well as the above... Figure 12 The data processing device 1200 describes the functions of the feature extraction module 1210 and the task prediction module 1220. The memory can also store data sets; for example, a portion of the storage resources in memory 1301 is divided into a region for storing programs that implement the functions of the neural network model in the embodiments of this application.

[0224] Processor 1302 may be a general-purpose CPU, an application-specific integrated circuit (ASIC), a GPU, or any combination thereof. Processor 1302 may include one or more chips. Processor 1302 may include an AI accelerator, such as an NPU.

[0225] The communication interface 1303 uses a transceiver module, such as, but not limited to, a transceiver, to enable communication between the computing device 1300 and other devices or communication networks. For example, the computing device 1300 can obtain training data and feedback on the trained neural network through the communication interface 1303.

[0226] Bus 1304 may include a pathway for transmitting information between various components of computing device 1300 (e.g., memory 1301, processor 1302, communication interface 1303).

[0227] The computing device 1300 can be a computer (e.g., a server) in a cloud data center, or a computer in an edge data center, or a terminal.

[0228] The functionality of the training device 420 can be deployed on each computing device 1300. For example, a GPU is used to implement the functionality of the training device 420.

[0229] For the functions of the training device 420 and the execution device 410 deployed within the same computing device 1300, the training device 420 can communicate with the execution device 410 via the bus 1304.

[0230] Regarding the functions of the training device 420 and the execution device 410 deployed within different computing devices 1300, the training device 420 can communicate with the execution device 410 through a communication network.

[0231] The method steps in this embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Additionally, the ASIC can reside in a terminal device. Alternatively, the processor and storage medium can exist as discrete components in a network device or terminal device.

[0232] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are performed entirely or partially. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user equipment, or other programmable device. The computer program or instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; it can also be an optical medium, such as a digital video disc (DVD); or it can be a semiconductor medium, such as a solid-state drive (SSD). The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data processing method, characterized in that, include: The statistical features, categorical features, and attribute features of at least one source sequence are combined into a comprehensive sequence representation. Each source sequence in the at least one source sequence includes at least one element belonging to the same category. The statistical features are the statistical features of the at least one source sequence, and the categorical features and attribute features are the features of the elements contained in each source sequence. Based on the comprehensive sequence representation, a first sequence representation and a second sequence representation are determined. The first sequence representation is used to represent the expected revenue at different positions in the output sequence, and the second sequence representation is used to represent the probability of user behavior occurring when different types of elements are placed at different positions in the output sequence. Under constraints, the order of elements contained in the at least one source sequence in the output sequence is determined based on the first sequence representation and the second sequence representation. The constraints are used to indicate the weight of the expected benefit factor and the user behavior factor corresponding to the different types of elements in influencing the decision when determining the element placed at each position in the output sequence.

2. The method according to claim 1, characterized in that, The method of combining the statistical characteristics, categorical characteristics, and attribute characteristics of at least one source sequence into a comprehensive sequence representation includes: A third sequence representation is determined based on the statistical characteristics and the categorical characteristics. The third sequence representation is used to represent the probability of placing elements of different types at different positions in the output sequence. A fourth sequence representation is determined based on the attribute features, and the fourth sequence representation is used to represent the probability of placing different elements at different positions in the output sequence; The third sequence representation and the fourth sequence representation are fused to obtain the comprehensive sequence representation.

3. The method according to claim 2, characterized in that, The step of determining the third sequence characterization based on the statistical characteristics and the categorical characteristics includes: The third sequence representation is determined using a gated loop unit based on the statistical features and the categorical features; The gated recurrent unit belongs to the hidden layer of the rearranged neural network model. The gated recurrent unit includes an activation function, which is used to determine the sequence probability of placing elements of different types of source sequences at different positions in the output sequence according to the category features and the statistical features. The sequence probability is used to calculate the candidate hidden state.

4. The method according to claim 2 or 3, characterized in that, Each source sequence further includes text features and user features of at least one element. The text features are used to represent user behavior information for the element. The user behavior information includes the time, location and / or user device model of the user behavior. The user features are used to represent a user profile. The user profile includes the user's gender, occupation and / or age. The feature fusion of the third sequence representation and the fourth sequence representation to obtain the comprehensive sequence representation includes: Perform a dot product on the third sequence representation and the fourth sequence representation to obtain the dot product result; The text features, user features, and dot product results are fused to obtain the comprehensive sequence representation.

5. The method according to claim 4, characterized in that, Before combining the statistical characteristics, categorical characteristics, and attribute characteristics of at least one source sequence into a comprehensive sequence characterization, the method further includes: The at least one source sequence is merged to obtain a mixed sequence; The statistical features are obtained by embedding the sequence features of the at least one source sequence. The features of the elements contained in the mixed sequence are embedded to obtain the category features and the attribute features.

6. The method according to claim 5, characterized in that, The merging operation is performed by the input layer in the rearranged neural network model, and the embedding operation is performed by the embedding layer in the rearranged neural network model. The input layer is connected to the hidden layer of the rearranged neural network model through the embedding layer.

7. The method according to claim 6, characterized in that, The hidden layer further includes a fully connected layer, which comprises a first multilayer perceptron and a second multilayer perceptron. The step of determining the first sequence representation and the second sequence representation based on the synthesized sequence representation includes: Based on the comprehensive sequence list and the user behavior benefits at different locations, the first sequence representation is determined using the first multilayer perceptron, whereby the user behavior benefits at different locations in the output sequence are used to represent the benefits of user behavior occurring at different locations. The second sequence representation is determined using the second multilayer perceptron based on the comprehensive sequence list.

8. The method according to claim 7, characterized in that, Determining the order of elements contained in the at least one source sequence in the output sequence based on the first sequence representation and the second sequence representation under constraints includes: Under the constraints, the second sequence representation is sorted according to the first sequence representation to obtain a fifth sequence representation. The fifth sequence representation is used to represent the probability that different elements are placed at different positions in the output sequence after combining the factors of the expected return and the factors of user behavior corresponding to the different types of elements. For each position in the output sequence, the element placed at each position is determined to be the element with the highest probability of being the element placed at each position, based on the fifth sequence characterization.

9. The method according to claim 7, characterized in that, Determining the order of elements contained in the at least one source sequence in the output sequence based on the first sequence representation and the second sequence representation under constraints includes: Under the constraints, the second sequence representation is sorted according to the first sequence representation to obtain a fifth sequence representation. The fifth sequence representation is used to represent the probability of different types of elements being placed at different positions in the output sequence after combining the factors of the expected return and the factors of user behavior corresponding to the different types of elements. For the first position of the output sequence, the type of the element placed at the first position is determined to be the type of the element placed at the first position with the highest probability according to the fifth sequence characterization, and the first position is any position in the output sequence. The element to be placed is determined from the unplaced elements in the source sequence corresponding to the type of the element placed at the first position according to a specified order, wherein the element to be placed is the first element among the unplaced elements arranged in the specified order.

10. The method according to claim 7, characterized in that, The rearranged neural network model further includes an output layer. The rearranged neural network is trained based on multiple loss functions. The multiple loss functions are used to characterize the constraints. The multiple loss functions include a first loss function and a second loss function. The first loss function is used to calculate the loss corresponding to the occurrence of user behavior when different types of elements are placed at different positions in the output sequence. The second loss function is used to calculate the loss corresponding to the gain when different types of elements are placed at different positions in the output sequence.

11. The method according to claim 10, characterized in that, The multiple loss function further includes a third loss function, which is used to calculate the loss between the distribution of different types of elements in the output sequence and a preset distribution, wherein the preset distribution includes a preset proportion of different types of elements in the output sequence.

12. The method according to claim 10 or 11, characterized in that, The first loss function and the second loss function in the multiple loss functions are each set with weight hyperparameters.

13. The method according to claim 12, characterized in that, The first loss function is obtained by calculating the cross-entropy loss function between the user behavior corresponding to the elements in the output sequence and the first sequence representation during the training phase of the rearranged neural network model.

14. The method according to claim 13, characterized in that, The second loss function is obtained during the training phase of the rearranged neural network model by calculating the cross-entropy loss function based on the labels constructed according to the second sequence representation and the fifth sequence representation. The fifth sequence representation is used to represent the probability of different types of elements being placed at different positions in the output sequence after combining the factors of the expected return and the factors of user behavior corresponding to the different types of elements.

15. A data processing apparatus, characterized in that, include: The feature extraction module is used to combine the statistical features, category features and attribute features of at least one source sequence into a comprehensive sequence representation. Each source sequence in the at least one source sequence includes at least one element belonging to the same category. The statistical features are the statistical features of the at least one source sequence, and the category features and attribute features are the features of the elements contained in each source sequence. The task prediction module is used to determine a first sequence representation and a second sequence representation based on the comprehensive sequence representation. The first sequence representation is used to represent the expected reward at different positions in the output sequence, and the second sequence representation is used to represent the probability of user behavior occurring when different types of elements are placed at different positions in the output sequence. The task prediction module is further configured to determine, under constraints, the order of elements contained in the at least one source sequence in the output sequence based on the first sequence representation and the second sequence representation, wherein the constraints are used to indicate the weight of the expected benefit factor and the factor of user behavior corresponding to the different types of elements influencing the decision when determining the element placed at each position in the output sequence.

16. The apparatus according to claim 15, characterized in that, The feature extraction module is specifically used for: A third sequence representation is determined based on the statistical characteristics and the categorical characteristics. The third sequence representation is used to represent the probability of placing elements of different types at different positions in the output sequence. A fourth sequence representation is determined based on the attribute features, and the fourth sequence representation is used to represent the probability of placing different elements at different positions in the output sequence; The third sequence representation and the fourth sequence representation are fused to obtain the comprehensive sequence representation.

17. The apparatus according to claim 16, characterized in that, The feature extraction module is specifically used for: The third sequence representation is determined using a gated loop unit based on the statistical features and the categorical features; The gated recurrent unit belongs to the hidden layer of the rearranged neural network model. The gated recurrent unit includes an activation function, which is used to determine the sequence probability of placing elements of different types of source sequences at different positions in the output sequence according to the category features and the statistical features. The sequence probability is used to calculate the candidate hidden state.

18. The apparatus according to claim 16 or 17, characterized in that, Each source sequence further includes text features and user features of at least one element. The text features represent user behavior information for the element, including the time, location, and / or user device model of the user behavior. The user features represent a user profile, including the user's gender, occupation, and / or age. The feature extraction module is specifically used for: Perform a dot product on the third sequence representation and the fourth sequence representation to obtain the dot product result; The text features, user features, third sequence representation, and fourth sequence representation are fused to obtain the comprehensive sequence representation.

19. The apparatus according to claim 18, characterized in that, The feature extraction module is also used for: The at least one source sequence is merged to obtain a mixed sequence; The statistical features are obtained by embedding the sequence features of the at least one source sequence. The features of the elements contained in the mixed sequence are embedded to obtain the category features and the attribute features.

20. The apparatus according to claim 19, characterized in that, The merging operation is performed by the input layer in the rearranged neural network model, and the embedding operation is performed by the embedding layer in the rearranged neural network model. The input layer is connected to the hidden layer of the rearranged neural network model through the embedding layer.

21. The apparatus according to claim 20, characterized in that, The hidden layer further includes a fully connected layer, which comprises a first multilayer perceptron and a second multilayer perceptron. The feature extraction module is specifically used for: Based on the comprehensive sequence list and the user behavior benefits at different locations, the first sequence representation is determined using the first multilayer perceptron, whereby the user behavior benefits at different locations in the output sequence are used to represent the benefits of user behavior occurring at different locations. The second sequence representation is determined using the second multilayer perceptron based on the comprehensive sequence list.

22. The apparatus according to claim 21, characterized in that, The task prediction module is specifically used for: Under the constraints, the second sequence representation is sorted according to the first sequence representation to obtain a fifth sequence representation. The fifth sequence representation is used to represent the probability that different elements are placed at different positions in the output sequence after combining the factors of the expected return and the factors of user behavior corresponding to the different types of elements. For each position in the output sequence, the element placed at each position is determined to be the element with the highest probability of being the element placed at each position, based on the fifth sequence characterization.

23. The apparatus according to claim 22, characterized in that, The task prediction module is specifically used for: Under the constraints, the second sequence representation is sorted according to the first sequence representation to obtain a fifth sequence representation. The fifth sequence representation is used to represent the probability of different types of elements being placed at different positions in the output sequence after combining the factors of the expected return and the factors of user behavior corresponding to the different types of elements. For the first position of the output sequence, the type of the element placed at the first position is determined to be the type of the element placed at the first position with the highest probability according to the fifth sequence characterization, and the first position is any position in the output sequence. The element to be placed is determined from the unplaced elements in the source sequence corresponding to the type of the element placed at the first position according to a specified order, wherein the element to be placed is the first element among the unplaced elements arranged in the specified order.

24. The apparatus according to claim 21, characterized in that, The rearranged neural network model further includes an output layer. The rearranged neural network is trained based on multiple loss functions. The multiple loss functions are used to characterize the constraints. The multiple loss functions include a first loss function and a second loss function. The first loss function is used to calculate the loss corresponding to the occurrence of user behavior when different types of elements are placed at different positions in the output sequence. The second loss function is used to calculate the loss corresponding to the gain when different types of elements are placed at different positions in the output sequence.

25. The apparatus according to claim 24, characterized in that, The multiple loss function further includes a third loss function, which is used to calculate the loss between the distribution of different types of elements in the output sequence and a preset distribution, wherein the preset distribution includes a preset proportion of different types of elements in the output sequence.

26. The apparatus according to claim 24 or 25, characterized in that, The first loss function and the second loss function in the multiple loss functions are each set with weight hyperparameters.

27. The apparatus according to claim 26, characterized in that, The first loss function is obtained by calculating the cross-entropy loss function between the user behavior corresponding to the elements in the output sequence and the first sequence representation during the training phase of the rearranged neural network model.

28. The apparatus according to claim 27, characterized in that, The second loss function is obtained during the training phase of the rearranged neural network model by calculating the cross-entropy loss function based on the labels constructed according to the second sequence representation and the fifth sequence representation. The fifth sequence representation is used to represent the probability of different types of elements being placed at different positions in the output sequence after combining the factors of the expected return and the factors of user behavior corresponding to the different types of elements.

29. A computing device, characterized in that, The computing device includes a memory and at least one processor, the memory being used to store a set of computer instructions; when the processor executes the set of computer instructions, it performs the operational steps of the method described in any one of claims 1-14.

30. A computer system, characterized in that, The computer system includes a training device and an execution device. The training device is used to train a model to obtain a rearranged neural network model, and the execution device is used to execute the operation steps of the method described in any one of claims 1-14 using the rearranged neural network model.

31. A readable storage medium, characterized in that, The readable storage medium includes a computer program or instructions that, when executed on a computer, cause the computer to perform the operational steps of the method described in any one of claims 1-14.

Images