Behavior prediction, model training method, device and storage medium

Abstract

This application provides a behavior prediction, model training method, device, and storage medium. In the behavior prediction method, when predicting the behavior outcome of a target user, the task type can be input as a prompt into the behavior prediction model. The behavior prediction model can perform conditional attention encoding on the data to be predicted based on the task type and the importance of pre-learned user data of different dimensions in different tasks, thereby accurately extracting the features relevant to that task type from the data to be predicted. Based on the extracted features, the behavior prediction model can accurately predict the behavior outcome of the data to be predicted at the corresponding behavioral stage of the task type.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a behavior prediction, model training method, device and storage medium. Background Technology

[0002] In the financial lending sector, loan transactions mainly include application, disbursement, and repayment. In the application stage, users must submit a loan application. If the loan application is approved, the disbursement stage begins, where users can withdraw the funds. In the repayment stage, some users may default on payments. Therefore, the financial lending sector faces credit risk issues.

[0003] To avoid credit risk, a risk behavior prediction model can be trained and used to detect credit risk during the application process. However, the number of users who actually default is relatively small, meaning there is insufficient sample size for training the risk behavior prediction model. This results in low accuracy of the trained model and an inability to accurately output risk detection results. Therefore, a new solution is needed. Summary of the Invention

[0004] This application provides a method, device, and storage medium for behavior prediction and model training in a multi-stage service scenario, which can accurately predict user behavior and improve the accuracy of behavior prediction results.

[0005] This application provides a behavior prediction method in a multi-stage service scenario, comprising: responding to a behavior prediction request for a target user and obtaining data to be predicted corresponding to the behavior prediction request; the data to be predicted includes: a task type corresponding to the prediction request and multi-dimensional user data of the target user; the task type corresponds to a target behavior stage in the multi-stage service scenario; using a behavior prediction model, taking the task type as a condition, extracting features from the data to be predicted based on a conditional attention mechanism to obtain conditional attention features of the data to be predicted; and determining a prediction label for the data to be predicted under the task type based on the conditional attention features to obtain the behavior prediction result of the target user in the target behavior stage.

[0006] Optionally, using the task type as a condition, feature extraction is performed on the data to be predicted based on a conditional attention mechanism to obtain conditional attention features of the data to be predicted. This includes: vectorizing the multi-dimensional user data and task type of the data to be predicted to obtain user feature vectors and task feature vectors; inputting the user feature vectors and task feature vectors into at least one transformation network; using the at least one transformation network, employing a first weight parameter corresponding to the task type and a second attention weight corresponding to the multi-dimensional user data, performing conditional attention-based feature encoding on the task feature vectors and user feature vectors to obtain at least one encoding result; wherein, in two adjacent transformation networks, the encoding result output by the previous transformation network is input into the next transformation network along with the user feature vectors and task feature vectors; and obtaining the encoding result output by the last transformation network in the at least one transformation network as the conditional attention feature.

[0007] Optionally, using a first weight parameter corresponding to the task type and a second weight parameter corresponding to the multi-dimensional user data, conditional attention-based feature encoding is performed on the task feature vector and the user feature vector respectively to obtain at least one encoding result, including: in any transformation network, embedding the identification information of the transformation network into the task feature vector using an embedding layer to obtain an updated task feature vector; using a linear transformation layer, performing linear transformations on the updated task feature vector and the user feature vector respectively using the first weight parameter and the second weight parameter to obtain a transformation matrix of the updated task feature vector and a transformation matrix of the user feature vector; concatenating the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector to obtain a conditional encoding matrix; using a multi-head attention layer, performing multi-head attention calculation on the conditional encoding matrix to obtain a first conditional attention distribution vector of the data to be predicted; and using an encoding output layer, outputting the conditional attention features corresponding to the transformation network based on the first conditional attention distribution vector and the residual information input to the transformation network, as the encoding result of the transformation network.

[0008] Optionally, linear transformations are performed on the updated task feature vector and the user feature vector using the first weight parameter and the second weight parameter, respectively, to obtain the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector. This includes: performing a linear transformation calculation on the task feature vector using the first weight parameter to obtain the key matrix and value matrix of the task feature vector; performing a linear transformation calculation on the user feature vector using the second weight parameter to obtain the key matrix, query matrix, and value matrix of the user feature vector; concatenating the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector to obtain a conditional encoding matrix, including: concatenating the key matrix corresponding to the task feature vector and the key matrix corresponding to the user feature vector to obtain a target key matrix containing the task type; concatenating the value matrix corresponding to the task feature vector and the value matrix corresponding to the user feature vector to obtain a target value matrix containing the task type; and using the target key matrix, the target value matrix, and the query matrix as the conditional encoding matrix.

[0009] Optionally, outputting the conditional attention features of the data to be predicted based on the first conditional attention distribution vector and the residual information input to the transform network includes: performing a residual concatenation between the first conditional attention distribution vector and the target residual information; normalizing the first conditional attention distribution vector after residual concatenation to obtain a second conditional attention distribution vector; performing feedforward calculation on the second conditional attention distribution vector to obtain a third conditional attention distribution vector; performing a residual concatenation between the second conditional attention distribution vector and the third conditional attention distribution vector; and normalizing the third conditional attention distribution vector after residual concatenation to obtain the conditional attention features.

[0010] Optionally, before adding the target residual information to the first conditional attention distribution vector of the data to be predicted, the method further includes: if the transformation network is the first transformation network among the at least one transformation network, then the user feature vector is determined as the target residual information; or, if the transformation network is not the first transformation network among the at least one transformation network, then the conditional attention feature output by the previous transformation network of the transformation network is determined as the target residual information.

[0011] Optionally, the multi-stage service scenario includes: a credit scenario; the task type includes: an application approval result prediction task, a disbursement behavior prediction task, or a repayment behavior prediction task.

[0012] This application embodiment also provides a method for training a behavior prediction model in a multi-stage service scenario, comprising: acquiring sample data in a multi-stage service scenario, wherein any sample data includes multi-dimensional user data of a single user, a task type, and a true label of the sample data under the task type; wherein the task type is determined according to the behavior stage corresponding to the sample data in the multi-stage service scenario; the multiple sample data correspond to multiple behavior stages; using a behavior prediction model, taking the task type of each of the multiple sample data as a condition, performing feature extraction on the multiple sample data based on a conditional attention mechanism to obtain conditional attention features of the multiple sample data; determining the predicted label of the multiple sample data under their respective task types based on the conditional attention features of the multiple sample data; and training the behavior prediction model with the goal of the specified range to which the difference between the true label and the predicted label of the multiple sample data under their respective task types converges.

[0013] This application also provides an electronic device, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to perform the steps in the method provided in this application.

[0014] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the method provided in this application.

[0015] In this embodiment, when predicting the behavioral outcome of a target user, the task type can be input as a prompt into the behavior prediction model. The behavior prediction model can perform conditional attention encoding on the data to be predicted based on the task type and the importance of pre-learned user data of different dimensions in different tasks, thereby accurately extracting the features relevant to that task type from the data to be predicted. Based on the extracted features, the behavior prediction model can accurately predict the behavioral outcome of the data to be predicted at the behavioral stage corresponding to the task type. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 A flowchart illustrating a behavior prediction model training method in a multi-stage service scenario provided as an exemplary embodiment of this application;

[0018] Figure 2 A schematic diagram of the structure of a behavior prediction model provided in an exemplary embodiment of this application;

[0019] Figure 3 A flowchart illustrating a behavior prediction method in a multi-stage service scenario provided as an exemplary embodiment of this application;

[0020] Figure 4 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of this application. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0023] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0024] It should be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0025] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0026] In view of the technical problem that the accuracy of risk behavior prediction models is low due to sample data selection bias in the prior art, this application provides a solution in some embodiments. The technical solutions provided by the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0027] Figure 1 This is a flowchart illustrating a behavior prediction model training method in a multi-stage service scenario provided by an exemplary embodiment of this application. The method may include, for example: Figure 1 The steps shown are as follows:

[0028] Step 101: Obtain multiple sample data in a multi-stage service scenario. Each sample data includes multi-dimensional user data of a single user, task type, and the real label of the sample data under the task type. The task type is determined based on the behavioral stage corresponding to the sample data in the multi-stage service scenario. The multiple sample data correspond to multiple behavioral stages.

[0029] Step 102: Using the behavior prediction model, the task type of each of the multiple sample data is used as a condition, and feature extraction is performed on the multiple sample data based on the conditional attention mechanism to obtain the conditional attention features of the multiple sample data.

[0030] Step 103: Determine the predicted label of the sample data under their respective task types based on the conditional attention features.

[0031] Step 104: Train the behavior prediction model with the goal of converging the gap between the true labels and predicted labels of the multiple sample data under their respective task types to a specified range.

[0032] The training method provided in this embodiment is used to train a behavior prediction model, which predicts the behavioral outcomes of users in different behavioral stages within a multi-stage service scenario. This multi-stage service scenario may include multiple behavioral stages that are related to each other. For example, this relationship may manifest as the behavioral outcome of a previous stage serving as a prerequisite for entering a subsequent stage. Multi-stage service scenarios may include: credit scenarios, advertising and marketing scenarios, online education scenarios, etc.

[0033] For example, in a credit scenario, the behavioral stages can be divided into the application stage, the disbursement stage, or the repayment stage, based on the process steps. The outcome of the application stage can include: approval or rejection. The outcome of the disbursement stage can include: funds disbursed or not disbursed. The outcome of the repayment stage can include: overdue repayment or no overdue repayment. Sample data in a credit scenario typically consists of user data from loan applicants across historical periods. Based on user data from multiple stages, a behavioral prediction model for the credit scenario can be trained to predict the user's behavioral outcomes at different stages upon receiving a credit application, thereby keeping credit risk within a controllable and perceptible range.

[0034] For example, in advertising and marketing scenarios, the behavioral stages can be divided into ad browsing and purchase stages. The behavioral outcome in the ad clicking stage can include: clicking the ad link or not clicking the ad link. The behavioral outcome in the purchase stage can include: purchasing or not purchasing. User behavior in the ad browsing stage and user behavior in the purchase stage are somewhat correlated; that is, after a user clicks an ad link in the ad browsing stage, they can proceed to the purchase stage. When training a behavioral detection model for advertising and marketing scenarios, joint modeling can be performed based on related prediction tasks for multiple behavioral stages.

[0035] For example, in online education scenarios, behavioral stages can be divided into: introductory, beginner, intermediate, and advanced stages. The behavioral outcome in the introductory stage can include passing or failing the introductory exam. The behavioral outcome in the beginner stage can include passing or failing the beginner exam, and so on. The behaviors across these stages are correlated; that is, a user who passes the introductory exam can enter the beginner stage; a user who passes the beginner exam can enter the intermediate stage; and a user who passes the intermediate exam can enter the advanced stage. When training a behavioral detection model for online education scenarios, joint modeling can be performed based on related prediction tasks across multiple behavioral stages.

[0036] Sample data in multi-stage service scenarios includes sample data corresponding to multiple different behavioral stages. For example, in a credit scenario, some users enter the application stage but fail to pass the approval process, thus preventing them from entering the disbursement stage. Other users pass the approval process and enter the disbursement stage, but have not yet disbursed funds, thus preventing them from entering the repayment stage. Sample data generated in credit scenarios can include sample data from the application stage and sample data from the disbursement stage. Based on the sample data corresponding to different behavioral stages, a behavior detection model can be trained to learn the intrinsic relationship between different user data and behavioral outcomes at different behavioral stages. The sample data for different behavioral stages is labeled with the actual labels corresponding to that stage. For example, in a credit scenario, sample data stopping at the application stage is labeled with the actual label of whether the approval was passed. Sample data stopping at the disbursement stage is labeled with the actual label of whether funds were disbursed. Sample data stopping at the repayment stage is labeled with the actual label of whether the payment is overdue.

[0037] The following will use any sample data as an example to illustrate the training process of the behavior prediction model. When there is batch sample data, the training process of each sample data in the batch sample data can be performed with reference to the training method provided in this embodiment. Any sample data includes multi-dimensional user data of a single user and task type. Multi-dimensional user data refers to data describing the characteristics of a loan applicant user from multiple dimensions. These multiple dimensions may include: monthly sales, average daily order count, debt information, etc. Different dimensions of user data contribute differently to the prediction task at different behavioral stages. The task type is determined based on the behavioral stage corresponding to the sample data in the multi-stage service scenario. Different task types correspond to different behavioral stages of the sample data. For example, if the user corresponding to the sample data stops at the application stage, the task type corresponding to the sample data is the application approval result prediction task. If the user corresponding to the sample data stops at the withdrawal stage, the task type corresponding to the sample data is the withdrawal behavior prediction task. If the user corresponding to the sample data stops at the repayment stage, the task type corresponding to the sample data is the repayment behavior prediction task. The task type can be used as a feature of the sample data for the task dimension to provide task prompts to the behavior prediction model.

[0038] After obtaining sample data from a multi-stage service scenario, the sample data can be input into a behavior prediction model. In this embodiment, the behavior prediction model can be a transformation model based on an attention mechanism. In step 202, the behavior prediction model can be used to extract features from the sample data based on a conditional attention mechanism, using the task type as a condition, to obtain conditional attention features of the sample data. Here, the conditional attention mechanism refers to a feature extraction mechanism that selectively extracts information of interest to a specific condition from the input data under certain conditions. Based on the conditional attention mechanism, the knowledge preferences of different task types can be captured, thereby accurately extracting the knowledge information preferred by that task type when a given task type is used as a prompt word, and accurately outputting the prediction result under that task type. In this embodiment, using the task type as a condition allows for the selective extraction of information of interest to that task from the sample data during feature extraction based on the attention mechanism, resulting in conditional attention features of the sample data.

[0039] In some exemplary embodiments, the behavior prediction model includes at least one transformation network, any of which can use task type as a condition and extract features from sample data based on a conditional attention mechanism to obtain conditional attention features of the sample data.

[0040] Before inputting the sample data into the at least one transformation network, the multi-dimensional user data and task type of the sample data can be vectorized to obtain user feature vectors and task feature vectors. Vectorization maps the multi-dimensional user data and task type to a vector space, facilitating feature extraction in that space. For example, the vectorization result of sample data a may include a task feature vector eT and a user feature vector Ex, where Ex = [e1, e2, ..., em]. Here, e1, e2, ..., em are the vectorization results of m dimensions of the user data, where m is a positive integer. Optionally, the user feature vector and task feature vector can be input into the at least one transformation network. Each transformation network includes weight parameters corresponding to the task type and weight parameters corresponding to the multi-dimensional user data. For ease of description and distinction, the weight parameters corresponding to the task type are described as first weight parameters, and the weight parameters corresponding to the multi-dimensional user data are described as second weight parameters. These "first" and "second" designations are only used to distinguish similar or identical descriptive objects and do not limit the number of parameters. The first weight parameter corresponding to the task type and the second weight parameter corresponding to the multi-dimensional user data are used to extract the attention features of that task type from the sample data. During training, the first and second weight parameters in the network are transformed into parameters to be learned.

[0041] In the behavior prediction model, at least one transformation network can be used to encode the task feature vector corresponding to the task type and the user feature vector corresponding to the multi-dimensional user data using a first weight parameter corresponding to the task type and a second weight parameter corresponding to the multi-dimensional user data, respectively, to obtain at least one encoding result. In adjacent transformation networks, the encoding result output by the preceding transformation network is used as residual information and input into the following transformation network along with the user feature vector and the task feature vector. That is, when multiple transformation networks exist, in adjacent transformation networks, the output layer of the preceding transformation network is connected to the residual connection and the normalization layer of the following transformation network, thereby deepening the behavior prediction model and improving its performance. Based on the above structure, the encoding result output by the last transformation network in the at least one transformation network can be obtained as the conditional attention feature of the sample data.

[0042] The following will take any one of the at least one transformation network as an example to further illustrate the structure of the transformation network and the data processing method.

[0043] Optionally, any transformation network may include at least: an embedding layer, a linear transformation layer, a multi-head attention layer, and an encoding output layer.

[0044] In any transform network, an embedding layer can be used to embed the network's identifier information into the task feature vector to obtain an updated task feature vector. The identifier information of the transform network can be randomly initialized. When multiple transform networks exist, their identifier information differs. By inputting the same task feature vector into different transform networks and embedding the identifier information of each network into the input feature vector, random perturbation can be added to the task vector, distinguishing the processing objects of different transform networks and thus improving the model's prediction performance.

[0045] Optionally, embedding the identifier information of the transform network into the task feature vector can be achieved by: initializing the layer encoding vector corresponding to the identifier information of the transform network; and concatenating the layer encoding vector with the task feature vector to obtain an updated task feature vector. For example, a behavior prediction model includes three transform networks: transform network L1, transform network L2, and transform network L3. The identifier information for transform networks L1, L2, and L3 is L1, L2, and L3, respectively. L1, L2, and L3 are randomly initialized to obtain layer encoding vectors eL1, eL2, and eL3. After the task feature vector eT of sample data a is input into transform network L1, the layer encoding vector eL1 and the task feature vector eT can be concatenated in the embedding layer of transform network L1 to obtain an updated task feature vector eTL1. Similarly, after the task feature vector eT of sample data a is input into the transform network L2, the layer encoding vector eL2 and the task feature vector eT can be concatenated in the embedding layer of the transform network L2 to obtain the updated task feature vector eTL2, which will not be elaborated further.

[0046] The user feature vector output from the embedding layer and the updated task feature vector are input to the linear transformation layer. In the linear transformation layer, a first weight parameter corresponding to the task type and a second weight parameter corresponding to the multi-dimensional user data are used to perform linear transformations on the input updated task feature vector and user feature vector, respectively, to obtain the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector. The linear transformation is used to generate queries, keys, and values ​​based on the input vector. The implementation of the linear transformation will be illustrated below. Optionally, in the linear transformation layer, the first weight parameter can be used to perform a linear transformation calculation on the updated task feature vector to obtain the key matrix and value matrix of the updated task feature vector. The first weight parameter may include two weight parameter matrices: one weight parameter matrix is ​​used to perform a linear transformation calculation on the updated task feature vector to obtain the key matrix; the other weight parameter matrix is ​​used to perform a linear transformation calculation on the updated task feature vector to obtain the value matrix. For example, the weight parameter matrices included in the first weight parameter can be represented as Wkl and Wvl. The linear transformation of the updated task feature vector eTL can be represented as: key matrix Pk = eTL * Wkl, value matrix Pv = eTL * Wvl.

[0047] In the linear transformation layer, a second weight parameter can be used to perform a linear transformation on the user feature vector to obtain the key matrix, query matrix, and value matrix of the user feature vector. The second weight parameter can include three weight parameter matrices: one weight parameter matrix is ​​used to perform the linear transformation on the user feature vector to obtain the key matrix; another weight parameter matrix is ​​used to perform the linear transformation on the user feature vector to obtain the query matrix; and yet another weight parameter matrix is ​​used to perform the linear transformation on the user feature vector to obtain the value matrix. For example, the weight parameter matrices included in the second weight parameter can be represented as: Wk2, Wq, Wv2. The linear transformation process of the user feature vector Ex can be represented as: key matrix K = Ex * Wk2, query matrix Q = Ex * Wq, value matrix V = Ex * Wv2.

[0048] Among them, the first weight parameter and the second weight parameter are parameters to be learned during the training process. They can be continuously optimized as the model iterates during training to learn the features that are of interest to different task types.

[0049] After obtaining the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector based on the above implementation method, the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector can be concatenated to obtain the conditional encoding matrix. Optionally, the key matrix corresponding to the task feature vector can be concatenated with the key matrix corresponding to the user feature vector to obtain a target key matrix containing the task type; that is, the key matrix Pk and the key matrix K are concatenated to obtain the target key matrix KT. The value matrix corresponding to the task feature vector is concatenated with the value matrix corresponding to the user feature vector to obtain a target value matrix containing the task type; that is, the value matrix Pv and the value matrix V are concatenated to obtain the target value matrix VT. The target key matrix KT, the target value matrix VT, and the query matrix Q can be used as the conditional encoding matrix.

[0050] After obtaining the conditional encoding matrix based on the above implementation method, a multi-head attention layer can be used to perform multi-head attention calculation on the conditional encoding matrix to obtain the first conditional attention distribution vector of the data to be predicted. Optionally, the target key matrix, target value matrix, and query matrix obtained in the above steps can be input into the multi-head attention layer. In the multi-head attention layer, multi-head attention calculation is performed on the target key matrix, target value matrix, and query matrix to obtain the first conditional attention distribution vector of the sample data.

[0051] The first conditional attention distribution vector is input to the encoding output layer. In the encoding output layer, the conditional attention features corresponding to the transform network are output based on the first conditional attention distribution vector and the residual information input to the transform network. Optionally, the encoding output layer may include at least a first residual connection and normalization layer (Add&Normlayer), a feedforward layer, and a second residual connection and normalization layer connected in sequence.

[0052] Optionally, after the first conditional attention distribution vector is input into the encoding output layer, a first residual connection and a normalization layer can be used to perform a residual connection between the first conditional attention distribution vector and the target residual information. Here, a residual connection refers to adding the network input (i.e., the first conditional attention distribution vector) and the target residual information. In cases of deep network structures, adding residual information to the network input helps reduce the probability of the vanishing gradient problem during gradient backpropagation to update model parameters.

[0053] Optionally, for any transform network, if the transform network is the first transform network among the at least one transform networks, then the user feature vector is determined as the target residual information. That is, the first residual connection and normalization layer of the first transform network can be connected to the embedding layer to obtain the user feature vector from the embedding layer as the target residual information. Optionally, if the transform network is not the first transform network among the at least one transform networks, then the conditional attention features output by the previous transform network are determined as the target residual information. That is, the first residual connection and normalization layer of the non-first transform network can be connected to the encoding output layer of the previous transform network to obtain the conditional attention features from the encoding output layer of the previous transform network as the target residual information. The first residual connection and normalization layer can normalize the first conditional attention distribution vector after residual connection to obtain the second conditional attention distribution vector.

[0054] The second conditional attention distribution vector is input into the feedforward layer. In the feedforward layer, the second conditional attention distribution vector is fed forward to obtain the third conditional attention distribution vector. The third conditional attention distribution vector can then be input into the second residual connection and normalization layer.

[0055] In the residual connection and normalization layer, the second conditional attention distribution vector and the third conditional attention distribution vector can be residually connected, and the third conditional attention distribution vector after residual connection can be normalized to obtain the conditional attention features.

[0056] After obtaining the conditional attention features, the predicted label for the sample data at the corresponding behavioral stage of the task type can be calculated using these features. Once the predicted label is obtained, the difference between the true label and the predicted label for the sample data at that behavioral stage can be calculated as the training loss of the model. The model is then trained with the goal of the difference between the true label and the predicted label for the sample data at that behavioral stage converging to a specified range. The training loss of the model can be calculated using a binary classification loss function, such as the cross-entropy loss function or the mean squared error loss function, etc., as included but not limited to in this embodiment.

[0057] In this implementation, by adding task types to the sample data, explicit modeling of behavior prediction tasks at different behavioral stages in a multi-stage service scenario is achieved. The behavior prediction model can learn the importance of different dimensions of user data in different tasks based on task types, reducing learning errors caused by sample bias. On the one hand, there is no need to set up multi-task branch networks for behavior prediction tasks at different behavioral stages; a unified model structure can be used to achieve different prediction tasks, simplifying the model structure. On the other hand, based on the learned importance of different dimensions of user data in different tasks, the model can accurately predict the behavioral outcome of the input data at the corresponding behavioral stage of the task type during the inference stage, according to the task type and the input data.

[0058] The following will combine Figure 2 Taking a multi-stage service scenario as an example of a credit scenario, the training method of the behavior prediction model provided in this application embodiment will be further illustrated.

[0059] During training, sample data can be obtained from multiple behavioral stages within the credit scenario. These stages include the application stage, disbursement stage, and repayment stage. Assume the number of sample data points for the application, disbursement, and repayment stages are N1, N2, and N3, respectively. Each sample data point includes m dimensions of feature data, where N1, N2, N3, and m are positive integers. Each dimension describes the feature of the sample data from a specific aspect. For example, in the application stage, user data can be obtained from m dimensions, resulting in multiple sample data points, each with m dimensions. Each sample data point may include m user feature columns, which can be described in the following format: {x1 = xzx, x2 = xxx, ..., xm = xxx}. For example, in this embodiment, the m dimensions may include: the user's monthly sales revenue, average daily order count, debt information, etc.

[0060] Each sample data is labeled with a scenario label. The N1 sample data in the application stage are labeled with whether the application was approved, the N2 sample data in the disbursement stage are labeled with whether there was a disbursement behavior, and the N3 sample data in the repayment stage are labeled with whether an overdue payment occurred.

[0061] In this embodiment, in addition to the aforementioned m dimensions, a task feature dimension is added to the sample data based on the stage to which the sample data belongs. The task feature dimension uses different tasks corresponding to the sample data as prompt words to train the risk behavior prediction model's ability to capture knowledge from the sample data under different task types. Specifically, the N1 sample data points in the application stage are mainly used to train the risk behavior prediction model's ability to predict application results; the N2 sample data points in the disbursement stage are mainly used to train the risk behavior prediction model's ability to predict disbursement behavior; and the N3 sample data points in the repayment stage are mainly used to train the risk detection module's ability to predict overdue behavior. Based on this, a task feature column can be added to the N1 sample data points in the application stage, with the value being "application"; a task feature column can be added to the N2 sample data points in the disbursement stage, with the value being "disbursement"; and a task feature column can be added to the N3 sample data points in the repayment stage, with the value being "risk". Taking the repayment stage as an example, after adding the task feature column, the description of any sample data can be updated to: {task feature column = risk, x1 = xzx, x2 = xxx, ..., xm = xxx}.

[0062] After determining the sample data for each stage based on the above implementation method, the sample data for each stage can be mixed to obtain N sample data and N labels corresponding to each sample data, where N = N1 + N2 + N3.

[0063] Input N sample data into the network model. The following will use any one of the N sample data as an example to illustrate the processing procedure of the network model.

[0064] In the network model, the embedding layer encodes the sample data, resulting in m+1 encoded vectors corresponding to the sample data. In this embodiment, for ease of subsequent calculation, the encoded vectors corresponding to the sample data are divided into two parts: the encoded vector eT for the task feature columns and the encoded vector Ex = [e1, e2, ..., em] for the user feature columns. Each encoded vector e has a dimension of 1×d. The encoded vector eT for the task feature columns can be randomly initialized. When the task values ​​corresponding to the task feature columns are different, different task values ​​can correspond to different initialized encoded vectors eT to distinguish between different task values.

[0065] Based on the task feature encoding vectors and user feature encoding vectors of the sample data obtained through the above steps, a multi-layer task-conditional transformation network (hereinafter referred to as the transformation network) can be input. Multiple transformation networks have similar structures; the following will use any one of these transformation networks as an example for illustrative purposes.

[0066] In a transform network, a vector eL can be used to represent the transform network, and eL can be 1×d-dimensional. After inputting the encoding vector corresponding to the sample data into the transform network, the encoding vector eT of the task feature column and the vector eL corresponding to the transform network can be concatenated to obtain the concatenated vector eTL. The value of vector eL is obtained through initialization. When the prediction model includes multiple transform networks, different transform networks can correspond to different vectors eL. This allows random noise to be added to the encoding vectors of the feature columns input to different transform networks based on different vectors eL, thereby improving the generalization performance of the transform network.

[0067] The concatenated eTLs are input as follows: Figure 2 The linear transformation layer shown yields the key matrix Pk and value matrix Pv of the task feature columns in that layer; the dimensions of Pk and Pv can be 1×d. This linear transformation layer may include two Multilayer Perceptron (MLP) networks. In the MLP, the calculation process of Pk and Pv can be expressed as: Pkl = eTL*Wkl, Pvl = eTL*Wvl. Here, Wkl and Wvl are the model parameters to be learned.

[0068] The encoding vector Ex of the user feature column can be input as follows: Figure 2 The linear transformation layer shown is used to compute the key matrix K, query matrix K, and value matrix K of the user feature columns. Here, K = Ex * Wk², Q = Ex * Wq, V = Ex * Wv², where Wk, Wq, and Wv are d×d matrix parameters to be learned. In the first training iteration, the values ​​of Wk², Wq, and Wv² are initialized. In subsequent iterations, the values ​​of Wk², Wq, and Wv² are continuously optimized based on the network model's loss.

[0069] Matrix Pk and matrix K can be concatenated to obtain matrix KT containing task types, with a dimension of (m+1)×d; matrix Pv and matrix V can be concatenated to obtain matrix VT containing task types, with a dimension of (m+1)×d. Based on this step, task features can be added to user features, thereby providing suggestions on the task types corresponding to user features.

[0070] After obtaining vectors Q, KT, and VT through the aforementioned steps, the attention value is calculated based on a multi-head attention mechanism. Vectors Q, KT, and VT can be input... Figure 2 The multi-head attention layer is shown. After weighting vectors Q, KT, and VT, the multi-head attention layer outputs a conditional attention distribution vector Ax,L, where the dimension of the conditional attention distribution vector Ax,L is m×d.

[0071] The conditional attention distribution vector Ax,L is input as follows Figure 2 The first residual connection and normalization layer shown yields Axn,L. In this residual connection and normalization layer, residual information can be added to the conditional attention distribution vector Ax,L, and normalization is performed after adding the residual information, i.e., Axn,L = LayerNorm(Ax + Ax,L). Here, LayerNorm() represents the normalization operation. Ax represents the residual information. When the network model contains multiple transform networks, for the first transform network, the residual information is the encoded vector of the user feature column, i.e., Ax = Ex; for subsequent transform networks, the residual information Ax = Axo,L, where Axo,L is the output of the previous transform network.

[0072] Next, input the normalized matrix Axn,L as follows: Figure 2 The feedforward layer shown yields the matrix Axf,L.

[0073] The matrix Axf,L output by the feedforward layer and the matrix Axn,L output by the first residual connection and normalization layer can be input into the second residual connection and normalization layer to obtain the output matrix Axo,L, where Axo,L = LayerNorm(Axn,L + Axf,L).

[0074] Based on the above calculation process, the matrix Axo,L output by the last transformed network in the network model can be obtained, and this matrix Axo,L can be input as follows: Figure 2 The classification layer (Sigmoid) is shown. In the classification layer, the matrix Axo,L can be calculated to obtain the predicted label for the sample data. Based on the predicted and true labels of the sample data, the binary classification loss function can be calculated.

[0075] Based on the above method, the binary classification loss can be calculated for each of the N sample data, and the total classification loss for the N sample data can be obtained. The binary classification loss L for the N sample data can be expressed by the following function:

[0076]

[0077] Where i represents the sequence number of the sample data. Let y represent the predicted label of the i-th sample data. i Let represent the true label of the i-th sample data. Based on the total classification loss, the gradient can be calculated and the network updated in reverse until the model converges.

[0078] In this embodiment, based on the scenario of the sample data, tasks such as application, disbursement, and risk detection are modeled as different query tasks. During training, the model learns the weights of different feature dimensions in different tasks, so that when given a prompt word (i.e., task type), it can output detection results based on the learned weights.

[0079] In addition to the training method for the behavior prediction model provided in the foregoing embodiments, this application also provides a behavior prediction method in a multi-stage service scenario, such as... Figure 3 As shown, the method mainly includes the following steps:

[0080] Step 301: Respond to the behavior prediction request for the target user and obtain the data to be predicted corresponding to the behavior prediction request; the data to be predicted includes: the task type corresponding to the prediction request and the multi-dimensional user data of the target user; the task type corresponds to the target behavior stage in the multi-stage service scenario.

[0081] Step 302: Using a behavior prediction model, the task type is used as a condition, and feature extraction is performed on the data to be predicted based on the conditional attention mechanism to obtain the conditional attention features of the data to be predicted.

[0082] Step 303: Based on the conditional attention features, determine the prediction label of the data to be predicted under the task type, so as to obtain the behavior prediction result of the target user in the target behavior stage.

[0083] In some exemplary embodiments, the multi-stage service scenarios include: credit scenarios, advertising and marketing scenarios, online education scenarios, etc. Taking a credit scenario as an example, the task types include: application approval result prediction tasks, disbursement behavior prediction tasks, or repayment behavior prediction tasks. Taking an advertising and marketing scenario as an example, the task types include: ad click result prediction tasks or purchase result prediction tasks.

[0084] Taking a credit scenario as an example, the behavior prediction request can be initiated by an administrator user related to the credit service. Upon receiving a loan request from a credit applicant, the administrator user can initiate a behavior prediction request to the electronic device. The electronic device can then perform behavior prediction operations based on a behavior prediction model. This behavior prediction model is trained using the model training methods described in the preceding embodiments, which will not be repeated here. This behavior prediction request can be used to predict the behavior of the credit applicant at different stages. Based on the preceding embodiments, it is known that, according to the process steps of the credit scenario, the behavior stages can be divided into the application stage, the disbursement stage, or the repayment stage. The task types differ at each behavior stage. During behavior prediction, the electronic device can obtain the task type as a prompt to guide the behavior prediction model to execute the corresponding behavior prediction task, thereby obtaining the behavior prediction result of the credit applicant at the corresponding behavior stage for that task type.

[0085] The target behavior stage can be any behavior stage in a multi-stage scenario.

[0086] In some optional embodiments, administrator users can provide task types corresponding to prediction requests based on the behavior prediction needs at different behavioral stages. For example, they can provide application approval result prediction tasks for the application stage (i.e., the target behavior stage), disbursement behavior prediction tasks for the disbursement stage, or repayment behavior prediction tasks for the repayment stage.

[0087] In some alternative embodiments, the electronic device can determine the target behavior stage based on the current behavior stage of the credit applicant to automatically determine the task type. In this embodiment, if the current behavior stage of the credit applicant is the application stage (i.e., the target behavior stage), the task type can be determined as an application approval result prediction task to predict whether the credit applicant will be approved. If the current behavior stage of the credit applicant is the disbursement stage, the task type can be determined as a disbursement behavior prediction task to predict whether the credit applicant will disburse funds. If the current behavior stage of the credit applicant is the repayment stage, the task type can be determined as a repayment behavior prediction task to predict whether the credit applicant will default on repayments.

[0088] In some exemplary embodiments, the task type is used as a condition, and feature extraction is performed on the data to be predicted based on a conditional attention mechanism to obtain conditional attention features of the data to be predicted. This includes: vectorizing the multi-dimensional user data and task type of the data to be predicted to obtain user feature vectors and task feature vectors; inputting the user feature vectors and task feature vectors into at least one transformation network; using the at least one transformation network, employing a first weight parameter corresponding to the task type and a second attention weight corresponding to the multi-dimensional user data, performing conditional attention-based feature encoding on the task feature vectors and user feature vectors to obtain at least one encoding result; wherein, in two adjacent transformation networks, the encoding result output by the previous transformation network is input into the next transformation network along with the user feature vectors and task feature vectors; and obtaining the encoding result output by the last transformation network in the at least one transformation network as the conditional attention feature.

[0089] In some exemplary embodiments, using a first weight parameter corresponding to the task type and a second weight parameter corresponding to the multi-dimensional user data, conditional attention-based feature encoding is performed on the task feature vector and the user feature vector respectively to obtain at least one encoding result. This includes: in any transform network, embedding the identification information of the transform network into the task feature vector using an embedding layer to obtain an updated task feature vector; using a linear transform layer, applying the first weight parameter and the second weight parameter to perform linear transformations on the updated task feature vector and the user feature vector respectively to obtain a transformation matrix of the updated task feature vector and a transformation matrix of the user feature vector; concatenating the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector to obtain a conditional encoding matrix; using a multi-head attention layer, performing multi-head attention calculation on the conditional encoding matrix to obtain a first conditional attention distribution vector of the data to be predicted; and using an encoding output layer, outputting the conditional attention features corresponding to the transform network based on the first conditional attention distribution vector and the residual information input to the transform network, as the encoding result of the transform network.

[0090] In some exemplary embodiments, linear transformations are performed on the updated task feature vector and the user feature vector using the first weight parameter and the second weight parameter, respectively, to obtain the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector. This includes: performing a linear transformation calculation on the task feature vector using the first weight parameter to obtain the key matrix and value matrix of the task feature vector; performing a linear transformation calculation on the user feature vector using the second weight parameter to obtain the key matrix, query matrix, and value matrix of the user feature vector; concatenating the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector to obtain a conditional encoding matrix, including: concatenating the key matrix corresponding to the task feature vector with the key matrix corresponding to the user feature vector to obtain a target key matrix containing the task type; concatenating the value matrix corresponding to the task feature vector with the value matrix corresponding to the user feature vector to obtain a target value matrix containing the task type; and using the target key matrix, the target value matrix, and the query matrix as the conditional encoding matrix.

[0091] In some exemplary embodiments, outputting the conditional attention features of the data to be predicted based on the first conditional attention distribution vector and the residual information input to the transform network includes: performing a residual concatenation between the first conditional attention distribution vector and the target residual information; normalizing the first conditional attention distribution vector after residual concatenation to obtain a second conditional attention distribution vector; performing feedforward calculation on the second conditional attention distribution vector to obtain a third conditional attention distribution vector; performing a residual concatenation between the second conditional attention distribution vector and the third conditional attention distribution vector; and normalizing the third conditional attention distribution vector after residual concatenation to obtain the conditional attention features.

[0092] In some exemplary embodiments, before adding target residual information to the first conditional attention distribution vector of the data to be predicted, the method further includes: if the transformation network is the first transformation network among the at least one transformation network, then determining the user feature vector as the target residual information; or, if the transformation network is not the first transformation network among the at least one transformation network, then determining the conditional attention feature output by the previous transformation network of the transformation network as the target residual information.

[0093] In this implementation, when predicting the behavioral outcome of a target user, the task type can be input as a prompt into the behavior prediction model. The behavior prediction model can then perform conditional attention encoding on the data to be predicted based on the task type and the importance of pre-learned user data across different dimensions in various tasks. This allows it to accurately extract the features relevant to that task type from the data to be predicted. Based on these extracted features, the behavior prediction model can accurately predict the behavioral outcome of the data to be predicted at the corresponding behavioral stage of the task type.

[0094] It should be noted that the execution subject of each step of the method provided in the above embodiments can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 101 to 104 can be device A; or the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0095] Furthermore, some processes described in the above embodiments and accompanying drawings include multiple operations appearing in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not represent any execution order. Additionally, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first" and "second" in this document are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to different types.

[0096] Figure 4 This application illustrates a schematic diagram of the structure of an electronic device provided in an exemplary embodiment, as shown below. Figure 4 As shown, the electronic device includes a memory 401 and a processor 402.

[0097] Memory 401 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device.

[0098] In some embodiments, Figure 4The illustrated electronic device can be used to execute a behavior prediction method in a multi-stage service scenario. A processor 402, coupled to a memory 401, executes a computer program in the memory 401 to: respond to a behavior prediction request for a target user; obtain data to be predicted corresponding to the behavior prediction request; the data to be predicted includes: a task type corresponding to the prediction request and multi-dimensional user data of the target user; the task type corresponds to a target behavior stage in the multi-stage service scenario; using a behavior prediction model, taking the task type as a condition, extract features from the data to be predicted based on a conditional attention mechanism to obtain conditional attention features of the data to be predicted; and determining a prediction label for the data to be predicted under the task type based on the conditional attention features to obtain the behavior prediction result of the target user in the target behavior stage.

[0099] Optionally, when the processor 402 extracts features from the data to be predicted based on a conditional attention mechanism using the task type as a condition to obtain the conditional attention features of the data to be predicted, it specifically performs the following steps: vectorizing the multi-dimensional user data and task type of the data to be predicted to obtain user feature vectors and task feature vectors; inputting the user feature vectors and task feature vectors into at least one transformation network; using the at least one transformation network, employing a first weight parameter corresponding to the task type and a second attention weight corresponding to the multi-dimensional user data, performing conditional attention-based feature encoding on the task feature vectors and user feature vectors to obtain at least one encoding result; wherein, in two adjacent transformation networks, the encoding result output by the previous transformation network is input into the next transformation network along with the user feature vectors and task feature vectors; and obtaining the encoding result output by the last transformation network in the at least one transformation network as the conditional attention features.

[0100] Optionally, when the processor 402 performs conditional attention-based feature encoding on the task feature vector and the user feature vector using the first weight parameter corresponding to the task type and the second weight parameter corresponding to the multi-dimensional user data to obtain at least one encoding result, it specifically performs the following: In any transformation network, it embeds the identification information of the transformation network into the task feature vector using an embedding layer to obtain an updated task feature vector; it performs linear transformation on the updated task feature vector and the user feature vector using the first weight parameter and the second weight parameter using a linear transformation layer to obtain a transformation matrix of the updated task feature vector and a transformation matrix of the user feature vector; it concatenates the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector to obtain a conditional encoding matrix; it performs multi-head attention calculation on the conditional encoding matrix using a multi-head attention layer to obtain a first conditional attention distribution vector of the data to be predicted; and it outputs the conditional attention features corresponding to the transformation network using an encoding output layer based on the first conditional attention distribution vector and the residual information input to the transformation network, as the encoding result of the transformation network.

[0101] Optionally, when the processor 402 performs linear transformations on the updated task feature vector and the user feature vector using the first weight parameter and the second weight parameter respectively to obtain the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector, it specifically performs the following: it performs a linear transformation calculation on the task feature vector using the first weight parameter to obtain the key matrix and value matrix of the task feature vector; when the processor 402 performs a linear transformation calculation on the user feature vector using the second weight parameter to obtain the key matrix, query matrix, and value matrix of the user feature vector; and concatenates the transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector to obtain a conditional encoding matrix, it specifically performs the following: it concatenates the key matrix corresponding to the task feature vector with the key matrix corresponding to the user feature vector to obtain a target key matrix containing the task type; it concatenates the value matrix corresponding to the task feature vector with the value matrix corresponding to the user feature vector to obtain a target value matrix containing the task type; and it uses the target key matrix, the target value matrix, and the query matrix as the conditional encoding matrix.

[0102] Optionally, when the processor 402 outputs the conditional attention features of the data to be predicted based on the first conditional attention distribution vector and the residual information input to the transform network, it specifically performs the following steps: performing a residual concatenation between the first conditional attention distribution vector and the target residual information; normalizing the first conditional attention distribution vector after residual concatenation to obtain a second conditional attention distribution vector; performing feedforward calculation on the second conditional attention distribution vector to obtain a third conditional attention distribution vector; performing a residual concatenation between the second conditional attention distribution vector and the third conditional attention distribution vector; and normalizing the third conditional attention distribution vector after residual concatenation to obtain the conditional attention features.

[0103] Optionally, before adding the target residual information to the first conditional attention distribution vector of the data to be predicted, the processor 402 is further configured to: if the transformation network is the first transformation network among the at least one transformation network, then determine the user feature vector as the target residual information; or, if the transformation network is not the first transformation network among the at least one transformation network, then determine the conditional attention feature output by the previous transformation network of the transformation network as the target residual information.

[0104] Optionally, the multi-stage service scenario includes: a credit scenario; the task type includes: an application approval result prediction task, a disbursement behavior prediction task, or a repayment behavior prediction task.

[0105] In this implementation, when predicting the behavioral outcome of a target user, the task type can be input as a prompt into the behavior prediction model. The behavior prediction model can then perform conditional attention encoding on the data to be predicted based on the task type and the importance of pre-learned user data across different dimensions in various tasks. This allows it to accurately extract the features relevant to that task type from the data to be predicted. Based on these extracted features, the behavior prediction model can accurately predict the behavioral outcome of the data to be predicted at the corresponding behavioral stage of the task type.

[0106] In some embodiments, Figure 4The illustrated electronic device can be used to execute a behavior prediction model training method in a multi-stage service scenario. A processor 402, coupled to a memory 401, executes a computer program in the memory 401 to: acquire multiple sample data points in the multi-stage service scenario, each sample data point including multi-dimensional user data of a single user, a task type, and a true label for the sample data under the task type; wherein the task type is determined based on the behavior stage corresponding to the sample data in the multi-stage service scenario; the multiple sample data points correspond to multiple behavior stages; using a behavior prediction model, taking the task type of each of the multiple sample data points as a condition, extracts features from the multiple sample data points based on a conditional attention mechanism to obtain conditional attention features of the multiple sample data points; determines the predicted labels of the multiple sample data points under their respective task types based on the conditional attention features of the multiple sample data points; and trains the behavior prediction model with the goal of the specified range to which the difference between the true labels and predicted labels of the multiple sample data points under their respective task types converges.

[0107] In this embodiment, by adding task types to the sample data, explicit modeling of behavior prediction tasks at different behavioral stages in a multi-stage service scenario is achieved. The behavior prediction model can learn the importance of different dimensions of user data in different tasks based on task types, reducing learning errors caused by sample bias. On the one hand, there is no need to set up multi-task branch networks for behavior prediction tasks at different behavioral stages; a unified model structure can be used to achieve different prediction tasks, simplifying the model structure. On the other hand, based on the learned importance of different dimensions of user data in different tasks, the model can accurately predict the behavioral outcome of the input data at the corresponding behavioral stage of the task type during the inference stage, according to the task type and the input data.

[0108] Furthermore, such as Figure 4 As shown, the electronic device also includes other components such as a communication component 403, a power supply component 404, a display component 405, and an audio component 406. Figure 4 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 4 The components shown.

[0109] The memory 401 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random-access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0110] The communication component 403 is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as Wi-Fi, 2G (e.g., Global System for Mobile Communications (GSM)), 3G (e.g., Wideband Code Division Multiple Access (WCDMA), 4G (e.g., Long Term Evolution (LTE)), 4G+ (e.g., LTE-Advanced (LTE-A)), or 5G (5th Generation Mobile Communication Technology), or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, the communication component may be implemented based on Near Field Communication (NFC), Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wide Band (UWB), Bluetooth (BT), and other technologies.

[0111] The power supply component 404 is used to provide power to various components of the device in which the power supply component is located. The power supply component may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device in which the power supply component is located.

[0112] The display component 405 includes a screen, which may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation.

[0113] The audio component 406 can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0114] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed, can implement the steps that can be performed by an electronic device in the above method embodiments.

[0115] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM (Compact Disc Read-Only Memory), optical storage, etc.) containing computer-usable program code.

[0116] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0117] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0118] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0119] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interfaces, network interfaces, and memory.

[0120] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0121] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0122] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0123] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A method for behavior prediction in a multi-stage service scenario, characterized in that, include: In response to a behavior prediction request for a target user, obtain the data to be predicted corresponding to the behavior prediction request; The data to be predicted includes: the task type corresponding to the prediction request and the multi-dimensional user data of the target user; the task type corresponds to the target behavior stage in the multi-stage service scenario. Using a behavior prediction model, with the task type as a condition, feature extraction is performed on the data to be predicted based on a conditional attention mechanism to obtain conditional attention features of the data to be predicted. This includes: vectorizing the multi-dimensional user data and the task type to obtain user feature vectors and task feature vectors; using at least one transformation network, employing a first weight parameter corresponding to the task type and a second weight parameter corresponding to the multi-dimensional user data, performing conditional attention-based feature encoding on the task feature vectors and user feature vectors to obtain at least one encoding result; obtaining the encoding result output by the last transformation network in the at least one transformation network as the conditional attention feature; and in any transformation network, conditional attention-based feature encoding includes: embedding the identification information of the transformation network into the task feature vector to obtain an updated task feature vector; performing linear transformations on the updated task feature vector and the user feature vector; concatenating the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector obtained after the linear transformation to obtain a conditional encoding matrix, and performing multi-head attention calculation on the conditional encoding matrix. Based on the conditional attention features, the prediction label of the data to be predicted under the task type is determined, so as to obtain the behavior prediction result of the target user in the target behavior stage.

2. The method of claim 1, wherein, In two adjacent transformation networks, the encoding result output by the previous transformation network is input into the next transformation network along with the user feature vector and the task feature vector.

3. The method of claim 2, wherein, Using the at least one transformation network, and employing the first weight parameter corresponding to the task type and the second weight parameter corresponding to the multi-dimensional user data, feature encoding based on conditional attention is performed on the task feature vector and the user feature vector respectively, to obtain at least one encoding result, including: In any transformation network, the identification information of the transformation network is embedded into the task feature vector using an embedding layer to obtain an updated task feature vector; Using a linear transformation layer, the updated task feature vector and the user feature vector are linearly transformed using the first weight parameter and the second weight parameter, respectively, to obtain the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector. The transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector are concatenated to obtain the conditional coding matrix; By using a multi-head attention layer, multi-head attention calculation is performed on the conditional coding matrix to obtain the first conditional attention distribution vector of the data to be predicted; Using the encoding output layer, based on the first conditional attention distribution vector and the residual information input to the transform network, the conditional attention features corresponding to the transform network are output as the encoding result of the transform network.

4. The method of claim 3, wherein, Using the first weight parameter and the second weight parameter, linear transformations are performed on the updated task feature vector and the user feature vector, respectively, to obtain the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector, including: The updated task feature vector is linearly transformed using the first weight parameter to obtain the key matrix and value matrix of the task feature vector. The user feature vector is subjected to a linear transformation using the second weight parameter to obtain the key matrix, query matrix, and value matrix of the user feature vector. The transformation matrix of the user feature vector and the transformation matrix of the updated task feature vector are concatenated to obtain the conditional coding matrix, which includes: The key matrix corresponding to the task feature vector is concatenated with the key matrix corresponding to the user feature vector to obtain a target key matrix containing the task type; The value matrix corresponding to the task feature vector is concatenated with the value matrix corresponding to the user feature vector to obtain a target value matrix containing the task type; The target key matrix, the target value matrix, and the query matrix are used as the condition encoding matrix.

5. The method of claim 3, wherein, Based on the first conditional attention distribution vector and the residual information input to the transformation network, the conditional attention features of the data to be predicted are output, including: Perform a residual concatenation between the first conditional attention distribution vector and the target residual information; The first conditional attention distribution vector after residual connection is normalized to obtain the second conditional attention distribution vector; The second conditional attention distribution vector is fed forward to obtain the third conditional attention distribution vector; Perform a residual concatenation between the second conditional attention distribution vector and the third conditional attention vector; The conditional attention features are obtained by normalizing the third conditional attention distribution vector after residual connection.

6. The method of claim 5, wherein, Before adding the target residual information to the first conditional attention distribution vector of the data to be predicted, the method further includes: If the transformation network is the first transformation network among the at least one transformation network, then the user feature vector is determined as the target residual information; or, If the transformation network is not the first transformation network among the at least one transformation networks, then the conditional attention feature output by the previous transformation network is determined as the target residual information.

7. The method according to any one of claims 1 to 6, characterized in that, The multi-stage service scenarios include: credit scenarios; the task types include: application approval result prediction tasks, disbursement behavior prediction tasks, or repayment behavior prediction tasks in credit scenarios.

8. A method for training a behavior prediction model in a multi-stage service scenario, characterized in that, include: Multiple sample data points are acquired in a multi-stage service scenario. Each sample data point includes multi-dimensional user data of a single user, task type, and the actual label of the sample data under the task type. The task type is determined based on the behavioral stage corresponding to the sample data in the multi-stage service scenario. The multiple sample data points correspond to multiple behavioral stages. Using a behavior prediction model, the task types of the multiple sample data are used as conditions, and feature extraction is performed on the multiple sample data based on a conditional attention mechanism to obtain conditional attention features of the multiple sample data. This includes: for any sample data, vectorizing the multi-dimensional user data and task type in the sample data to obtain user feature vectors and task feature vectors; using at least one transformation network, employing a first weight parameter corresponding to the task type and a second weight parameter corresponding to the multi-dimensional user data, performing conditional attention-based feature encoding on the task feature vectors and user feature vectors respectively to obtain at least one encoding result; and acquiring the at least one transformation network. The encoding result output by the last transform network in the transform network is used as the conditional attention feature. In any transform network, the feature encoding based on conditional attention includes: embedding the identification information of the transform network into the task feature vector to obtain an updated task feature vector; performing linear transformations on the updated task feature vector and the user feature vector respectively; concatenating the transformation matrix of the updated task feature vector and the transformation matrix of the user feature vector obtained after the linear transformation to obtain a conditional encoding matrix, and performing multi-head attention calculation on the conditional encoding matrix; and determining the predicted label of the multiple sample data under their respective task types based on the conditional attention features of the multiple sample data. The behavior prediction model is trained with the goal of converging the gap between the true and predicted labels of the multiple sample data under their respective task types to a specified range.

9. An electronic device, comprising: include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions for performing the steps of the method according to any one of claims 1-8.

10. A computer readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it is able to implement the behavior prediction method according to any one of claims 1-8.

Images