Model training method, behavior prediction method, device and electronic equipment
By combining masking and labeling of user behavior data and employing a semi-supervised training method, the problem of poor model training quality in existing technologies is solved, achieving more accurate user behavior representation and prediction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- DUXIAOMAN TECH (BEIJING) CO LTD
- Filing Date
- 2023-01-28
- Publication Date
- 2026-06-26
AI Technical Summary
In existing technologies, when using natural language processing models to train user behavior, there are problems such as poor training quality and inability to accurately represent user behavior.
By acquiring users' historical behavior data and masking it, and combining it with label identification, the model is trained using a semi-supervised method to improve the training quality of the model.
A well-trained model can better represent the user's label information, improving the training quality and prediction accuracy of the model.
Smart Images
Figure CN116011554B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, specifically to a model training method, behavior prediction method, device, and electronic equipment. Background Technology
[0002] In the field of internet finance, user behavior sequence modeling is an important task. Based on user behavior data, such as lists of mobile applications (APPs) installed and used by users, and user search records, sequence modeling methods can be used to deeply mine credit-related characteristics of users. This can be applied to various scenarios in the internet finance field, such as customer acquisition, risk control, and existing customer management. Therefore, effective sequence modeling methods are extremely crucial for fully mining user data.
[0003] In related technologies, models designed for Natural Language Processing (NLP) are typically used to learn user behavior. However, there is a significant difference between user behavior and natural language. Therefore, this training method results in poor training quality of the model, and the trained model cannot accurately represent user behavior. Summary of the Invention
[0004] In order to overcome the shortcomings of the prior art, this invention proposes a model training method, behavior prediction method, device and electronic device, which can improve the training quality of the model and obtain a more accurate user representation through the trained model.
[0005] A first aspect of the present invention provides a model training method, the method comprising:
[0006] The system acquires historical behavior data from multiple users, including historical behavior operation data and tag identifiers. The tag identifiers are used to indicate whether the user has tag information, and the tag information is used to indicate whether the user has performed a preset behavior.
[0007] The historical behavior data is masked to obtain masked historical behavior data;
[0008] The model is trained based on the historical behavior data behind the mask.
[0009] The model training method provided by this invention acquires historical behavior data from multiple users, masks this data to obtain masked historical behavior data, and trains the model based on this masked data to obtain a trained model. Since the historical behavior data includes user tags, which indicate whether a user has tag information and whether the user has performed a preset behavior (such as overdue payment), this invention trains the model based on user tag information. Therefore, the trained model can better represent users and learn user tag information more effectively. Furthermore, this method of training the model based on user tag information is a semi-supervised training method, which improves the training quality compared to unsupervised training methods.
[0010] Optionally, the step of obtaining historical behavior data from multiple users includes:
[0011] Obtain the historical behavior operation data;
[0012] The tag identifier is obtained, and when the user has the tag information and the user performs the preset behavior, the tag identifier is a first preset value; when the user has the tag information and the user does not perform the preset behavior, the tag identifier is a second preset value; when the user does not have the tag information, the tag identifier is a third preset value.
[0013] The historical behavior data is obtained by combining the historical behavior data and the tag identifier.
[0014] Optionally, the historical behavior operation data includes multiple data units;
[0015] The step of masking the historical behavior data to obtain masked historical behavior data includes:
[0016] When the user has the tag information, a portion of the data units in the tag identifier and the historical behavior operation data are masked.
[0017] If the user does not have the tag information, a portion of the data units in the historical operation behavior data are masked.
[0018] Optionally, the step of training the model based on the masked historical behavior data includes:
[0019] The model is pre-trained using the historical behavior data behind the mask to obtain the pre-trained model.
[0020] The pre-trained model is trained using the historical behavior data to obtain a trained model.
[0021] Optionally, the step of pre-training the model using the masked historical behavior data to obtain the pre-trained model includes:
[0022] Determine the initial network parameters for the initial model;
[0023] The masked historical behavior data of each user is input into the initial model;
[0024] Extract the correlation features of the historical behavior data after the masking, whereby the correlation features represent the correlation between each data unit in the historical behavior operation data after the masking or the weight between the label identifier and the data unit;
[0025] Predict the masked data units and / or the label identifiers to obtain prediction results;
[0026] Based on the prediction results, the true values, and the preset loss function, the initial network parameters of the model are adjusted. If the number of iterations is less than the preset iteration threshold, the process returns to the step of inputting the historical behavior data of each user into the initial model.
[0027] A second aspect of the present invention provides a behavior prediction method, the method comprising:
[0028] Obtain the target user's historical behavioral data;
[0029] The target's historical behavioral data is input into a model trained using any of the model training methods described in the first aspect above to obtain the predicted behavior of the target user, which includes overdue behavior, investment behavior, or investment stability indicators.
[0030] The behavior prediction method provided by this invention involves training a model using the aforementioned model training method with input data of the target's historical behavior. This model training method acquires historical behavior data from multiple users, masks this data to obtain masked historical behavior data, and then trains the model based on this masked data. Since the historical behavior data includes user tags indicating whether a user possesses tag information, which in turn indicates whether the user has performed a preset behavior (such as overdue payment), this invention trains the model based on user tag information. Therefore, the trained model better represents the user and learns the user's tag information more effectively. Furthermore, this method of training the model based on user tag information is a semi-supervised training method, which improves the training quality compared to unsupervised training methods.
[0031] A third aspect of the present invention provides a model training apparatus, the apparatus comprising:
[0032] The first acquisition module is used to acquire historical behavior data of multiple users. The historical behavior data includes historical behavior operation data and tag identifiers. The tag identifiers are used to indicate whether the user has tag information, and the tag information is used to indicate whether the user has performed a preset behavior.
[0033] The masking module is used to mask the historical behavior data to obtain masked historical behavior data.
[0034] The training module is used to train the model based on the historical behavior data behind the mask.
[0035] A fourth aspect of the present invention provides a behavior prediction device, the device comprising:
[0036] The second acquisition module is used to acquire the target user's target historical behavior data.
[0037] The prediction module is used to input the target's historical behavioral operation data into a model trained using the model training method described in any one of the first aspects, to obtain the predicted behavior of the target user, wherein the predicted behavior includes overdue behavior, investment behavior, or investment stability indicators.
[0038] In a third aspect of the present invention, an electronic device is also provided, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus.
[0039] Memory, used to store computer programs;
[0040] When a processor executes a program stored in memory, it implements the method steps of any of the above-described model training methods or behavior prediction methods.
[0041] In a fourth aspect of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the method steps of any of the model training methods or the behavior prediction methods described above.
[0042] In a fifth aspect of the invention, a computer program product comprising instructions is also provided, which, when run on a computer, causes the computer to perform the method steps of any of the model training methods or behavior prediction methods described above. Attached Figure Description
[0043] Figure 1 A schematic flowchart of a model training method provided in an embodiment of the present invention;
[0044] Figure 2a for Figure 1 The specific process of step S101 in the illustrated embodiment;
[0045] Figure 2b A schematic diagram illustrating the process of obtaining historical behavior data of users with tagged information after masking.
[0046] Figure 2c A schematic diagram illustrating the process of obtaining historical behavior data of users without tag information after masking.
[0047] Figure 2d This is a diagram illustrating the process of obtaining masked historical behavior data when a user installs or uses an app.
[0048] Figure 3 for Figure 1 The specific process of step S103 in the illustrated embodiment;
[0049] Figure 4 for Figure 3 The specific process of step S301 in the illustrated embodiment;
[0050] Figure 5 A flowchart illustrating a behavior prediction method provided in an embodiment of the present invention;
[0051] Figure 6 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present invention;
[0052] Figure 7This is a schematic diagram of the structure of a behavior prediction device provided in an embodiment of the present invention;
[0053] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation
[0054] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.
[0056] In related technologies, models applied in NLP are typically used to learn user behavior, such as long short-term memory (LSTM) models or Transformer models. In the NLP field, the emergence of pre-trained language models, such as Bidirectional Encoder Representations from Transformers (BERT), which are pre-trained using massive amounts of unlabeled sequence corpora, has significantly improved the performance of many NLP tasks and has been widely adopted.
[0057] Given the similarities between user behavior data and NLP corpus data, the industry has begun to try using NLP models to pre-train user behavior data, mine the correlations between user behaviors, obtain a general modeling model of users, and then fine-tune it in subsequent downstream supervision tasks. This pre-training-fine-tuning model has significantly improved user behavior data modeling.
[0058] However, there is a huge difference between user behavior data in the Internet finance field and corpus data in NLP: in scenarios such as risk control and old customer management, there is a large amount of user tag information, which is basically in the tens of millions of data. In contrast, typical downstream supervised tasks in NLP, such as text classification, have a smaller scale of labeled data. Therefore, current pre-trained models are based on the unsupervised paradigm, learning general knowledge from massive unlabeled corpora, which can improve the performance of downstream tasks.
[0059] However, this training method results in poor training quality of the model, and the trained model cannot accurately represent the user's behavior.
[0060] In view of this, embodiments of the present invention provide a model training method that can be applied to a server.
[0061] like Figure 1 As shown, model training methods may include:
[0062] S101, retrieves historical behavior data from multiple users.
[0063] In this embodiment, historical behavior data may include historical behavior operation data and tag identifiers. The historical behavior operation data can be data on apps installed or used by the user, data on apps searched by the user, or data on users searching for financial information. When creating a model, multiple models can be created, each targeting one type of data. For example, three models can be created: one targeting data on apps installed or used by the user, one targeting data on apps searched by the user, and another targeting data on users searching for financial information.
[0064] Tag identifiers indicate whether a user has tag information, and tag information indicates whether a user has committed a preset behavior, which can be an overdue payment. In other words, when a user's past data indicates that they have previously committed overdue payments, or when a user has never committed overdue payments, tag information can be set for that user. The tag information corresponding to overdue payments can be different from the tag information corresponding to never having committed overdue payments; this difference in tag information indicates whether a user has committed overdue payments.
[0065] S102, mask the historical behavior data to obtain the masked historical behavior data.
[0066] Historical behavior can be masked, that is, some data in the historical behavior data can be erased to obtain the masked historical behavior data.
[0067] S103, the model is trained based on masked historical behavior data.
[0068] After obtaining the masked historical behavior data, the model is trained based on the masked historical behavior data to obtain a trained model.
[0069] The model training method provided in this invention acquires historical behavior data from multiple users, masks the historical behavior data to obtain masked historical behavior data, and trains the model based on the masked historical behavior data to obtain a trained model. Since the historical behavior data includes user tags, which indicate whether a user has tag information and whether the user has performed a preset behavior (such as overdue payment), this invention trains the model based on user tag information. Therefore, the trained model can better represent users and learn user tag information more effectively. Furthermore, this method of training the model based on user tag information is a semi-supervised training method, which improves the training quality compared to unsupervised training methods.
[0070] Optionally, such as Figure 2a As shown, Figure 1 Step S101 in the illustrated embodiment, the step of obtaining historical behavior data of multiple users, may include:
[0071] S201, Obtain historical behavior operation data.
[0072] It can obtain behavioral data from multiple users, such as Figure 2b and Figure 2c As shown, behavioral data can include behavioral data 1, behavioral data 2, ..., behavioral data n, with each behavioral data being a data unit. For example, behavioral data could be data about installing or using a mobile app. Figure 2d As shown, the user's APP1, APP2, and APPn can be obtained.
[0073] like Figure 2b and Figure 2c As shown, after obtaining the behavior data, a start symbol can be added to the front of the behavior data and an end symbol can be added to the back of the behavior data to obtain historical behavior operation data.
[0074] S202, Obtain the tag identifier.
[0075] User tags can be pre-saved as follows: When a user's historical credit behavior or other behaviors can be obtained, tag information can be set for that user. Based on the user's historical credit behavior, the tag information is determined. If the user has performed a preset behavior, the tag identifier can be set to a first preset value; if the user has not performed a preset behavior, the tag identifier can be set to a second preset value. If the user's historical credit behavior or other behaviors cannot be obtained, the user can be determined as a user without tag information, and the user's tag identifier can be set to a third preset value. The first, second, and third preset values are all different. For example, the first preset value can be 1, the second preset value can be 0, and the third preset value can be -100. Of course, it is understood that in other embodiments, the first, second, and third preset values can also be other values, as long as these three values are different.
[0076] During model training, the tags of each user can be obtained.
[0077] S203 combines historical behavior operation data and tag identifiers to obtain historical behavior data.
[0078] It can pre-save the correspondence between each data unit in historical behavior operation data and the behavior data identity document (ID), and obtain the behavior data ID corresponding to each data unit in historical behavior operation data. For example, it can look up the target ID corresponding to each APP based on a pre-saved dictionary of APP and ID.
[0079] Furthermore, the start symbol can be set as a label identifier to obtain historical behavior data. In other words, the historical behavior data includes the label identifier, historical operation behavior data, and the end symbol.
[0080] Optionally, the historical behavior operation data includes multiple data units, with each operation record corresponding to one data unit. For example, each time a mobile APP is installed corresponds to one data unit.
[0081] Figure 1 Step S102 in the illustrated embodiment, which involves masking the historical behavior data to obtain the masked historical behavior data, may include:
[0082] When a user has tag information, some data units in the tag identifier and historical behavior operation data are masked.
[0083] The system can determine whether a user has tag information based on the tag identifier. When a user has tag information, the tag identifier and certain data units in the historical behavior data can be masked. For example, when the tag identifier is 1 or 0, it indicates that the user has tag information.
[0084] Figure 2b The historical behavior data shown is the historical behavior data of users with tag information, from... Figure 2b As can be seen, the label identifier and behavior data 2 have been masked. Figure 2d The historical behavior data shown is the historical behavior data of users with tag information. Therefore, it can be seen from the masked historical behavior data that the tag identifier, APP2, and APPn have been masked. When masking a portion of the data units in the historical behavior operation data, 15% of the data units in the historical behavior operation data can be masked, and these 15% of data units are randomly selected.
[0085] In cases where users do not have tag information, some data units in the historical operation behavior data are masked.
[0086] The system can determine whether a user has tag information based on the tag identifier. When a user does not have tag information, certain data units in the historical behavior data can be masked. For example, when the tag identifier is -100, it indicates that the user does not have tag information.
[0087] like Figure 2c As shown, Figure 2c This represents historical behavioral data of users with tagged information, from Figure 2c As can be seen, behavior data 2 and data behavior n were masked. When masking a portion of the data units in the historical behavior operation data, 15% of the data units in the historical behavior operation data can be masked, and these 15% of data units are randomly selected.
[0088] When a user has tag information, some data units in the tag identifier and historical behavior data are masked. This facilitates the prediction of the tag identifier each time during model training, and the adjustment of the model's network parameters based on the prediction results and the actual tag identifier, thereby improving the training accuracy of the model. Moreover, in this embodiment of the invention, the model can be trained simultaneously based on the historical behavior data of users with tag information and users without tag information, without increasing the training complexity.
[0089] Optionally, such as Figure 3 As shown, Figure 1Step S103 in the illustrated embodiment, the step of training the model based on the masked historical behavior data, may include:
[0090] S301, using masked historical behavior data to pre-train the model, and obtain the pre-trained model.
[0091] The model can be trained twice: the first training is pre-training, and the second training is fine-tuning. Pre-training can be performed using masked historical behavior data to obtain a pre-trained model.
[0092] S302, using historical behavioral operation data to train the pre-trained model to obtain a trained model.
[0093] After obtaining the pre-trained model, a classification layer can be added to it. Since the model has already undergone masking and prediction during pre-training, when the user's historical behavior data is input into the pre-trained model with the added classification layer, the classification layer can be used directly to classify the user's behavior and obtain labels. Then, the network parameters of the model are adjusted using the labels obtained from the classification, the user's actual labels, and the loss function until training is complete.
[0094] Optionally, such as Figure 4 As shown, Figure 3 Step S301 in the illustrated embodiment, which involves pre-training the model using masked historical behavior data to obtain the pre-trained model, may include:
[0095] S401, Determine the initial network parameters of the initial model.
[0096] For this model, its initial network parameters can be randomly initialized.
[0097] In this embodiment, the model may include an encoder and a prediction layer, wherein the encoder may be a position encoder. The prediction layer may be a masked language model (MLM) layer.
[0098] S402, input the masked historical behavior data of each user into the initial model.
[0099] After obtaining the masked historical behavior data of each user, the masked historical behavior data can be input into the initial model.
[0100] S403, Extract the correlation features of historical behavior data after masking.
[0101] In this embodiment, the encoder in the model can extract the correlation features of the masked historical behavior data. These correlation features can represent the correlation between data units in the masked historical behavior operation data or the weight between the label and the data unit. The correlation features can include the relationship between each data unit and the label, as well as the internal correlation between each data unit.
[0102] S404 predicts the masked data units and / or tag identifiers to obtain the prediction results.
[0103] In this embodiment, the prediction layer in the model can predict the masked data units and / or label identifiers based on the correlation characteristics of the masked historical behavior data, and obtain the prediction results. The masked data units are the erased data units, and the masked label identifiers are the erased label identifiers. Therefore, the masked data units and the masked label identifiers can be predicted to obtain prediction results, which can be the predicted data units and the predicted label identifiers.
[0104] For users with tag information, since the masking process involves both the tag identifier and a portion of the data units, the prediction result includes the predicted tag identifier and the predicted portion of the data units. For users without tag information, since the masking process involves only a portion of the data units, only the portion of the data units needs to be predicted, and the tag identifier does not need to be predicted. Therefore, the prediction result includes the predicted portion of the data units.
[0105] S405, determine whether the number of iterations is greater than or equal to the preset number of iterations threshold. If yes, proceed to step S406; otherwise, proceed to step S407.
[0106] S406 indicates that model training is complete.
[0107] Since the training process of the model requires multiple iterations, an iteration threshold can be set in advance. If the number of iterations reaches the preset iteration threshold during the training process, the model is considered to have completed training.
[0108] S407, based on the prediction results, the true value, and the preset loss function, adjust the initial network parameters of the model, and return to S402, the step of inputting the historical behavior data of each user into the initial model.
[0109] If the number of iterations is less than the preset iteration threshold, the initial network parameters of the feature prediction model are adjusted using the prediction results, ground truth, preset loss function, and network parameter adjustment algorithm. Then, the process proceeds to the next iteration, i.e., returning to step S402 to input the historical behavior data of each user into the initial model. The loss function can be the cross-entropy loss function. During the adjustment of the initial network parameters, the adaptive moment estimation (ADAM) optimizer can be used to optimize and adjust the model parameters.
[0110] For a masked data unit, the data unit before the mask is the true value of the masked data unit; for a masked label information, the label information before the mask is the true value of the corresponding label information.
[0111] Understandably, the prediction layer in the model comprises multiple prediction units, each responsible for predicting one masked data unit or label, or each prediction unit responsible for predicting multiple masked data units or labels, or multiple prediction units responsible for predicting one masked data unit or label. Therefore, in each iteration, the network parameters can be adjusted only for the prediction unit corresponding to the masked data unit or label, thereby reducing computational load and improving the model's pre-training efficiency.
[0112] In addition, during the pre-training process of the model, the Dropout technique can be used to prevent overfitting.
[0113] like Figure 5 As shown, a second aspect of the present invention also provides a behavior prediction method, which can be applied to scenarios such as risk control, new customer acquisition, and existing customer maintenance. The behavior prediction method may include:
[0114] S501, obtain the target user's target historical behavior operation data.
[0115] It can acquire target user behavior data, which can include behavior data 1, behavior data 2, ..., behavior data n, with each target behavior data being a data unit. For example, target behavior data can be data on installing or using a mobile app; in this case, it can acquire the user's APP1, APP2, APP3, and APPn.
[0116] After obtaining the target behavior data, a start symbol can be added to the front of the target behavior data and an end symbol can be added to the back of the target behavior data to obtain the target historical behavior operation data.
[0117] S502, input the target's historical behavior data into the model to obtain the predicted behavior of the target user.
[0118] In this embodiment, the model is a model trained using any of the methods described above. The predicted behavior may include overdue behavior, investment behavior, or investment stability indicators.
[0119] The behavior prediction method provided by this invention involves training a model using the aforementioned model training method with input data of the target's historical behavior. This model training method acquires historical behavior data from multiple users, masks this data to obtain masked historical behavior data, and then trains the model based on this masked data. Since the historical behavior data includes user tags indicating whether a user possesses tag information, which in turn indicates whether the user has performed a preset behavior (such as overdue payment), this invention trains the model based on user tag information. Therefore, the trained model better represents the user and learns the user's tag information more effectively. Furthermore, this method of training the model based on user tag information is a semi-supervised training method, which improves the training quality compared to unsupervised training methods.
[0120] This invention also provides a specific embodiment of a model training device, which is related to... Figure 1 The process shown corresponds to the one described above. (Refer to the process description.) Figure 6 , Figure 6 This is a schematic diagram of a model training device according to an embodiment of the present invention. The model training device may include:
[0121] The first acquisition module 601 is used to acquire historical behavior data of multiple users. The historical behavior data includes historical behavior operation data and tag identifiers. The tag identifiers are used to indicate whether the user has tag information, and the tag information is used to indicate whether the user has performed a preset behavior.
[0122] The masking module 602 is used to mask historical behavior data to obtain masked historical behavior data.
[0123] Training module 603 is used to train the model based on masked historical behavior data.
[0124] Optionally, the first acquisition module 601 described above may include:
[0125] The first acquisition submodule is used to acquire historical behavior operation data.
[0126] The second acquisition submodule is used to acquire the tag identifier. When the user has tag information and a preset behavior occurs, the tag identifier is a first preset value; when the user has tag information but the user has not performed a preset behavior, the tag identifier is a second preset value; when the user does not have tag information, the tag identifier is a third preset value.
[0127] The combination submodule is used to combine historical behavior operation data and tag identifiers to obtain historical behavior data.
[0128] Optionally, historical behavior data may include multiple data units.
[0129] The mask module 602 mentioned above may include:
[0130] The first masking submodule is used to mask some data units in the tag identifier and historical behavior operation data when the user has tag information.
[0131] The second masking submodule is used to mask some data units in historical operation behavior data when the user does not have tag information.
[0132] Optionally, the training module 603 described above may include:
[0133] The pre-training submodule is used to pre-train the model using masked historical behavior data to obtain the pre-trained model.
[0134] The training submodule is used to train the pre-trained model using historical behavior data to obtain a trained model.
[0135] Optionally, the aforementioned pre-training submodule may include:
[0136] The defining unit is used to determine the initial network parameters of the initial model.
[0137] The input unit is used to input the masked historical behavior data of each user into the initial model.
[0138] The extraction unit is used to extract similarity features from the masked historical behavior data. The similarity features represent the similarity between data units in the masked historical behavior operation data or the similarity between the label identifier and the data unit.
[0139] The prediction unit is used to predict the masked data units and / or label identifiers to obtain the prediction results.
[0140] The adjustment unit is used to adjust the initial network parameters of the model based on the prediction results, the true values, and the preset loss function. When the number of iterations is less than the preset threshold, the input unit is triggered to perform the step of inputting the historical behavior data of each user into the initial model.
[0141] This invention also provides a specific embodiment of a behavior prediction device, which is related to... Figure 5 The process shown corresponds to the one described above. (Refer to the process description.) Figure 7 , Figure 7 This is a schematic diagram of the structure of a behavior prediction device according to an embodiment of the present invention. The behavior prediction device may include:
[0142] The second acquisition module 701 is used to acquire the target user's target historical behavior operation data.
[0143] The prediction module 702 is used to input the target's historical behavioral operation data into the model trained using any of the model training methods described above, and to obtain the predicted behavior of the target user, including overdue behavior, investment behavior, or investment stability indicators.
[0144] This invention also provides an electronic device, such as... Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803, and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.
[0145] The memory 803 is used to store computer programs.
[0146] When processor 801 executes a program stored in memory 803, it performs the following steps:
[0147] Acquire historical behavior data from multiple users. The historical behavior data includes historical behavior operation data and tag identifiers. Tag identifiers are used to indicate whether a user has tag information, and tag information is used to indicate whether a user has performed a preset behavior.
[0148] The historical behavior data is masked to obtain the masked historical behavior data.
[0149] The model is trained based on masked historical behavior data.
[0150] Alternatively, processor 801 is used to implement the following steps:
[0151] Obtain the target user's historical behavioral data.
[0152] Input the target's historical behavioral data into the model trained using any of the above methods to obtain the target user's predicted behavior, which includes overdue behavior, investment behavior, or investment stability indicators.
[0153] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.
[0154] The communication interface is used for communication between the aforementioned terminal and other devices.
[0155] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.
[0156] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.
[0157] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements any of the model training methods or behavior prediction methods described in the above embodiments.
[0158] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the model training methods or behavior prediction methods described in the above embodiments.
[0159] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
[0160] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0161] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.
[0162] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.
Claims
1. A model training method, characterized in that, The method includes: The system acquires historical behavior data from multiple users, including historical behavior operation data and tag identifiers. The tag identifiers are used to indicate whether the user has tag information, and the tag information is used to indicate whether the user has performed a preset behavior. The historical behavior data is masked to obtain masked historical behavior data; The model is trained based on the historical behavior data behind the mask; The step of obtaining historical behavior data from multiple users includes: Obtain the historical behavior operation data; The tag identifier is obtained, and when the user has the tag information and the user performs the preset behavior, the tag identifier is a first preset value; when the user has the tag information and the user does not perform the preset behavior, the tag identifier is a second preset value; when the user does not have the tag information, the tag identifier is a third preset value. The historical behavior data is obtained by combining the historical behavior data and the tag identifier. The historical behavior operation data includes multiple data units; The step of masking the historical behavior data to obtain masked historical behavior data includes: When the user has the tag information, a portion of the data units in the tag identifier and the historical behavior operation data are masked. If the user does not have the tag information, a portion of the data units in the historical behavior operation data are masked.
2. The method according to claim 1, characterized in that, The step of training the model based on the masked historical behavior data includes: The model is pre-trained using the historical behavior data behind the mask to obtain the pre-trained model. The pre-trained model is trained using the historical behavior data to obtain a trained model.
3. The method according to claim 2, characterized in that, The step of pre-training the model using the masked historical behavior data to obtain the pre-trained model includes: Determine the initial network parameters for the initial model; The masked historical behavior data of each user is input into the initial model; Extract the correlation features of the historical behavior data after the masking, whereby the correlation features represent the correlation between each data unit in the historical behavior operation data after the masking or the weight between the label identifier and the data unit; Predict the masked data units and / or the label identifiers to obtain prediction results; Based on the prediction results, the true values, and the preset loss function, the initial network parameters of the model are adjusted. If the number of iterations is less than the preset iteration threshold, the process returns to the step of inputting the historical behavior data of each user into the initial model.
4. A behavior prediction method, characterized in that, The method includes: Obtain the target user's historical behavioral data; The target's historical behavioral data is input into a model trained using the method described in any one of claims 1-3 to obtain the predicted behavior of the target user, which includes overdue behavior, investment behavior, or investment stability indicators.
5. A model training device, characterized in that, The device includes: The first acquisition module is used to acquire historical behavior data of multiple users. The historical behavior data includes historical behavior operation data and tag identifiers. The tag identifiers indicate whether the user has tag information, and the tag information indicates whether the user has performed a preset behavior. The step of acquiring historical behavior data of multiple users includes: Obtain the historical behavior operation data; The tag identifier is obtained, and when the user has the tag information and the user performs the preset behavior, the tag identifier is a first preset value; when the user has the tag information and the user does not perform the preset behavior, the tag identifier is a second preset value; when the user does not have the tag information, the tag identifier is a third preset value. The historical behavior data is obtained by combining the historical behavior data and the tag identifier. The masking module is used to mask the historical behavior data to obtain masked historical behavior data; the historical behavior operation data includes multiple data units. The step of masking the historical behavior data to obtain masked historical behavior data includes: When the user has the tag information, a portion of the data units in the tag identifier and the historical behavior operation data are masked. In the absence of the user having the tag information, a portion of the data units in the historical behavior operation data are masked. The training module is used to train the model based on the historical behavior data behind the mask.
6. A behavior prediction device, characterized in that, The device includes: The second acquisition module is used to acquire the target user's target historical behavior data. The prediction module is used to input the target's historical behavioral operation data into a model trained using the method described in any one of claims 1-5 to obtain the predicted behavior of the target user, wherein the predicted behavior includes overdue behavior, investment behavior, or investment stability indicators.
7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-3 or 4.
8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-3 or 4.