A model training method and device

Abstract

This disclosure relates to the field of artificial intelligence technology, and provides a model training method, apparatus, computer device, and computer-readable storage medium. The method determines training sample sets, validation sample sets, and test sample sets based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of multiple users. Therefore, this embodiment can adjust the duration of the time period for acquiring each sample set to obtain more positive sample business information, thereby expanding the quantity of positive sample business information in each sample set. This helps the classification model acquire more learnable positive sample business information, thus improving the learning ability of the classification model and consequently enhancing its prediction accuracy and generalization ability.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and in particular to a model training method and apparatus. Background Technology

[0002] With the advent of the big data era, the research and application of data-driven modeling methods in the modern data analysis industry have attracted widespread attention.

[0003] In recent years, the data analytics industry has continuously invested in information technology, accumulating massive amounts of business information data, which has laid a solid data foundation for the application of data-driven modeling methods. The rapid development of technologies such as big data and deep learning has provided a wealth of efficient algorithms, which are constantly improving. The performance of intelligent chips such as GPUs and TPUs is constantly improving, laying a technical foundation for the large-scale industry application of data-driven modeling.

[0004] However, the market environment has changed, and new business has encountered bottlenecks. How to fully utilize and leverage the value of existing business information has become a crucial issue for marketing operations. Currently popular classification algorithms (such as LGBM) perform well in classification; however, due to the nature of the business, the number of positive sample business information is far less than the total amount of business information. Furthermore, marketing operations focus on identifying high-quality business based on project dimensions, resulting in a significant imbalance between positive and negative samples. Consequently, models trained on a small sample set with few positive samples have very low prediction accuracy. Therefore, it is urgent to address the problem of imbalanced positive and negative samples. Summary of the Invention

[0005] In view of this, the present disclosure provides a model training method, apparatus, computer device, and computer-readable storage medium to solve the problem in the prior art that the number of positive sample business information is much smaller than the total number of business information, and marketing business is to mine high-quality business according to the project dimension, which leads to a very unbalanced positive and negative sample, resulting in very low prediction accuracy of the model trained on a sample set with a small number of positive samples.

[0006] A first aspect of this disclosure provides a model training method, the method comprising:

[0007] Obtain historical business information from multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information;

[0008] Based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users, a training sample set, a verification sample set, and a test sample set are determined; wherein, the training sample set, the verification sample set, and the test sample set each include the historical business information of at least one user;

[0009] The preset classification model is trained using the training sample set, the validation sample set, and the test sample set to obtain the trained classification model.

[0010] A second aspect of this disclosure provides a model training apparatus, the apparatus comprising:

[0011] The information acquisition unit is used to acquire historical business information of multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information;

[0012] The set determination unit is used to determine a training sample set, a verification sample set, and a test sample set based on the time information of business behavior information, the actual business result information, and the time information of actual business result information in the historical business information of the multiple users; wherein the training sample set, the verification sample set, and the test sample set each include the historical business information of at least one user;

[0013] The model training unit is used to train a preset classification model using the training sample set, the validation sample set, and the test sample set to obtain a trained classification model.

[0014] A third aspect of this disclosure provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described above.

[0015] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0016] The beneficial effects of this disclosure embodiment compared to the prior art are as follows: This disclosure embodiment can first obtain historical business information of multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information. Then, a training sample set, a validation sample set, and a test sample set can be determined based on the time information of the business behavior information, the actual business result information, and the time information of the actual business result information in the historical business information of the multiple users; wherein, the training sample set, the validation sample set, and the test sample set each include the historical business information of at least one user. Next, a preset classification model can be trained using the training sample set, the validation sample set, and the test sample set to obtain a trained classification model. As can be seen, in this embodiment, the training sample set, validation sample set, and test sample set can be determined based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users. Therefore, this embodiment can adjust the duration of the time period for obtaining each sample set (for example, lengthening the duration of the training set acquisition period, such as taking data from the past 60 days or 180 days up to the data usage date) to obtain more positive sample business information. This can expand the quantity of positive sample business information in each sample set, helping the classification model to obtain more learnable positive sample business information. In this way, more positive sample business information can be used to improve the learning ability of the classification model, thereby improving the prediction accuracy and prediction generalization ability of the classification model. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating an application scenario of an embodiment of this disclosure;

[0019] Figure 2 This is a flowchart of the model training method provided in the embodiments of this disclosure;

[0020] Figure 3 This is a block diagram of the model training apparatus provided in the embodiments of this disclosure;

[0021] Figure 4 This is a schematic diagram of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0022] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0023] A model training method and apparatus according to an embodiment of the present disclosure will now be described in detail with reference to the accompanying drawings.

[0024] In existing technologies, popular classification algorithms (such as LGBM) perform well in classification. However, due to the nature of the business, the number of positive business information samples is far less than the total number of business information samples. Furthermore, marketing business involves mining high-quality business based on project dimensions, resulting in a significant imbalance between positive and negative samples. Consequently, models trained on a sample set with a small number of positive samples have very low prediction accuracy. Therefore, it is urgent to address the problem of imbalanced positive and negative samples.

[0025] To address the aforementioned problems, this invention provides a model training method. In this method, historical business information from multiple users is first acquired. Each user's historical business information includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information. Then, a training sample set, a validation sample set, and a test sample set are determined based on the time information of the business behavior information, the actual business result information, and the time information of the actual business result information from the historical business information of the multiple users. Each of the training sample set, the validation sample set, and the test sample set includes historical business information from at least one user. Next, a preset classification model can be trained using the training sample set, the validation sample set, and the test sample set to obtain a trained classification model. As can be seen, in this embodiment, the training sample set, validation sample set, and test sample set can be determined based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users. Therefore, this embodiment can adjust the duration of the time period for obtaining each sample set (for example, lengthening the duration of the training set acquisition period, such as taking data from the past 60 days or 180 days up to the data usage date) to obtain more positive sample business information. This can expand the quantity of positive sample business information in each sample set, helping the classification model to obtain more learnable positive sample business information. In this way, more positive sample business information can be used to improve the learning ability of the classification model, thereby improving the prediction accuracy and prediction generalization ability of the classification model.

[0026] For example, embodiments of the present invention can be applied to, for example... Figure 1 The application scenario shown can include terminal device 1 and server 2.

[0027] Terminal device 1 can be hardware or software. When terminal device 1 is hardware, it can be various electronic devices with a display screen that support communication with server 2, including but not limited to smartphones, tablets, laptops, and desktop computers; when terminal device 1 is software, it can be installed in the aforementioned electronic devices. Terminal device 1 can be implemented as multiple software programs or software modules, or as a single software program or software module, and this disclosure does not limit this. Furthermore, various applications can be installed on terminal device 1, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0028] Server 2 can be a server that provides various services, such as a backend server that receives requests sent by terminal devices with which it has established communication connections. This backend server can receive and analyze the requests sent by the terminal devices and generate processing results. Server 2 can be a single server, a server cluster consisting of several servers, or a cloud computing service center; this disclosure does not limit this.

[0029] It should be noted that server 2 can be either hardware or software. When server 2 is hardware, it can be various electronic devices that provide various services to terminal device 1. When server 2 is software, it can be multiple software programs or software modules that provide various services to terminal device 1, or it can be a single software program or software module that provides various services to terminal device 1. This disclosure does not impose any limitations on this aspect.

[0030] Terminal device 1 and server 2 can communicate via a network. The network can be a wired network using coaxial cable, twisted pair, or fiber optic connection, or a wireless network that enables interconnection of various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), or Infrared. This disclosure does not limit the scope of the embodiments.

[0031] Specifically, users can input training sample bipartite graphs through terminal device 1; terminal device 1 sends historical business information of multiple users to server 2. Server 2 stores classification models to be trained; server 2 can first determine training sample sets, validation sample sets, and test sample sets based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users; then, server 2 can use the training sample set, the validation sample set, and the test sample set to train the preset classification model to obtain the trained classification model. Thus, since this application can determine the training sample set, validation sample set, and test sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users, this embodiment can adjust the duration of the time period for obtaining each sample set (for example, lengthening the duration of the time period for obtaining the training set, such as taking data from the past 60 days or 180 days up to the data usage date) to obtain more positive sample business information. This can expand the quantity of positive sample business information in each sample set, helping the classification model to obtain more learnable positive sample business information. In this way, more positive sample business information can be used to improve the learning ability of the classification model, thereby improving the prediction accuracy and prediction generalization ability of the classification model.

[0032] It should be noted that the specific types, quantities, and combinations of terminal device 1, server 2, and network can be adjusted according to the actual needs of the application scenario, and this disclosure embodiment does not impose any restrictions on this.

[0033] It should be noted that the above application scenarios are shown only for the purpose of understanding this disclosure, and the implementation of this disclosure is not limited in any way. On the contrary, the implementation of this disclosure can be applied to any applicable scenario.

[0034] Figure 2 This is a flowchart of a model training method provided in an embodiment of this disclosure. Figure 2 One model training method can be derived from Figure 1 The terminal device or server executes the command. For example... Figure 2 As shown, the model training method includes:

[0035] S201: Obtain historical business information for multiple users.

[0036] The historical business information of each user may include business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information.

[0037] In this implementation, a user's historical business information can be understood as business information that the user has already engaged in. A user's business information can be understood as all information generated by the user within a business scenario, which may include business behavior information and actual business result information. Actual business result information can be understood as the business result information that was actually generated.

[0038] It should be noted that business behavior information can be understood as user interaction behavior information generated in a business scenario; for example, in a house sales scenario, user interaction behavior information may include the target user's house viewing records; as another example, in a product sales scenario, user interaction behavior information may include the target user's clicks, favorites, and / or purchases of products. The time information of business behavior information can be understood as the time when the business behavior occurs.

[0039] Actual business result information can be understood as the outcome of a user's interactive behavior. For example, when business behavior information includes a target user's house viewing records, the actual business result information could be "house purchased" or "house not purchased." Similarly, when business behavior information includes interactive behavior information such as the target user's click count, favorites, and / or purchases of products, the actual business result information could be "product purchased" or "product not purchased." The time information in actual business result information can be understood as the time when the actual business result occurred.

[0040] S202: Based on the time information of business behavior information, actual business result information, and time information of actual business result information in the historical business information of the multiple users, determine the training sample set, the verification sample set, and the test sample set.

[0041] In this embodiment, after obtaining the historical business information of multiple users, the historical business information of multiple users can be divided into a training sample set, a verification sample set, and a test sample set based on the time information of the business behavior information, the actual business result information, and the time information of the actual business result information in the historical business information of multiple users. Each of the training sample set, the verification sample set, and the test sample set includes the historical business information of at least one user.

[0042] It is understandable that, since the duration of the time period for acquiring each sample set can be adjusted in this embodiment, the time information of business behavior information and the time information of real business results can both satisfy the historical business information of the collection time periods of the training sample set, the validation sample set, and the test sample set, which will also be different. Therefore, this embodiment can adjust the duration of the time period for acquiring each sample set (for example, lengthening the acquisition time period of the training set, such as taking data from the past 60 days or 180 days up to the data usage date) to acquire more positive sample business information, thereby expanding the quantity of positive sample business information in each sample set. In this way, data augmentation of small samples can be achieved, which can help the classification model acquire more learnable positive sample business information. Here, positive sample business information can be understood as the historical business information of the target business result, such as the purchased goods or the subscribed house.

[0043] S203: The preset classification model is trained using the training sample set, the validation sample set, and the test sample set to obtain the trained classification model.

[0044] After obtaining the training sample set, the validation sample set, and the test sample set, the preset classification model can be initially trained using the training sample set. Then, the model parameters of the preset classification model can be optimized and adjusted using the validation sample set. Finally, the trained classification model can be tested using the test sample set to ensure that the prediction accuracy of the trained classification model meets the requirements.

[0045] It should be noted that in this embodiment, the learning classification model can be an ensemble learning classification model, which is an ensemble learning model based on multiple classifiers (weak classifiers) to improve classification performance. The learning classification model combines multiple classifiers to achieve better classification results, rather than using only a single classifier. In one implementation, the learning classification model can be at least one of the following: Random Forest, Gradient Boosting Tree, AdaBoost, XGBoost (Extreme Gradient Boosting), and other machine learning classification models and deep learning classification models. Taking XGBoost (Extreme Gradient Boosting) as an example, XGBoost is a widely used model for regression, classification, and other tasks. The advantages of XGBoost are that it can handle non-linear data and supports parallel computation. Its key feature is that it can combine multiple models to improve accuracy and performance.

[0046] The beneficial effects of this disclosure embodiment compared to the prior art are as follows: This disclosure embodiment can first obtain historical business information of multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information. Then, a training sample set, a validation sample set, and a test sample set can be determined based on the time information of the business behavior information, the actual business result information, and the time information of the actual business result information in the historical business information of the multiple users; wherein, the training sample set, the validation sample set, and the test sample set each include the historical business information of at least one user. Next, a preset classification model can be trained using the training sample set, the validation sample set, and the test sample set to obtain a trained classification model. As can be seen, in this embodiment, the training sample set, validation sample set, and test sample set can be determined based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users. Therefore, this embodiment can adjust the duration of the time period for obtaining each sample set (for example, lengthening the duration of the training set acquisition period, such as taking data from the past 60 days or 180 days up to the data usage date) to obtain more positive sample business information. This can expand the quantity of positive sample business information in each sample set, helping the classification model to obtain more learnable positive sample business information. In this way, more positive sample business information can be used to improve the learning ability of the classification model, thereby improving the prediction accuracy and prediction generalization ability of the classification model.

[0047] In some embodiments, the step of "determining the training sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users" in S202 may include the following steps:

[0048] S2101: Obtain the preset first fixed time period.

[0049] In this embodiment, the preset first fixed time period can be understood as a time period with a pre-set start and end time. For example, in one implementation, the end time T of the first fixed time period is the date the dataset is used, such as October 28, 2022, and the start time of the first fixed time period is September 7, 2022, that is, the first fixed time period is from September 7, 2022 to October 28, 2022.

[0050] S2102: For each user's historical business information, if the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the first fixed time period, and the actual business result information is the target business result, then the user's historical business information is taken as the first positive sample business information; if the time information of the business behavior information in the user's historical business information is within the first fixed time period, and the time information of the actual business result information in the user's historical business information is not within the first fixed time period or the actual business result information is not the target business result, then the user's historical business information is taken as the first negative sample business information.

[0051] The target business outcome can be understood as a pre-set outcome that meets the user's expectations, such as the purchase of goods or the subscription of a house.

[0052] It is understandable that when the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the first fixed time period, and the actual business result information is the target business result, it indicates that the user's historical business information is a positive sample, and the user's historical business information can be used as the first positive sample business information.

[0053] If the time information of the business behavior information in the user's historical business information is within the first fixed time period, and the time information of the actual business result information in the user's historical business information is not within the first fixed time period or the actual business result information is not the target business result, it indicates that the user did not achieve the target business result within the first fixed time period. Therefore, the user's historical business information can be determined as a negative sample, and the user's historical business information can be used as the first negative sample business information.

[0054] S2103: Use all the first positive sample service information and a preset first number of first negative sample service information as the training sample set.

[0055] In this embodiment, to ensure a balance between positive and negative samples in the training sample set, all the first positive sample service information and a portion of the first negative sample service information can be used as the training sample set. The number of first negative sample service information items in the training sample set can be a preset first number. It should be noted that the preset first number can be fixed or determined based on the number of first positive sample service information items. For example, the number of first positive sample service information items and the number of first negative sample service information items in the training sample set can be the same, or the ratio of the number of first positive sample service information items to the number of first negative sample service information items in the training sample set can be 4:6.

[0056] In some embodiments, the step of "determining the verification sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the plurality of users" in S202 may include the following steps:

[0057] S2201: Obtain the preset second fixed time period.

[0058] In this embodiment, the preset second fixed time period can be understood as a time period with a pre-set start and end time; wherein, the end time of the second fixed time period is the same as the end time of the first fixed time period. For example, in one implementation, the end time T of the second fixed time period is the date the dataset is used, such as October 28, 2022, and the start time of the second fixed time period is October 7, 2022, that is, the second fixed time period is from October 7, 2022 to October 28, 2022.

[0059] S2202: For each user's historical business information, if the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the second fixed time period, and the actual business result information is the target business result, then the user's historical business information is used as the second positive sample business information; if the time information of the business behavior information in the user's historical business information is within the second fixed time period, and the time information of the actual business result information in the user's historical business information is not within the second fixed time period or the actual business result information is not the target business result, then the user's historical business information is used as the negative sample business information, and the user's historical business information is used as the second negative sample business information.

[0060] It is understandable that when the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the second fixed time period, and the actual business result information is the target business result, it indicates that the user's historical business information is a positive sample, and the user's historical business information can be used as the second positive sample business information.

[0061] If the time information of the business behavior information in the user's historical business information is within the second fixed time period, and the time information of the actual business result information in the user's historical business information is not within the second fixed time period or the actual business result information is not the target business result, it indicates that the user did not achieve the target business result within the second fixed time period. Therefore, the user's historical business information can be determined as a negative sample, and the user's historical business information can be used as the second negative sample business information.

[0062] S2203: Use all the second positive sample service information and the preset second number of second negative sample service information as the verification sample set.

[0063] In this embodiment, to ensure a balance between positive and negative samples in the training sample set, all the second positive sample service information and a portion of the second negative sample service information can be used as the verification sample set. The number of second negative sample service information items in the verification sample set can be a preset second number. It should be noted that the preset second number can be fixed or determined based on the number of second positive sample service information items. For example, the number of second positive sample service information items and the number of second negative sample service information items in the verification sample set can be the same, or the ratio of the number of second positive sample service information items to the number of second negative sample service information items in the verification sample set can be 4:6.

[0064] In some embodiments, the step of "determining the test sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the plurality of users" in S202 may include the following steps:

[0065] S2301: Obtain a preset third fixed time period; wherein the end time of the third fixed time period is the same as the end time of the first fixed time period.

[0066] In this embodiment, the preset third fixed time period can be understood as a time period with a pre-set start and end time; wherein, the end time of the third fixed time period is the same as the end time of the first fixed time period. For example, in one implementation, the end time T of the third fixed time period is the date the dataset is used, such as October 28, 2022, and the start time of the second fixed time period is October 14, 2022, that is, the third fixed time period is from October 14, 2022 to October 28, 2022.

[0067] S2302: For each user's historical business information, if the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the third fixed time period, and the actual business result information is the target business result, then the user's historical business information is used as the third positive sample business information; if the time information of the business behavior information in the user's historical business information is within the third fixed time period, and the time information of the actual business result information in the user's historical business information is not within the second fixed time period or the actual business result information is not the target business result, then the user's historical business information is used as the third negative sample business information.

[0068] It is understandable that when the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the third fixed time period, and the actual business result information is the target business result, it indicates that the user's historical business information is a positive sample, and the user's historical business information can be used as the third positive sample business information.

[0069] If the time information of the business behavior information in the user's historical business information is within the third fixed time period, and the time information of the actual business result information in the user's historical business information is not within the third fixed time period or the actual business result information is not the target business result, it indicates that the user did not achieve the target business result within the third fixed time period. Therefore, the user's historical business information can be determined as a negative sample, and the user's historical business information can be used as the third negative sample business information.

[0070] S2303: Use all third positive sample service information and a preset third number of third negative sample service information as the test sample set.

[0071] In this embodiment, to ensure a balance between positive and negative samples in the training sample set, all the second positive sample service information and a portion of the second negative sample service information can be used as the verification sample set. The number of second negative sample service information items in the verification sample set can be a preset second number. It should be noted that the preset second number can be fixed or determined based on the number of second positive sample service information items. For example, the number of second positive sample service information items and the number of second negative sample service information items in the verification sample set can be the same, or the ratio of the number of second positive sample service information items to the number of second negative sample service information items in the verification sample set can be 4:6.

[0072] It should be noted that, in one implementation, the duration of the third fixed time period can be the same as the duration of the second fixed time period, or the duration of the third fixed time period can be shorter than the duration of the second fixed time period; the duration of the first fixed time period can be longer than the duration of the second fixed time period and the duration of the third fixed time period.

[0073] In some embodiments, the step of "determining the training sample set based on the time information of business behavior information, the actual business result information, and the time information of actual business result information in the historical business information of the multiple users" in S202 may include the following steps:

[0074] S2401: Get the preset first duration.

[0075] In this embodiment, the preset first duration can be understood as the collection duration of training samples that is fixed in advance, for example, the first duration can be 55 days.

[0076] S2402: For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the first dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the fourth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the first dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the fourth negative sample business information.

[0077] In this embodiment, the first dynamic time period is a time period that starts with the time information of the business behavior information in the user's historical business information and has a duration equal to the first duration. For example, if the time information T of the business behavior information in the user's historical business information is October 28, 2022, and the first duration is 55 days, then the end time of the first dynamic time period can be September 3, 2022. It should be noted that if a user has multiple business behavior information, the first dynamic time period can be determined separately based on the time information of each business behavior information. In this way, a user can collect historical business information within multiple first dynamic time periods, thereby obtaining multiple historical business information of a user, which further increases the probability of obtaining the fourth positive sample business information.

[0078] It is understandable that when the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the first dynamic time period, and the actual business result information is the target business result, it indicates that the user's historical business information is a positive sample, and the user's historical business information can be used as the fourth positive sample business information.

[0079] If the time information of the business behavior information in the user's historical business information is within the first dynamic time period, and the time information of the actual business result information in the user's historical business information is not within the first dynamic time period or the actual business result information is not the target business result, it indicates that the user did not achieve the target business result within the first dynamic time period. Therefore, the user's historical business information can be determined as a negative sample, and the user's historical business information can be used as the fourth negative sample business information.

[0080] S2403: Use all the fourth positive sample business information and a preset fourth number of fourth negative sample business information as the training sample set.

[0081] In this embodiment, to ensure a balance between positive and negative samples in the training sample set, all the fourth positive sample service information and a portion of the fourth negative sample service information can be concatenated to form the training sample set. The number of fourth negative sample service information items in the training sample set can be a preset fourth number. It should be noted that the preset fourth number can be fixed or determined based on the number of fourth positive sample service information items. For example, the number of fourth positive sample service information items and the number of fourth negative sample service information items in the training sample set can be the same, or the ratio of the number of fourth positive sample service information items to the number of fourth negative sample service information items in the training sample set can be 4:6.

[0082] In some embodiments, the step of "determining the verification sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the plurality of users" in S202 may include the following steps:

[0083] S2501: Get the preset second duration.

[0084] In this embodiment, the preset second duration can be understood as the pre-fixed collection duration of the verification sample, for example, the second duration can be 28 days.

[0085] S2502: For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the second dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the fifth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the second dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the fifth negative sample business information.

[0086] In this embodiment, the second dynamic time period is a time period that starts with the time information of the business behavior information in the user's historical business information and has a duration equal to the second duration. For example, if the time information T of the business behavior information in the user's historical business information is October 28, 2022, and the second duration is 28 days, then the end time of the second dynamic time period can be September 30, 2022. It should be noted that if a user has multiple business behavior information, the second dynamic time period can be determined separately based on the time information of each business behavior information. In this way, a user can collect historical business information within multiple second dynamic time periods, thereby obtaining multiple historical business information of a user, which further increases the probability of obtaining the fifth positive sample business information.

[0087] It is understandable that when the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the second dynamic time period, and the actual business result information is the target business result, it indicates that the user's historical business information is a positive sample, and the user's historical business information can be used as the fifth positive sample business information.

[0088] If the time information of the business behavior information in the user's historical business information is within the second dynamic time period, and the time information of the actual business result information in the user's historical business information is not within the second dynamic time period or the actual business result information is not the target business result, it indicates that the user did not achieve the target business result within the second dynamic time period. Therefore, the user's historical business information can be determined as a negative sample, and the user's historical business information can be used as the fifth negative sample business information.

[0089] S2503: Use all the fifth positive sample business information and the preset fifth number of fifth negative sample business information as the verification sample set.

[0090] In this embodiment, to ensure a balance between positive and negative samples in the training sample set, all fifth positive sample service information and a portion of fifth negative sample service information can be concatenated to form the training sample set. The number of fifth negative sample service information in the training sample set can be a preset fifth number. It should be noted that the preset fifth number can be fixed or determined based on the number of fifth positive sample service information. For example, the number of fifth positive sample service information and the number of fifth negative sample service information in the training sample set can be the same, or the ratio of fifth positive sample service information to fifth negative sample service information in the training sample set can be 4:6.

[0091] In some embodiments, the step of "determining the test sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the plurality of users" in S202 may include the following steps:

[0092] S2601: Get the preset third duration.

[0093] In this embodiment, the preset second duration can be understood as the pre-fixed collection duration of the test sample, for example, the second duration can be 14 days.

[0094] S2602: For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the third dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the sixth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the third dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the sixth negative sample business information.

[0095] In this embodiment, the third dynamic time period is a time period that starts with the time information of the business behavior information in the user's historical business information and has a duration equal to the third duration. For example, if the time information T of the business behavior information in the user's historical business information is October 28, 2022, and the third duration is 14 days, then the end time of the third dynamic time period can be October 14, 2022. It should be noted that if a user has multiple business behavior information, the third dynamic time period can be determined separately based on the time information of each business behavior information. In this way, a user can collect historical business information within multiple third dynamic time periods, thereby obtaining multiple historical business information of a user, which further increases the probability of obtaining the sixth positive sample business information.

[0096] It is understandable that when the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are both within the third dynamic time period, and the actual business result information is the target business result, it indicates that the user's historical business information is a positive sample, and the user's historical business information can be used as the sixth positive sample business information.

[0097] If the time information of the business behavior information in the user's historical business information is within the third dynamic time period, and the time information of the actual business result information in the user's historical business information is not within the third dynamic time period or the actual business result information is not the target business result, it indicates that the user did not achieve the target business result within the third dynamic time period. Therefore, the user's historical business information can be determined as a negative sample, and the user's historical business information can be used as the sixth negative sample business information.

[0098] S2603: Use all sixth positive sample service information and a preset sixth number of sixth negative sample service information as the test sample set.

[0099] In this embodiment, to ensure a balance between positive and negative samples in the training sample set, all the fifth positive sample service information and a portion of the sixth negative sample service information can be concatenated to form the training sample set. The number of sixth negative sample service information items in the training sample set can be a preset sixth number. It should be noted that the preset sixth number can be fixed or determined based on the number of sixth positive sample service information items. For example, the number of sixth positive sample service information items and the number of sixth negative sample service information items in the training sample set can be the same, or the ratio of the number of sixth positive sample service information items to the number of sixth negative sample service information items in the training sample set can be 4:6.

[0100] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0101] The following are embodiments of the apparatus disclosed herein, which can be used to execute embodiments of the method disclosed herein. For details not disclosed in the apparatus embodiments of this disclosure, please refer to the embodiments of the method disclosed herein.

[0102] Figure 3 This is a schematic diagram of the model training apparatus provided in an embodiment of this disclosure. Figure 3 As shown, the model training device includes:

[0103] The information acquisition unit 301 is used to acquire historical business information of multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information;

[0104] The set determination unit 302 is used to determine a training sample set, a verification sample set, and a test sample set based on the time information of the business behavior information, the actual business result information, and the time information of the actual business result information in the historical business information of the multiple users; wherein the training sample set, the verification sample set, and the test sample set all include the historical business information of at least one user;

[0105] The model training unit 303 is used to train a preset classification model using the training sample set, the validation sample set, and the test sample set to obtain a trained classification model.

[0106] Optionally, the set determination unit 302 is used for:

[0107] Get the preset first fixed time period;

[0108] For each user's historical business information, if both the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are within the first fixed time period, and the actual business result information is the target business result, then the user's historical business information is used as the first positive sample business information; if the time information of the business behavior information in the user's historical business information is within the first fixed time period, and the time information of the actual business result information in the user's historical business information is not within the first fixed time period or the actual business result information is not the target business result, then the user's historical business information is used as the first negative sample business information.

[0109] All first positive sample business information and a preset first number of first negative sample business information are used as the training sample set.

[0110] Optionally, the set determination unit 302 is used for:

[0111] Obtain a preset second fixed time period; wherein the end time of the second fixed time period is the same as the end time of the first fixed time period;

[0112] For each user's historical business information, if both the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are within the second fixed time period, and the actual business result information is the target business result, then the user's historical business information is used as the second positive sample business information; if the time information of the business behavior information in the user's historical business information is within the second fixed time period, and the time information of the actual business result information in the user's historical business information is not within the second fixed time period or the actual business result information is not the target business result, then the user's historical business information is used as the negative sample business information, and the user's historical business information is used as the second negative sample business information.

[0113] All second positive sample business information and a preset second number of second negative sample business information are used as the verification sample set.

[0114] Optionally, the set determination unit 302 is used for:

[0115] Obtain a preset third fixed time period; wherein the end time of the third fixed time period is the same as the end time of the first fixed time period;

[0116] For each user's historical business information, if both the time information of the business behavior information and the time information of the actual business result information in the user's historical business information are within the third fixed time period, and the actual business result information is the target business result, then the user's historical business information is used as the third positive sample business information; if the time information of the business behavior information in the user's historical business information is within the third fixed time period, and the time information of the actual business result information in the user's historical business information is not within the second fixed time period or the actual business result information is not the target business result, then the user's historical business information is used as the third negative sample business information.

[0117] All third positive sample business information and a preset third number of third negative sample business information are used as the test sample set.

[0118] Optionally, the set determination unit 302 is used for:

[0119] Get the preset first duration;

[0120] For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the first dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the fourth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the first dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the fourth negative sample business information; wherein, the first dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the first duration;

[0121] All fourth positive sample business information and a preset fourth number of fourth negative sample business information are used as the training sample set.

[0122] Optionally, the set determination unit 302 is used for:

[0123] Get the preset second duration;

[0124] For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the second dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the fifth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the second dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the fifth negative sample business information; wherein, the second dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the second duration;

[0125] All fifth positive sample business information and a preset fifth number of fifth negative sample business information are used as the verification sample set.

[0126] Optionally, the set determination unit 302 is used for:

[0127] Get the preset third duration;

[0128] For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the third dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the sixth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the third dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the sixth negative sample business information; wherein, the third dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the third duration;

[0129] All sixth positive sample business information and a preset sixth number of sixth negative sample business information are used as the test sample set.

[0130] The beneficial effects of this disclosure embodiment compared with the prior art are as follows: This disclosure embodiment provides a model training device, the device comprising: an information acquisition unit, configured to acquire historical business information of multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information; a set determination unit, configured to determine a training sample set, a verification sample set, and a test sample set based on the time information of the business behavior information, the actual business result information, and the time information of the actual business result information in the historical business information of the multiple users; wherein, the training sample set, the verification sample set, and the test sample set each include the historical business information of at least one user; and a model training unit, configured to train a preset classification model using the training sample set, the verification sample set, and the test sample set to obtain a trained classification model. As can be seen, in this embodiment, the training sample set, validation sample set, and test sample set can be determined based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users. Therefore, this embodiment can adjust the duration of the time period for obtaining each sample set (for example, lengthening the duration of the training set acquisition period, such as taking data from the past 60 days or 180 days up to the data usage date) to obtain more positive sample business information. This can expand the quantity of positive sample business information in each sample set, helping the classification model to obtain more learnable positive sample business information. In this way, more positive sample business information can be used to improve the learning ability of the classification model, thereby improving the prediction accuracy and prediction generalization ability of the classification model.

[0131] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this disclosure.

[0132] Figure 4 This is a schematic diagram of the computer device 4 provided in an embodiment of this disclosure. Figure 4 As shown, the computer device 4 in this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / module in the various device embodiments described above.

[0133] Exemplarily, computer program 403 may be divided into one or more modules, which are stored in memory 402 and executed by processor 401 to perform the present disclosure. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in computer device 4.

[0134] Computer device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other similar computer device. Computer device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of computer device 4 and does not constitute a limitation on computer device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, computer device may also include input / output devices, network access devices, buses, etc.

[0135] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0136] The memory 402 can be an internal storage module of the computer device 4, such as a hard disk or RAM of the computer device 4. The memory 402 can also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 4. Furthermore, the memory 402 can include both internal and external storage modules of the computer device 4. The memory 402 is used to store computer programs and other programs and data required by the computer device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described functional modules are used as examples. In practical applications, the functions described above can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The functional modules in the embodiments can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules can be implemented in hardware or as software functional modules. Furthermore, the specific names of the functional modules are only for easy differentiation and are not intended to limit the scope of protection of this disclosure. The specific working process of the modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0138] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0139] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0140] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or modules is only a logical functional division, and in actual implementation, there may be other division methods. Multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or modules may be electrical, mechanical, or other forms.

[0141] The modules described as separate components may or may not be physically separate. Similarly, the components shown as modules may or may not be physical modules; they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0142] Furthermore, the functional modules in the various embodiments of this disclosure can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0143] If an integrated module / module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in a computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0144] The above embodiments are only used to illustrate the technical solutions of this disclosure, and are not intended to limit it. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be included within the protection scope of this disclosure.

Claims

1. A model training method, characterized in that, The method includes: Obtain historical business information from multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information; Based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users, a training sample set, a verification sample set, and a test sample set are determined. The process of determining the training sample set, the verification sample set, and the test sample set involves adjusting the duration of the time period for acquiring each sample set to obtain more positive sample business information. The training sample set, the verification sample set, and the test sample set all include the historical business information of at least one user. The preset classification model is trained using the training sample set, the validation sample set, and the test sample set to obtain the trained classification model; The step of determining the training sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users includes: Get the preset first duration; For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within a first dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the fourth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the first dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the fourth negative sample business information; wherein, the first dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the first duration; All fourth positive sample business information and a preset fourth number of fourth negative sample business information are used as the training sample set.

2. The method according to claim 1, characterized in that, The step of determining a verification sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users includes: Get the preset second duration; For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the second dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the fifth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the second dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the fifth negative sample business information; wherein, the second dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the second duration; All fifth positive sample business information and a preset fifth number of fifth negative sample business information are used as the verification sample set.

3. The method according to claim 1, characterized in that, The step of determining a test sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users includes: Get the preset third duration; For each user's historical business information, if the time information of the actual business result information in the user's historical business information is within the third dynamic time period, and the actual business result information is the target business result, then the user's historical business information is used as the sixth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the third dynamic time period or the actual business result information is not the target business result, then the user's historical business information is used as the sixth negative sample business information; wherein, the third dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the third duration; All sixth positive sample business information and a preset sixth number of sixth negative sample business information are used as the test sample set.

4. A model training device, characterized in that, The device includes: The information acquisition unit is used to acquire historical business information of multiple users; wherein, the historical business information of each user includes business behavior information, time information of the business behavior information, actual business result information, and time information of the actual business result information; The set determination unit is used to determine a training sample set, a verification sample set, and a test sample set based on the time information of business behavior information, actual business result information, and the time information of actual business result information in the historical business information of the multiple users. The process of determining the training sample set, the verification sample set, and the test sample set is to obtain more positive sample business information by adjusting the duration of the time period for obtaining each sample set. The training sample set, the verification sample set, and the test sample set all include the historical business information of at least one user. The model training unit is used to train a preset classification model using the training sample set, the validation sample set, and the test sample set to obtain a trained classification model. The set determination unit is specifically used for: obtaining a preset first duration; for each user's historical business information, if the time information of the actual business result information in the user's historical business information is within a first dynamic time period, and the actual business result information is a target business result, then the user's historical business information is used as the fourth positive sample business information; if the time information of the actual business result information in the user's historical business information is not within the first dynamic time period or the actual business result information is not a target business result, then the user's historical business information is used as the fourth negative sample business information; wherein, the first dynamic time period is a time period with the time information of the business behavior information in the user's historical business information as the starting time and the duration being the first duration; and using all the fourth positive sample business information and a preset fourth number of fourth negative sample business information as the training sample set.

5. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 3.

Images