Training methods, devices, media, and equipment for data extraction task duration prediction models

Abstract

This specification provides a training method, apparatus, medium, and device for a data extraction task duration prediction model. The method includes: acquiring historical data extraction tasks; generating a corresponding feature vector for each historical data extraction task; wherein each element in the feature vector corresponds to a type of data extraction parameter, and the value of each element is the sum of the key values ​​of that type of data extraction parameter in various advertising campaigns for that historical data extraction task; acquiring the actual time consumed for each historical data extraction task, and using the feature vector and the corresponding actual time consumed for each historical data extraction task as a training sample; training the model based on multiple training samples to obtain a data extraction task duration prediction model. The data extraction task duration prediction model trained using the method provided in this embodiment can accurately and stably predict the time consumed by data extraction tasks.

Description

TECHNICAL FIELD

[0001] The one or more embodiments of the specification relate to the technical field of model training, and in particular to a training method and device of a data extraction task duration prediction model, a medium and equipment. BACKGROUND

[0002] In the current era of data explosion, as the channels for data generation continue to increase, data gradually increases, and data analysis for data follows, which requires data extraction. The conventional way is to use SQL language for data extraction, but for users, it is not possible to perceive how long the data extraction task will take to complete.

[0003] The existing solution is to make a judgment based on experience, which has randomness and low accuracy. Moreover, as the data and business situations of each company are different, specific analysis needs to be made according to the data and business situations of each company, so it is not universal. SUMMARY

[0004] To solve at least one of the above problems, the embodiments of the specification describe a training method and device of a data extraction task duration prediction model, a medium and equipment.

[0005] According to a first aspect, the embodiments of the application provide a training method of a data extraction task duration prediction model, comprising:

[0006] obtaining historical data extraction tasks; wherein the historical data extraction task is a task of extracting data from advertising data in a preset historical time period, each historical data extraction task includes an activity set and key values of at least two types of data extraction parameters, and the activity set includes an identifier of at least one advertising activity;

[0007] According to each historical data extraction task, a corresponding feature vector is generated; wherein each element in the feature vector corresponds to a type of data extraction parameter, and the value of each element is the sum of the key values of the type of data extraction parameter corresponding to the element in each advertising activity in the historical data extraction task;

[0008] obtaining the actual time consumption corresponding to each historical data extraction task, and taking the feature vector corresponding to each historical data extraction task and the actual time consumption corresponding to each historical data extraction task as a training sample;

[0009] According to the plurality of training samples, model training is performed to obtain a data extraction task duration prediction model.

[0010] According to a second aspect, the embodiments of the application provide a training device of a data extraction task duration prediction model, comprising:

[0011] The task acquisition module is configured to acquire historical extraction tasks, wherein the historical extraction task is a task of extracting data from advertisement launching data in a preset historical time period, each historical extraction task includes an activity set and key values of at least two types of extraction parameters, and the activity set includes an identifier of at least one advertisement launching activity.

[0012] The vector generation module is configured to generate a corresponding feature vector according to each historical extraction task, wherein each element in the feature vector corresponds to one type of extraction parameter, and a value of each element is a sum of key values of the type of extraction parameter corresponding to the element in each advertisement launching activity in the historical extraction task.

[0013] The time consumption acquisition module is configured to acquire an actual time consumption corresponding to each historical extraction task, and use the feature vector corresponding to each historical extraction task and the actual time consumption corresponding to each historical extraction task as a training sample.

[0014] The model training module is configured to perform model training according to the plurality of training samples to obtain an extraction task time length prediction model.

[0015] According to a third aspect, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program causes a computer to execute the method provided in the first aspect when the computer program is executed in the computer.

[0016] According to a fourth aspect, a computing device is provided, and the computing device includes a memory and a processor, the memory stores executable code, and the processor executes the executable code to implement the method provided in the first aspect.

[0017] The method and device for training an extraction task time length prediction model, the medium, the equipment, the combination or each of the embodiments of the present specification have the following technical effects:

[0018] (1) In the embodiments of the present application, historical extraction tasks are first acquired, then a corresponding feature vector is generated according to each historical extraction task, an actual time consumption corresponding to each historical extraction task is acquired, and the feature vector corresponding to each historical extraction task and the actual time consumption corresponding to each historical extraction task are used as a training sample. Finally, model training is performed according to the plurality of training samples to obtain an extraction task time length prediction model. Through the embodiments of the present application, an extraction task time length prediction model can be trained, and then the model is used to predict the time consumption. Compared with the experience judgment in the prior art, the time consumption can be more accurately, effectively and stably evaluated. Even if the data and business of each company are different, the above method can be used to train an extraction task time length prediction model suitable for the company.

[0019] (2) In the embodiment of the present application, an initial first data table is constructed, then the key values of the at least two types of bid parameters in each advertising placement activity are extracted from each historical bid task, and the key values are imported into the first data table at the intersection position of the row corresponding to the advertising placement activity of the historical bid task and the column corresponding to each type of bid parameter, and the positions in the first data table where no data is imported are treated as 0, so as to realize the filling of the first data table. Then, for each row of data corresponding to each advertising placement activity of each historical bid task in the first data table, the sum of each key value located in the same column is calculated to form a feature vector. In this way, the embodiment of the present application constructs a feature vector, so that the feature vector can reflect the characteristics of the historical bid task from multiple dimensions, and accurately represent the historical bid task.

[0020] (3) In the embodiment of the present application, when the bid task duration prediction model is used, the prediction error is determined according to the difference between the predicted duration and the actual duration of the bid task in a period of time, and then the model update condition is determined according to the prediction error, and the model is updated when the model update condition is met. It can be seen that intelligent analysis of whether to update the model can be realized. By updating the model, the model has high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0022] Figure 1 is a flow diagram of a training method of a bid task duration prediction model of the present application;

[0023] Figure 2 is a structural block diagram of a training device of a bid task duration prediction model of the present application. DETAILED DESCRIPTION

[0024] The schemes provided by the present application will be described below in combination with the drawings.

[0025] In a first aspect, the embodiment of the present application provides a training method of a bid task duration prediction model, which can be executed by any computing device, referring to Figure 1 The method comprises the following steps S110-S140:

[0026] S110, acquire historical extraction tasks; wherein the historical extraction task is a task of extracting data from advertisement delivery data in a preset historical time period, each historical extraction task includes an activity set and key values of at least two types of extraction parameters, and the activity set includes identification of at least one advertisement delivery activity;

[0027] It can be understood that the extraction task refers to a task of data extraction, and the role of the extraction task duration prediction model is to predict the duration consumed by the extraction task. The training process of the model depends on historical extraction tasks, so historical extraction tasks are acquired in S110.

[0028] The application scenario of the embodiment of the application is advertisement delivery, and the historical extraction task refers to a task of extracting data from advertisement delivery data in a preset historical time period. In this scenario, each historical extraction task includes an activity set and key values of multiple types of extraction parameters.

[0029] The activity set includes identification of at least one advertisement delivery activity. For example, the activity set in one historical extraction task is {1, 2, 3}, and the activity set in another historical extraction task is {1, 2, 3, 4}. The former activity set includes three advertisement delivery activities, and the latter activity set includes four advertisement delivery activities.

[0030] It can be understood that at least one advertisement delivery activity will be performed when advertisement delivery is performed. For example, one advertisement delivery activity is performed for an office building in a geographical location corresponding to a certain latitude and longitude, and the identification of the advertisement delivery activity is 1; one advertisement delivery activity is performed for a restaurant in the geographical location corresponding to the latitude and longitude, and the identification of the advertisement delivery activity is 2; one advertisement delivery activity is performed for an office building in a geographical location corresponding to another latitude and longitude, and the identification of the advertisement delivery activity is 3; one advertisement delivery activity is performed for a restaurant in the geographical location corresponding to the other latitude and longitude, and the identification of the advertisement delivery activity is 4. When data extraction is performed, at least one of the above four identifications can be included in the historical extraction task, for example, the activity set in each historical extraction task is {1, 2, 3}, so that related data in the advertisement delivery activities with identifications 1, 2 and 3 can be extracted.

[0031] The multiple types of extraction parameters can represent different dimension parameters of the task. For example, the start and end time corresponding to the advertisement delivery activity, the advertiser ID, the data management platform, the exposure frequency, the query engine, and the filtering condition.

[0032] The start and end time of one advertisement delivery activity is the start time and the end time, and the advertiser ID can be used to identify different advertisers.

[0033] The data management platform is a third-party data management platform, which is a management platform of a population, and stores related data of the population, such as age, gender, and the like. The data management platform is a data management platform from which a target population for an advertisement placement activity in the historical number task comes. Different advertisers can prefer to select different data management platforms, for example, advertiser A prefers to use data management platform a, and then selects a target population from data management platform a for advertisement placement for advertiser A. Correspondingly, in the corresponding historical number task, the data management platform corresponding to the advertisement placement activity is data management platform a. Advertiser B prefers to use data management platform b, and then selects a target population from data management platform B for advertisement placement for advertiser B. Correspondingly, in the corresponding historical number task, the data management platform corresponding to the advertisement placement activity is data management platform b.

[0034] The exposure frequency refers to the exposure frequency of the target population, for example, the target population browses the advertisement placed by the advertiser for three times during the advertisement placement period, and the exposure frequency is 3.

[0035] The historical number task can be stored in different databases, and different databases correspond to different query engines, for example, the historical number task in database C can be obtained through query engine c, so the query engine here refers to the query engine corresponding to the database where the historical number task is located.

[0036] The filtering condition refers to the setting condition of the advertisement placement activity, for example, the time period from 10 am to 8 pm is set as the time period of the advertisement placement activity.

[0037] It can be understood that each advertisement placement activity corresponds to the key value of the number parameter of multiple categories, for example, the key value of the number parameter corresponding to the advertisement placement activity identified as 1 includes the key value of the advertiser ID, the key value of the data management platform, and the key value of the exposure frequency. The key value of the number parameter corresponding to the advertisement placement activity identified as 2 includes the key value of the advertiser ID, the key value of the data management platform, the key value of the query engine, and the key value of the filtering condition.

[0038] The obtained historical number task is specifically a successfully executed historical number task, and after the historical number task is obtained, the preparation of the basic data is realized.

[0039] S120, according to each historical number task, a corresponding feature vector is generated; wherein each element in the feature vector corresponds to a type of number parameter, and the value of each element is the sum of the key values of the type of number parameter corresponding to each advertisement placement activity in the historical number task.

[0040] That is, for each historical bid task, a corresponding feature vector is generated, which can be used to represent the historical bid task. In a feature vector, multiple elements are included, each element corresponding to a category of bid parameter. Specifically, the value of an element is the sum of the key values of the bid parameter of the corresponding category in each advertising campaign included in a historical bid task.

[0041] In one embodiment, S120 can specifically include steps S121-S124:

[0042] S121, constructing an initial first data table; wherein the first row of the first data table is identified as the identification of each historical bid task, the second row of the first data table is identified as the identification of N1 advertising campaigns, each first row identification corresponds to N1 second row identification; the column of the first data table is identified as the identification of N2 categories of bid parameters corresponding to each advertising campaign, N1 and N2 are positive integers, N1 is the preset maximum number of advertising campaigns in the activity set, N2 is the preset maximum number of categories of bid parameters;

[0043] Wherein the first data table, for example, see Table 1 below:

[0044] Table 1

[0045]

[0046]

[0047] In Table 1 above, N1 is 4 and N2 is 6.

[0048] S122, extracting the key values of the at least two categories of bid parameters in each advertising campaign from each historical bid task, and importing the key values into the first data table at the intersection position of the row corresponding to the advertising campaign of the historical bid task and each column corresponding to each category of bid parameter;

[0049] S123, after the key values of each historical bid task are imported, the positions in the first data table where no data is imported are treated as 0;

[0050] For example, the various category number parameters corresponding to the advertising activity 1 in the historical number task e1 include: the advertiser ID, the data management platform, and the exposure frequency. The key value of the advertiser ID is filled into the cell at the intersection of the row where the advertising activity 1 is located and the column where the advertiser ID is located in the above table 1. The key value of the data management platform is filled into the cell at the intersection of the row where the advertising activity 1 is located and the column where the data management platform is located in the above table 1. The key value of the exposure frequency is filled into the cell at the intersection of the row where the advertising activity 1 is located and the column where the exposure frequency is located in the above table 1. The other cells of the row where the advertising activity 1 is located are set to 0. The data can be filled in this way for other advertising data.

[0051] In S124, the sum of the key values in the same column is calculated for each row of data corresponding to each advertising activity of each historical number task in the first data table, and the sum of the key values in different columns forms a feature vector corresponding to the historical number task.

[0052] That is, after obtaining the filled first data table, the sum of the key values in the corresponding column in each row of data of each advertising activity is calculated for each historical advertising data to obtain an element in the feature vector.

[0053] For example, for the historical number task e1, the sum of the key values of the start and end times of the advertising activities 1-4 is obtained as the first sum of key values. The sum of the key values of the advertiser IDs of the advertising activities 1-4 is obtained as the second sum of key values. The sum of the key values of the data management platforms of the advertising activities 1-4 is obtained as the third sum of key values. The sum of the key values of the exposure frequencies of the advertising activities 1-4 is obtained as the fourth sum of key values. The sum of the key values of the query engines of the advertising activities 1-4 is obtained as the fifth sum of key values. The sum of the key values of the filtering conditions of the advertising activities 1-4 is obtained as the sixth sum of key values. The six sums of key values form a feature vector corresponding to the historical number task e1, which includes six elements.

[0054] As can be seen, the feature vector corresponding to each historical number task can be obtained by the above method. It can be understood that the above process is actually an analysis of the basic data to realize feature engineering. In addition to the above category number parameters, other category number parameters can also be included in the feature engineering, for example, the initiator of the historical number task, but this number parameter is meaningless and can not be used in the above feature engineering.

[0055] It can be understood that the above various data can exist in different forms in the advertising delivery data, and need to be extracted from the advertising campaign data and filled into the first data table. For example, some data is in the form of specific numerical values, such as advertiser ID, start and end time. Some data is in the form of a set, such as a campaign set. Some data is in the form of a dictionary, such as a filter condition.

[0056] In one embodiment, before generating the corresponding feature vector according to each historical sampling task, further comprising: eliminating the historical sampling task with actual time length exceeding the preset time length from the obtained historical sampling task.

[0057] The actual time length being too long indicates that the historical sampling task is a timeout task and is not representative, and therefore the historical sampling task with the actual time length being too long is deleted, thereby realizing cleaning of the historical sampling task.

[0058] S130, obtaining the actual time length corresponding to each historical sampling task, and taking the feature vector corresponding to each historical sampling task and the corresponding actual time length as a training sample;

[0059] That is, the actual time length corresponding to each historical sampling task is obtained, and the actual time length is the time actually consumed in executing the historical sampling task. The actual time length corresponding to a historical sampling task is taken as the label information of the feature vector corresponding to the historical sampling task, and the feature vector labeled with the actual time length is taken as a training sample. In this way, a training sample set can be formed.

[0060] S140, model training is performed according to the plurality of training samples to obtain a sampling task time length prediction model.

[0061] That is, the plurality of training samples obtained by S130 are used for model training to obtain a sampling task time length prediction model. The input information of the sampling task time length prediction model is a feature vector corresponding to a sampling task, and the output information is a predicted time length corresponding to the sampling task.

[0062] In one embodiment, S140 can specifically include steps S141-S143:

[0063] S141, constructing a framework of the sampling task time length prediction model based on a gradient boosting decision tree algorithm;

[0064] The gradient boosting decision tree algorithm, namely Light Gradient Boosting Machine (LightGBM), is a distributed gradient boosting framework algorithm based on a decision tree algorithm. LigthGBM is an excellent algorithm model in the boosting ensemble model, and uses the negative gradient of the loss function as the residual approximation of the current decision tree to fit a new decision tree. LigthGBM has the advantages of strong training efficiency, lower memory occupation, and distributed large-scale data training.

[0065] S142, selecting hyperparameters by using a Bayesian optimization model;

[0066] The Bayesian optimization model is the SOTA (State of the Art) means in the current hyperparameter optimization field, and can be considered as the most advanced optimization framework. Selecting the Bayesian optimization model for hyperparameter selection can quickly search for hyperparameters, so that the time consumption prediction model can more accurately predict the time consumption.

[0067] The hyperparameters are configuration parameters outside the model, and the so-called "parameter tuning" in machine learning is actually "hyperparameter tuning". Hyperparameters, for example, learning rate, iteration number, batch size in gradient descent method, k in k-nearest neighbor method, tree depth in decision tree model, and tree number.

[0068] S143, based on the selected hyperparameters, model training is performed on the framework according to a plurality of training samples, to obtain the time consumption prediction model.

[0069] It can be understood that after the hyperparameters of the framework are set, the framework can be trained by using a plurality of training samples, so as to obtain the time consumption prediction model.

[0070] In one embodiment, the method provided by the embodiment of the application can further include:

[0071] Obtaining state data corresponding to each historical time consumption task; wherein the state data is the running state data of the server cluster.

[0072] Correspondingly, the feature vector corresponding to each historical time consumption task and the corresponding actual time consumption are taken as a training sample, including: the feature vector corresponding to each historical time consumption task, the corresponding state data, and the corresponding actual time consumption are taken as a training sample.

[0073] That is, in addition to including the feature vector and the actual consumption time length in a training sample, the running state data of the server cluster can also be included, for example, the memory usage of the server, the cpu usage, etc. The server cluster has corresponding running state data when performing each historical sampling task, and the running state data is recorded. The recorded running state data is obtained before forming the training sample.

[0074] In this way, when predicting by using the sampling task time length prediction model, in addition to the feature vector corresponding to the sampling task, the running state data at this time is also included in the input information. Based on the feature vector corresponding to the sampling task and the running state data, the corresponding consumption time length is predicted, so that the predicted consumption time length takes into account the current server pressure situation, meets the actual scene demand, and improves the prediction accuracy.

[0075] In an embodiment, the method provided by the embodiment of the present application can further include, before the model training according to the plurality of training samples to obtain the sampling task time length prediction model:

[0076] The discrete data in each training sample is encoded, and the continuous data in each training sample is normalized, so as to realize the correction processing of each training sample.

[0077] Correspondingly, the model training according to the plurality of training samples to obtain the sampling task time length prediction model includes: training the model by using the plurality of training samples after the correction processing to obtain the sampling task time length prediction model.

[0078] That is, the training sample is corrected, and the training sample after the correction is used for model training. Different data is corrected in different ways. For example, the key value corresponding to the advertiser ID can be encoded. For example, the continuous data can be normalized, for example, the key value corresponding to the start and end time can be normalized.

[0079] It can be understood that the purpose of the normalization processing here is to facilitate the model training.

[0080] It can be understood that after the sampling task time length prediction model is trained, for a new sampling task, the feature vector of the sampling task is generated, and the feature vector is input into the sampling task time length prediction model, so that the predicted consumption time length can be obtained. The generation process of the feature vector of the new sampling task is consistent with the above process.

[0081] In an embodiment, the method provided by the embodiment of the present application can further include:

[0082] After outputting a predicted time length of the number-of-picking task by using the number-of-picking task time length prediction model, a prediction error of the number-of-picking task time length prediction model for the current number-of-picking task is determined according to the predicted time length and an actual time length;

[0083] Every unit time period, whether the model update condition is met is determined according to the prediction error of the number-of-picking task in the current unit time period;

[0084] If yes, the number-of-picking task time length prediction model is updated according to the actual time length of the number-of-picking task in the current unit time period.

[0085] For example, whether the model update condition is met is determined every one hour. The prediction error is determined by the difference between the predicted time length and the actual time length of the number-of-picking task output by the number-of-picking task time length prediction model for the number-of-picking task in the one hour. If the prediction error is large, the model needs to be updated, that is, the model update condition is met, and then the number-of-picking task time length prediction model is updated according to the actual time length of the number-of-picking task in the one hour. If the prediction error is small, the model does not need to be updated, that is, the model update condition is not met. If there are multiple number-of-picking tasks in the one hour, the average of the prediction errors of the number-of-picking tasks is used for determination.

[0086] It can be seen that after the initial number-of-picking task time length prediction model is trained, the number-of-picking task time length prediction model needs to be updated, but the update time and update effect of the number-of-picking task time length prediction model cannot be effectively guaranteed, so the embodiment of the present application proposes an intelligent model update condition, that is, intelligent analysis is performed every unit time, and if the analysis result is that the model needs to be updated, the model is updated. In addition, the model can also be updated at a fixed time every day, for example, the model is updated at a fixed time every morning, so that the effectiveness and the accuracy of each day are guaranteed.

[0087] It can be understood that the method provided by the embodiment of the present application can train a number-of-picking task time length prediction model, and then the model is used for prediction of the time length. Compared with the experience judgment in the prior art, the time length can be more accurately, effectively and stably evaluated, and the error is generally within 5%. Even if the data and business of each company are different, the above method can also be used to train a number-of-picking task time length prediction model suitable for the company.

[0088] In a second aspect, the embodiment of the present application provides a training device of a number-of-picking task time length prediction model, which is shown in Figure 2 The device 100 comprises:

[0089] The task acquisition module 110 is configured to acquire historical extraction tasks, wherein the historical extraction task is a task of extracting data from advertisement launching data in a preset historical time period, each historical extraction task includes an activity set and key values of at least two types of extraction parameters, and the activity set includes an identifier of at least one advertisement launching activity;

[0090] The vector generation module 120 is configured to generate a corresponding feature vector according to each historical extraction task, wherein each element in the feature vector corresponds to one type of extraction parameter, and a value of each element is a sum of key values of the type of extraction parameter corresponding to the element in each advertisement launching activity in the historical extraction task;

[0091] The time consumption acquisition module 130 is configured to acquire an actual time consumption corresponding to each historical extraction task, and take the feature vector corresponding to each historical extraction task and the actual time consumption corresponding to the historical extraction task as a training sample.

[0092] The model training module 140 is configured to perform model training according to a plurality of training samples to obtain an extraction task time length prediction model.

[0093] In one embodiment, the vector generation module 120 is further configured to eliminate, before generating the corresponding feature vector according to each historical extraction task, a historical extraction task with an actual time consumption exceeding a preset time length from the acquired historical extraction tasks.

[0094] In one embodiment, the vector generation module 120 includes:

[0095] The first construction unit is configured to construct an initial first data table, wherein a first row of the first data table is identified as an identifier of each historical extraction task, a second row of the first data table is identified as an identifier of N1 advertisement launching activities, each first row identifier corresponds to N1 second row identifiers, a column of the first data table is identified as an identifier of N2 types of extraction parameters corresponding to each advertisement launching activity, N1 and N2 are positive integers, N1 is a preset maximum number of advertisement launching activities in the activity set, and N2 is a preset maximum number of types of extraction parameters;

[0096] The first filling unit is configured to extract key values of the at least two types of extraction parameters in each advertisement launching activity from each historical extraction task, and import the key values into a cross position of a row corresponding to the advertisement launching activity and each column corresponding to each type of extraction parameter in the historical extraction task in the first data table;

[0097] The second filling unit is configured to dispose a position in the first data table where no data is imported as 0 after the key values of all historical extraction tasks are imported.

[0098] A vector forming unit is configured to calculate a sum of key values in the same column for each row of data corresponding to each advertising delivery activity of each historical sampling task in the first data table, and a sum of key values in different columns forms a feature vector corresponding to the historical sampling task.

[0099] In one embodiment, the model training module 140 comprises:

[0100] A second construction unit is configured to construct a framework of the sampling task duration prediction model based on a gradient boosting decision tree algorithm.

[0101] A first selection unit is configured to select hyperparameters by using a Bayesian optimization model.

[0102] A model training unit is configured to perform model training on the framework based on the selected hyperparameters and the plurality of training samples, to obtain the sampling task duration prediction model.

[0103] In one embodiment, the apparatus further comprises:

[0104] A state acquisition module is configured to acquire state data corresponding to each historical sampling task, wherein the state data is running state data of a server cluster.

[0105] Correspondingly, the time consumption acquisition module is specifically configured to take the feature vector corresponding to each historical sampling task, the corresponding state data, and the corresponding actual time consumption as a training sample.

[0106] In one embodiment, the apparatus further comprises:

[0107] A correction processing module is configured to perform encoding processing on discrete data in each training sample and normalization processing on continuous data in each training sample before the model training module 140 performs model training on the plurality of training samples to obtain the sampling task duration prediction model, so as to realize correction processing on each training sample.

[0108] Correspondingly, the model training module 140 is specifically configured to perform model training on the plurality of training samples after correction processing, to obtain the sampling task duration prediction model.

[0109] In one embodiment, the apparatus further comprises:

[0110] An error calculation module is configured to determine a prediction error of the sampling task duration prediction model for a sampling task after outputting a predicted time consumption of the sampling task by using the sampling task duration prediction model, according to the predicted time consumption and an actual time consumption.

[0111] The updating judgment module is configured to determine whether a model updating condition is met according to the prediction error of the number of times of picking up corresponding to the number of times of picking up task in the unit time period.

[0112] The model updating module is configured to update the number of times of picking up task duration prediction model according to the actual time duration of the number of times of picking up task in the unit time period if the model updating condition is met.

[0113] In a third aspect, an embodiment of the present specification provides a computer readable storage medium, which stores a computer program, and the computer program causes a computer to execute the method in any one of the embodiments of the present specification when the computer program is executed in the computer.

[0114] In a fourth aspect, an embodiment of the present specification provides a computing device, which comprises a memory and a processor, and the memory stores executable code, and the processor executes the executable code to implement the method in any one of the embodiments of the present specification.

[0115] It can be understood that the structure illustrated in the embodiments of the present specification does not constitute a specific limitation on the device of the embodiments of the present specification. In other embodiments of the present specification, the device can include more or fewer components than the illustration, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0116] The information interaction, execution process and the like between the modules in the above device and system are based on the same concept as the method embodiments of the present specification, and the specific content can be referred to the description in the method embodiments of the present specification, which will not be described here.

[0117] Each of the embodiments in the present specification describes in a progressive manner, and the same and similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the difference from other embodiments. Especially for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the related parts can be referred to the part of the method embodiments.

[0118] Those skilled in the art should realize that in one or more examples described above, the functions described in the present application can be realized by hardware, software, plug-in or any combination thereof. When realized by software, these functions can be stored in a computer readable medium or transmitted as one or more instructions or codes on a computer readable medium.

[0119] The above detailed description of the specific embodiments of the present application has been given to illustrate the purpose, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not used to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solutions of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a data extraction task duration prediction model, characterized in that, The method comprises the following steps: obtaining a historical extraction task; wherein the historical extraction task is a task of extracting data from a plurality of advertisement delivery data within a preset historical time period, each historical extraction task comprises a set of activities and key values of at least two types of extraction parameters, and the set of activities comprises identifiers of a plurality of advertisement delivery activities; wherein each advertisement delivery activity corresponds to key values of a plurality of types of extraction parameters; generating a corresponding feature vector according to each historical extraction task; wherein each feature vector comprises a plurality of elements, each element in the feature vector corresponds to a type of extraction parameter, and the value of each element is the sum of the key values of the corresponding type of extraction parameter in each advertisement delivery activity in the historical extraction task; obtaining the actual time length corresponding to each historical extraction task, and taking the feature vector corresponding to each historical extraction task and the actual time length corresponding to each historical extraction task as a training sample; training a model according to a plurality of training samples to obtain an extraction task time length prediction model; wherein the generating a corresponding feature vector according to each historical extraction task comprises: constructing an initial first data table; wherein the first row identifier of the first data table is the identifier of each historical extraction task, the second row identifier of the first data table is the identifier of N1 advertisement delivery activities, each first row identifier corresponds to N1 second row identifiers; the column identifier of the first data table is the identifier of N2 types of extraction parameters corresponding to each advertisement delivery activity, N1 and N2 are positive integers, N1 is a preset maximum number of advertisement delivery activities in the set of activities, and N2 is a preset maximum number of types of extraction parameters; extracting the key values of the at least two types of extraction parameters in each advertisement delivery activity from each historical extraction task, and importing the key values into the intersection position of the row corresponding to the advertisement delivery activity of the historical extraction task and the column corresponding to each type of extraction parameter in the first data table; after the key values of all historical extraction tasks are imported, the positions in the first data table where no data is imported are treated as 0; for each row of data corresponding to each advertisement delivery activity of each historical extraction task in the first data table, calculating the sum of the key values in the same column, and the sum of the key values in different columns forms a feature vector corresponding to the historical extraction task.

2. The method of claim 1, wherein, Before the generating a corresponding feature vector according to each historical extraction task, the method further comprises: eliminating historical extraction tasks with actual time lengths exceeding a preset time length from the obtained historical extraction tasks.

3. The method of claim 1, wherein, The training a model according to a plurality of training samples to obtain an extraction task time length prediction model comprises: constructing a framework of the extraction task time length prediction model based on a gradient boosting decision tree algorithm; selecting hyperparameters using a Bayesian optimization model; based on the selected hyperparameters, training the framework according to the plurality of training samples to obtain the extraction task time length prediction model.

4. The method of claim 1, wherein, The method further comprises: obtaining state data corresponding to each historical extraction task; wherein the state data is the running state data of a server cluster. Correspondingly, the step of using the feature vector and the actual time consumed for each historical data extraction task as a training sample includes: using the feature vector, the state data, and the actual time consumed for each historical data extraction task as a training sample.

5. The method of claim 1, wherein, Before training the model based on multiple training samples to obtain the data extraction task duration prediction model, the method further includes: The discrete data in each training sample is encoded, and the continuous data in each training sample is normalized to achieve the correction processing for each training sample. Correspondingly, the step of training the model based on multiple training samples to obtain the data extraction task duration prediction model includes: training the model using the corrected multiple training samples to obtain the data extraction task duration prediction model.

6. The method of claim 1, wherein, Also includes: After using the data extraction task duration prediction model to output the predicted time of a data extraction task, the prediction error of the data extraction task duration prediction model for this data extraction task is determined based on the predicted time and the actual time. At each unit time interval, based on the prediction error corresponding to the data extraction task within that unit time interval, determine whether the model update conditions are met; If so, the data extraction task duration prediction model is updated based on the actual time consumed by the data extraction task within this unit time period.

7. A training device for a data extraction task duration prediction model, characterized in that, include: The task acquisition module is used to acquire historical data extraction tasks; wherein, the historical data extraction tasks are tasks that extract data from multiple advertising data within a preset historical time period, and each historical data extraction task includes an activity set and key values ​​of at least two types of data extraction parameters, wherein the activity set includes identifiers of multiple advertising activities; wherein, each advertising activity corresponds to key values ​​of multiple types of data extraction parameters. The vector generation module is used to generate a corresponding feature vector for each historical data extraction task. Each feature vector includes multiple elements, and each element in the feature vector corresponds to a type of data extraction parameter. The value of each element is the sum of the key values ​​of the corresponding data extraction parameter in each advertising campaign of the historical data extraction task. The time consumption acquisition module is used to acquire the actual time consumption of each historical data extraction task, and use the feature vector and the actual time consumption of each historical data extraction task as a training sample. The model training module is used to train the model based on multiple training samples to obtain a data extraction task duration prediction model. The vector generation module includes: The first construction unit is used to construct an initial first data table; wherein, the first row of the first data table is identified by the identifier of each historical data extraction task, the second row of the first data table is identified by the identifier of N1 advertising campaigns, and each first row identifier corresponds to N1 second row identifiers; the column identifiers of the first data table are the identifiers of N2 types of data extraction parameters corresponding to each advertising campaign, N1 and N2 are positive integers, N1 is the preset maximum number of advertising campaigns in the campaign set, and N2 is the preset maximum number of categories of data extraction parameters; a first filling unit configured to extract key values of the at least two types of bid parameters in each advertising campaign from each historical bid task and introduce the key values into the first data table at intersecting positions of a row corresponding to the advertising campaign of the historical bid task and columns corresponding to the bid parameters; a second filling unit configured to set positions in the first data table where no data is introduced as 0 after the key values of all the historical bid tasks are introduced; a vector forming unit configured to calculate a sum of key values in the same column and a sum of key values in different columns for each row of data corresponding to each advertising campaign of each historical bid task in the first data table to form a feature vector corresponding to the historical bid task.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer readable medium, and when the computer program is executed in the computer, the computer is caused to execute the method of any one of claims 1-6.

9. A computing device, comprising: A device includes a memory and a processor, the memory stores executable code, and the processor executes the executable code to implement the method of any one of claims 1-6.

Images