Model building method and device, storage medium and electronic device

Abstract

The present application provides a model building method and device, storage medium and electronic device, which obtain the model type and evaluation index type selected by the user in the type selection interface, or the model type, algorithm type and evaluation index type; obtain the preset modeling wide table corresponding to the model type selected by the user; determine the target algorithm from the preset multiple model algorithms; based on the modeling wide table, build multiple initial models corresponding to each target algorithm; perform model evaluation on each initial model to obtain multiple evaluation values, and use the initial model corresponding to the maximum evaluation value as the target model. This technical solution presets multiple modeling wide tables and model algorithms. The user only needs to select the model type and evaluation index type, or select the model type, algorithm type and evaluation index type in the type selection interface to call the corresponding modeling wide table and model algorithm to build the target model without the need for programmers to program and implement, thereby improving the speed and efficiency of model building.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a model building method and device, a storage medium and an electronic device. Background Art

[0002] With the rapid development of science and technology, machine learning models are widely used in computer fields, vision fields, artificial intelligence fields, etc. Machine learning models can be used to identify and process pictures, audio, video, etc.

[0003] In the prior art, the construction of machine learning models requires programmers to implement them through programming, which results in slow model construction and low efficiency. Summary of the invention

[0004] The present application provides a model building method and device, a storage medium and an electronic device, aiming to solve the problem that the construction of existing machine learning models requires programmers to implement them through programming, resulting in slow model building speed and low efficiency.

[0005] In order to achieve the above objectives, this application provides the following technical solutions:

[0006] A model building method, comprising:

[0007] In response to a modeling task request from a user, a type selection interface is generated; the type selection interface includes multiple model types, multiple algorithm types, and multiple evaluation index types;

[0008] Obtaining the model type and evaluation index type, or the model type, algorithm type, and evaluation index type selected by the user in the type selection interface;

[0009] Obtaining a preset modeling wide table corresponding to the model type selected by the user;

[0010] If the user selects an algorithm type in the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user is determined as a target algorithm; if the user does not select an algorithm type in the type selection interface, each preset model algorithm is determined as a target algorithm;

[0011] Based on the modeling wide table, construct multiple initial models corresponding to each target algorithm;

[0012] Based on the target evaluation algorithm, each of the initial models is evaluated to obtain the evaluation value corresponding to each initial model, and the initial model corresponding to the maximum evaluation value is used as the target model; wherein the target evaluation algorithm is the evaluation algorithm corresponding to the evaluation indicator type selected by the user.

[0013] In the above method, optionally, the step of constructing multiple initial models corresponding to each target algorithm based on the modeling wide table includes:

[0014] Performing data preprocessing on each sample data included in the modeling wide table to obtain a plurality of first sample data;

[0015] According to a preset clustering method, the first sample data are grouped into multiple sets; the correlation between the first sample data in each set and other first sample data in the set is greater than the correlation between the first sample data and each first sample data in other sets, and the other sets are sets in each set except the set to which the first sample data belongs;

[0016] Selecting one first sample data from each first sample data included in each set as the second sample data;

[0017] For each of the second sample data, calculate the information volume IV value of the second sample data, and when the IV value is greater than a preset threshold, use the second sample data as target sample data;

[0018] Divide each target sample data into bins;

[0019] Based on the result data of the binning process, each target sample data is encoded to obtain the encoding result corresponding to each target sample data;

[0020] Based on the encoding results of each target sample data, multiple initial models corresponding to each target algorithm are constructed.

[0021] In the above method, optionally, the step of performing data preprocessing on each sample data included in the modeling wide table to obtain a plurality of first sample data includes:

[0022] Eliminate abnormal data from each sample data in the modeling width;

[0023] Data filling is performed on the missing sample data in the modeling wide table, and each sample data after the data filling is used as the first sample data.

[0024] In the above method, optionally, the step of selecting one first sample data from each first sample data included in each set as the second sample data includes:

[0025] For each first sample data included in each set, a first correlation between the first sample data and the set is calculated, and a second correlation between the first sample data and each other set is calculated, and based on the first correlation and a target correlation, a statistic of the first sample data is calculated; wherein the target correlation is the second correlation with the largest value among the second correlations;

[0026] For each set, the first sample data with the smallest statistic among the first sample data in the set is used as the second sample data.

[0027] In the above method, optionally, obtaining a preset modeling wide table corresponding to the model type selected by the user includes:

[0028] Determine whether there is a modeling wide table corresponding to the model type selected by the user in the cache database;

[0029] If a modeling wide table corresponding to the model type selected by the user exists in the cache database, obtaining the modeling wide table corresponding to the model type selected by the user in the cache database;

[0030] If the modeling wide table corresponding to the model type selected by the user does not exist in the cache database, the modeling wide table corresponding to the model type selected by the user is obtained from the data warehouse, and the modeling wide table is stored in the cache database, and the modeling wide table corresponding to the model type selected by the user in the cache data is obtained.

[0031] The above method may optionally further include:

[0032] The operation logs generated during the model evaluation and model building process are acquired in real time, and the acquired operation logs are stored in the cache database.

[0033] The above method, optionally, after taking the initial model corresponding to the maximum evaluation value as the target model, further comprises:

[0034] Storing the target model in a NAS disk;

[0035] The target model is visualized.

[0036] A model building device, comprising:

[0037] A generating unit, configured to generate a type selection interface in response to a modeling task request from a user; the type selection interface includes a plurality of model types, a plurality of algorithm types, and a plurality of evaluation index types;

[0038] A first acquisition unit, used to acquire the model type and evaluation index type, or the model type, algorithm type and evaluation index type selected by the user in the type selection interface;

[0039] A second acquisition unit, configured to acquire a preset modeling wide table corresponding to the model type selected by the user;

[0040] a determination unit, configured to determine, if the user selects an algorithm type on the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user as a target algorithm, and, if the user does not select an algorithm type on the type selection interface, determine each preset model algorithm as a target algorithm;

[0041] A construction unit, used to construct a plurality of initial models corresponding to each target algorithm based on the modeling wide table;

[0042] An evaluation unit is used to perform model evaluation on each of the initial models based on a target evaluation algorithm, obtain an evaluation value corresponding to each initial model, and take the initial model corresponding to the maximum evaluation value as the target model; wherein the target evaluation algorithm is an evaluation algorithm corresponding to the evaluation indicator type selected by the user.

[0043] A storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned model building method.

[0044] An electronic device includes a memory and one or more instructions, wherein the one or more instructions are stored in the memory and are configured to be executed by one or more processors to execute the above-mentioned model building method.

[0045] Compared with the prior art, this application has the following advantages:

[0046] The present application provides a fault event method and device, a storage medium and an electronic device, the method comprising: generating a type selection interface in response to a user's modeling task request; obtaining the model type and evaluation index type, or the model type, algorithm type and evaluation index type selected by the user in the type selection interface; obtaining a preset modeling wide table corresponding to the model type selected by the user; if the user selects an algorithm type in the type selection interface, the preset model algorithm corresponding to the algorithm type selected by the user is determined as a target algorithm, and if the user does not select an algorithm type in the type selection interface, each preset model algorithm is determined as a target algorithm; based on the modeling wide table, multiple initial models corresponding to each target algorithm are constructed; based on the target evaluation algorithm, each initial model is evaluated to obtain an evaluation value corresponding to each initial model, and the initial model corresponding to the maximum evaluation value is used as the target model; the target evaluation algorithm is an evaluation algorithm corresponding to the evaluation index type selected by the user. This technical solution presets multiple modeling wide tables and multiple model algorithms. Users only need to select the model type and evaluation index type in the type selection interface, or select the model type, algorithm type and evaluation index type, to call the corresponding modeling wide table and model algorithm to build the target model without the need for programmers to implement programming, thereby improving the speed of model construction and improving the efficiency of model construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0048] Figure 1 A method flow chart of a model building method provided in this application;

[0049] Figure 2 A flow chart of another method for constructing a model provided in this application;

[0050] Figure 3 An example diagram of a model building method provided in this application;

[0051] Figure 4 A schematic diagram of the structure of a model building device provided in this application;

[0052] Figure 5 A schematic diagram of the structure of an electronic device provided in this application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] The present application can be used in many general or special computing device environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multi-processor devices, distributed computing environments including any of the above devices or equipment, etc.

[0055] The present application embodiment provides a model building method, which can be applied to a model building system. The method flow chart of the method is as follows: Figure 1 As shown, specifically including:

[0056] S101. In response to a modeling task request from a user, a type selection interface is generated.

[0057] When a user needs to build a model, he sends a modeling task request to the model building system. The user's modeling task request can be generated by triggering model building related function keys in the model building system.

[0058] After receiving the modeling task request from the user, the model building system responds to the modeling task request from the user and generates a type selection interface. The type selection interface includes multiple model types, multiple algorithm types and multiple evaluation index types. The user can select the model type and evaluation index type related to the model to be built according to the needs, or select the model type, algorithm type and evaluation index type related to the model to be built according to the needs. Among them, the model type is the identification information of the modeling wide table, the modeling wide table is a sample data table, and the modeling wide table includes multiple sample data; the algorithm type is the identification information of the model algorithm, and the model algorithm is a machine learning algorithm, including but not limited to: Logistic Regression, Decision Tree, Random Forest and Neural Network algorithm; the evaluation index type is the identification information of the evaluation algorithm, and the evaluation algorithm includes but is not limited to: ROC curve, KS curve, recall rate (Recall), precision rate (Precision), lift (Lift), gain (Gain), accuracy (Accuracy) and variable importance.

[0059] S102: Obtain the model type and evaluation index type, or the model type, algorithm type, and evaluation index type selected by the user in the type selection interface.

[0060] If the user selects the model type and evaluation index type in the type selection interface, the model type and evaluation index type selected by the user in the type selection interface are obtained; if the user selects the model type, algorithm type and evaluation index type in the type selection interface, the model type, algorithm type and evaluation index type selected by the user in the type selection interface are obtained.

[0061] S103: Obtain a preset modeling wide table corresponding to the model type selected by the user.

[0062] In the method provided in the embodiment of the present application, multiple modeling wide tables are preset and stored in the data warehouse (Greenplum).

[0063] In the method provided in the embodiment of the present application, a preset wide modeling table corresponding to the model type selected by the user is obtained. Specifically, the wide modeling table corresponding to the model type selected by the user may be obtained from a cache database.

[0064] The process of obtaining a preset modeling wide table corresponding to the model type selected by the user specifically includes:

[0065] Determine whether there is a modeling wide table corresponding to the model type selected by the user in the cache database;

[0066] If a modeling wide table corresponding to the model type selected by the user exists in the cache database, obtaining the modeling wide table corresponding to the model type selected by the user in the cache database;

[0067] If the modeling wide table corresponding to the model type selected by the user does not exist in the cache database, the modeling wide table corresponding to the model type selected by the user is obtained from the data warehouse, and the modeling wide table is stored in the cache database, and the modeling wide table corresponding to the model type selected by the user in the cache data is obtained.

[0068] In the method provided in the embodiment of the present application, after the user selects the model type, the data exploration model in the model building system determines whether there is a modeling wide table corresponding to the model type selected by the user in the database based on the model type selected by the user. If not, the modeling wide table corresponding to the model type selected by the user is obtained from the data warehouse, and the obtained modeling wide table is stored in the cache database. It should be noted that the cache database periodically cleans up the modeling wide table stored in itself, that is, after the modeling wide table stored in the cache database reaches the set time, the cache database deletes it.

[0069] The algorithm engine module in the model building system obtains the modeling wide table corresponding to the model type selected by the user from the cache database.

[0070] S104. If the user selects an algorithm type in the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user is determined as a target algorithm; if the user does not select an algorithm type in the type selection interface, each preset model algorithm is determined as a target algorithm.

[0071] In the method provided in the embodiment of the present application, multiple model algorithms are preset, each model algorithm corresponds to multiple groups of algorithm parameters, and each group of algorithm parameters is used to construct a model, that is, each model algorithm can construct multiple models.

[0072] In the method provided in the embodiment of the present application, the user can select any algorithm type in the type selection interface, or may not select an algorithm type. If the user selects an algorithm type in the type selection interface, the model algorithm corresponding to the algorithm type selected by the user is determined from multiple preset model algorithms, and the model algorithm corresponding to the algorithm type selected by the user is determined as the target algorithm; if the user does not select an algorithm type in the type selection interface, each preset algorithm type is determined as the target algorithm.

[0073] S105. Based on the modeling wide table, construct multiple initial models corresponding to each target algorithm.

[0074] In the method provided in the embodiment of the present application, for each target algorithm, based on the modeling wide table corresponding to the model type selected by the user, each set of algorithm parameters corresponding to the model algorithm is used to construct an initial model corresponding to each set of algorithm parameters, thereby obtaining multiple initial models corresponding to the target algorithm.

[0075] See also Figure 2 , based on the modeling wide table, the process of building multiple initial models corresponding to each target algorithm includes:

[0076] S201: Perform data preprocessing on each sample data included in the modeling wide table to obtain a plurality of first sample data.

[0077] According to the modeling wide table corresponding to the model type selected by the user, data preprocessing is performed on each sample data included in the modeling wide table, so as to obtain a plurality of first sample data.

[0078] The process of preprocessing the sample data contained in the modeling wide table includes:

[0079] Eliminate abnormal data from each sample data in the modeling width;

[0080] Data filling is performed on the missing sample data in the modeling wide table, and each sample data after the data filling is used as the first sample data.

[0081] In the method provided in the embodiment of the present application, based on the data type of each sample data contained in the modeling wide table, each sample data is divided into a continuous data group and a discrete data group, wherein the data type of each sample data in the continuous data group is continuous, and the data type of each sample data in the discrete data group is discrete. For each sample data in the continuous data group, according to the first calculation strategy, the first calculation value of the sample data is calculated, and it is determined whether the first calculation value satisfies the first abnormal rule. If it does, the sample data is used as abnormal data. Among them, the first calculation value includes multiple first calculation sub-values, and the first calculation value includes but is not limited to quantile, skewness, kurtosis, standard deviation and missing rate. The process of determining whether the first calculation value satisfies the first abnormal rule is specifically: determining whether each first calculation sub-value satisfies the first abnormal rule corresponding to each other, and if they all meet, it is determined that the first calculation value satisfies the first abnormal rule.

[0082] For each sample data in the discrete data group, a second calculation value of the sample data is calculated according to a second calculation strategy, and it is determined whether the second calculation value satisfies the second abnormal rule. If so, the sample data is used as abnormal data. The second calculation value includes a plurality of second calculation sub-values, and the second calculation value includes but is not limited to the missing rate, the number of levels, and the sample proportion of each level. The process of determining whether the second calculation value satisfies the second abnormal rule is specifically as follows: determining whether each second calculation sub-value satisfies the corresponding second abnormal rule. If all of them are satisfied, it is determined that the second calculation value satisfies the second abnormal rule.

[0083] In the method provided in the embodiment of the present application, abnormal data in the determined modeling wide table is eliminated, and data filling is performed on the missing sample data in the modeling wide table. Each sample data remaining after the abnormal data is eliminated in the modeling wide table, and each data obtained after data filling is used as the first sample data, thereby obtaining multiple first sample data.

[0084] S202: Group the first sample data into multiple sets according to a preset clustering method.

[0085] According to a preset clustering method, each first sample data is grouped into multiple sets, and the correlation between the first sample data in each set and other first sample data in the set is greater than the correlation between the first sample data and each first sample data in other sets, and other sets are sets other than the set to which the first sample data belongs in each set. In other words, the correlation between each first sample data in each set is greater than a first threshold, and the correlation between each first sample data in each set and first sample data in other sets is less than a second threshold, wherein the first threshold is greater than the second threshold.

[0086] Optionally, the preset clustering method may be R-type clustering.

[0087] S203: Select one first sample data from each first sample data included in each set as the second sample data.

[0088] Thus, one first sample data is selected from each first sample data contained in each set as the second sample data.

[0089] The process of selecting one first sample data from each first sample data included in each set as the second sample data specifically includes:

[0090] For each first sample data contained in each set, a first correlation between the first sample data and the set is calculated, and a second correlation between the first sample data and each other set is calculated, and based on the first correlation and a target correlation, a statistic of the first sample data is calculated; wherein the target correlation is the second correlation with the largest value among the second correlations;

[0091] For each set, the first sample data with the smallest statistic among the first sample data in the set is used as the second sample data.

[0092] In the method provided in the embodiment of the present application, for each first sample data contained in each set, a first correlation between the first sample number and the set is calculated. The first correlation can be represented by the symbol R 2 own_cluster, and calculate the second correlation between the first sample data and other sets. The second correlation between the sets of other sets except the set to which the first sample data belongs can be represented by the symbol R 2 Next_cluster indicates that the second correlation with the largest value is selected from each second correlation as the target correlation, and the statistics of the first sample data are calculated based on the first correlation and the target correlation by using the statistics calculation formula. The statistics calculation formula is: Among them, 1-R 2 ratio represents the statistic of the first sample data.

[0093] In the method provided in the embodiment of the present application, for each set, based on the statistics of each first sample data in the set, the first sample data with the smallest statistics is used as the second sample data, thereby achieving fast unsupervised dimensionality reduction and eliminating multicollinearity between the first sample data.

[0094] S204: For each second sample data, calculate the information volume IV value of the second sample data, and when the IV value is greater than a preset threshold, use the second sample data as target sample data.

[0095] For each second sample data, the information IV value of the second sample data is calculated. It should be noted that the calculation process of the IV value can refer to the existing IV value calculation process, which will not be repeated here.

[0096] For each second sample data, the IV of the second sample data is compared with a preset threshold. If the IV value of the second sample data is greater than the preset threshold, the second sample data is used as the target sample data. It should be noted that the preset threshold can be modified according to needs, and this application does not limit the value of the preset threshold.

[0097] S205: bin each target sample data.

[0098] Based on the decision tree algorithm, each target sample data is binned. Specifically, based on the business correlation between each second target sample data, each target sample data is binned to obtain multiple sample boxes. It should be noted that the condition for the end of the binning process is that the number of sample boxes is greater than the binning threshold.

[0099] It should be noted that the correlation between each target sample data in each sample box and other target sample data in the sample box is greater than the correlation between the target sample data and the target sample data in other sample boxes. Figure 3 , B1-B10 are all target sample data. After binning the target sample data, we get the first sample box composed of B1 and B2, the second sample box composed of B3 itself, the third sample box composed of B4, B5 and B6, the fourth sample box composed of B7 itself, and the fifth sample box composed of B8, B9 and B10. It can be seen that for B1 in the first sample box, the correlation between B1 and B2 is greater than the correlation between B1 and any target sample data from B3 to B10. For B2 in the first sample box, the correlation between B2 and B1 is greater than the correlation between B2 and any target sample data from B3 to B10.

[0100] S206: Based on the result data of the binning process, encode each target sample data to obtain an encoding result corresponding to each target sample data.

[0101] Based on the result data of the binning process, each target sample data is encoded, that is, based on each sample box obtained after the binning process, each target sample data contained in each sample box is encoded to obtain the encoding result corresponding to each target sample data. Optionally, encoding each target sample data may be woe encoding of each target sample data.

[0102] Optionally, a coding mapping table is generated based on the result data of the binning processing and the coding result corresponding to each target sample data.

[0103] S207. Based on the encoding result of each target sample data, construct multiple initial models corresponding to each target algorithm.

[0104] For each target algorithm, based on the encoding results of each target sample data, an initial model corresponding to each set of algorithm parameters is constructed using each set of algorithm parameters corresponding to the model algorithm, thereby obtaining multiple initial models corresponding to the target algorithm.

[0105] S106. Based on the target evaluation algorithm, each initial model is evaluated to obtain the evaluation value corresponding to each initial model, and the initial model corresponding to the maximum evaluation value is used as the target model; wherein the target evaluation algorithm is the evaluation algorithm corresponding to the evaluation indicator type selected by the user.

[0106] In the method provided in the embodiment of the present application, multiple evaluation algorithms are preset, and based on the evaluation indicator type selected by the user, an evaluation algorithm corresponding to the evaluation indicator type is obtained, and the algorithm is determined as the target evaluation algorithm, and a model evaluation is performed on each initial model based on the target evaluation algorithm to obtain an evaluation value of each initial model, and the evaluation values ​​are compared to determine the evaluation value with the largest value, and the evaluation value with the largest value is used as the maximum evaluation value, and the initial model corresponding to the maximum evaluation value is determined as the target model.

[0107] In the method provided in the embodiment of the present application, model evaluation can also be performed on each initial model based on each evaluation algorithm to obtain multiple evaluation results corresponding to each initial model, generate an evaluation result table for each evaluation result corresponding to each model, and send the evaluation result table to the device corresponding to the user.

[0108] In the model building method provided in the embodiment of the present application, the model type and evaluation index type selected by the user in the type selection interface, or the model type, algorithm type and evaluation index type are obtained; a preset modeling wide table corresponding to the model type selected by the user is obtained; a target algorithm is determined from a plurality of preset model algorithms; based on the modeling wide table, a plurality of initial models corresponding to each target algorithm are constructed; a model evaluation is performed on each initial model to obtain a plurality of evaluation values, and the initial model corresponding to the maximum evaluation value is used as the target model. Applying the model building method provided in the embodiment of the present application, a plurality of modeling wide tables and model algorithms are preset, and the user only needs to select the model type and evaluation index type, or select the model type, algorithm type and evaluation index type in the type selection interface to realize the call of the corresponding modeling wide table and model algorithm, and construct the target model, without the need for programming by the programmer, thereby improving the speed and efficiency of model building.

[0109] The model building method provided in the embodiment of the present application may also include: obtaining the operation logs generated in the model evaluation and model building process in real time, and storing the obtained operation logs in the cache database. By storing the operation logs generated in the model building process and the model evaluation process in the cache database, the user can retrieve the operation logs at any time to view them, thereby achieving real-time monitoring of the operation of model building and model evaluation, and when the operation log indicates that a fault has occurred in the model building and model evaluation process, the execution of the model building or model evaluation process is stopped.

[0110] In the model building method provided in the embodiment of the present application, after the initial model corresponding to the maximum evaluation value is used as the target model, it also includes: storing the target model in a NAS disk; and visually displaying the target model.

[0111] In the method provided in the embodiment of the present application, the target model is stored in a NAS disk to achieve data sharing and ensure data synchronization. The target model is visualized, and the target scoring algorithm is deployed to the model algorithm prediction module in the model building system, so that the target evaluation algorithm can be called later to perform model evaluation on the target model.

[0112] and Figure 1 Corresponding to the method described above, the embodiment of the present application also provides a model building device for Figure 1 The specific implementation of the method is shown in the following diagram: Figure 4 As shown, specifically including:

[0113] A generating unit 401 is used to generate a type selection interface in response to a modeling task request from a user; the type selection interface includes multiple model types, multiple algorithm types, and multiple evaluation index types;

[0114] A first acquisition unit 402 is used to acquire the model type and evaluation index type, or the model type, algorithm type and evaluation index type selected by the user in the type selection interface;

[0115] A second acquisition unit 403 is used to acquire a preset modeling wide table corresponding to the model type selected by the user;

[0116] A determination unit 404 is configured to determine, if the user selects an algorithm type in the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user as a target algorithm, and if the user does not select an algorithm type in the type selection interface, determine each preset model algorithm as a target algorithm;

[0117] A construction unit 405 is used to construct a plurality of initial models corresponding to each target algorithm based on the modeling wide table;

[0118] The evaluation unit 406 is used to perform model evaluation on each of the initial models based on a target evaluation algorithm, obtain an evaluation value corresponding to each initial model, and use the initial model corresponding to the maximum evaluation value as the target model; wherein the target evaluation algorithm is an evaluation algorithm corresponding to the evaluation indicator type selected by the user.

[0119] The model building device provided in the embodiment of the present application presets multiple modeling wide tables and multiple model algorithms. The user only needs to select the model type and the evaluation index type in the type selection interface, or select the model type, the algorithm type and the evaluation index type, to call the corresponding modeling wide table and the model algorithm to build the target model without the need for programming by the programmer, thereby improving the speed of model building and improving the efficiency of model building.

[0120] In one embodiment of the present application, based on the above solution, the construction unit 405 is configured as follows:

[0121] A preprocessing subunit, configured to perform data preprocessing on each sample data contained in the modeling wide table to obtain a plurality of first sample data;

[0122] a combining subunit, configured to group the first sample data into a plurality of sets according to a preset clustering method; the correlation between the first sample data in each set and other first sample data in the set is greater than the correlation between the first sample data and each first sample data in other sets, and the other sets are sets in each set except the set to which the first sample data belongs;

[0123] A selection subunit, used to select one first sample data from each first sample data included in each set as the second sample data;

[0124] a calculation subunit, configured to calculate, for each of the second sample data, an information volume IV value of the second sample data, and when the IV value is greater than a preset threshold, use the second sample data as target sample data;

[0125] A binning unit is used to bin each target sample data;

[0126] The encoding subunit is used to encode each target sample data based on the result data of the binning process to obtain the encoding result corresponding to each target sample data;

[0127] The construction subunit is used to construct multiple initial models corresponding to each target algorithm based on the encoding result of each target sample data.

[0128] In one embodiment of the present application, based on the above solution, the preprocessing subunit performs data preprocessing on each sample data contained in the modeling wide table to obtain a plurality of first sample data for:

[0129] Eliminate abnormal data from each sample data in the modeling width;

[0130] Data filling is performed on the missing sample data in the modeling wide table, and each sample data after the data filling is used as the first sample data.

[0131] In one embodiment of the present application, based on the above solution, the selection subunit selects one first sample data from each first sample data included in each set as the second sample data, for:

[0132] For each first sample data included in each set, a first correlation between the first sample data and the set is calculated, and a second correlation between the first sample data and each other set is calculated, and based on the first correlation and a target correlation, a statistic of the first sample data is calculated; wherein the target correlation is the second correlation with the largest value among the second correlations;

[0133] For each set, the first sample data with the smallest statistic among the first sample data in the set is used as the second sample data.

[0134] In one embodiment of the present application, based on the above solution, the second acquisition unit 403 executes acquisition of a modeling wide table preset corresponding to the model type selected by the user, for:

[0135] Determine whether there is a modeling wide table corresponding to the model type selected by the user in the cache database;

[0136] If a modeling wide table corresponding to the model type selected by the user exists in the cache database, obtaining the modeling wide table corresponding to the model type selected by the user in the cache database;

[0137] If the modeling wide table corresponding to the model type selected by the user does not exist in the cache database, the modeling wide table corresponding to the model type selected by the user is obtained from the data warehouse, and the modeling wide table is stored in the cache database, and the modeling wide table corresponding to the model type selected by the user in the cache data is obtained.

[0138] In one embodiment of the present application, based on the above solution, it can also be configured as follows:

[0139] The third acquisition unit is used to acquire the operation logs generated in the model evaluation and model construction process in real time, and store the acquired operation logs in the cache database.

[0140] In one embodiment of the present application, based on the above solution, it can also be configured as follows:

[0141] A storage unit, used for storing the target model in a NAS disk;

[0142] A display unit is used to visually display the target model.

[0143] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the above-mentioned model building method.

[0144] The present application also provides an electronic device, whose structural diagram is as follows: Figure 5 As shown, it specifically includes a memory 501 and one or more instructions 502, wherein the one or more instructions 502 are stored in the memory 501 and are configured to be executed by one or more processors 503 to perform the following operations:

[0145] In response to a modeling task request from a user, a type selection interface is generated; the type selection interface includes multiple model types, multiple algorithm types, and multiple evaluation index types;

[0146] Obtaining the model type and evaluation index type, or the model type, algorithm type, and evaluation index type selected by the user in the type selection interface;

[0147] Obtaining a preset modeling wide table corresponding to the model type selected by the user;

[0148] If the user selects an algorithm type in the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user is determined as a target algorithm; if the user does not select an algorithm type in the type selection interface, each preset model algorithm is determined as a target algorithm;

[0149] Based on the modeling wide table, construct multiple initial models corresponding to each target algorithm;

[0150] Based on the target evaluation algorithm, each of the initial models is evaluated to obtain the evaluation value corresponding to each initial model, and the initial model corresponding to the maximum evaluation value is used as the target model; wherein the target evaluation algorithm is the evaluation algorithm corresponding to the evaluation indicator type selected by the user.

[0151] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0152] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0153] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0154] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0155] The above is a detailed introduction to a model building method and device, storage medium and electronic device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A model building method, characterized in that: include: In response to a modeling task request from a user, generating a type selection interface; The type selection interface includes multiple model types, multiple algorithm types and multiple evaluation index types; Obtaining the model type and evaluation index type, or the model type, algorithm type, and evaluation index type selected by the user in the type selection interface; Obtaining a preset modeling wide table corresponding to the model type selected by the user; If the user selects an algorithm type in the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user is determined as a target algorithm; if the user does not select an algorithm type in the type selection interface, each preset model algorithm is determined as a target algorithm, wherein each model algorithm corresponds to multiple groups of algorithm parameters, and each group of algorithm parameters is used to construct a model; Based on the modeling wide table, construct multiple initial models corresponding to each target algorithm; Based on the target evaluation algorithm, each of the initial models is evaluated to obtain the evaluation value corresponding to each initial model, and the initial model corresponding to the maximum evaluation value is used as the target model; wherein the target evaluation algorithm is the evaluation algorithm corresponding to the evaluation indicator type selected by the user; The step of constructing a plurality of initial models corresponding to each target algorithm based on the modeling wide table includes: Performing data preprocessing on each sample data included in the modeling wide table to obtain a plurality of first sample data; According to a preset clustering method, the first sample data are grouped into multiple sets; the correlation between the first sample data in each set and other first sample data in the set is greater than the correlation between the first sample data and each first sample data in other sets, and the other sets are sets in each set except the set to which the first sample data belongs; Selecting one first sample data from each first sample data included in each set as the second sample data; For each of the second sample data, calculate the information volume IV value of the second sample data, and when the IV value is greater than a preset threshold, use the second sample data as target sample data; Divide each target sample data into bins; Based on the result data of the binning process, each target sample data is encoded to obtain the encoding result corresponding to each target sample data; Based on the encoding results of each target sample data, multiple initial models corresponding to each target algorithm are constructed.

2. The method according to claim 1, characterized in that The performing data preprocessing on each sample data included in the modeling wide table to obtain a plurality of first sample data includes: Eliminate abnormal data from each sample data in the modeling width; Data filling is performed on the missing sample data in the modeling wide table, and each sample data after the data filling is used as the first sample data.

3. The method according to claim 1, characterized in that The selecting one first sample data from each first sample data included in each set as the second sample data includes: For each first sample data included in each set, a first correlation between the first sample data and the set is calculated, and a second correlation between the first sample data and each other set is calculated, and based on the first correlation and a target correlation, a statistic of the first sample data is calculated; wherein the target correlation is the second correlation with the largest value among the second correlations; For each set, the first sample data with the smallest statistic among the first sample data in the set is used as the second sample data.

4. The method according to claim 1, characterized in that: The obtaining of a modeling wide table preset to correspond to the model type selected by the user includes: Determine whether there is a modeling wide table corresponding to the model type selected by the user in the cache database; If a modeling wide table corresponding to the model type selected by the user exists in the cache database, obtaining the modeling wide table corresponding to the model type selected by the user in the cache database; If the modeling wide table corresponding to the model type selected by the user does not exist in the cache database, the modeling wide table corresponding to the model type selected by the user is obtained from the data warehouse, and the modeling wide table is stored in the cache database, and the modeling wide table corresponding to the model type selected by the user in the cache data is obtained.

5. The method according to claim 4, characterized in that Also includes: The operation logs generated during the model evaluation and model building process are acquired in real time, and the acquired operation logs are stored in the cache database.

6. The method according to claim 1, characterized in that After the initial model corresponding to the maximum evaluation value is used as the target model, the method further includes: Storing the target model in a NAS disk; The target model is visualized.

7. A model building device, characterized in that: include: A generating unit, used to generate a type selection interface in response to a modeling task request from a user; The type selection interface includes multiple model types, multiple algorithm types and multiple evaluation index types; A first acquisition unit, used to acquire the model type and evaluation index type, or the model type, algorithm type and evaluation index type selected by the user in the type selection interface; A second acquisition unit, configured to acquire a preset modeling wide table corresponding to the model type selected by the user; a determination unit, configured to determine, if the user selects an algorithm type in the type selection interface, a preset model algorithm corresponding to the algorithm type selected by the user as a target algorithm, and if the user does not select an algorithm type in the type selection interface, determine each preset model algorithm as a target algorithm, wherein each model algorithm corresponds to multiple groups of algorithm parameters, and each group of algorithm parameters is used to construct a model; A construction unit, used to construct a plurality of initial models corresponding to each target algorithm based on the modeling wide table; An evaluation unit, configured to perform model evaluation on each of the initial models based on a target evaluation algorithm, obtain an evaluation value corresponding to each initial model, and use the initial model corresponding to the maximum evaluation value as the target model; wherein the target evaluation algorithm is an evaluation algorithm corresponding to the evaluation indicator type selected by the user; The construction unit is configured as follows: A preprocessing subunit, configured to perform data preprocessing on each sample data contained in the modeling wide table to obtain a plurality of first sample data; a combining subunit, configured to group the first sample data into a plurality of sets according to a preset clustering method; the correlation between the first sample data in each set and other first sample data in the set is greater than the correlation between the first sample data and each first sample data in other sets, and the other sets are sets in each set except the set to which the first sample data belongs; A selection subunit, used to select one first sample data from each first sample data included in each set as the second sample data; a calculation subunit, configured to calculate, for each of the second sample data, an information volume IV value of the second sample data, and when the IV value is greater than a preset threshold, use the second sample data as target sample data; The binning unit is used to bin each target sample data; The encoding subunit is used to encode each target sample data based on the result data of the binning process to obtain the encoding result corresponding to each target sample data; The construction subunit is used to construct multiple initial models corresponding to each target algorithm based on the encoding result of each target sample data.

8. A storage medium, characterized in that: The storage medium includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the model building method according to any one of claims 1 to 6.

9. An electronic device, characterized in that: The system comprises a memory and one or more instructions, wherein the one or more instructions are stored in the memory and configured to be executed by one or more processors to execute the model building method according to any one of claims 1 to 6.

Images