Information processing method and device, electronic equipment and computer readable storage medium
By acquiring the configuration information table to process feature variables and train the modeling data, the problem of low prediction efficiency of broadband satisfaction model is solved, realizing fast and automated model building and prediction, improving efficiency and reducing costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA TELECOM CORP LTD
- Filing Date
- 2022-12-28
- Publication Date
- 2026-04-14
AI Technical Summary
Existing broadband satisfaction models have low prediction efficiency and cannot quickly respond to business needs.
By obtaining the configuration information table, feature variable processing is performed on the modeling data according to its settings, including outlier detection, missing value handling, discretization, encoding, and dimensionless processing. Then, model training and evaluation are performed, and finally, the target model is used for prediction.
It enables automatic model building and prediction, improves the efficiency of broadband satisfaction prediction, shortens the development cycle, and reduces labor input costs.
Smart Images

Figure CN116070095B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of information processing technology, and more specifically, to an information processing method, apparatus, electronic device, and computer-readable storage medium. Background Technology
[0002] With the gradual improvement of enterprise informatization, various industries have accumulated a large amount of business data. The development of information technology has gradually shifted from the previous focus on data accumulation to the extraction of data value. Data mining and data analysis have played a crucial role in this process, providing effective data support for operational strategies and becoming key to enhancing the core competitiveness of various industries.
[0003] Traditional data modeling development is time-consuming and requires significant manpower. A complete modeling process includes a series of steps such as business understanding, data preparation, data processing, model building, model evaluation, model deployment, and model optimization. With increasing market competition and a faster pace of development, business demands are becoming more numerous and frequent. Therefore, enabling technical personnel to respond quickly to business needs and complete model building in the shortest possible time has become a pressing issue. Taking a broadband satisfaction prediction scenario in a certain city within the telecom operator sector as an example, models built based on existing technologies have low efficiency in predicting broadband satisfaction.
[0004] There is currently no effective solution to the problem of low efficiency in predicting broadband satisfaction using models built in related technologies. Summary of the Invention
[0005] The main objective of this application is to provide an information processing method, apparatus, electronic device, and computer-readable storage medium to address the problem that models constructed in related technologies are inefficient in predicting broadband satisfaction.
[0006] To achieve the above objectives, according to one aspect of this application, an information processing method is provided. The method includes: obtaining a configuration information table, wherein the configuration information table includes at least a method for obtaining configuration of modeling data; obtaining modeling data for broadband satisfaction based on the configuration information table; performing feature variable processing on the modeling data according to the settings in the configuration information table to obtain processed modeling data; training a model based on the processed modeling data to obtain a target model; and using the target model to predict broadband satisfaction to obtain a predicted satisfaction level.
[0007] Furthermore, after processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data, the method further includes: dividing the processed modeling data into test set data and training set data; training the model based on the training set data to obtain the target model; and evaluating the target model based on the training set data.
[0008] Further, the modeling data is processed according to the settings in the configuration information table to obtain processed modeling data, including: outlier detection of the modeling data according to the settings in the configuration information table to obtain detected modeling data, wherein the outlier detection method is at least one of the following: mean squared error, quantiles, or isolated forest; missing value processing of the detected modeling data to obtain first modeling data, wherein the missing value processing method is at least one of the following: mean, median, mode, preset value imputation, or model-based multiple imputation; discretization of the first modeling data to obtain second modeling data, wherein the discretization processing method is at least one of the following: unsupervised custom interval method, binary method, equal frequency method, equal interval method, or supervised binning method; encoding of the second modeling data to obtain third modeling data, wherein the encoding processing method is at least one of the following: one-hot encoding, frequency encoding, or WOE encoding; and dimensionless processing of the third modeling data to obtain processed modeling data, wherein the dimensionless processing method is at least one of the following: data scaling, data standardization, or data regularization.
[0009] Furthermore, after training the model based on the processed modeling data to obtain the target model, the method further includes: acquiring inference data for broadband satisfaction; processing the inference data for feature variables to obtain processed inference data; and performing inference analysis on the target model's files based on the processed inference data to obtain a first evaluation result for the target model.
[0010] Furthermore, the method further includes: acquiring inference data for broadband satisfaction; processing the inference data for feature variables to obtain processed inference data; and performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model.
[0011] Furthermore, after performing reasoning analysis on the file of the initial model based on the processed reasoning data to obtain a second evaluation result for the initial model, the method further includes: determining whether to update the file of the target model and the file of the initial model based on the first evaluation result and the second evaluation result.
[0012] Further, determining whether to update the target model file and the initial model file based on the first evaluation result and the second evaluation result includes: determining whether the model update condition is met based on the first evaluation result and the second evaluation result; if the model update condition is met, then the broadband satisfaction model is updated to the target model; if the model update condition is not met, then the broadband satisfaction model is kept as the initial model.
[0013] To achieve the above objectives, according to another aspect of this application, an information processing apparatus is provided. The apparatus includes: a first acquisition unit for acquiring a configuration information table, wherein the configuration information table includes at least a method for acquiring modeling data configuration; a second acquisition unit for acquiring modeling data for broadband satisfaction based on the configuration information table; a first processing unit for performing feature variable processing on the modeling data according to the settings in the configuration information table to obtain processed modeling data; a first training unit for training a model based on the processed modeling data to obtain a target model; and a first prediction unit for predicting broadband satisfaction using the target model to obtain the predicted satisfaction level.
[0014] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, including a memory and a processor, wherein the memory stores a program and the processor is configured to execute the program to implement the information processing method described in any of the preceding claims.
[0015] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, characterized in that the storage medium stores a program, wherein the program, when running, controls the device where the computer-readable storage medium is located to execute the information processing method described in any one of the above claims.
[0016] This application employs the following steps: obtaining a configuration information table, which includes at least the method for obtaining modeling data configuration; obtaining modeling data for broadband satisfaction based on the configuration information table; processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data; training the model based on the processed modeling data to obtain a target model; and using the target model to predict broadband satisfaction to obtain the predicted satisfaction level. This solves the problem of low efficiency in predicting broadband satisfaction in related technologies. By automatically constructing the model based on a pre-set configuration information table, and then predicting broadband satisfaction based on the constructed target model, the efficiency of broadband satisfaction prediction is improved. Attached Figure Description
[0017] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:
[0018] Figure 1 This is a flowchart of an information processing method provided according to an embodiment of this application;
[0019] Figure 2 This is a flowchart of an optional information processing method provided according to an embodiment of this application. Figure 1 ;
[0020] Figure 3 This is a flowchart of an optional information processing method provided according to an embodiment of this application. Figure 2 ;
[0021] Figure 4 This is a flowchart of an optional information processing method provided according to an embodiment of this application. Figure 3 ;
[0022] Figure 5 This is a schematic diagram of an information processing apparatus provided according to an embodiment of this application. Detailed Implementation
[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0024] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0026] It should be noted that all relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) involved in this invention are information and data authorized by the user or fully authorized by all parties. For example, this system has an interface with the relevant user or organization. Before obtaining relevant information, it needs to send an acquisition request to the aforementioned user or organization through the interface, and obtain the relevant information after receiving consent from the aforementioned user or organization.
[0027] The present invention will now be described in conjunction with preferred implementation steps. Figure 1 This is a flowchart of an information processing method provided according to an embodiment of this application, such as... Figure 1 As shown, the method includes the following steps:
[0028] Step S101: Obtain the configuration information table, wherein the configuration information table includes at least the method for obtaining the modeling data configuration.
[0029] The configuration information table mentioned above can include: specifying the modeling data source, configuring the modeling data source, and describing the modeling data acquisition. Specifically, regarding specifying the modeling data source, the data source can be in formats such as df (Python data format), csv, txt, xlsx, etc., or common databases such as Hive, MySQL, Oracle, GBase, PostgreSQL, etc. In the broadband satisfaction prediction scenario, the data source can be Oracle. Regarding the modeling data source configuration, if the data source is a file, the data source configuration is a file reading configuration. If the data source is a database, the data source configuration is a database connection configuration. In the broadband satisfaction prediction scenario, the configuration information can be the Oracle connection configuration for the specific site. Regarding the description of modeling data acquisition, if the data source is a file, the data acquisition description is the absolute path of the data file. If the data source is a database, the data acquisition description is the complete data reading SQL command. In the broadband satisfaction prediction scenario, the configuration information in the configuration information table can be:
[0030]
[0031]
[0032] Step S102: Obtain modeling data for broadband satisfaction based on the configuration information table.
[0033] Obtaining modeling data for broadband satisfaction can be understood as preparing modeling samples. The configuration information table mentioned above also specifies the preparation process for modeling samples, including, for example, the ratio of positive to negative samples and the sampling method. The ratio of positive to negative samples can be common, such as 1:1 or 1:3. The sampling method can be chosen based on whether the positive and negative samples are balanced, such as random sampling, downsampling, or oversampling. Through the above sampling methods, accurate modeling data for broadband satisfaction can be obtained.
[0034] Step S102: Perform feature variable processing on the modeling data according to the settings in the configuration information table to obtain the processed modeling data.
[0035] Optionally, in the information processing method provided in this application embodiment, processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data includes: performing outlier detection on the modeling data according to the settings in the configuration information table to obtain detected modeling data, wherein the outlier detection method is at least one of the following: mean squared error, quantiles, or isolated forest; and performing missing value processing on the detected modeling data to obtain first modeling data, wherein the missing value processing method is at least one of the following: mean, median, mode, imputation of preset values, or model-based multiple imputation. Supplement; Discretize the first modeling data to obtain the second modeling data, wherein the discretization method is at least one of the following: unsupervised custom interval method, binary method, equal frequency method, equal interval method, and supervised binning method; Encode the second modeling data to obtain the third modeling data, wherein the encoding method is at least one of the following: one-hot encoding, frequency encoding, and WOE encoding; Perform dimensionless processing on the third modeling data to obtain the processed modeling data, wherein the dimensionless processing method is at least one of the following: data scaling, data standardization, and data regularization.
[0036] In this application, the configuration information table mentioned above defines the processing content for feature variables. This includes variable type description, whether a variable is included in the model, variable anomaly detection, variable missing value imputation, variable discretization, variable encoding, and dimensionless variable processing. Variable anomaly detection supports methods such as mean squared error, quantiles, and isolated forests. Variable missing value imputation supports single imputation methods such as mean, median, and mode, as well as model-based multiple imputation and imputation using default values combined with business experience. Variable discretization supports unsupervised custom interval methods, binary methods, equal-frequency methods, and equal-interval methods, as well as supervised methods such as Chi2 and Woe binning. Variable encoding supports commonly used methods such as one-hot encoding, frequency encoding, and Woe encoding. Dimensionless variable processing includes data scaling, data standardization, and data regularization.
[0037] After obtaining the processed modeling data, feature filtering can be performed on it. The configuration information table mentioned above specifies the feature filtering methods. The configuration information table can include methods such as Filter, Wrapper, and Embedded, which combine divergence or correlation.
[0038] Step S103: Train the model based on the processed modeling data to obtain the target model.
[0039] When training a model on the processed modeling data, the selection of a training algorithm is involved. The configuration information table mentioned above includes the algorithms used to train the model. For example, it includes the algorithm type and algorithm name. Algorithm types include classification, regression, and clustering. Common classification algorithms include: LR, SVM, decision tree, random forest, GBDT, XGB, LGB, etc. Common regression algorithms include: linear regression, GBDT, XGB, LGB, etc. Common clustering algorithms include K-means, DBSCAN, Hcluster, etc.
[0040] In addition, the configuration information table above specifies the hyperparameter search method. Common hyperparameter search methods include: random search, grid search, Bayesian search, and genetic algorithm search.
[0041] Step S103: Use the target model to predict broadband satisfaction and obtain the predicted satisfaction level.
[0042] The information processing method provided in this application involves obtaining a configuration information table, which includes at least the method for obtaining modeling data configuration; obtaining modeling data for broadband satisfaction based on the configuration information table; processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data; training a model based on the processed modeling data to obtain a target model; and using the target model to predict broadband satisfaction to obtain the predicted satisfaction level. This method solves the problem of low efficiency in predicting broadband satisfaction using models constructed in related technologies. By automatically constructing the model based on a pre-set configuration information table, and then predicting broadband satisfaction based on the constructed target model, the method improves the efficiency of broadband satisfaction prediction.
[0043] Optionally, in the information processing method provided in the embodiments of this application, after processing the modeling data for feature variables according to the settings in the configuration information table to obtain the processed modeling data, the method further includes: dividing the processed modeling data into test set data and training set data; training the model according to the training set data to obtain the target model; and evaluating the target model according to the training set data.
[0044] In the above scheme, the training set data and test set data are split, and the ratio of the training set data to the test set data can be specified in the configuration information table mentioned above. For example, the ratio of training set data to test set data can be 8:2, or 7:3, etc. Through this scheme, the model is trained using the training set data to obtain the target model, and the target model is evaluated using the training set data, ensuring the accuracy of the trained target model.
[0045] In this application, the configuration information table above also specifies the model evaluation method. For classification scenarios, common evaluation methods include: confusion matrix, accuracy, precision, recall, F1 score, AUC, KS score, etc. For regression scenarios, common evaluation methods include: MSE, RMSE, R2, etc. For clustering scenarios, common evaluation methods include: silhouette coefficient, CH score, DBI, mutual information, etc. Additionally, the save path for the target model file after training can be either a pkl or bin file.
[0046] Optionally, in the information processing method provided in the embodiments of this application, after training the model based on the processed modeling data to obtain the target model, the method further includes: obtaining inference data for broadband satisfaction; processing the inference data for feature variables to obtain processed inference data; and performing inference analysis on the target model's files based on the processed inference data to obtain a first evaluation result for the target model.
[0047] In the above scheme, the inference methods for the model may include inference data acquisition, feature variable processing, feature variable selection, model file specification, inference root cause analysis, inference performance evaluation, and inference result storage. All of these are configured using common files (such as xlsx, txt, json, etc.). The automated inference tool will read the configuration file and perform automatic inference based on the configuration information.
[0048] Regarding the acquisition of inference data, the configuration information table above specifies the methods for acquiring inference data. This can include specifying the data source, configuring the data source, and providing data acquisition instructions. The data source can be in formats such as CSV or TXT, or a database such as Hive or MySQL. The data source configuration can be either a file reading configuration or a database connection configuration. The data acquisition instructions can be the absolute path to the data file or a complete SQL command for reading the data.
[0049] Regarding the handling of feature variables, the handling of feature variables for inference sample data will be consistent with that for modeling sample data. Regarding the selection of feature variables, the selection of feature variables for inference sample data will be consistent with that for modeling sample data, and will not be elaborated further here.
[0050] Regarding the specification of model files, the configuration information table above specifies the storage path of model files used during model inference, for example, mainly pkl or bin files.
[0051] Regarding root cause analysis of inference, the configuration information table above specifies whether to perform root cause analysis on the inference results, that is, to output one or more core feature variables that determine the inference results.
[0052] Regarding the evaluation of inference performance, the configuration information table above specifies the model evaluation methods. For classification scenarios, common evaluation methods include confusion matrix, accuracy, precision, recall, F1 score, AUC, KS score, etc. For regression scenarios, common evaluation methods include MSE, RMSE, R2, etc. For clustering scenarios, common evaluation methods include silhouette coefficient, CH score, DBI, mutual information, etc.
[0053] Regarding the storage of inference results, specifically the storage of the first evaluation results mentioned above, the configuration information table specifies the storage method for the inference result data. This includes specifying the data source, configuring the data source, and specifying the data acquisition instructions. The data source can be in formats such as CSV or TXT, or a database such as Hive or MySQL. The data acquisition instructions can be the absolute path to the data file, the database table name, or the HDFS storage path.
[0054] Optionally, in the information processing method provided in the embodiments of this application, the method further includes: acquiring inference data for broadband satisfaction; performing feature variable processing on the inference data to obtain processed inference data; and performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model.
[0055] In the above scheme, the reasoning method for the initial model, i.e. the old model, is the same as the reasoning method for the target model, and will not be repeated here.
[0056] Optionally, in the information processing method provided in this application embodiment, after performing reasoning analysis on the file of the initial model based on the processed reasoning data to obtain a second evaluation result of the initial model, the method further includes: determining whether to update the file of the target model and the file of the initial model based on the first evaluation result and the second evaluation result.
[0057] By using the above scheme, based on the first and second evaluation results, it is possible to accurately determine whether to update the target model file and the initial model file, thereby ensuring the accuracy of the model file update.
[0058] Optionally, in the information processing method provided in this application embodiment, determining whether to update the target model file and the initial model file based on the first evaluation result and the second evaluation result includes: determining whether the model update condition is met based on the first evaluation result and the second evaluation result; if the model update condition is met, then the model for broadband satisfaction is updated to the target model; if the model update condition is not met, then the model for broadband satisfaction is kept as the initial model.
[0059] The configuration information table above defines the model update conditions. When the model evaluation metrics meet the conditions, automatic iterative optimization of the model can be achieved. These model update conditions can be fixed thresholds, percentages compared to the old model, improvements compared to the old model, etc.
[0060] In other words, this application provides an automatic model optimization method, which includes: training data acquisition, automatic model construction, validation data acquisition, automatic inference based on new model files, automatic inference based on old model files, and model update conditions. All of the above are configured using common files (such as xlsx, txt, json, etc.). The automated optimization tool will read the configuration file and automatically optimize the model based on the configuration information.
[0061] The configuration information table above specifies the methods for acquiring training data, used for the automatic training of the new model (target model). The method for acquiring training data can include specifying the data source, configuring the data source, and providing data acquisition instructions. The data source can be in formats such as CSV or TXT, or a database such as Hive or MySQL. The data source configuration can be a file reading configuration or a database connection configuration. The data acquisition instructions can be the absolute path to the data file or a complete SQL command for reading the data. The model is automatically built, outputting a new model file.
[0062] The configuration information table above specifies the method for obtaining validation data, used for automatic inference between the old and new models. This includes specifying the data source, configuring the data source, and providing data acquisition instructions. The data source can be in formats such as CSV or TXT, or a database such as Hive or MySQL. The data source configuration can be either a file reading configuration or a database connection configuration. The data acquisition instructions can be the absolute path to the data file or a complete SQL command for reading the data.
[0063] Automatic inference based on the new model file and output of the new model's performance.
[0064] Automatic inference is performed based on the model (initial model) from the old model file, and the old model's performance is output.
[0065] The configuration information table above defines the conditions for model optimization and updating. When the model evaluation index meets the conditions for model optimization and updating, the model can be automatically iteratively optimized.
[0066] Compared with the prior art, the solution in the embodiments of this application has the following beneficial effects:
[0067] In contrast to existing technologies that emphasize platforms and systems (such as drag-and-drop modeling tools), which include numerous functions and are technically complex (involving at least Java / SQL / Python, and even other big data components) and require a complete system, this invention emphasizes "lightweight modeling." It offers a small tool with significant capabilities, achieving automated modeling using only Python source code and configuration files. The tool is lightweight, highly portable, has low maintenance costs, and focuses on a single functionality.
[0068] This invention emphasizes its accessibility to business personnel in traditional industries, requiring no prior knowledge or expertise. However, it is not only suitable for those without prior experience but also for those with some understanding or extensive experience in data mining, aiming to improve development efficiency.
[0069] In contrast to existing automated modeling frameworks (such as AutoML), whose internal processing mechanisms allow users to intervene only to a limited extent based on open parameters, or even not at all, this invention provides professional customization for the entire modeling process. Users with some understanding or knowledge of data anomalies, missing data, algorithm usage, and algorithm hyperparameters can directly adjust configurations such as feature processing, algorithm selection, and hyperparameter search.
[0070] Existing technologies emphasize the automation of certain stages of the modeling process, such as automated data reading, automated feature selection, automated model selection, and automated model parameter tuning. This invention covers all automatable stages and provides three core functions: complete automated modeling, automated inference, and model self-optimization. It covers the vast majority of data mining work across most industries, significantly reducing manual input costs, shortening development cycles, and improving productivity.
[0071] The automatic reasoning module in this invention not only provides automatic reasoning capabilities but also performs root cause analysis, outputting one or more core characteristic variables that determine the reasoning result. For example, if a user is predicted to have a 97% chance of churning next month, the main reasons might be zero usage of voice / data / SMS this month, or more than five escalated complaints. Root cause analysis helps business personnel understand the model's output, enabling them to implement targeted operational strategies.
[0072] The automatic inference module in this invention can perform inference tasks periodically by setting scheduled tasks, which is essential for most business needs. For example, in the field of telecom operators, the identification model for potential home broadband users typically needs to perform inference based on the previous month's data at the beginning of each month to predict the potential home broadband users for the current month, supporting the business side to make regular outbound marketing calls.
[0073] This invention presents an automatic model optimization module that addresses the issues of insufficient model timeliness and reduced model effectiveness resulting from changes in business processes and data evolution. In use, model update conditions can be configured. When the evaluation metrics of a new model meet the update conditions, the model optimization function is triggered, enabling automatic iterative optimization of the model. This ensures continuous model effectiveness without manual intervention.
[0074] Specifically, taking a broadband satisfaction prediction scenario in a certain city in the operator field as an example, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the implementation of the present invention is not limited thereto.
[0075] Broadband satisfaction prediction is a conventional binary classification model. Model training requires a series of processes, including data reading, data cleaning, feature selection, model selection, and hyperparameter selection.
[0076] like Figure 2 As shown, the automatic model construction method in this invention includes the following steps:
[0077] Step S11: Modeling data acquisition, configuring the method for acquiring modeling data. Configuration includes: specifying the modeling data source, configuring the modeling data source, and instructions for acquiring modeling data.
[0078] Step S111: Specify the data source for modeling. The data source can be in formats such as df (Python data format), csv, txt, xlsx, etc., or common databases such as Hive, MySQL, Oracle, GBase, PostgreSQL, etc. In the example broadband satisfaction prediction scenario, the data source is Oracle.
[0079] Step S112: Modeling data source configuration. If the data source is a file, the data source is configured as a file read configuration. If the data source is a database, the data source is configured as a database connection configuration. In the example broadband satisfaction prediction scenario, the configuration information is the on-site Oracle connection configuration.
[0080] Step S113: Data Acquisition Instructions for Modeling. If the data source is a file, the data acquisition instructions are the absolute path to the data file. If the data source is a database, the data acquisition instructions are the complete SQL commands for reading the data. In the example broadband satisfaction prediction scenario, the configuration information is as follows:
[0081]
[0082] Step S12: Modeling Sample Preparation. Configure the preparation method for modeling samples. Configuration includes: positive and negative sample ratio and sample sampling method.
[0083] Step S121: Model the positive and negative sample ratio. If the requirement scenario is binary classification, configure the positive and negative sample ratio, commonly from 1:1 to 1:10. Optionally, this configuration has a default value. In the example broadband satisfaction prediction scenario, the modeled positive and negative sample ratio is 1:5.
[0084] Step S122: Modeling Sample Sampling Method. If the scenario involves binary classification and the positive and negative samples are relatively balanced, common random sampling methods in data mining are supported here, including random sampling and stratified sampling. If the scenario involves binary classification and the positive and negative samples are relatively unbalanced, common non-random sampling methods in data mining are supported here, including downsampling, oversampling, and downsampling + oversampling. Optionally, this configuration has a default value. In the example broadband satisfaction prediction scenario, positive samples are comprehensively sampled, i.e., downsampling + oversampling, to balance the ratio of positive to negative samples.
[0085] Step S13: Feature variable processing, configuring the processing of feature variables. Configuration includes: variable type description, whether a variable is included in the model, variable anomaly detection, variable missing value imputation, variable discretization processing, variable encoding processing, and dimensionless variable processing.
[0086] Step S131: Variable type description. If it is a character type, then it is string. If it is a numeric type, then it is number.
[0087] Step S132: Check if the variable is included in the model. If not included, the value is 0. If included, the value is 1. If it is a modeling label, the value is 2.
[0088] Step S133: Variable anomaly detection. This step supports all common outlier detection methods in the data mining field, including mean squared error, quantiles, and isolated forests. Optionally, this configuration has default values.
[0089] Step S134: Missing value imputation. This supports all common missing value imputation methods in the data mining field, including single imputation using the mean, median, mode, and default values based on business experience, as well as model-based multiple imputation. Optionally, this configuration has default values set.
[0090] Step S135: Variable Discretization. This step supports all common variable discretization methods in the data mining field, including unsupervised custom interval methods, binary methods, equal-frequency methods, equal-interval methods, and supervised methods such as Chi2 and Woe binning. Optionally, this configuration has default values.
[0091] Step S136: Variable encoding processing. This supports all common variable encoding methods in the data mining field, including one-hot encoding, frequency encoding, WOE encoding, etc. Optionally, this configuration has default values.
[0092] Step S137: Dimensionless variable processing. This supports all common dimensionless variable methods in the data mining field, including data scaling, data standardization, and data regularization. Optionally, this configuration has default values.
[0093] The configuration information for step S13 is as follows. Taking the broadband satisfaction prediction scenario as an example, the configuration information table is shown in Table 1 below.
[0094] Table 1
[0095]
[0096]
[0097] Step S14: Feature Variable Selection. Configure the feature variable selection method. This supports all common variable selection methods in the data mining field, including filtering methods (such as those based on variable variance or correlation with the target variable), wrapping methods (such as forward selection, backward shaving, and iterative shaving), and embedding methods (such as those based on tree models). Optionally, this configuration has default values. In the example broadband satisfaction prediction scenario, the random forest ensemble tree algorithm is selected for variable selection.
[0098] Step S15: Split the training / test set and configure the ratio of training to test sets. Common ratios are 8:2, 7:3, etc. Optionally, this configuration has a default value. In the example broadband satisfaction prediction scenario, the training and test sets are split in a 7:3 ratio.
[0099] Step S16: Algorithm selection, configuring the selected algorithm. Configuration includes: algorithm type and algorithm name.
[0100] Step S161: Algorithm Type. Common algorithm types in the data mining field are supported here, including classification, regression, clustering, and dimensionality reduction. In the example broadband satisfaction prediction scenario, the algorithm type is classification.
[0101] Step S162: Algorithm Name. If the algorithm type is classification, common classification algorithms in the data mining field are supported here, including LR, SVM, decision trees, random forests, GBDT, XGB, LGB, etc. If the algorithm type is regression, common regression algorithms in the data mining field are supported here, including linear regression, GBDT, XGB, LGB, etc. If the algorithm type is clustering, common clustering algorithms in the data mining field are supported here, including K-means, DBSCAN, Hcluster, etc. If the algorithm type is dimensionality reduction, common dimensionality reduction algorithms in the data mining field are supported here, including PCA, etc. In the example broadband satisfaction prediction scenario, the algorithm name is XGB.
[0102] Step S17: Algorithm Hyperparameter Search, configuring the hyperparameter search method. Common hyperparameter search methods in the data mining field are supported here, including random search, grid search, Bayesian search, and genetic algorithm search. In the example broadband satisfaction prediction scenario, the algorithm hyperparameter search method is Bayesian search.
[0103] Step S18: Model Performance Evaluation. Configure the model evaluation method. If the algorithm type is classification, common evaluation methods for classification algorithms in the data mining field are supported here, including confusion matrix, accuracy, precision, recall, F1 score, AUC, KS score, etc. If the algorithm type is regression, common evaluation methods for regression algorithms in the data mining field are supported here, including MSE, RMSE, R2, etc. If the algorithm type is clustering, common evaluation methods for clustering algorithms in the data mining field are supported here, including silhouette coefficient, CH score, DBI, mutual information, etc. In the example broadband satisfaction prediction scenario, multiple evaluation indicators such as accuracy, precision, recall, and F1 score are used for comprehensive evaluation.
[0104] Step S19: Save the model file and configure the model file save path. Common model saving methods in the data mining field are supported here, including pkl, bin, etc. In the example broadband satisfaction prediction scenario, the model file save path is . / model / bb_satisfy_xgb_model.pkl
[0105] The configuration information for steps S11-S19 (except for S13) is as follows, taking the broadband satisfaction prediction scenario as an example, as shown in Table 2 below.
[0106] Table 2
[0107]
[0108]
[0109] At this point, the training of the broadband satisfaction prediction model has been completed, and the model can then be used for inference.
[0110] like Figure 3 As shown, the automatic model reasoning method in this application embodiment includes the following steps:
[0111] Step S21: Inference Data Acquisition. Configure the method for acquiring inference data. Configuration includes: specifying the data source, configuring the data source, and specifying the data acquisition method.
[0112] Step S211: Read the specified inference data source. The data source can be in formats such as df (Python data format), csv, txt, xlsx, etc., or common databases such as Hive, MySQL, Oracle, GBase, PostgreSQL, etc. In the example broadband satisfaction prediction scenario, the data source is also Oracle.
[0113] Step S212: Read the inference data source configuration. If the data source is a file, the data source configuration is a file reading configuration. If the data source is a database, the data source configuration is a database connection configuration. In the example broadband satisfaction prediction scenario, the configuration information is the on-site Oracle connection configuration.
[0114] Step S213: Inference Data Acquisition Instructions. If the data source is a file, the data acquisition instructions are the absolute path of the data file. If the data source is a database, the data acquisition instructions are the complete SQL command for reading the data. In the example broadband satisfaction prediction scenario, the configuration information is "select t1.*from predict_table t1".
[0115] Step S22: Feature Variable Processing. The feature variable processing for the inference sample data will be consistent with that for the modeling sample data.
[0116] Step S23: Feature Variable Selection. The feature variable selection for the inference sample data will be consistent with the feature variable selection for the modeling sample data.
[0117] Step S24: Model file specification. Configure the storage path for the model file used during model inference. You need to specify the model file already generated by the automatic modeling module. In the example broadband satisfaction prediction scenario, the model file used for inference is . / model / bb_satisfy_xgb_model.pkl
[0118] Step S25: Root cause analysis, configuring whether to output one or more core feature variables that determine the inference result. In the example broadband satisfaction prediction scenario, the number of variables used in root cause analysis is 10.
[0119] Step S26: Evaluate the inference performance and configure the model evaluation method. If the algorithm type is classification, common classification algorithm evaluation methods in the data mining field are supported here, including confusion matrix, accuracy, precision, recall, F1 score, AUC, KS score, etc. If the algorithm type is regression, common regression algorithm evaluation methods in the data mining field are supported here, including MSE, RMSE, R2, etc. If the algorithm type is clustering, common clustering algorithm evaluation methods in the data mining field are supported here, including silhouette coefficient, CH score, DBI, mutual information, etc. In the example broadband satisfaction prediction scenario, multiple indicators such as accuracy, precision, recall, and F1 score are used to evaluate the performance of the inference data.
[0120] Step S27: Inference data storage, configure the storage method for inference data. Configuration includes: specifying the storage data source, configuring the storage data source, and describing the data storage.
[0121] Step S271: Specify the data source for storing inference data. The data source can be in formats such as df (Python data format), csv, txt, xlsx, etc., or common databases such as Hive, MySQL, Oracle, GBase, PostgreSQL, etc. In the example broadband satisfaction prediction scenario, the inference data is stored in a local csv file.
[0122] Step S272: Configure the storage inference data source. If the data source is a file, configure it as a file read device. If the data source is a database, configure it as a database connection device. In the example broadband satisfaction prediction scenario, the configuration information is sep='|', encoding='utf-8'.
[0123] Step S273: Infer the data storage description. If the data source is a file, the data storage description is the absolute path of the data file. If the data source is a database, the data storage description is the database table name.
[0124] In the example broadband satisfaction prediction scenario, the local storage path for the inference data is . / data / bb_satisfy_xgb_predict_result.csv.
[0125] The configuration information for steps S21-S27 is as follows, taking the broadband satisfaction prediction scenario as an example, as shown in Table 3 below.
[0126] Table 3
[0127]
[0128]
[0129] To ensure that the model performance remains at a relatively stable level, the self-optimization ability needs to be utilized to regularly evaluate the model performance and optimize the broadband satisfaction prediction model.
[0130] As Figure 4 shown, the model automatic optimization method in the embodiment of the present application includes the following steps:
[0131] Step S31: Training data acquisition, configuring the acquisition method of modeling data. The configuration content includes: Specifying the training data source, configuring the training data source, and instructions for training data acquisition.
[0132] Step S311: Specifying the training data source. The data source can be in formats such as df (python data format), csv, txt, xlsx, etc., or can be common databases such as hive, mysql, oracle, gbase, postgre, etc. In the exemplified broadband satisfaction prediction scenario, the data source is also oracle.
[0133] Step S312: Configuring the training data source. If the data source is a file, the data source configuration is the file reading configuration. If the data source is a database, the data source configuration is the database connection configuration. In the exemplified broadband satisfaction prediction scenario, the configuration information is the oracle connection configuration for this site.
[0134] Step S313: Instructions for training data acquisition. If the data source is a file, the data acquisition instruction is the absolute path of the data file. If the data source is a database, the data acquisition instruction is the complete data reading sql command. In the exemplified broadband satisfaction prediction scenario, the training data acquisition statement is:
[0135]
[0136] where t2.label is not null
[0137] and(x1!='21 years 10th issue' or (x1 = '21 years 10th issue' and tag in (0, 1, 2)))"
[0138] Step S32: Automatic model construction, repeating step S1. Among them, steps S11 - S19 in the automatic model construction process can be adjusted according to needs.
[0139] Step S33: Verification data acquisition, configuring the acquisition method of modeling data. The configuration content includes: Specifying the verification data source, configuring the verification data source, and instructions for verification data acquisition.
[0140] Step S331: Verify the data source specification. The data source can be in formats such as df (Python data format), csv, txt, xlsx, etc., or common databases such as hive, mysql, oracle, gbase, postgre, etc. In the broadband satisfaction prediction scenario example, the data source is also oracle.
[0141] Step S332: Verify the data source configuration. If the data source is a file, the data source configuration is the file reading configuration. If the data source is a database, the data source configuration is the database connection configuration. In the broadband satisfaction prediction scenario example, the configuration information is the oracle connection configuration for that site.
[0142] Step S333: Verify the data acquisition description. If the data source is a file, the data acquisition description is the absolute path of the data file. If the data source is a database, the data acquisition description is the complete data reading sql command. In the broadband satisfaction prediction scenario example, the verified data acquisition statement is: "select t2.*
[0143] from(select mod(t1.phonenum209,5)as tag
[0144] ,t1.*
[0145] ,case when X6>=0and X6<=7then'1'when X6>=9and X6<=10then'0'elseNULL end as label
[0146] from optimize_table t1)t2
[0147] where t2.label is not null
[0148] and(x1='21 years 10th issue'and tag not in(0,1,2))".
[0149] Step S34: Based on the model automatic reasoning of the new model file (i.e., the first module above), repeat Step S2 and output the new model effect. Among them, Steps S21 - S27 in the model automatic reasoning process can be adjusted according to needs.
[0150] Step S35: Based on the model automatic reasoning of the old model file (i.e., the first module above), repeat Step S2 and output the old model effect. Among them, Steps S21 - S27 in the model automatic reasoning process can be adjusted according to needs.
[0151] Step S36: Model Update Conditions. Configure model optimization and update conditions. When the model evaluation metrics meet the conditions, automatic iterative optimization of the model can be achieved. Update conditions can be fixed thresholds, percentages compared to the old model, improvement ratios compared to the old model, etc. In the example broadband satisfaction prediction scenario, the condition that the new model precision > the old model precision, and the new model recall >= the old model recall * 0.9 is used to determine whether the model can be updated.
[0152] The configuration information for steps S31-S36 is as follows, taking the broadband satisfaction prediction scenario as an example, as shown in Table 4 below.
[0153] Table 4
[0154]
[0155] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0156] This application also provides an information processing apparatus. It should be noted that the information processing apparatus of this application can be used to execute the information processing method provided in this application. The information processing apparatus provided in this application will be described below.
[0157] Figure 5 This is a schematic diagram of an information processing apparatus according to an embodiment of this application. Figure 5 As shown, the device includes: a first acquisition unit 501, a second acquisition unit 502, a first processing unit 503, a first training unit 504, and a first prediction unit 505.
[0158] Specifically, the first acquisition unit 501 is used to acquire a configuration information table, wherein the configuration information table includes at least the acquisition method for the modeling data configuration;
[0159] The second acquisition unit 502 is used to acquire modeling data for broadband satisfaction based on the configuration information table;
[0160] The first processing unit 503 is used to process the feature variables of the modeling data according to the settings in the configuration information table to obtain the processed modeling data.
[0161] The first training unit 504 is used to train the model based on the processed modeling data to obtain the target model.
[0162] The first prediction unit 505 is used to predict broadband satisfaction using the target model to obtain the predicted satisfaction level.
[0163] The information processing apparatus provided in this application embodiment acquires a configuration information table through a first acquisition unit 501, wherein the configuration information table includes at least the acquisition method for configuring modeling data; a second acquisition unit 502 acquires modeling data for broadband satisfaction based on the configuration information table; a first processing unit 503 processes the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data; a first training unit 504 trains the model based on the processed modeling data to obtain a target model; and a first prediction unit 505 uses the target model to predict broadband satisfaction to obtain the predicted satisfaction level. This solves the problem of low efficiency in predicting broadband satisfaction using models constructed in related technologies. It achieves automatic model construction based on a pre-set configuration information table, thereby predicting broadband satisfaction based on the constructed target model and obtaining the predicted satisfaction level, thus improving the efficiency of predicting broadband satisfaction.
[0164] Optionally, in the information processing apparatus provided in the embodiments of this application, the apparatus further includes: a second processing unit, used to process the modeling data for feature variables according to the settings in the configuration information table, and after obtaining the processed modeling data, divide the processed modeling data into test set data and training set data; a second training unit, used to train the model according to the training set data to obtain the target model; and a third processing unit, used to evaluate the target model according to the training set data.
[0165] Optionally, in the information processing apparatus provided in this application embodiment, the first processing unit 503 includes: a detection subunit, used to perform outlier detection on the modeling data according to the settings in the configuration information table, to obtain the detected modeling data, wherein the outlier detection method is at least one of the following: mean squared error, quantiles, and isolated forest; a first processing subunit, used to perform missing value processing on the detected modeling data, to obtain first modeling data, wherein the missing value processing method is at least one of the following: mean, median, mode, preset value imputation, and model-based multiple imputation; and a second processing subunit, used to process the first modeling data. The system performs discretization processing to obtain second modeling data, wherein the discretization processing method is at least one of the following: unsupervised custom interval method, binary method, equal frequency method, equal interval method, and supervised binning method; a third processing subunit is used to encode the second modeling data to obtain third modeling data, wherein the encoding processing method is at least one of the following: one-hot encoding, frequency encoding, and WOE encoding; a fourth processing subunit is used to perform dimensionless processing on the third modeling data to obtain processed modeling data, wherein the dimensionless processing method is at least one of the following: data scaling, data standardization, and data regularization.
[0166] Optionally, in the information processing apparatus provided in the embodiments of this application, the apparatus further includes: a third acquisition unit, used to train a model based on the processed modeling data, and after obtaining a target model, acquire inference data for broadband satisfaction; a fourth processing unit, used to process the inference data for feature variables, and obtain processed inference data; and a first analysis unit, used to perform inference analysis on the target model's file based on the processed inference data, and obtain a first evaluation result for the target model.
[0167] Optionally, in the information processing apparatus provided in the embodiments of this application, the apparatus further includes: a fourth acquisition unit, used to acquire inference data for broadband satisfaction; a fifth processing unit, used to process the inference data for feature variables to obtain processed inference data; and a second analysis unit, used to perform inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model.
[0168] Optionally, in the information processing apparatus provided in the embodiments of this application, the apparatus further includes: a determining unit, configured to, after performing reasoning analysis on the file of the initial model based on the processed reasoning data to obtain a second evaluation result of the initial model, determine whether to update the file of the target model and the file of the initial model based on the first evaluation result and the second evaluation result.
[0169] Optionally, in the information processing apparatus provided in the embodiments of this application, the determining unit includes: a first determining subunit, used to determine whether the model update condition is met based on the first evaluation result and the second evaluation result; a fifth processing subunit, used to update the model of broadband satisfaction to the target model if the model update condition is met; and a sixth processing subunit, used to keep the model of broadband satisfaction as the initial model if the model update condition is not met.
[0170] The information processing device includes a processor and a memory. The first acquisition unit 501, the second acquisition unit 502, the first processing unit 503, the first training unit 504, and the first prediction unit 505 are all stored in the memory as program units. The processor executes the program units stored in the memory to realize the corresponding functions.
[0171] The processor contains a kernel, which retrieves the corresponding program units from memory. One or more kernels can be configured, and adjusting kernel parameters can improve the efficiency of predicting bandwidth satisfaction.
[0172] The memory may include non-permanent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.
[0173] This invention provides a storage medium storing a program that, when executed by a processor, implements the information processing method.
[0174] This invention provides a processor for running a program, wherein the program executes the information processing method during runtime.
[0175] This invention provides a device including a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, it performs the following steps: obtaining a configuration information table, wherein the configuration information table includes at least a method for obtaining configuration of modeling data; obtaining modeling data for broadband satisfaction based on the configuration information table; performing feature variable processing on the modeling data according to the settings in the configuration information table to obtain processed modeling data; training a model based on the processed modeling data to obtain a target model; and using the target model to predict broadband satisfaction to obtain the predicted satisfaction level.
[0176] When the processor executes the program, it performs the following steps: after processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data, the method further includes: dividing the processed modeling data into test set data and training set data; training the model based on the training set data to obtain the target model; and evaluating the target model based on the training set data.
[0177] When the processor executes the program, it performs the following steps: Processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data includes: performing outlier detection on the modeling data according to the settings in the configuration information table to obtain detected modeling data, wherein the outlier detection method is at least one of the following: mean squared error, quantiles, or isolated forest; and performing missing value processing on the detected modeling data to obtain first modeling data, wherein the missing value processing method is at least one of the following: mean, median, mode, preset value imputation, or model-based multiple imputation. The first modeling data is discretized to obtain the second modeling data, wherein the discretization method is at least one of the following: unsupervised custom interval method, binary method, equal frequency method, equal interval method, and supervised binning method; the second modeling data is encoded to obtain the third modeling data, wherein the encoding method is at least one of the following: one-hot encoding, frequency encoding, and WOE encoding; the third modeling data is dimensionless to obtain the processed modeling data, wherein the dimensionless processing method is at least one of the following: data scaling, data standardization, and data regularization.
[0178] When the processor executes the program, it performs the following steps: after training the model based on the processed modeling data to obtain the target model, the method further includes: acquiring inference data for broadband satisfaction; processing the inference data for feature variables to obtain processed inference data; and performing inference analysis on the target model's file based on the processed inference data to obtain a first evaluation result for the target model.
[0179] When the processor executes the program, it performs the following steps: acquiring inference data for broadband satisfaction; processing the inference data for feature variables to obtain processed inference data; and performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model.
[0180] When the processor executes the program, it performs the following steps: after performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model, the method further includes: determining whether to update the file of the target model and the file of the initial model based on the first evaluation result and the second evaluation result.
[0181] When the processor executes the program, it performs the following steps: determining whether to update the target model file and the initial model file based on the first evaluation result and the second evaluation result includes: determining whether the model update condition is met based on the first evaluation result and the second evaluation result; if the model update condition is met, then the broadband satisfaction model is updated to the target model; if the model update condition is not met, then the broadband satisfaction model is kept as the initial model. The device in this document can be a server, PC, PAD, mobile phone, etc.
[0182] This application also provides a computer program product, which, when executed on a data processing device, is suitable for executing an initialization program having the following method steps: obtaining a configuration information table, wherein the configuration information table includes at least a method for obtaining configuration of modeling data; obtaining modeling data for broadband satisfaction based on the configuration information table; performing feature variable processing on the modeling data according to the settings in the configuration information table to obtain processed modeling data; training a model based on the processed modeling data to obtain a target model; and using the target model to predict broadband satisfaction to obtain the predicted satisfaction level.
[0183] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: after processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data, the method further includes: dividing the processed modeling data into test set data and training set data; training the model according to the training set data to obtain the target model; and evaluating the target model according to the training set data.
[0184] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: processing the modeling data for feature variables according to the settings in the configuration information table to obtain processed modeling data, including: performing outlier detection on the modeling data according to the settings in the configuration information table to obtain detected modeling data, wherein the outlier detection method is at least one of the following: mean squared error, quantiles, or isolated forest; and performing missing value processing on the detected modeling data to obtain first modeling data, wherein the missing value processing method is at least one of the following: mean, median, mode, imputation of preset values, or base case. For multiple model imputation; the first modeling data is discretized to obtain the second modeling data, wherein the discretization method is at least one of the following: unsupervised custom interval method, binary method, equal frequency method, equal interval method, and supervised binning method; the second modeling data is encoded to obtain the third modeling data, wherein the encoding method is at least one of the following: one-hot encoding, frequency encoding, and WOE encoding; the third modeling data is dimensionless to obtain the processed modeling data, wherein the dimensionless processing method is at least one of the following: data scaling, data standardization, and data regularization.
[0185] When executed on a data processing device, it is also suitable to execute an initialization program with the following method steps: after training a model based on the processed modeling data to obtain a target model, the method further includes: acquiring inference data for broadband satisfaction; performing feature variable processing on the inference data to obtain processed inference data; and performing inference analysis on the target model file based on the processed inference data to obtain a first evaluation result of the target model.
[0186] When executed on a data processing device, it is also suitable to execute an initialization program with the following steps: acquiring inference data for broadband satisfaction; performing feature variable processing on the inference data to obtain processed inference data; and performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model.
[0187] When executed on a data processing device, the method is also adapted to execute an initialization procedure having the following steps: after performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model, the method further includes: determining whether to update the file of the target model and the file of the initial model based on the first evaluation result and the second evaluation result.
[0188] When executed on a data processing device, it is also suitable to execute an initialization program with the following method steps: determining whether to update the target model file and the initial model file based on the first evaluation result and the second evaluation result, including: determining whether the model update condition is met based on the first evaluation result and the second evaluation result; if the model update condition is met, updating the broadband satisfaction model to the target model; if the model update condition is not met, keeping the broadband satisfaction model as the initial model.
[0189] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0190] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0191] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0192] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0193] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0194] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0195] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0196] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0197] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0198] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.
Claims
1. An information processing method, characterized in that, include: Obtain a configuration information table, wherein the configuration information table includes at least the acquisition method of modeling data configuration, the modeling sample sampling method, the ratio of positive and negative modeling samples, the feature selection method, the model evaluation method, and the model optimization and update conditions; Based on the configuration information table, obtain modeling data for broadband satisfaction; Based on the settings in the configuration information table, the modeling data is processed using feature variables to obtain the processed modeling data; The target model is obtained by training the model based on the processed modeling data. The target model is used to predict broadband satisfaction, and the predicted satisfaction level is obtained. The method further includes, after training the model based on the processed modeling data to obtain the target model, the method also includes: Obtain inference data regarding broadband satisfaction; The inference data is processed by feature variable processing to obtain processed inference data; Based on the processed inference data, inference analysis is performed on the target model's file to obtain a first evaluation result for the target model; The method further includes: Obtain inference data regarding broadband satisfaction; The inference data is processed by feature variable processing to obtain processed inference data; Based on the processed inference data, inference analysis is performed on the file of the initial model to obtain a second evaluation result of the initial model; Based on the first evaluation result and the second evaluation result, determining whether to update the files of the target model and the initial model includes: Based on the first evaluation result and the second evaluation result, determine whether the model update conditions are met; If the model update conditions are met, the model for broadband satisfaction will be updated to the target model. If the model update conditions are not met, the model for broadband satisfaction will remain the initial model.
2. The method according to claim 1, characterized in that, After processing the modeling data for feature variables according to the settings in the configuration information table to obtain the processed modeling data, the method further includes: The processed modeling data is divided into test set data and training set data; The target model is obtained by training the model based on the training set data. The target model is evaluated based on the test set data.
3. The method according to claim 1, characterized in that, Based on the settings in the configuration information table, the modeling data is processed using feature variables to obtain the processed modeling data, which includes: Based on the settings in the configuration information table, outlier detection is performed on the modeling data to obtain the detected modeling data. The outlier detection method is at least one of the following: mean squared error, quantiles, or isolated forest. The detected modeling data is processed for missing values to obtain the first modeling data. The missing value processing method is at least one of the following: mean, median, mode, imputation of preset values, and model-based multiple imputation. The first modeling data is discretized to obtain the second modeling data. The discretization method is at least one of the following: unsupervised custom interval method, binary method, equal frequency method, equal interval method, and supervised binning method. The second modeling data is encoded to obtain the third modeling data, wherein the encoding method is at least one of the following: one-hot encoding, frequency encoding, or WOE encoding; The third modeling data is subjected to dimensionless processing to obtain the processed modeling data. The dimensionless processing method is at least one of the following: data scaling, data standardization, and data regularization.
4. An information processing device, characterized in that, include: The first acquisition unit is used to acquire a configuration information table, wherein the configuration information table includes at least the acquisition method of the modeling data configuration, the modeling sample sampling method, the ratio of positive and negative modeling samples, the feature selection method, the model evaluation method, and the model optimization and update conditions. The second acquisition unit is used to acquire modeling data for broadband satisfaction based on the configuration information table; The first processing unit is used to process the modeling data according to the settings in the configuration information table to obtain the processed modeling data; The first training unit is used to train the model based on the processed modeling data to obtain the target model. The first prediction unit is used to predict broadband satisfaction using the target model to obtain the predicted satisfaction level. The device further includes: a third acquisition unit, used to train a model based on the processed modeling data, and after obtaining the target model, acquire inference data for broadband satisfaction; a fourth processing unit, used to process the inference data for feature variables, and obtain processed inference data; and a first analysis unit, used to perform inference analysis on the target model file based on the processed inference data, and obtain a first evaluation result of the target model. The device further includes: a fourth acquisition unit for acquiring inference data on broadband satisfaction; a fifth processing unit for performing feature variable processing on the inference data to obtain processed inference data; and a second analysis unit for performing inference analysis on the file of the initial model based on the processed inference data to obtain a second evaluation result of the initial model. The device further includes: a first determining subunit, configured to determine whether the model update conditions are met based on the first evaluation result and the second evaluation result; a fifth processing subunit, configured to update the model of broadband satisfaction to the target model if the model update conditions are met; and a sixth processing subunit, configured to keep the model of broadband satisfaction as the initial model if the model update conditions are not met.
5. An electronic device, characterized in that, It includes a memory and a processor, wherein the memory stores a program and the processor executes the program to implement the information processing method according to any one of claims 1 to 3.
6. A computer-readable storage medium, characterized in that, The storage medium stores a program, wherein when the program is executed, it controls the device containing the computer-readable storage medium to perform the information processing method according to any one of claims 1 to 3.
Citation Information
Patent Citations
Data processing method, device, terminal device and storage medium
CN108764273A
Evaluation model generation method and device, evaluation data processing method and device, equipment and medium
CN110956224A
Information determination method and device, equipment and storage medium
CN114139451A