Multi-source heterogeneous data-oriented monomerized model self-adaptive automatic modeling technology

Abstract

The invention discloses a multi-source heterogeneous data-oriented monomer model self-adaptive automatic modeling technology, relates to the technical field of data processing and machine learning, and solves the technical problem that it is difficult to combine the accuracy, the F1 score and the calculation cost and dynamically adjust an objective function through a weight coefficient to adapt to different business requirements. Comprising the steps of obtaining multi-source heterogeneous data; extracting features from the multi-source heterogeneous data to generate a preliminary feature set; missing values are processed, low-variance features are filtered, and a first feature set is obtained based on correlation filtering and feature removal; selecting a basic model for training, evaluating feature importance, and removing features of which the importance is lower than a threshold value to obtain a second feature set; defining a target function, selecting an optimal feature subset, training a model by using the optimal feature subset, and introducing an adaptive learning mechanism to dynamically adjust hyper-parameters; a modeling process is decomposed into a plurality of stages, and flow management and exception handling are set.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing and machine learning, and specifically relates to a monolithic model adaptive automatic modeling technology for multi-source heterogeneous data. BACKGROUND

[0002] With the rapid development of information technology, the sources and types of data are increasingly diverse, and multi-source heterogeneous data widely exist in various fields.

[0003] However, these data differ in format, quality, semantics, etc., bringing great challenges to data analysis and modeling. Traditional modeling methods often require manual data preprocessing, feature engineering and model tuning, which is not only inefficient but also easily affected by human factors, leading to unstable model performance.

[0004] Therefore, there is an urgent need for a technology that can automatically process multi-source heterogeneous data and build high-performance monolithic models. SUMMARY

[0005] The present application aims to at least solve one of the technical problems existing in the prior art; for this purpose, the present application proposes a monolithic model adaptive automatic modeling technology for multi-source heterogeneous data, which is used to solve the following technical problems:

[0006] Traditional modeling methods often require manual data preprocessing, feature engineering and model tuning, which is not only inefficient but also easily affected by human factors, leading to unstable model performance.

[0007] To solve the above problems, the present application provides a monolithic model adaptive automatic modeling technology for multi-source heterogeneous data, comprising:

[0008] S1: Obtain multi-source heterogeneous data and perform preprocessing operations thereon;

[0009] S2: Extract features from the multi-source heterogeneous data respectively to generate a preliminary feature set;

[0010] S3: Perform missing rate processing, variance filtering, correlation filtering and irrelevant feature removal on the preliminary feature set to obtain a first feature set;

[0011] S4: Select a base model, train the model using the first feature set and target variable, and perform feature importance evaluation to remove features with importance lower than a threshold to obtain a second feature set;

[0012] S5: Sort features by importance, gradually remove the least important features, evaluate the performance of different feature subsets through grid search, define the objective function based on the evaluation parameters and corresponding weights, select the feature subset with the best performance, train the model using the optimal feature subset, introduce an adaptive learning mechanism, and dynamically adjust the model's hyperparameters by monitoring the evaluation indicators of the model during training;

[0013] S6: Decompose the modeling process into multiple stages, set up process management and exception handling, and monitor the process status in real time;

[0014] Defining the objective function according to the evaluation parameters and corresponding weights includes:

[0015] Z=argmax(ω a ·a+ω f ·f-ω c c)

[0016] Where Z is the objective function, a, f, and c are accuracy, F1 score, and computational cost, respectively. a 、ω f and ω c are the corresponding weight coefficients respectively.

[0017] Preferably, the step S1 includes:

[0018] Comprehensively scan multi-source heterogeneous data to identify and process noise, outliers, and missing values ​​in the data;

[0019] Convert data from different sources into a unified, pre-set standard format;

[0020] Select the corresponding normalization method according to the characteristics and distribution of the data;

[0021] At the same time, data from different data sources are integrated by establishing data association rules and mapping relationships.

[0022] Preferably, the step S2 includes:

[0023] Extract features from multi-source heterogeneous data and generate a preliminary feature set;

[0024] The multi-source heterogeneous data includes structured data, text data, image data and time series data;

[0025] The first feature set includes statistical features, interactive features, polynomial features, TF-IDF, word vectors, topic models, deep learning features, manual features, rolling features, frequency domain features and autocorrelation coefficients.

[0026] Preferably, the step S3 includes:

[0027] Traverse all features, calculate the missing rate of each feature, remove features with missing rates higher than the preset threshold, and fill the remaining feature values ​​using the mean filling, median filling, or interpolation filling methods;

[0028] Calculate the variance of each feature and remove features whose variance is lower than a preset threshold;

[0029] Calculate the Pearson correlation coefficient between each feature and the target variable, remove features whose absolute value of the correlation coefficient is lower than the preset threshold, and calculate the correlation coefficient between features, remove features whose absolute value of the correlation coefficient is higher than the preset threshold, and retain one feature;

[0030] For classification problems, use the chi-square test or mutual information method to remove irrelevant features; for regression problems, use the F test or p-value method to remove irrelevant features;

[0031] The feature set obtained after a screening of the preliminary feature set is recorded as the first feature set.

[0032] Preferably, the step S4 includes:

[0033] Select one or more models from the linear model or the tree model as a base model, and use the first feature set and the target variable to train the base model to obtain a trained model;

[0034] For linear models, feature importance is evaluated by regularization weights; for tree models, feature importance is evaluated by the frequency of use of the feature in the split node;

[0035] According to the feature importance sorting, features with importance higher than the preset threshold are selected. For the Lasso model, features with a coefficient of 0 are directly removed.

[0036] The feature set after secondary screening is recorded as the second feature set.

[0037] Preferably, the step S5 includes:

[0038] Using the second feature set as initial input, select a base model, train the base model on the second feature set, and evaluate the importance of each feature;

[0039] According to the feature importance ranking, the least important features are gradually removed and the remaining features are retained;

[0040] Repeat the above process until the preset number of features is reached;

[0041] Use grid search to evaluate the performance of different feature subsets; define the objective function based on the evaluation parameters and corresponding weights to select the optimal feature subset;

[0042] Divide the dataset into training and test sets, use the optimal feature subset as input features, train the model, evaluate the model performance on the test set, and output the final results;

[0043] At the same time, during the model construction process, an adaptive learning mechanism is introduced to dynamically adjust the model's hyperparameters by monitoring the model's evaluation indicators during training.

[0044] Preferably, the objective function is defined according to the evaluation parameters and the corresponding weights, further comprising:

[0045] Consider multiple evaluation metrics simultaneously in the objective function, including accuracy, F1 score, and computational cost, and normalize the computational cost;

[0046] The dynamic adjustment of the model's hyperparameters includes treating weights as hyperparameters, automatically searching for the optimal weight combination through Bayesian optimization, and dynamically adjusting the weights based on real-time data feedback after the model is deployed.

[0047] Preferably, the optimal feature subset includes:

[0048] Generate multiple candidate feature subsets from the initial feature set, and calculate the accuracy, F1 score and computational cost for each candidate feature subset;

[0049] Calculate the score of each feature subset through the objective function;

[0050] From all candidate feature subsets, the feature subset that maximizes the objective function is selected as the optimal feature subset.

[0051] Preferably, the step S6 includes:

[0052] Decompose the entire modeling process into multiple stages and define clear inputs, outputs, and operations for each stage;

[0053] Define a process management system for each stage, including task scheduling, logging, and status tracking. Task scheduling is used to manage the execution order of the process, logging is used to record logs for each stage, and status tracking is used to record the execution status of each stage. If a stage fails, the subsequent stages will automatically stop.

[0054] Introducing an exception handling mechanism into the process to automatically identify and process abnormal data;

[0055] At the same time, monitoring tools are used to monitor the execution status of the process in real time, and a process management interface is provided to view process status, logs and error information.

[0056] Beneficial effects of the present invention:

[0057] The present invention extracts diversified features from structured, text, image and time series data, covering different dimensions of the data, and fuses them through multiple methods to effectively reduce redundant features, retain the features that contribute most to the target variable, use basic models to evaluate feature importance, and combine the advantages of linear models and tree models to ensure the scientificity and rationality of feature selection.

[0058] The present invention selects the feature subset with the best performance through grid search and objective function optimization, thereby improving the accuracy and efficiency of the model, dynamically adjusting the model hyperparameters, and continuously optimizing the model during training to adapt to changes in data distribution and improve generalization ability. Combining accuracy, F1 score and computational cost, the objective function is dynamically adjusted through weight coefficients to balance model performance and computing resources to meet different business needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 Schematic diagram of the method of the present invention. DETAILED DESCRIPTION

[0060] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0061] See also Figure 1 As shown, the present invention is a single model adaptive automatic modeling technology for multi-source heterogeneous data, including:

[0062] S1: Acquire multi-source heterogeneous data and perform preprocessing operations on it;

[0063] S2: Extract features from multi-source heterogeneous data and generate a preliminary feature set;

[0064] S3: The first feature set is obtained by performing missing rate processing, variance filtering, correlation filtering and irrelevant feature removal on the preliminary feature set;

[0065] S4: Select a basic model, train the model using the first feature set and the target variable, evaluate the feature importance, remove features whose importance is lower than the threshold, and obtain the second feature set;

[0066] S5: Sort features by importance, gradually remove the least important features, evaluate the performance of different feature subsets through grid search, define the objective function based on the evaluation parameters and corresponding weights, select the feature subset with the best performance, train the model using the optimal feature subset, introduce an adaptive learning mechanism, and dynamically adjust the model's hyperparameters by monitoring the evaluation indicators of the model during training;

[0067] S6: Decompose the modeling process into multiple stages, set up process management and exception handling, and monitor the process status in real time;

[0068] Defining the objective function according to the evaluation parameters and corresponding weights includes:

[0069] Z=argmax(ω a ·a+ω f ·f-ω c c)

[0070] Where Z is the objective function, a, f, and c are accuracy, F1 score, and computational cost, respectively. a 、ω f and ω c are the corresponding weight coefficients respectively.

[0071] In one embodiment of the present invention, step S1 includes:

[0072] Comprehensively scan multi-source heterogeneous data to identify and process noise, outliers, and missing values ​​in the data;

[0073] Convert data from different sources into a unified, pre-set standard format;

[0074] Select the corresponding normalization method according to the characteristics and distribution of the data;

[0075] At the same time, data from different data sources are integrated by establishing data association rules and mapping relationships.

[0076] Specifically, conduct a preliminary inspection of multi-source heterogeneous data to understand the data distribution, format, content and quality issues, and perform corresponding noise processing, outlier processing and missing value processing on numerical data, image data and text data respectively; determine the standard format according to the data type and task requirements, and define the field name, data type, unit and other specifications; use Pandas or SQL to export structured data into CSV or JSON format, convert images and text into standard format and JSON format, convert geospatial data into GeoTIFF or Shapefile format, and unify the projection coordinate system; select the appropriate normalization method according to data distribution and task requirements, apply the selected normalization method to numerical features, perform one-hot encoding on categorical features, and normalize image data; use GIS tools to overlay image, topographic map and other data, use Pandas' merge or join function to merge tables based on association keys, and integrate numerical, text, image and other data into a unified data structure.

[0077] In one embodiment of the present invention, step S2 includes:

[0078] Extract features from multi-source heterogeneous data and generate a preliminary feature set;

[0079] The multi-source heterogeneous data includes structured data, text data, image data and time series data;

[0080] The first feature set includes statistical features, interactive features, polynomial features, TF-IDF, word vectors, topic models, deep learning features, manual features, rolling features, frequency domain features and autocorrelation coefficients.

[0081] In one embodiment of the present invention, step S3 includes:

[0082] Traverse all features, calculate the missing rate of each feature, remove features with missing rates higher than the preset threshold, and fill the remaining feature values ​​using the mean filling, median filling, or interpolation filling methods;

[0083] Calculate the variance of each feature and remove features whose variance is lower than a preset threshold;

[0084] Calculate the Pearson correlation coefficient between each feature and the target variable, remove features whose absolute value of the correlation coefficient is lower than the preset threshold, and calculate the correlation coefficient between features, remove features whose absolute value of the correlation coefficient is higher than the preset threshold, and retain one feature;

[0085] For classification problems, use the chi-square test or mutual information method to remove irrelevant features; for regression problems, use the F test or p-value method to remove irrelevant features;

[0086] The feature set obtained after a screening of the preliminary feature set is recorded as the first feature set.

[0087] Specifically, the missing rate is obtained by dividing the number of missing values ​​by the total number of samples, and a preset threshold is set, such as 0.3, to remove features with a missing rate higher than the threshold; for numerical data, the mean of the feature is used to replace the missing value, and for data that is insensitive to outliers, the median of the feature is used to replace the missing value, and for time series data, the previous value, the next value or the moving average is used to fill the missing value; for classification problems, features irrelevant to the target variable are removed, including using the chi-square test to evaluate the correlation between classification features and target variables, using mutual information to evaluate the correlation between continuous features and target variables, setting a preset threshold, and removing irrelevant features; for regression problems, features irrelevant to the target variable are removed, including using the F test to evaluate the significance of features and target variables, using the p-value method to evaluate the correlation between features and target variables, setting a preset threshold, and removing irrelevant features; the above features that have undergone missing rate processing, variance filtering, correlation filtering, and irrelevant feature removal are combined into a first feature set.

[0088] In one embodiment of the present invention, step S4 includes:

[0089] Select one or more models from the linear model or the tree model as a base model, and use the first feature set and the target variable to train the base model to obtain a trained model;

[0090] For linear models, feature importance is evaluated by regularization weights; for tree models, feature importance is evaluated by the frequency of use of the feature in the split node;

[0091] According to the feature importance sorting, features with importance higher than the preset threshold are selected. For the Lasso model, features with a coefficient of 0 are directly removed.

[0092] The feature set after secondary screening is recorded as the second feature set.

[0093] Specifically, one or more models are selected as the basic model from the linear model or tree model. The linear model includes logistic regression, linear regression or Lasso regression; the tree model includes decision tree, random forest or gradient boosting tree; the first feature set and the target variable are divided into a training set and a validation set, the basic model is trained using the training set data, the hyperparameters are adjusted to optimize the performance, and the trained model is recorded for subsequent feature importance evaluation; for logistic regression or linear regression, the coefficients (weights) of the model are extracted, the larger the absolute value, the higher the feature importance, the coefficients are normalized to obtain the feature importance score; for Lasso regression, the coefficients of irrelevant features are automatically compressed to 0, and the features with coefficients of 0 are directly removed; for decision tree, random forest or gradient boosting Boosting tree uses the model's feature importance attribute to obtain the importance score of each feature. The importance score is usually based on the number of times the feature is used in the split node and its explanatory power for the target variable. The importance score is normalized to facilitate subsequent sorting and screening; all features are sorted from high to low according to the importance score. For linear models, the absolute value of the coefficient or the normalized score can be used directly; for tree models, the normalized importance score is used; a preset threshold is set to select features with an importance score higher than the threshold. For the Lasso model, features with a coefficient of 0 are directly removed without additional screening. For the tree model, the threshold can be adjusted according to business needs to retain the most important features; features with importance higher than the threshold are merged to form a feature set after secondary screening.

[0094] In one embodiment of the present invention, step S5 includes:

[0095] Using the second feature set as initial input, select a base model, train the base model on the second feature set, and evaluate the importance of each feature;

[0096] According to the feature importance ranking, the least important features are gradually removed and the remaining features are retained;

[0097] Repeat the above process until the preset number of features is reached;

[0098] Use grid search to evaluate the performance of different feature subsets; define the objective function based on the evaluation parameters and corresponding weights to select the optimal feature subset;

[0099] Divide the dataset into training and test sets, use the optimal feature subset as input features, train the model, evaluate the model performance on the test set, and output the final results;

[0100] At the same time, during the model construction process, an adaptive learning mechanism is introduced to dynamically adjust the model's hyperparameters by monitoring the model's evaluation indicators during training.

[0101] Specifically, a basic model is selected as the starting point for feature importance evaluation and optimization, the basic model is trained using the second feature set and the target variable, and feature importance is extracted, for example, the absolute value of the coefficient or the feature importance score, and the features are sorted from high to low in importance, and a preset feature quantity threshold is set as the termination condition; the least important features are removed from the current feature subset, the basic model is retrained on the remaining feature subsets, the performance of the new feature subset is evaluated, and the above process is repeated until the number of features reaches the preset threshold; the candidate range of the feature subset is defined, and for each feature subset, the model performance is evaluated using cross-validation, the performance indicators (evaluation indicators) of each feature subset are recorded, the objective function is defined, weights are assigned to different evaluation indicators, the objective function value of each feature subset is calculated, and the feature subset with the optimal value is selected.

[0102] In one embodiment of the present invention, defining the objective function according to the evaluation parameters and the corresponding weights further includes:

[0103] Consider multiple evaluation metrics simultaneously in the objective function, including accuracy, F1 score, and computational cost, and normalize the computational cost;

[0104] The dynamic adjustment of the model's hyperparameters includes treating weights as hyperparameters, automatically searching for the optimal weight combination through Bayesian optimization, and dynamically adjusting the weights based on real-time data feedback after the model is deployed.

[0105] In one embodiment of the present invention, the optimal feature subset includes:

[0106] Generate multiple candidate feature subsets from the initial feature set, and calculate the accuracy, F1 score and computational cost for each candidate feature subset;

[0107] Calculate the score of each feature subset through the objective function;

[0108] From all candidate feature subsets, the feature subset that maximizes the objective function is selected as the optimal feature subset.

[0109] In one embodiment of the present invention, step S6 includes:

[0110] Decompose the entire modeling process into multiple stages and define clear inputs, outputs, and operations for each stage;

[0111] Define a process management system for each stage, including task scheduling, logging, and status tracking. Task scheduling is used to manage the execution order of the process, logging is used to record logs for each stage, and status tracking is used to record the execution status of each stage. If a stage fails, the subsequent stages will automatically stop.

[0112] Introducing an exception handling mechanism into the process to automatically identify and process abnormal data;

[0113] At the same time, monitoring tools are used to monitor the execution status of the process in real time, and a process management interface is provided to view process status, logs and error information.

[0114] Specifically, the modeling process is decomposed into multiple independent stages, and the input, output and operation of each stage are clarified; Stage 1: Input: original data set, Output: cleaned data set, Operation: data cleaning, missing value filling, outlier detection and processing; Stage 2: Input: cleaned data set, Output: first feature set, Operation: missing rate calculation, variance filtering, correlation analysis, irrelevant feature removal; Stage 3: Input: first feature set, Output: second feature set, Operation: training basic model; Stage 4: Input: second feature set, Output: optimal model, Operation: Bayesian optimization, grid search, model training and verification; Stage 5: Input: optimal model, Output: : Deployed models and real-time updated weights, operations: model deployment, real-time data feedback, dynamic weight adjustment; use task scheduling tools to define stage dependencies, set stage execution order, and support parallel execution; use the log library to record the start time, end time, operation details and results of each stage, and store log files by stage; use status tracking tools to record the execution status of each stage. If a stage fails, the subsequent stages will automatically stop and trigger the exception handling mechanism; add an exception capture mechanism to each stage, record error logs and terminate the process, define a retry mechanism (such as retrying 3 times after failure) and a rollback mechanism (such as restoring to the last successful state).

[0115] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A self-adaptive automated modeling technology for single models of multi-source heterogeneous data, characterized by: include: S1: Acquire multi-source heterogeneous data and perform preprocessing operations on it; S2: Extract features from multi-source heterogeneous data and generate a preliminary feature set; S3: The first feature set is obtained by performing missing rate processing, variance filtering, correlation filtering and irrelevant feature removal on the preliminary feature set; S4: Select a basic model, use the first feature set and the target variable to train the model, and perform feature importance evaluation. Remove features whose importance is lower than the threshold to obtain the second feature set. S5: Sort features by importance, gradually remove the least important features, evaluate the performance of different feature subsets through grid search, define the objective function based on the evaluation parameters and corresponding weights, select the feature subset with the best performance, train the model using the optimal feature subset, introduce an adaptive learning mechanism, and dynamically adjust the model's hyperparameters by monitoring the evaluation indicators of the model during training; S6: Decompose the modeling process into multiple stages, set up process management and exception handling, and monitor the process status in real time; Defining the objective function according to the evaluation parameters and corresponding weights includes: Z=argmax(ω a ·a+ω f ·f-ω c ·c) Where Z is the objective function, a, f, and c are accuracy, F1 score, and computational cost, respectively. a 、ω f and ω c are the corresponding weight coefficients respectively.

2. The adaptive automated modeling technology for single models of multi-source heterogeneous data according to claim 1 is characterized in that: The step S1 comprises: Comprehensively scan multi-source heterogeneous data to identify and process noise, outliers, and missing values ​​in the data; Convert data from different sources into a unified, pre-set standard format; Select the corresponding normalization method according to the characteristics and distribution of the data; At the same time, data from different data sources are integrated by establishing data association rules and mapping relationships.

3. The adaptive automated modeling technology for single models of multi-source heterogeneous data according to claim 1 is characterized in that: The step S2 includes: Extract features from multi-source heterogeneous data and generate a preliminary feature set; The multi-source heterogeneous data includes structured data, text data, image data and time series data; The first feature set includes statistical features, interactive features, polynomial features, TF-IDF, word vectors, topic models, deep learning features, manual features, rolling features, frequency domain features and autocorrelation coefficients.

4. The adaptive automated modeling technology for single models of multi-source heterogeneous data according to claim 1 is characterized in that: The step S3 includes: Traverse all features, calculate the missing rate of each feature, remove features with missing rates higher than the preset threshold, and fill the remaining feature values ​​using the mean filling, median filling, or interpolation filling methods; Calculate the variance of each feature and remove features whose variance is lower than a preset threshold; Calculate the Pearson correlation coefficient between each feature and the target variable, remove features whose absolute value of the correlation coefficient is lower than the preset threshold, and calculate the correlation coefficient between features, remove features whose absolute value of the correlation coefficient is higher than the preset threshold, and retain one feature; For classification problems, use the chi-square test or mutual information method to remove irrelevant features; for regression problems, use the F test or p-value method to remove irrelevant features; The feature set obtained after a screening of the preliminary feature set is recorded as the first feature set.

5. The adaptive automated modeling technology for single models of multi-source heterogeneous data according to claim 1 is characterized in that: The step S4 comprises: Select one or more models from the linear model or the tree model as a base model, and use the first feature set and the target variable to train the base model to obtain a trained model; For linear models, feature importance is evaluated by regularization weights; for tree models, feature importance is evaluated by the frequency of use of the feature in the split node; According to the feature importance sorting, features with importance higher than the preset threshold are selected. For the Lasso model, features with a coefficient of 0 are directly removed. The feature set after secondary screening is recorded as the second feature set.

6. The self-adaptive automated modeling technology for multi-source heterogeneous data according to claim 1 is characterized in that: The step S5 comprises: Using the second feature set as initial input, select a base model, train the base model on the second feature set, and evaluate the importance of each feature; According to the feature importance ranking, the least important features are gradually removed and the remaining features are retained; Repeat the above process until the preset number of features is reached; Use grid search to evaluate the performance of different feature subsets; define the objective function based on the evaluation parameters and corresponding weights to select the optimal feature subset; Divide the dataset into training and test sets, use the optimal feature subset as input features, train the model, evaluate the model performance on the test set, and output the final results; At the same time, during the model construction process, an adaptive learning mechanism is introduced to dynamically adjust the model's hyperparameters by monitoring the model's evaluation indicators during training.

7. The adaptive automated modeling technology for single models of multi-source heterogeneous data according to claim 6 is characterized in that: Defining the objective function according to the evaluation parameters and the corresponding weights further includes: Consider multiple evaluation metrics simultaneously in the objective function, including accuracy, F1 score, and computational cost, and normalize the computational cost; The dynamic adjustment of the model's hyperparameters includes treating weights as hyperparameters, automatically searching for the optimal weight combination through Bayesian optimization, and dynamically adjusting the weights based on real-time data feedback after the model is deployed.

8. The adaptive automated modeling technology for single models of multi-source heterogeneous data according to claim 6 is characterized in that: The optimal feature subset includes: Generate multiple candidate feature subsets from the initial feature set, and calculate the accuracy, F1 score and computational cost for each candidate feature subset; Calculate the score of each feature subset through the objective function; From all candidate feature subsets, the feature subset that maximizes the objective function is selected as the optimal feature subset.

9. The self-adaptive automated modeling technology for multi-source heterogeneous data according to claim 1, characterized in that: The step S6 comprises: Decompose the entire modeling process into multiple stages and define clear inputs, outputs, and operations for each stage; Define a process management system for each stage, including task scheduling, logging, and status tracking. Task scheduling is used to manage the execution order of the process, logging is used to record logs for each stage, and status tracking is used to record the execution status of each stage. If a stage fails, the subsequent stages will automatically stop. Introducing an exception handling mechanism into the process to automatically identify and process abnormal data; At the same time, monitoring tools are used to monitor the execution status of the process in real time, and a process management interface is provided to view process status, logs and error information.

Images