Data warehouse intelligent construction method and system
By employing a multi-technology collaborative intelligent data warehouse construction method, the problems of low efficiency in the fusion and construction of multi-source heterogeneous data are solved, achieving efficient and accurate data warehouse construction, adapting to multi-source data scenarios, and reducing labor costs.
Patent Information
- Application Number
- CN202511772025.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-28
- Publication Date
- 2026-03-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
In existing technologies, the deep fusion of multi-source heterogeneous data and the construction of data warehouses are inefficient, require a lot of manual intervention, and are difficult to achieve efficient and accurate data warehouse construction.
A multi-technology collaborative approach is adopted, including data classification, feature extraction, multimodal feature fusion, data cleaning and standardization, and data quality assessment. By combining the joint Kalman filter model and reinforcement learning, a data warehouse model is automatically generated and dynamically optimized.
It enables automated integration and intelligent modeling of multi-source heterogeneous data, improves the efficiency of data warehouse construction and data quality, adapts to various multi-source data scenarios, reduces labor costs, and supports rapid response to changes in business needs.
Smart Images
Figure CN121597660A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data warehouse technology, and in particular to a method and system for intelligent construction of a data warehouse. Background Technology
[0002] With the widespread adoption of digital technologies across various fields, data sources are becoming increasingly diversified, encompassing relational databases (such as MySQL and Oracle), log files (JSON / XML), real-time sensor data (such as IoT device output), and unstructured documents (such as PDF reports and surveillance images). This has resulted in a multi-source, heterogeneous data system where structured, semi-structured, and unstructured data coexist. Traditional data warehouse construction relies on manual processes for data source integration, data cleaning rule definition, model design (star / snowflake schema), and ETL script writing, which suffers from core problems such as high data integration difficulty and low efficiency.
[0003] In existing technologies, some solutions use metadata tools to assist data management, but fail to address the deep integration of multi-source heterogeneous data; others employ a single algorithm (such as Kalman filtering) to optimize data processing, but do not cover the entire data warehouse construction process. Therefore, there is an urgent need for a fully intelligent, end-to-end method for constructing multi-source heterogeneous data warehouses, breaking through scenario limitations and achieving efficient and accurate data warehouse construction. Summary of the Invention
[0004] The purpose of this invention is to provide a data warehouse intelligent construction method and system to solve the problems existing in the background technology mentioned above. By using multiple technologies to achieve automated integration and intelligent modeling of multi-source heterogeneous data, manual intervention is reduced and the construction efficiency and data quality of the data warehouse are improved.
[0005] To achieve the above objectives, the present invention provides a method for intelligent construction of a data warehouse, comprising the following steps: S1. Collect heterogeneous data from multiple sources, and perform data classification and feature extraction based on data types to provide a foundation for subsequent data fusion; S2. Eliminate data heterogeneity through multimodal feature fusion algorithms, including feature-level fusion, data cleaning and standardization, and data quality assessment, thereby outputting a standardized fusion dataset; S3. Based on the data fused in step S2, automatically divide the subject areas and optimize the model structure, and generate an executable script to realize the intelligent construction of the data warehouse model; S4. Deploy the constructed data warehouse model and monitor and dynamically optimize its operation to ensure the long-term stable operation of the data warehouse and its adaptation to business changes.
[0006] Preferably, step S1 specifically includes: S11. Automatically match the acquisition method based on the data format and transmission protocol of the acquired data; S12. A hybrid classification method is used to divide the data into text, numerical, and image categories. For text data, a pre-trained BERT model is used to extract semantic features, and the semantic feature confidence score of each token is calculated using the following formula: ; in, Output the first BERT model The probability distribution of each token, when If the semantic feature extraction is deemed valid, the feature is retained. For image data, the VGG-19 model is used to extract visual features. The cosine similarity between the image patch and the features in the sample database is calculated as the confidence score, using the following formula: ; in, For the first Cosine similarity between each image patch and features in the sample database. For the first Feature vectors of image patches For the feature vectors of similar images in the sample library, when When the visual feature extraction is deemed effective, it is determined that the extraction is valid. Numerical data retains its original statistical characteristics and proceeds directly to the subsequent fusion step; S13. Generate a standardized feature set and label the extraction confidence of each feature.
[0007] Preferably, in step S2, a joint Kalman filter model is constructed to achieve feature-level fusion of data for the three types of features: text, image, and numerical data. Specifically, this includes: State prediction equation: Let the first... The system state vector at time step is The state transition matrix is The control input matrix is The input vector is ,but: ; in, This is process noise; For the first The system state vector at each time step; For the first The control input vector for the time step; Observation equation: Let the observation matrix be... The observation vector is The observation noise is ,but: ; Calculate the Kalman gain: Let the prior estimate covariance matrix be... Then the Kalman gain for: ; in, for The transpose of the matrix; For the first The observation noise covariance matrix at each time step; State update: Calculate the correlation weights between text and numerical features, and between image and numerical features. The formula is: ; in, , The evidence classification results are for the two types of features, , The dimensions of the two types of features; combined Update post-test status : ; Covariance update: ;in, It is an identity matrix.
[0008] Preferably, the data cleaning and standardization in step S2 specifically includes: The Isolation Forest algorithm is used to calculate the anomaly score for each sample by randomly splitting the data. The formula is as follows: ; in, For a single sample to be tested, The total number of samples in the dataset. This represents the average path length of the sample in the isolated forest. for The average path length of each sample in a random tree, and , For the first A harmonic number; when When an outlier occurs, it is identified as an outlier and removed. For missing numerical data, the sliding window mean method is used to impute missing values: ; in, For window size, For non-missing values within the window; for missing text data, fill with unknown labels. Numerical data is standardized using Min-Max, with the following formula: ; Classification data uses one-hot encoding to convert class features into 3D binary vectors.
[0009] Preferably, the data quality assessment in step S2 specifically includes: Definition of integrity ,consistency ,accuracy Three core metrics: ; ; ; The weights of each indicator are calculated using the entropy weight method: ; ; in, For the first The entropy value of each indicator; For the first The first sample Normalized values of each indicator; A comprehensive score will be given: ; when If the data quality is deemed acceptable, the process returns to the data cleaning and standardization steps in step S2.
[0010] Preferably, step S3 specifically includes: S31. Based on business requirement keywords, the K-means algorithm is used to cluster the fields of the fused data; S32. Construct a reinforcement learning model and generate a data warehouse model with the optimization objectives of the shortest query response time and the lowest storage cost. S33. Based on the generated model structure, automatically generate data definition statements, generate DDL scripts, and generate ETL scripts based on the data dictionary definition of extraction, transformation, and loading rules; S34, Output data warehouse model, DDL script and ETL script.
[0011] Preferably, step S31 specifically includes: For each field, construct a field feature vector. ,in, For data volume, To update frequency, It is a semantic vector; Using the elbow rule, the contour coefficients for different K values are calculated, and the K with the largest contour coefficient is selected as the number of subject domains. For the clustered fields, calculate the correlation between the fields using the following formula: ; in, For semantic weight coefficients, For fields and semantic similarity, The percentage of records associated with a field; when When, the field and Fields are grouped into the same fact table, while descriptive fields are grouped into dimension tables.
[0012] Preferably, step S32 specifically includes: Define the agent responsible for selecting model structure parameters and the data warehouse operating environment; Define the state space: ,in, For data volume; Query frequency; For storage resources; Define the action space: ,in: Dimensional hierarchy; Partitioning method; For index type; Set the reward function: ; in, As time weight, As cost weight, and ; This refers to the actual query response time. This is the threshold for query response; Actual storage cost; Cost threshold; The DQN algorithm is used for training iterations to enable the agent to learn the optimal action strategy and generate a data warehouse model.
[0013] Preferably, step S4 specifically includes: S41. Execute the DDL script in the target data warehouse engine, deploy the ETL script to the scheduling tool, and set the scheduling cycle; S42. Monitor data warehouse operational metrics, including storage metrics, computation metrics, and data metrics; when a metric exceeds a threshold, update the model parameters using the following formula: ; in, For the first Model parameters for the next iteration For the model parameter variables to be optimized, For the number of iterations, For the first The loss function for the next iteration. For the first Input features for the next iteration For the first The true label of the next iteration For parameters The L1 norm, For parameters The square of the L2 norm, , The regularization coefficient is used to re-optimize the model structure; S43. Set up an interactive interface to receive user feedback and categorize and process the feedback; thereby outputting a stable data warehouse, dynamic optimization logs, and user feedback processing reports.
[0014] This invention also provides a data warehouse intelligent construction system, comprising: The data acquisition module includes a multi-source connector, a data classification unit, and a feature extraction unit, enabling unified access and feature extraction of multi-source heterogeneous data; The data fusion module includes a joint Kalman filter unit, a data cleaning unit, and a quality assessment unit, which realizes multi-source feature fusion and data quality control. The model generation module is used to generate optimal data models and executable scripts; Deploy the monitoring module to enable model deployment, metric monitoring, and dynamic optimization; The interaction and knowledge base module provides an interactive interface, stores a general data dictionary, historical model parameters, and industry rule base, and supports system iteration and scenario adaptation.
[0015] Therefore, the present invention adopts the above-mentioned intelligent data warehouse construction method and system, which solves the problems of low accuracy, low construction efficiency and insufficient intelligence in the fusion of multi-source heterogeneous data, effectively improves data quality and shortens query response time; at the same time, it is compatible with various multi-source data scenarios, supports mainstream data warehouse engines, reduces labor costs, can quickly respond to changes in business needs, and provides stable and high-quality data support for data-driven decision-making.
[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0017] Figure 1 This is a flowchart of a data warehouse intelligent construction method according to the present invention; Figure 2This is a flowchart illustrating the construction process of the data warehouse model according to an embodiment of the present invention. Figure 3 This is a schematic diagram of the structure of a data warehouse intelligent construction system according to the present invention. Detailed Implementation
[0018] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.
[0019] Please see Figures 1-2 A data warehouse intelligent construction method includes the following steps: S1. Collect multi-source heterogeneous data, and perform data classification and feature extraction based on data type to provide a foundation for subsequent data fusion; specifically including: S11. Automatically match the data acquisition method based on the data format (structured / semi-structured / unstructured) and transmission protocol (JDBC / ODBC / Kafka / FTP / API); Structured data (such as MySQL tables and Excel files): Supports timed collection (minimum granularity of 10 minutes) or real-time synchronization (based on CDC technology) via JDBC / ODBC connector. Semi-structured data (such as JSON logs and XML configuration files): accessed through the Kafka message queue, with the number of partitions (default 6) and replicas (default 3) set to ensure high throughput (supports 100,000 data entries per second). Unstructured data (such as surveillance images and PDF reports): retrieved in batches via FTP protocol (once a day) or pulled in real time via API interface and stored in a distributed file system (such as HDFS).
[0020] S12. Using a hybrid classification method combining rules and machine learning, the data is divided into text, numerical, and image categories. For text data, a pre-trained BERT model is used to extract semantic features, and the semantic feature confidence score for each token is calculated using the following formula: ; in, Output the first BERT model The probability distribution of each token, when If the semantic feature extraction is deemed valid, the feature is retained. For image data, the VGG-19 model is used to extract visual features. The cosine similarity between the image patch and the features in the sample database is calculated as the confidence score, using the following formula: ; in, For the first Cosine similarity between each image patch and features in the sample database. For the first Feature vectors of image patches For the feature vectors of similar images in the sample library, when When the visual feature extraction is deemed effective, it is determined that the extraction is valid. Numerical data retains its original statistical characteristics (mean, variance, extreme values) and does not require additional feature transformation; it can directly proceed to the subsequent fusion steps. S13. Generate a standardized feature set (including text semantic features, image visual features, and numerical statistical features), and label the extraction confidence of each feature.
[0021] S2. Eliminate data heterogeneity through multimodal feature fusion algorithms, including feature-level fusion, data cleaning and standardization, and data quality assessment, to output a standardized fused dataset; for text, image, and numerical features, construct a joint Kalman filter model to achieve feature-level fusion, specifically including: State prediction equation: Let the first... The system state vector at time step is The state transition matrix is The control input matrix is The input vector is ,but: ; in, This is process noise; For the first The system state vector at each time step; For the first The control input vector for the time step; Observation equation: Let the observation matrix be... The observation vector is The observation noise is ,but: ; Calculate the Kalman gain: Let the prior estimate covariance matrix be... Then the Kalman gain for: ; in, for The transpose of the matrix; For the first The observation noise covariance matrix at each time step; State update: Calculate the correlation weights between text and numerical features, and between image and numerical features. The formula is: ; in, , The evidence classification results are for the two types of features, , The dimensions of the two types of features; combined Update post-test status : ; Covariance update: ;in, It is an identity matrix.
[0022] Data cleaning and standardization specifically include: The Isolation Forest algorithm is used to calculate the anomaly score for each sample by randomly splitting the data. The formula is as follows: ; in, For a single sample to be tested, The total number of samples in the dataset. This represents the average path length of the sample in the isolated forest. for The average path length of each sample in a random tree, and , For the first A harmonic number; when When an outlier occurs, it is identified as an outlier and removed. For missing numerical data, the sliding window mean method is used to impute missing values: ; in, For window size, For non-missing values within the window; for missing text data, fill with unknown labels. Numerical data is standardized using Min-Max, with the following formula: ; Classification data uses one-hot encoding to convert class features into 3D binary vectors.
[0023] The data quality assessment specifically includes: Definition of integrity ,consistency ,accuracy Three core metrics: ; ; ; The weights of each indicator are calculated using the entropy weight method: ; ; in, For the first The entropy value of each indicator; For the first The first sample Normalized values of each indicator; A comprehensive score will be given: ; when If the data quality is deemed acceptable, the process returns to the data cleaning and standardization steps in step S2.
[0024] S3. Based on the data fused in step S2, automatically divide the subject areas and optimize the model structure, and generate executable scripts to achieve intelligent construction of the data warehouse model; specifically including: S31. Based on business requirement keywords, the K-means algorithm is used to cluster the fields of the fused data; For each field, construct a field feature vector. ,in, For data volume, To update frequency, It is a semantic vector; Using the elbow rule, the contour coefficients for different K values are calculated, and the K with the largest contour coefficient is selected as the number of subject domains. For the clustered fields, calculate the correlation between the fields using the following formula: ; in, For semantic weight coefficients, For fields and semantic similarity, The percentage of records associated with a field; when When, the field and Fields are grouped into the same fact table, while descriptive fields are grouped into dimension tables.
[0025] S32. Construct a reinforcement learning model and generate a data warehouse model with the optimization objectives of the shortest query response time and the lowest storage cost. Define the agent responsible for selecting model structure parameters and the data warehouse operating environment; Define the state space: ,in, For data volume; Query frequency; For storage resources; Define the action space: ,in: Dimensional hierarchy; Partitioning method; For index type; Set the reward function: ; in, As time weight, As cost weight, and ; This refers to the actual query response time. This is the threshold for query response; Actual storage cost; Cost threshold; The DQN algorithm is used for training iterations to enable the agent to learn the optimal action strategy and generate a data warehouse model.
[0026] S33. Based on the generated model structure, automatically generate data definition statements, support mainstream data warehouse engines such as Doris, Hive, and Greenplum, generate DDL scripts, and generate ETL scripts based on the data dictionary definition of extraction, transformation, and loading rules. S34, Output data warehouse model, DDL script and ETL script.
[0027] S4. Deploy the constructed data warehouse model and monitor and dynamically optimize its operation to ensure long-term stable operation and adaptability to business changes; specifically: S41. Execute the DDL script in the target data warehouse engine, deploy the ETL script to the scheduling tool, and set the scheduling cycle; S42. Monitor data warehouse operational metrics, including storage metrics, computation metrics, and data metrics; when a metric exceeds a threshold, update the model parameters using the following formula: ; in, For the first Model parameters for the next iteration For the model parameter variables to be optimized, For the number of iterations, For the first The loss function for the next iteration. For the first Input features for the next iteration For the first The true label of the next iteration For parameters The L1 norm, For parameters The square of the L2 norm, , The regularization coefficient is used to re-optimize the model structure; S43. Set up an interactive interface to receive user feedback and categorize and process the feedback; thereby outputting a stable data warehouse, dynamic optimization logs, and user feedback processing reports.
[0028] like Figure 3 As shown, a data warehouse intelligent construction system includes: The data acquisition module includes a multi-source connector, a data classification unit, and a feature extraction unit, enabling unified access and feature extraction of multi-source heterogeneous data; The data fusion module includes a joint Kalman filter unit, a data cleaning unit, and a quality assessment unit, which realizes multi-source feature fusion and data quality control. The model generation module is used to generate optimal data models and executable scripts; Deploy the monitoring module to enable model deployment, metric monitoring, and dynamic optimization; The interaction and knowledge base module provides an interactive interface, stores a general data dictionary, historical model parameters, and industry rule base, and supports system iteration and scenario adaptation.
[0029] Therefore, the present invention adopts the above-mentioned intelligent data warehouse construction method and system, which realizes the automated integration and intelligent modeling of multi-source heterogeneous data through multi-technology collaboration, reduces manual intervention, and improves the construction efficiency, data quality and cross-scenario applicability of data warehouse.
[0030] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.
Claims
1. A method for intelligently constructing a data warehouse, characterized in that, Includes the following steps: S1. Collect heterogeneous data from multiple sources, and perform data classification and feature extraction based on data types to provide a foundation for subsequent data fusion; S2. Eliminate data heterogeneity through multimodal feature fusion algorithms, including feature-level fusion, data cleaning and standardization, and data quality assessment, thereby outputting a standardized fusion dataset; S3. Based on the data fused in step S2, automatically divide the subject areas and optimize the model structure, and generate an executable script to realize the intelligent construction of the data warehouse model; S4. Deploy the constructed data warehouse model and monitor and dynamically optimize its operation to ensure the long-term stable operation of the data warehouse and its adaptation to business changes.
2. The intelligent data warehouse construction method according to claim 1, characterized in that, Step S1 specifically includes: S11. Automatically match the acquisition method based on the data format and transmission protocol of the acquired data; S12. A hybrid classification method is used to divide the data into text, numerical, and image categories. For text data, a pre-trained BERT model is used to extract semantic features, and the semantic feature confidence score of each token is calculated using the following formula: ； in, Output the first BERT model The probability distribution of each token, when If the semantic feature extraction is deemed valid, the feature is retained. For image data, the VGG-19 model is used to extract visual features. The cosine similarity between the image patch and the features in the sample database is calculated as the confidence score, using the following formula: ； in, For the first Cosine similarity between each image patch and features in the sample database. For the first Feature vectors of image patches For the feature vectors of similar images in the sample library, when When the visual feature extraction is deemed effective, it is determined that the extraction is valid. Numerical data retains its original statistical characteristics and proceeds directly to the subsequent fusion step; S13. Generate a standardized feature set and label the extraction confidence of each feature.
3. The intelligent data warehouse construction method according to claim 2, characterized in that, In step S2, a joint Kalman filter model is constructed to achieve feature-level fusion of data for the three types of features: text, image, and numerical data. Specifically, this includes: State prediction equation: Let the first... The system state vector at time step is The state transition matrix is The control input matrix is The input vector is ,but: ； in, This is process noise; For the first The system state vector at each time step; For the first The control input vector for the time step; Observation equation: Let the observation matrix be... The observation vector is The observation noise is ,but: ； Calculate the Kalman gain: Let the prior estimate covariance matrix be... Then the Kalman gain for: ； in, for The transpose of the matrix; For the first The observation noise covariance matrix at each time step; State update: Calculate the correlation weights between text and numerical features, and between image and numerical features. The formula is: ； in, , The evidence classification results are for the two types of features, , The dimensions of the two types of features; combined Update post-test status : ； Covariance update: ;in, It is an identity matrix.
4. The intelligent data warehouse construction method according to claim 3, characterized in that, The data cleaning and standardization in step S2 specifically includes: The Isolation Forest algorithm is used to calculate the anomaly score for each sample by randomly splitting the data. The formula is as follows: ； in, For a single sample to be tested, The total number of samples in the dataset. This represents the average path length of the sample in an isolated forest. for The average path length of each sample in a random tree, and , For the first A harmonic number; when When this happens, it is identified as an outlier and removed. For missing numerical data, the sliding window mean method is used to impute missing values: ； in, For window size, For non-missing values within the window; for missing text data, fill with unknown labels. Numerical data is standardized using Min-Max, with the following formula: ； Classification data uses one-hot encoding to convert class features into 3D binary vectors.
5. The intelligent data warehouse construction method according to claim 1, characterized in that, The data quality assessment in step S2 specifically involves: Definition of integrity ,consistency ,accuracy Three core metrics: ； ； ； The weights of each indicator are calculated using the entropy weight method: ； ； in, For the first The entropy value of each indicator; For the first The first sample Normalized values of each indicator; A comprehensive score will be given: ; when If the data quality is deemed acceptable, the process returns to the data cleaning and standardization steps in step S2.
6. The intelligent data warehouse construction method according to claim 5, characterized in that, Step S3 specifically includes: S31. Based on business requirement keywords, the K-means algorithm is used to cluster the fields of the fused data; S32. Construct a reinforcement learning model and generate a data warehouse model with the optimization objectives of the shortest query response time and the lowest storage cost. S33. Based on the generated model structure, automatically generate data definition statements, generate DDL scripts, and generate ETL scripts based on the data dictionary definition of extraction, transformation, and loading rules; S34, Output data warehouse model, DDL script and ETL script.
7. The intelligent data warehouse construction method according to claim 6, characterized in that, Step S31 is as follows: For each field, construct a field feature vector. ,in, For data volume, To update frequency, It is a semantic vector; Using the elbow rule, the contour coefficients for different K values are calculated, and the K with the largest contour coefficient is selected as the number of subject domains. For the clustered fields, calculate the correlation between the fields using the following formula: ； in, For semantic weight coefficients, For fields and semantic similarity, The percentage of records associated with a field; when When, the field and Fields are grouped into the same fact table, while descriptive fields are grouped into dimension tables.
8. The intelligent data warehouse construction method according to claim 6, characterized in that, Step S32 is as follows: Define the agent responsible for selecting model structure parameters and the data warehouse operating environment; Define the state space: ,in, For data volume; Query frequency; For storage resources; Define the action space: ,in: For dimensional hierarchy; Partitioning method; For index type; Set the reward function: ; in, As time weight, As cost weight, and ; This refers to the actual query response time. This is the threshold for query response; Actual storage cost; Cost threshold; The DQN algorithm is used for training iterations to enable the agent to learn the optimal action strategy and generate a data warehouse model.
9. The intelligent data warehouse construction method according to claim 1, characterized in that, Step S4 is as follows: S41. Execute the DDL script in the target data warehouse engine, deploy the ETL script to the scheduling tool, and set the scheduling cycle; S42. Monitor data warehouse operational metrics, including storage metrics, computation metrics, and data metrics; when a metric exceeds a threshold, update the model parameters using the following formula: ； in, For the first Model parameters for the next iteration For the model parameter variables to be optimized, For the number of iterations, For the first The loss function for the next iteration. For the first Input features for the next iteration For the first The true label of the next iteration For parameters L1 norm, For parameters The square of the L2 norm, , The regularization coefficient is used to re-optimize the model structure; S43. Set up an interactive interface to receive user feedback and categorize and process the feedback; thereby outputting a stable data warehouse, dynamic optimization logs, and user feedback processing reports.
10. A data warehouse intelligent construction system, applied to the data warehouse intelligent construction method according to any one of claims 1-9, characterized in that, include: The data acquisition module includes a multi-source connector, a data classification unit, and a feature extraction unit, enabling unified access and feature extraction of multi-source heterogeneous data; The data fusion module includes a joint Kalman filter unit, a data cleaning unit, and a quality assessment unit, which realizes multi-source feature fusion and data quality control. The model generation module is used to generate optimal data models and executable scripts; Deploy the monitoring module to enable model deployment, metric monitoring, and dynamic optimization; The interaction and knowledge base module provides an interactive interface, stores a general data dictionary, historical model parameters, and industry rule base, and supports system iteration and scenario adaptation.