Method and system for constructing data reaction for periodic table of data elements

Abstract

The invention discloses a construction method of a data response for a periodic table of data elements. The construction method comprises the following steps: defining basic information of each data element; according to the properties and application scenes of the data, defining different types of data reactions, determining conditions of occurrence of the data reactions, determining the application scenes of the model, and determining problems to be solved by the model; a data reaction model is constructed, historical data and feature data are utilized to train and optimize the model, and the prediction accuracy and generalization ability of the model are improved; analyzing a reaction relationship among the data elements by utilizing the constructed data reaction model; outputting a data response analysis report, and outputting a prediction result of the model; and a prediction result is explained and presented through an artificial intelligence technology. According to the method, efficient organization, management and application of the data are realized, the accuracy of data reaction can be effectively improved, and more scientific and accurate data support is provided for business decision making.

Description

Technical Field

[0001] The present invention relates to the field of network technology, and in particular to a method and system for constructing a data reaction for a periodic table of data elements. Background Art

[0002] Data has been defined as the fifth largest production factor after land, labor, capital, and technology. The periodic table of data elements, as the name implies, is to break down data elements into more basic, easier to understand and operate data elements according to certain rules. Therefore, how to achieve efficient organization, management, and application of data based on the clear identification of data elements in the periodic table of data elements is currently being actively explored. Therefore, the present invention provides a method and system for constructing a data element periodic table using data reactions. Summary of the invention

[0003] To this end, the present invention provides a method and system for constructing a data element periodic table using data reactions to solve the problem of how to improve the efficient organization, management and application of data in the prior art.

[0004] In order to achieve the above object, the present invention provides the following technical solutions:

[0005] A method for constructing a data element periodic table using data reactions, comprising the following steps:

[0006] S1. Define data elements, collect all data elements involved in the business scenario, and define the basic information of each data element; classify the data elements into different categories according to their characteristics, and assign a unique code to each data element for identification and management in the system;

[0007] S2. Formulate data response rules. Define different types of data responses based on the nature of the data and application scenarios, clarify the conditions under which data responses occur, clarify the application scenarios of the model, and determine the problems that the model solves;

[0008] S3. Build a data response model and select appropriate machine learning or deep learning algorithms to build the data response model; use historical data and feature data to train and optimize the model to improve the prediction accuracy and generalization ability of the model;

[0009] S4. Data reaction analysis and prediction: Use the constructed data reaction model to analyze the reaction relationship between data elements; output the data reaction analysis report and the prediction results of the model;

[0010] S5. Data response prediction map is constructed, and the prediction results are interpreted and presented through artificial intelligence technology so that users can understand the interactions and changes between data elements.

[0011] Further: the data elements include user behavior data, transaction data, and device status data.

[0012] Further: The steps of constructing the data response model are as follows:

[0013] (1) Model architecture design: design the overall structure of the model, including the input layer, hidden layer, and output layer; select the appropriate model type based on the characteristics of the data and business needs;

[0014] (2) Algorithm selection and parameter setting: Select the appropriate algorithm based on the selected model type;

[0015] (3) Set the parameters of the algorithm and optimize these parameters.

[0016] Further: The algorithm selection principles are as follows:

[0017] (1) Accuracy: The algorithm needs to be able to accurately reflect the relationship between data to improve the accuracy of model prediction;

[0018] (2) Efficiency: The algorithm should have high computational efficiency to ensure that the model training and prediction can be completed within a reasonable time;

[0019] (3) Explainability: The algorithm should have a certain degree of explainability to facilitate business personnel to understand and use the model;

[0020] (4) Robustness: The algorithm needs to be able to adapt to different data sets and noise environments and maintain stable performance.

[0021] Further: The training method of the data response model is as follows:

[0022] (1) Dataset division: divide the dataset into training set, validation set, and test set to ensure the effectiveness of model training;

[0023] (2) Model training: use the training set to train the model and adjust the model parameters through mechanisms such as back propagation;

[0024] (3) Verification and tuning: Use the validation set to verify the model and adjust the model parameters and structure based on performance indicators;

[0025] (4) Testing and evaluation: Use the test set to test the model and evaluate the model’s generalization ability and overall performance.

[0026] Further: The specific implementation steps of S5 are as follows:

[0027] S5.1 Use NLP technology to convert the prediction results into easy-to-understand text form;

[0028] S5.2 Extract key information from the prediction results and generate relevant explanatory text through NLP technology;

[0029] S5.3 Use charts to visually display the forecast results and their changing trends;

[0030] S5.4 Visually demonstrate the prediction accuracy of the model by drawing a difference graph between the model prediction results and the actual results;

[0031] S5.5 Create interactive visualization interfaces that allow users to observe changes in prediction results in real time by adjusting the values ​​of data elements, thereby gaining a deeper understanding of the interactions between data elements;

[0032] S5.6 Analyze the causal relationship between data elements to help users understand why such prediction results occur;

[0033] S5.7 Use regression models to analyze the impact of data elements on prediction results, thereby identifying key influencing factors;

[0034] S5.8 integrates the explanatory text, visual charts, and results of causal inference and model regression generated by NLP; designs a user-friendly interactive interface so that users can easily view the prediction results and their explanations, and observe changes in the prediction results by adjusting data elements.

[0035] In order to achieve the above object, the present invention also provides a technical solution:

[0036] The system for constructing data reactions for the periodic table of data elements is characterized in that it includes a data acquisition layer module, a data storage module, a data processing module, a data reaction model module and an application layer module, and the modules exchange and communicate data through interfaces to ensure the scalability and maintainability of the system;

[0037] The data acquisition module is responsible for collecting data elements from various data sources and performing preliminary processing;

[0038] The data storage module is responsible for using the database structure to store the periodic table of data elements, data features, historical data, and model parameter information;

[0039] Data processing module, which uses distributed computing framework and parallel computing technology to improve the efficiency and speed of data processing;

[0040] Data response model module, responsible for the construction, training and prediction of data response models;

[0041] The application module provides a user interface and interactive functions, supports users to view the data periodic table, data reaction model, prediction result information, and provides interactive query and analysis functions.

[0042] Furthermore: the data processing module introduces an artificial intelligence computing framework to accelerate the training and prediction process of the data response model.

[0043] The present invention has the following advantages: the present invention implements a data response construction method by determining data elements, formulating data response rules, constructing data response models, and predicting data responses, thereby achieving efficient organization, management, and application of data, and can effectively improve the accuracy of data responses, providing more scientific and accurate data support for business decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more intuitively illustrate the prior art and the present application, exemplary drawings are given below. It should be understood that the specific shapes and structures shown in the drawings should not generally be regarded as limiting conditions for implementing the present application; for example, those skilled in the art are capable of easily making conventional adjustments or further optimizations to the addition / reduction / attribution division, specific shapes, positional relationships, connection methods, dimensional ratios, etc. of certain units (components) based on the technical concepts and exemplary drawings disclosed in the present application.

[0045] Figure 1 A flow chart of a method for constructing a data element periodic table using data reactions provided by the present invention.

[0046] Figure 2 Provided by the present invention. DETAILED DESCRIPTION

[0047] The following is an explanation of the implementation of the present invention by specific specific examples. People familiar with this technology can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of them. It should be understood that these embodiments are only to further illustrate the present invention and cannot be understood as limiting the scope of protection of the present invention. Technical engineers in this field can make some non-essential improvements and adjustments to the present invention based on the contents of the above invention; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] See also Figure 1 , a method for constructing a data element periodic table using data reactions, comprising the following steps:

[0049] S1. Define data elements and collect all data elements involved in business scenarios, including user behavior data, transaction data, device status data, etc.; define the name, type, attribute, source and other basic information of each data element; use machine learning algorithms to perform pre-processing operations such as data cleaning, deduplication, missing value filling, etc. to improve data quality; extract features useful for building data response models from raw data through feature engineering methods;

[0050] The pre-processed defined data elements are classified into different categories, such as user attributes, transaction behavior, product information, etc., according to the nature and purpose of the data elements; each data element is assigned a unique code for identification and management in the system;

[0051] According to business needs, determine the basic data elements that need to be included in the data element periodic table, such as population, electricity consumption, transaction amount, equipment status, etc. These data elements will constitute the basic columns of the periodic table;

[0052] According to the characteristics of data elements (such as time periodicity, change trend, etc.), design the structure of the data element periodic table, including time dimension (such as year, month, day), data element column, etc.;

[0053] Fill in periodic table data; collect and organize actual business data and fill it into the corresponding positions of the data element periodic table.

[0054] S2. Formulate data reaction rules. Define different types of data reactions, such as data fusion, data conversion, data association, etc., according to the nature of the data and application scenarios; clarify the conditions for data reactions, such as the type of data elements, attribute values, etc.;

[0055] Demand analysis and goal setting; clarify the application scenarios of the model, such as public data element management, power data analysis, or multi-tenant data management; determine the problems the model aims to solve, such as data association, data element analysis, data isolation and sharing, etc.

[0056] S3. Build a data response model and select appropriate machine learning or deep learning algorithms (such as neural networks, decision trees, support vector machines, etc.) to build a data response model; use historical data and feature data to train and optimize the model to improve the model's prediction accuracy and generalization ability.

[0057] The specific steps are as follows:

[0058] S3.1 Model architecture design, design the overall structure of the model, including input layer, hidden layer, output layer and other components; select the appropriate model type according to the characteristics of the data and business needs, such as neural network, decision tree or graph model;

[0059] Algorithm selection and parameter setting: Select the appropriate algorithm according to the selected model type, such as the BP algorithm in the neural network, the ID3 algorithm in the decision tree, etc.;

[0060] Set algorithm parameters, such as learning rate, number of iterations, tree depth, etc., and optimize these parameters through methods such as cross-validation;

[0061] Principles of algorithm selection:

[0062] (1) Accuracy: The algorithm needs to be able to accurately reflect the relationship between data to improve the accuracy of model predictions.

[0063] (2) Efficiency: The algorithm should have high computational efficiency to ensure that model training and prediction can be completed within a reasonable time.

[0064] (3) Explainability: The algorithm should have a certain degree of explainability to facilitate business personnel to understand and use the model.

[0065] (4) Robustness: The algorithm needs to be able to adapt to different data sets and noise environments and maintain stable performance.

[0066] S3.2 Model training method

[0067] (1) Dataset division: divide the dataset into training set, validation set, and test set to ensure the effectiveness of model training;

[0068] (2) Model training: use the training set to train the model and adjust the model parameters through mechanisms such as back propagation;

[0069] (3) Verification and tuning: Use the validation set to verify the model and adjust the model parameters and structure based on performance indicators (such as accuracy, recall, F1 value, etc.);

[0070] (4) Testing and evaluation: Use the test set to test the model and evaluate the model’s generalization ability and overall performance.

[0071] For example:

[0072] 1. In order to realize the construction of data response model, a hybrid architecture combining deep learning and traditional statistical model is adopted.

[0073] The architecture mainly includes the following parts:

[0074] 1. Data preprocessing layer:

[0075] Data cleaning: processing missing values, outliers, and standardizing or normalizing the data;

[0076] Feature engineering: Feature extraction and selection based on business knowledge, including the conversion of numerical features and categorical features.

[0077] Data preprocessing can be implemented using the following python code:

[0078] import pandas as pd

[0079] import numpy as np

[0080] # Load data

[0081] df = pd.read_csv('data.csv')

[0082] #Data cleaning

[0083] df.dropna(inplace=True)#delete missing values

[0084] df = pd.get_dummies(df, columns = ['category_column']) # Category feature unique hot encoding

[0085] df = (df-df.mean()) / df.std()#Feature standardization 0

[0087] 2. Deep learning model layer:

[0088] Embedding Layer: used to process category features and convert them into dense vectors;

[0089] Recurrent neural network layer (RNN / LSTM): used to process time series data and capture temporal dependencies;

[0090] Attention Layer: Used to focus on important parts of the input and improve the performance of the model.

[0091] 3. Traditional statistical model layer:

[0092] Linear regression model: used to capture the linear relationship between independent variables and dependent variables;

[0093] Decision Tree / Random Forest: used to capture nonlinear relationships and interactions between variables;

[0094] 4. Integrated learning layer:

[0095] Model fusion: Integrate the prediction results of multiple models through weighted averaging or stacking methods to improve prediction accuracy and robustness.

[0096] 5. Output layer: Outputs the final prediction result or response value.

[0097] 2. For the hybrid architecture combining deep learning and traditional statistical models, the algorithm selection and parameter settings are as follows:

[0098] 1. Deep learning model selection and parameter setting:

[0099] LSTM model: used to capture long-term dependencies in time series;

[0100] Input layer size: determined by the number of features. For example, if each sample has 10 features, the input layer size is 10.

[0101] Number of LSTM units: set to 128;

[0102] Dropout rate: set to 0.2 to prevent overfitting;

[0103] Learning rate: initialized to 0.001, using Adam optimizer;

[0104] Batch size: set to 32;

[0105] Number of training rounds: adjusted according to the performance on the validation set, usually 20-50 rounds.

[0106] 2. Traditional statistical model selection and parameter setting:

[0107] Linear regression model: used to capture simple linear relationships;

[0108] Regularization term: L2 regularization, coefficient is 0.01;

[0109] Random forest: used to capture complex nonlinear relationships;

[0110] Number of trees: set to 100;

[0111] Maximum depth: set to 10;

[0112] Number of feature selections: set to sqrt(total number of features).

[0113] 3. Train the model of the above architecture and algorithm. The specific method is as follows

[0114] 1. Data division:

[0115] The dataset is divided into training set, validation set and test set, with the ratio usually being 7:2:1.

[0116] 2. Model training: Use the training set to train the LSTM model and record the performance of each epoch on the validation set. If the loss on the validation set no longer decreases significantly, stop training in advance to prevent overfitting. A similar training process is also performed for the linear regression model and random forest model.

[0117] Model training can be implemented using the following python code:

[0118] 1 from tensorflow.keras.models import Sequential

[0119] 2 from tensorflow.keras.layers import LSTM,Dense,Embedding,Dropout

[0120] 3 from sklearn.linear_model import LinearRegression

[0121] 4 from sklearn.ensemble import RandomForestRegressor 5

[0123] 6 #LSTM model

[0124] 7 model_lstm=Sequential([

[0125] 8 LSTM(128,input_shape=(time_steps,features),return_sequences=False),

[0126] 9 Dropout(0.2),

[0127] 10 Dense(1)#Assuming it is a regression problem 11 ])

[0129] 12 model_lstm.compile(optimizer='adam',loss='mean_squared_error')

[0130] 13 model_lstm.fit(X_train,y_train,epochs=50,batch_size=32,validation_split=0.2) 14

[0132] 15 #Linear regression model

[0133] 16 model_lr=LinearRegression()

[0134] 17 model_lr.fit(X_train,y_train) 18

[0136] 19 # Random Forest Model

[0137] 20 model_rf=RandomForestRegressor(n_estimators=100,max_depth=10)

[0138] 21 model_rf.fit(X_train,y_train)

[0139] 3. Model evaluation and selection:

[0140] The performance of each model is evaluated on the validation set, and the best performing model is selected.

[0141] Evaluation indicators include mean square error (MSE), root mean square error (RMSE), etc.

[0142] 4. Model fusion: Use the Stacking method to fuse the selected models; in the Stacking process, linear regression is used as the meta-learner to perform a weighted combination of the prediction results of each base learner.

[0143] Model fusion can be implemented using the following python code:

[0144] from sklearn.linear_model import LinearRegression as MetaRegressor

[0145] #predict

[0146] y_pred_lstm=model_lstm.predict(X_val)

[0147] y_pred_lr=model_lr.predict(X_val)

[0148] y_pred_rf=model_rf.predict(X_val)

[0149] #Stacking

[0150] X_stack=np.column_stack((y_pred_lstm,y_pred_lr,y_pred_rf))

[0151] 0 meta_model=MetaRegressor()

[0152] 1 meta_model.fit(X_stack,y_val) 2

[0154] #Final prediction 3

[0156] y_final_pred=meta_model.predict(np.column_stack((y_pred_lstm,

[0157] 4 y_pred_lr,y_pred_rf)))

[0158] 5. Final model evaluation: The final fusion model is evaluated on the test set to ensure the generalization ability of the model.

[0159] Through the above steps, we can build a hybrid architecture that combines deep learning and traditional statistical models for the construction of data-responsive models. This model can capture complex relationships and patterns in the data and improve the accuracy and robustness of predictions.

[0160] S4. Data reaction analysis and prediction: Use the constructed data reaction model to analyze the reaction relationship between data elements; output data reaction analysis report, including information such as reaction intensity, impact range, potential value, etc., and output the prediction results of the model.

[0161] S5. Data response prediction graph is constructed, and the prediction results are interpreted and presented through artificial intelligence technology (such as natural language processing, visualization technology, etc.) so that users can understand the interactions and changes between data elements.

[0162] The specific steps are as follows:

[0163] S5.1 Use NLP technology to convert the prediction results into an easily understandable text form; for example, a descriptive text can be generated to explain the significance and impact of the prediction results;

[0164] S5.2 Extract key information from the prediction results and generate relevant explanatory text through NLP technology;

[0165] S5.3 Use charts (such as line charts, bar charts, scatter plots, etc.) to visually display the forecast results and their changing trends;

[0166] S5.4 Visually demonstrate the model's predictive accuracy by plotting the difference between the model's predictions and the true results (if any);

[0167] S5.5 creates an interactive visualization interface that allows users to observe changes in prediction results in real time by adjusting the values ​​of data elements, thereby gaining a deeper understanding of the interactions between data elements.

[0168] S5.6 Analyze the causal relationship between data elements to help users understand why such prediction results occur;

[0169] S5.7 Use regression models to analyze the impact of data elements on prediction results, thereby identifying key influencing factors.

[0170] S5.8 integrates the explanatory text, visual charts, and results of causal inference and model regression generated by NLP; designs a user-friendly interactive interface so that users can easily view the prediction results and their explanations, and observe changes in the prediction results by adjusting data elements.

[0171] This embodiment also provides a system for constructing data reactions for a periodic table of data elements, including a data acquisition layer module, a data storage module, a data processing module, a data reaction model module and an application layer module. The modules exchange and communicate data through interfaces to ensure the scalability and maintainability of the system.

[0172] The data acquisition module is responsible for collecting data elements from various data sources and performing preliminary processing.

[0173] The data storage module is responsible for using the database structure to store information such as the periodic table of data elements, data features, historical data, model parameters, etc.

[0174] The data processing module uses distributed computing frameworks (such as Apache Spark, Hadoop, etc.) and parallel computing technology to improve the efficiency and speed of data processing; introduces artificial intelligence computing frameworks (such as TensorFlow, PyTorch, etc.) to accelerate the training and prediction process of data response models.

[0175] The data response model module is responsible for the construction, training and prediction of the data response model.

[0176] The application module provides a user interface and interactive functions, supports users to view information such as the periodic table of data elements, data reaction models, prediction results, and provides interactive query and analysis functions.

[0177] The data response construction method and system of the present invention can effectively improve the accuracy of data response and provide more scientific and accurate data support for business decision-making; the technical solution realizes efficient organization, management and application of data by determining data elements, formulating data response rules, building data response models, predicting data response and designing and implementing corresponding systems.

[0178] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a data element periodic table using a data reaction, characterized in that: The following steps are involved: S1. Define data elements, collect all data elements involved in the business scenario, and define the basic information of each data element; classify the data elements into different categories according to their characteristics, and assign a unique code to each data element for identification and management in the system; S2. Formulate data response rules. Define different types of data responses based on the nature of the data and application scenarios, clarify the conditions under which data responses occur, clarify the application scenarios of the model, and determine the problems that the model solves; S3. Build a data response model and select appropriate machine learning or deep learning algorithms to build the data response model; use historical data and feature data to train and optimize the model to improve the prediction accuracy and generalization ability of the model; S4. Data reaction analysis and prediction: Use the constructed data reaction model to analyze the reaction relationship between data elements; output the data reaction analysis report and the prediction results of the model; S5. Data response prediction map is constructed, and the prediction results are interpreted and presented through artificial intelligence technology so that users can understand the interactions and changes between data elements.

2. The method for constructing a data element periodic table using data reactions according to claim 1, characterized in that: The data elements include user behavior data, transaction data, and device status data.

3. The method for constructing a data element periodic table using data reactions according to claim 1, characterized in that: The steps of constructing the data response model are as follows: (1) Model architecture design: design the overall structure of the model, including the input layer, hidden layer, and output layer; select the appropriate model type based on the characteristics of the data and business needs; (2) Algorithm selection and parameter setting: Select the appropriate algorithm based on the selected model type; (3) Set the parameters of the algorithm and optimize these parameters.

4. The method for constructing a data element periodic table using data reactions according to claim 3, characterized in that: The algorithm selection principles are as follows: (1) Accuracy: The algorithm needs to be able to accurately reflect the relationship between data to improve the accuracy of model prediction; (2) Efficiency: The algorithm should have high computational efficiency to ensure that the model training and prediction can be completed within a reasonable time; (3) Explainability: The algorithm should have a certain degree of explainability to facilitate business personnel to understand and use the model; (4) Robustness: The algorithm needs to be able to adapt to different data sets and noise environments and maintain stable performance.

5. The method for constructing a data element periodic table using data reactions according to claim 1, characterized in that: The training method of the data response model is as follows: (1) Dataset division: divide the dataset into training set, validation set, and test set to ensure the effectiveness of model training; (2) Model training: use the training set to train the model and adjust the model parameters through mechanisms such as back propagation; (3) Verification and tuning: Use the validation set to verify the model and adjust the model parameters and structure based on performance indicators; (4) Testing and evaluation: Use the test set to test the model and evaluate the model’s generalization ability and overall performance.

6. The method for constructing a data element periodic table using data reactions according to claim 1, characterized in that: The specific implementation steps of S5 are as follows: S5.1 Use NLP technology to convert the prediction results into easy-to-understand text form; S5.2 Extract key information from the prediction results and generate relevant explanatory text through NLP technology; S5.3 Use charts to visually display the forecast results and their changing trends; S5.4 Visually demonstrate the prediction accuracy of the model by drawing a difference graph between the model prediction results and the actual results; S5.5 Create interactive visualization interfaces that allow users to observe changes in prediction results in real time by adjusting the values ​​of data elements, thereby gaining a deeper understanding of the interactions between data elements; S5.6 Analyze the causal relationship between data elements to help users understand why such prediction results occur; S5.7 Use regression models to analyze the impact of data elements on prediction results, thereby identifying key influencing factors; S5.8 integrates the explanatory text, visual charts, and results of causal inference and model regression generated by NLP; designs a user-friendly interactive interface so that users can easily view the prediction results and their explanations, and observe changes in the prediction results by adjusting data elements.

7. A system for constructing a data element periodic table using data responses, characterized in that: It includes data acquisition layer module, data storage module, data processing module, data response model module and application layer module. The modules exchange data and communicate with each other through interfaces to ensure the scalability and maintainability of the system. The data acquisition module is responsible for collecting data elements from various data sources and performing preliminary processing; The data storage module is responsible for using the database structure to store the periodic table of data elements, data features, historical data, and model parameter information; Data processing module, which uses distributed computing framework and parallel computing technology to improve the efficiency and speed of data processing; Data response model module, responsible for the construction, training and prediction of data response models; The application module provides a user interface and interactive functions, supports users to view the data periodic table, data reaction model, prediction result information, and provides interactive query and analysis functions.

8. The system for constructing data element periodic table data reaction according to claim 7, characterized in that: The data processing module introduces an artificial intelligence computing framework to accelerate the training and prediction process of the data response model.

Images