Multi-source heterogeneous data cleaning method for diesel-electric hybrid power system

Abstract

The invention discloses a multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system, which improves a HoloCleanEX technology, adds a hierarchical reduction classification cleaning (HRSC) strategy in a core layer, monitors and controls the whole process of data cleaning in an interaction layer, and inputs cleaning data parameters and rules through the interaction layer. Data are accessed from the input layer, and format conversion and metadata extraction are carried out on the input layer. The method comprises the following steps of: performing data cleaning through a core layer, adding a hierarchical reduction classification cleaning (HRSC) strategy into the core layer, outputting through an export layer after processing, and converting the cleaned data into a format required by a user. According to the HRSC strategy, the workload of data cleaning is reduced through hierarchical reduction, the difficulty of data cleaning is reduced, and the accuracy of data cleaning is ensured. The multi-source heterogeneous data cleaning problem of the diesel-electric hybrid power system is solved, and support is provided for fault diagnosis of the hybrid power system.

Description

Technical Field

[0001] The invention provides a HoloHRSC-based data cleaning method, relates to the field of data cleaning, and aims to solve the problem of cleaning multi-source heterogeneous data in a diesel-electric hybrid power system. Background Art

[0002] In hybrid power systems, data comes from various sources, including sensors, monitoring equipment, user interactions, etc. These data differ in format, structure and quality, forming multi-source heterogeneous data. These data may contain errors and missing values. If they are not cleaned, the accuracy and reliability of data analysis will be affected, and the effectiveness of system decision-making will be affected. Data cleaning is a key step in ensuring data quality. It can identify and correct errors in data, fill missing values, eliminate inconsistencies, and thus improve data availability. Cleaned data can provide the system with more accurate and reliable information, support more effective fault diagnosis, performance optimization and predictive maintenance, and is an important basis for realizing the intelligence and automation of hybrid power systems. Therefore, cleaning the multi-source heterogeneous data of hybrid power systems can ensure the accuracy and availability of data, thereby improving the operating efficiency and reliability of the entire system. HoloCleanEX has significant advantages in processing multi-source heterogeneous data of diesel-electric hybrid power systems. Its powerful data cleaning capabilities can effectively handle missing values, noise and abnormal data, and support the fusion of multi-source heterogeneous data to improve data consistency. HoloCleanEX also has some shortcomings. It uses data cleaning technology based on weakly supervised learning strategies. When processing a large amount of redundant data, it cannot reduce the redundancy of large data sets. It runs slowly when processing large-scale data. At the same time, its model has poor interpretability, and the cleaning and processing process is not transparent enough. In this context, a HoloHRSC data cleaning method was invented based on HoloCleanEX, which added a real-time feedback interaction platform based on Web applications, strengthened the transparency of the cleaning and processing process, and innovatively added a hierarchical reduction classification cleaning (HRSC) strategy to the core layer, which fundamentally reduced irrelevant or redundant data sources, reduced the workload of data cleaning, completed the construction of the tree augmented Bayesian TAN network through attribute weights, and used probability values ​​to complete the classification and cleaning of data in different fields, thereby improving the accuracy of data cleaning. The key data of the diesel-electric hybrid system has been cleaned, greatly reducing the amount of information of the key data of each component of the system, so that the status monitoring of the system is more effective. Summary of the invention

[0003] In view of the above problems, the present invention provides a data cleaning method based on HoloHRSC, which is used to solve the cleaning problem of multi-source heterogeneous data of hybrid power system.

[0004] In order to achieve the above object, the present invention adopts the following technical solution:

[0005] Based on HoloHRSC data cleaning technology, it includes the following contents: interaction layer, output layer, core layer and export layer.

[0006] For the interaction layer, a real-time interactive feedback platform based on Web applications is provided. The functions of this platform are as follows:

[0007] The interactive layer realizes real-time two-way communication between the server and the client. Users can observe the detailed parameters of each important component of the hybrid system in the interactive layer. Set parameters for data cleaning.

[0008] The system can monitor log files and capture key information of data processing in real time. Users can browse the data set to be cleaned and view the basic structure and properties of the data, such as column name, data type, null value ratio, duplicate value ratio, etc. Common data types include: numerical data (engine speed, battery charge and discharge times), structured data (tabular data), and sensor data (temperature sensor, pressure sensor).

[0009] The interaction layer usually contains a graphical user interface (GUI), which allows users to interact with the system through visual elements (such as buttons, drop-down menus, charts, etc.), which can be easily understood and operated even by users without professional programming or data science background.

[0010] Users customize their own data cleaning solutions based on business knowledge and data characteristics, and input corresponding parameters and rules through the interactive layer. After cleaning, the results of data cleaning can be viewed in the interactive layer, including the comparison before and after cleaning, the detection and correction of abnormal values. The cleaned data observed in the interactive layer can be transmitted to the status monitoring module of the diesel-electric hybrid system for real-time status monitoring, reducing the amount of data processing for fault diagnosis.

[0011] For the input layer, each key component of the diesel-electric hybrid system is equipped with a sensor, which will transmit the specific parameters of the key components to the input layer. The input layer is responsible for receiving and preprocessing multi-source heterogeneous data from different sources to prepare for subsequent data cleaning work. Ensure that the data can be correctly understood and processed by the system, and extract metadata information that is critical to the cleaning process.

[0012] Data is first collected and imported into the system. The data of the diesel-electric hybrid system comes from different databases, file systems or real-time data streams, such as the current and voltage signals of the motor, the temperature of the diesel engine, etc. The formats include structured database tables, semi-structured CSV or JSON files, and unstructured text data. The input layer needs to have flexible data access capabilities to adapt to various data sources and formats.

[0013] The purpose of the HoloHRSC data cleaning method is to process data sets from different sources and with different standards. Therefore, before formally cleaning the data, the input layer must standardize the data. In view of Python's powerful ecological environment and easy-to-use data structure, the DataFrame data structure under Python's pandas library is selected as the standardized data structure.

[0014] For the core layer, it is a key component based on the HoloHRSC data cleaning method. It is responsible for executing the core logic and algorithm of data cleaning, and also interacts with the Web platform. According to the requirements of state detection under different working conditions of the diesel-electric hybrid system, the interactive layer will give different cleaning parameters to the core layer. The present invention innovatively adds a hierarchical reduction classification cleaning (HRSC) strategy.

[0015] The HRSC strategy fundamentally reduces irrelevant or redundant data sources to reduce the workload of data cleaning; then, by constructing a TAN network for classification cleaning, the problems of large-scale data association, intersection and fusion in the data cleaning process are solved, the construction of the tree augmented Bayesian TAN network is completed through attribute weights, and the classification cleaning of data in different fields is completed using probability values ​​to improve the accuracy of data cleaning.

[0016] The steps of the HRSC strategy are as follows: First, the irrelevant or redundant data sources are reduced by measuring the importance of the data source, then the data attributes and tuples are weighted by data density, the core tuples and edge tuples are retained, and the outlier tuples are reduced. Finally, the inaccurate data is classified and cleaned using the constructed TAN network and data probability values, thereby improving the efficiency and accuracy of data cleaning.

[0017] The core of the HRSC strategy is to reduce the workload of data cleaning and the difficulty of data cleaning through the idea of ​​hierarchical reduction, while ensuring the accuracy of data cleaning. This method is particularly suitable for processing inaccurate data in multi-source heterogeneous data environments, and can improve the consistency, accuracy, authenticity and availability of big data.

[0018] The core layer integrates a variety of data cleaning algorithms, and will continuously adjust and optimize cleaning strategies and algorithm parameters based on data quality assessment results and user feedback to improve cleaning effects.

[0019] After the data cleaning is completed, the HoloHRSC data cleaning method will automatically try to restore the data structure based on the metadata information. The data cleaning part of the HoloHRSC data cleaning method is built on the flask framework, so that it can better provide API services for open access to the interactive layer web platform.

[0020] For the export layer, the restored data is saved to the same path as the input data, and subsequent operations are performed on the cleaned data.

[0021] The export layer uses APM (Application Performance Management) tools to monitor the performance indicators of data export, such as latency, throughput, etc. Detailed logs of data export are recorded, including operation time, operator, operation results, etc., to facilitate troubleshooting and auditing. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 System architecture for HoloHRSC data cleaning method

[0023] Figure 2 Technical dependencies for the improved HoloHRSC data cleaning method

[0024] Figure 3 is the data tuple density map

[0025] Figure 4 It is a relationship diagram of H(X), H(Y), I(X, Y), etc.

[0026] Figure 5 Constructing a graph for a TAN network DETAILED DESCRIPTION

[0027] The principle and specific process of the present invention are further explained in conjunction with the accompanying drawings.

[0028] The main principle of the HoloHRSC data cleaning method is to use the data lake architecture to integrate multi-source heterogeneous data, innovatively use the hierarchical reduction classification cleaning (HRSC) strategy at the core layer to improve the efficiency of data cleaning in a multi-source heterogeneous environment and reduce the complexity of data cleaning, apply statistical and machine learning methods for error detection and repair, and allow users to participate in the data cleaning process through interactive cleaning solutions to improve data quality and cleaning efficiency, and prepare for the condition monitoring of diesel-electric hybrid power systems.

[0029] Figure 2 As shown, the HoloHRSC-based data cleaning method is mainly divided into core layer, output layer, input layer, and interaction layer.

[0030] The interaction layer is built based on the Web application framework (React), using the Bootstrap open source front-end framework to design and customize the required platform, and using WebSocket to achieve real-time two-way communication between the server and the client. When the hybrid system is in different working conditions, users can directly modify the data cleaning parameters in the browser to make data cleaning more in line with the requirements under different working conditions.

[0031] Metadata visualization interaction uses a graphical user interface (GUI) to improve the comprehensibility of data and enhance users' data analysis capabilities through interactive operations.

[0032] The visualization comparison of the cleaning structure of the interaction layer uses the ECharts open source visualization tool.

[0033] The interaction layer exchanges data between the client and the server through the API to achieve real-time data updates.

[0034] like Figure 1 The input layer automatically obtains the input data selected by the user (such as the location of the file in HDFS) and the negative constraint information (parses the abstract data of the graphical interface into program text data that can be recognized by the standardized program). Because it is necessary to connect to different data sources and formats, the database uses JDBC (Java Database) connection.

[0035] File system (operation log, maintenance record), use file I / O operations to read CSV, JSON, text files.

[0036] API data, use the HTTP client library (Python's requests library) to send requests and receive data.

[0037] Real-time data stream, use message queue client (RabbitMQ, Kafka client) to subscribe and receive messages.

[0038] DataFrame is used as a standardized data structure, and Python's preprocessing algorithm is used, which parses each data type into a Spark DataFrame. In order to restore the structured data to its original format, the name of the parent node is retained in the column naming, and the column names of all levels are linked using ":" as a connector. In addition to structured data from data sources such as SQL Server or semi-structured data such as JSON, NoSQL data is also considered an important data source. Taking the column family database HBase as an example, in order to rebuild a structured table, ":" is used as a connector to connect the column family and the columns under the column family. Due to the uniqueness of the row, this column will not participate in the entire data cleaning process.

[0039] The core layer is the core of the HoloHRSC data cleaning method for data processing, but the basic HoloCleanEX method has a huge amount of computation, so the present invention innovatively adds a hierarchical reduction classification cleaning (HRSC) strategy to the core layer.

[0040] Taking the diesel-electric hybrid system data as an example, the data are: engine performance parameters, battery status, fuel consumption data, cooling system temperature, transmission gear system vibration, cylinder vibration, and motor performance. One of the data sets: U = {X1, X2, ..., X n , V}, V is the class variable (data classification), and its value range is: {v1, v2, ..., v m}, m is the total number of types, including types v1: engine speed, v2: motor speed, v3: engine torque, v4: cylinder vibration, v5: engine oil inlet temperature, v5: fuel consumption, v6: air flow, v7: transmission gear system vibration, v8: battery power. {x1, x2, …, x n} is {X1, X2, …, X n} Displays the attribute values ​​of the classification features (the data input from the hybrid powertrain state detection system), where n is the number of attributes of the classification. x1, x2, ..., x n A set of data showing features, such as a set of motor speeds or engine speeds, the training data set is Q = {x n+1 , x n+2 , …, x 2n}, this data set is composed of the attribute values ​​of the previously cleaned classification features.

[0041] The first step of the hierarchical reduction classification cleaning (HRSC) strategy is to measure the importance of the data source. The algorithm is as follows: define core tuples and boundary tuples, determine the number of core tuples and the number of boundary tuples, and mark the weights of the tuples; then calculate the importance of the data attribute set based on the weight labels of the tuples, and mark the weights of the attributes; finally, return the attribute weights of the data, that is, the importance of the data.

[0042] In some data space S of the diesel-electric hybrid system database D, given a set of tuples Tu, set the object set O to be dense in the data space S, with P min The minimum number of objects that exist, given the radius of a neighborhood, is represented by ε. When a tuple is in a dataset, the number of objects contained in the neighborhood ε is at least P. min , it is called a core tuple, otherwise it is called a non-core tuple. Figure 1 In the example, when P min When =3, tuple P1 is the core tuple, tuple P2 is the boundary tuple, and tuple P3 is the outlier tuple.

[0043] The weight w(t k ) is the number of core tuples k divided by I C The total number of tuples: The weight w(t b ) is the number of boundary tuples b divided by I CThe total number of tuples: The weight w(C) of a data attribute is the weighted sum of the core tuples plus the sum of the weights of the boundary tuples representing existence: δ is the weight coefficient. The value of δ usually depends on the following factors: data distribution, business requirements, and model sensitivity. The value of δ may vary from 0.1 to 10 or more, depending on the above factors. In the diesel-electric hybrid system, the value of δ is 5. Finally, the importance of each set of data attributes C is returned (W, C) Through weight labeling, the HRSC strategy further narrows the scope of data cleaning and focuses on areas where similar duplicate data may exist, thereby improving the efficiency of data cleaning. In this way, the amount of data input into the state detection system from the interaction layer is greatly reduced, improving the efficiency of state monitoring.

[0044] The second step is to train and construct the TAN network using the training sample set. The TAN network is a tree-enhanced naive Bayes classifier that represents the dependency relationship between features by constructing a tree structure.

[0045] The steps of HRSC algorithm based on TAN network are as follows: Input: Source data attribute data set D(0) = {x1, x2, ..., x n}, The training sample set Q = {x n+1 , x n+2 , …, x 2n}, Output: TAN network node ① Sampling the source data attribute data set D(0); ② Reduce the data level of the sampled data according to the importance measurement algorithm; ③ According to the TAN network construction algorithm, the training sample set Q is trained to construct the TAN network; ④Use the training sample data set and use the mutual information value to score the constructed TAN network; ⑤Use the hierarchical reduction data set and score the constructed TAN network using the mutual information value; ⑥ Determine the results of step ④ and step ⑤. If the result of step ④ is greater than the result of step ⑤, return to step ③ until the result of step ⑤ is greater than the result of step ④. ⑦ Use ES to sort the mutual information values ​​of TAN network nodes in descending order according to expected correlation; ⑧ Determine the Top-k mutual information value I(X i , X j ) and empirical parameter values, H(X i ) measures the random variable X i The larger the entropy value, the greater the uncertainty of the random variable. i , x j ) is the variable (X i , X j ), p(x i ), p(x j ) are marginal distributions respectively. Mutual information refers to the degree of association between two random variables. In the diesel-electric hybrid system, the empirical parameter value of each type of data is taken as 0.5. When calculating mutual information, the degree of association between two random variables is given equal weight. This setting helps to balance the influence of different data sources and data types during data cleaning, and avoid excessive dominance of one type of data in the cleaning result. If the Top-k mutual information value I(X i , X j ) is less than the empirical parameter value, remove the top-level node, and query the score R of the result of removing the top-level node; ⑨ Sort the TAN network nodes in descending order according to the score R and output them.

[0046] Construction of TAN network: First, the weights between TAN network attributes are constructed by calculating the mutual information between the attributes of the variables. Mutual information refers to the degree of association between two random variables, that is, the degree to which the uncertainty of another random variable is weakened after a random variable is given, such as Figure 3 As shown, according to the entropy chain rule, we have H(X;Y)=H(X)+H(Y|X)=H(Y)+H(X|Y), Therefore, H(X)-H(X|Y)=H(Y)-H(Y|X) This difference is recorded as I(X; Y), and the mutual information I(X; Y) is defined as: Among them, p(x, y) is the joint distribution of variables (X, Y), and p(x) and p(y) are marginal distributions respectively. Figure 3 It is a relationship diagram of H(X), H(Y), I(X, Y), etc. The calculation of the probability of the classification result is to pass the query condition attributes into the classification model, and then calculate the probability values ​​under different class attributes. The classification attribute value with the maximum probability value is the final classification result, that is, the set with the largest probability of inaccurate data, that is, calculate its joint probability distribution: P(v i|argmax(v i ))This is the conditional probability, which means that given the classification result argmax(v i ), a feature v i The conditional probability of . argmax means selecting the class that maximizes the probability. In other words, it calculates the probabilities of different classes and selects the class with the largest probability value. First, according to the category predicted by the model (determined by maximizing the probability), all feature values ​​v1, v2, ...v are calculated under this predicted category. n The respective conditional probabilities. The mutual information between attributes is the correlation between attributes. Different attribute correlation values ​​are calculated differently. Considering the addition of TAN network class variable attributes, the mutual information formula of a certain classification attribute needs to be redefined. Therefore, the calculation formula of the mutual information of TAN network is: where v i , v j is the attribute variable, V is the class variable. Sort I(Xi, Xj) in descending order and output the node pairs in sequence. The multi-source heterogeneous data of the hybrid system is reduced by measuring the importance of the data source to reduce irrelevant or redundant data sources, weighting the data attributes and tuples, retaining the core tuples and edge tuples, reducing the outlier tuples, and using the TAN network and data probability values ​​to classify and clean the inaccurate data, which will greatly reduce the amount of data input to the interaction layer, thereby improving the efficiency and accuracy of data cleaning.

[0047] like Figure 2 The core layer also integrates a variety of data cleaning algorithms. The anomaly detection algorithm uses cluster analysis methods: K-Means, which is used to identify abnormal groups in the data.

[0048] The data correction and interpolation algorithm uses K nearest neighbor (KNN) to handle missing values ​​and polynomial interpolation to supplement missing values.

[0049] The core layer uses support vector machines (SVM) to find the boundaries in the data and distinguish between consistent and inconsistent data.

[0050] The HoloHRSC data cleaning method completes the main module of data cleaning on an independent server, including connecting and manipulating the SQL engine, various statistical calculations, and implementing a lightweight weight-sharing linear classification model based on pytorch. After the data cleaning is completed, the data structure is automatically restored based on the metadata information. The data cleaning part of the HoloHRSC data cleaning method is built on the flask framework, so that it can better provide API services for open access to the interactive layer Web platform.

[0051] like Figure 1 The export layer converts the cleaned data into a specific format required by the user or downstream system, JSON (which uses JavaScript Object Notation to convert data structures into text format).

[0052] At the beginning of the data cleaning work, HoloCleanEX will record the channel type of the category to which the data source belongs, covering data formats such as SQL, CSV, JSON, HBase, XML, Pandas DataFrame (Pandas.DF) and Spark DataFrame (Spark.DF). After the data cleaning is completed, the system will restore the data to its original format based on the recorded metadata information (such as the data storage path, file name, and type of data at each layer). If the original data is in CSV format, the cleaned data will be converted back to CSV format and saved to the specified storage path. It will also be compared with the input layer at the interaction layer to check whether the data is correctly restored to its original format. Data conversion processing steps: SQL / CSV format For structured data, no further processing is required. The Pandas library can be used to directly convert DataFrame to the corresponding format, such as exporting data to CSV or Excel files through the to_csv() or to_excel() methods. DataFrame (DF) format When receiving data of the DataFrame type, all DataFrames are converted to PandasDataFrame (Pandas.DF) for subsequent data format conversion in PyTorch. When returning data, the system uses the tools provided by Pandas to convert the format. JSON / HBase / XML format These formats often contain nested data. During data processing, the system parses the multi-layer nested structure into a single-layer structure and concatenates the column names with colons (:). When restoring the data, the system restores the original data format by splitting the colons (:) and recursively building a tree structure. This process simulates the tree structure through Python's dictionary data structure and converts the generated multi-layer dictionary back to the original nested data format. HBase and XML Data For data in HBase and XML formats, when the system converts each row into dictionary data, it can directly convert or store it using the tools that Python has encapsulated. For example, Python.happybase can recognize colons (:) in dictionaries and insert data into the newly created database through the put() method; and the ParseString tool in Python.xml.dom.minidom can restore the parsed nested data to the original XML format. The input data is cleaned by the core layer and restored by the export layer to reduce the amount of data and remove irrelevant data. The exported data set is: U1 = {X1, X2, ..., X n , V}, V is the class variable (data classification), and its value range is: {v1, v2, ..., v m}, m is the total number of types, including types v1: engine speed, v2: motor speed, v3: engine torque, v4: cylinder vibration, v5: engine oil inlet temperature, v5: fuel consumption, v6: air flow, v7: transmission gear vibration, v8: battery charge. {x1, x2, …, x n} is {X1, X2, …, X n} Displays the attribute values ​​of the classification features (the data input from the hybrid powertrain state detection system), where n is the number of attributes of the classification. x1, x2, ..., x n The data for displaying features is different from the input data set U in that the data volume is smaller and useless data is removed.

[0053] The interaction layer integrates data from different sources by using ETL (extract, transform, load) technology for unified management and analysis

[0054] The interaction layer is built on the Lakehouse architecture, combining the advantages of data lakes and data warehouses to provide a storage solution that supports both unstructured and structured data.

[0055] The undisclosed parts of the present invention belong to common knowledge.

Claims

1. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid system, characterized in that: Contains interaction layer, input layer, core layer, and export layer; The interaction layer provides a real-time interactive feedback platform based on Web applications, realizing real-time two-way communication between the server and the client; The input layer is responsible for receiving and preprocessing multi-source heterogeneous data from different sources to prepare for subsequent data cleaning work; many data cleaning algorithms are integrated in the core layer, and hierarchical reduction and classification cleaning strategies are also added; The export layer uses application performance management (APM) tools to monitor data export performance indicators, including latency and throughput.

2. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The visualization comparison of the cleaning structure of the interaction layer uses the ECharts open source visualization tool.

3. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The Data Frame is used as a normalized data structure and the preprocessing algorithm of MHDP is used, which parses each data type into a Spark Data Frame.

4. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: Through data level reduction, data attributes and tuples are weighted, core tuples and boundary tuples are retained according to the weights, and outlier tuples are reduced, which greatly reduces the workload of data cleaning.

5. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The construction of the tree augmented Bayesian TAN network is completed through attribute weights for data classification and cleaning, and the TAN network is used to complete the classification and cleaning of imprecise data and precise data.

6. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The core layer anomaly detection algorithm uses a cluster analysis method: K-Means, which is used to identify abnormal groups in the data.

7. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The data correction and interpolation algorithm uses K nearest neighbor (KNN) to handle missing values ​​and polynomial interpolation to supplement missing values.

8. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The core layer uses support vector machines (SVM) to find the boundaries in the data and distinguish between consistent and inconsistent data.

9. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: The export layer converts the cleaned data into the format required by the user or downstream system, and uses JavaScript object notation to convert the data structure into text format; the interaction layer integrates data from different sources by using ETL, namely extraction, transformation, and loading.

10. A multi-source heterogeneous data cleaning method for a diesel-electric hybrid power system as claimed in claim 1, characterized in that: In some data space S of the diesel-electric hybrid system database D, given a set of tuples Tu, set the object set O to be dense in the data space S, with P min The minimum number of objects that exist, given the radius of the neighborhood, is represented by ε; when a tuple is in a dataset, the number of objects contained in the neighborhood ε is at least P min When , it is called a core tuple, otherwise it is called a non-core tuple, and the one at the boundary is a boundary tuple; The weight w(t k ) is the number of core tuples k divided by I C The total number of tuples: The weight w(t b ) is the number of boundary tuples b divided by I C The total number of tuples: The weight w(C) of a data attribute is the weighted sum of the core tuples plus the sum of the weights of the boundary tuples representing presence: δ is the weight coefficient, and the value of δ is 5; Finally, the importance (W, C) of each set of data attributes C is returned; The second step is to train and construct the TAN network using the training sample set; the TAN network is a tree-enhanced naive Bayes classifier; The steps of HRSC algorithm based on TAN network are as follows: Input: Source data attribute data set D(0) = {x1, x2, ..., x n }, The training sample set Q = {x n+1 ,x n+2 ,…,x 2n }, Output: TAN network node ① Sampling the source data attribute data set D(0); ② Reduce the data level of the sampled data according to the importance measurement algorithm; ③ According to the TAN network construction algorithm, the training sample set Q is trained to construct the TAN network; ④Use the training sample data set and use the mutual information value to score the constructed TAN network; ⑤Use the hierarchical reduction data set and score the constructed TAN network using the mutual information value; ⑥ Determine the results of step ④ and step ⑤. If the result of step ④ is greater than the result of step ⑤, return to step ③ until the result of step ⑤ is greater than the result of step ④. ⑦ Use ES to sort the mutual information values ​​of TAN network nodes in descending order according to expected correlation; ⑧ Determine the Top-k mutual information value I(X i ,X j ) and empirical parameter values, H(X i ) measures the random variable X i The larger the entropy value, the greater the uncertainty of the random variable; p(x i ,x j ) is the variable (X i ,X j ), p(x i ), p(x j ) are marginal distributions respectively; Mutual information refers to the degree of association between two random variables. In the diesel-electric hybrid system, the empirical parameter value of each type of data is taken as 0.

5. If the Top-k mutual information value I(X i ,X j ) is less than the empirical parameter value, remove the top-level node, and query the score R of the result of removing the top-level node; ⑨ Sort the TAN network nodes in descending order according to the score R; Construction of TAN network: First, the weights between TAN network attributes are constructed by calculating the mutual information of attributes between variables; Mutual information refers to the degree of association between two random variables, that is, the degree to which the uncertainty of another random variable is weakened after a random variable is given. According to the entropy chain rule, there is H(X;Y)=H(X)+H(Y|X)=H(Y)+H(X|Y), Therefore, H(X)-H(X|Y)=H(Y)-H(Y|X) This difference is recorded as I(X; Y), and the mutual information I(X; Y) is defined as: Among them, p(x,y) is the joint distribution of variables (X,Y), p(x) and p(y) are marginal distributions respectively; Calculate the joint probability distribution: P(v i |argmax(v i ))This is the conditional probability, which means that given the classification result argmax(v i ), a feature v i The conditional probability of ; argmax means selecting the class that maximizes the probability; The predicted category is determined by maximizing the probability, and then all feature values ​​v1, v2, ...v are calculated under this predicted category. n their respective conditional probabilities; The calculation formula of mutual information of TAN network is: where v i , v j is the attribute variable, V is the class variable; sort I(Xi,Xj) in descending order and output the node pairs in sequence; Exported data set: U1 = {X1, X2, …, X n ,V}, V is the class variable, i.e. the classification of data, and its value range is: {v1,v2,…,v m }, m is the total number of types, including types v1: engine speed, v2: motor speed, v3: engine torque, v4: cylinder vibration, v5: engine oil inlet temperature, v5: fuel consumption, v6: air flow, v7: transmission gear system vibration.

Images