Efficient data processing platform and method for industrial internet

Abstract

The invention relates to the technical field of efficient data processing platforms for the industrial internet, and discloses an efficient data processing platform and method for the industrial internet, and the platform comprises a data collection module, a data preprocessing module, a data storage module, a data analysis module and a data visualization module. The data acquisition module acquires data from industrial equipment in real time; the data preprocessing module is used for cleaning, deduplication and formatting the received data; the data storage module stores the preprocessed data in a distributed database; the data analysis module uses a real-time data analysis engine to quickly analyze and process real-time data; the data visualization module uses a chart generator, a report generator and a visualization interface to display the analysis result in the form of charts and reports, and provides user interaction.

Description

Technical Field

[0001] This invention relates to the field of high-efficiency data processing platform technology for the Industrial Internet, specifically to a high-efficiency data processing platform and method for the Industrial Internet. Background Technology

[0002] In today's digital age, the Industrial Internet, acting as a bridge connecting industrial systems and the internet, is transforming the manufacturing industry at an unprecedented pace. By deploying sensors and networking technologies on industrial equipment, the Industrial Internet enables interconnectivity between devices, generating massive amounts of industrial data. This data contains immense value, helping companies optimize production processes, improve equipment utilization, reduce maintenance costs, and drive product innovation.

[0003] However, to fully unlock the value of industrial data, a series of challenges in the data processing process must first be addressed. These challenges mainly include: Real-time and accurate data acquisition: Industrial equipment comes in a wide variety of forms and uses different data formats and transmission protocols. How to acquire this data in real time and accurately is a key issue.

[0004] Data heterogeneity and complexity: Industrial data is often heterogeneous and complex, including structured data, semi-structured data and unstructured data. How to effectively process these different types of data is a challenge.

[0005] Data cleaning and preprocessing: Raw data often contains noise, missing values, and outliers, which can affect the accuracy of data analysis and modeling. Therefore, data cleaning and preprocessing are necessary.

[0006] Scalability of data storage and management: With the continuous development of the Industrial Internet, the amount of data is growing exponentially. How to store and manage this massive amount of data and ensure the scalability of the system is an important issue.

[0007] Efficiency of data analysis and mining: Industrial data is usually high-dimensional and time-sensitive. How to efficiently analyze and mine this data to extract valuable information and knowledge is the key to improving the level of industrial intelligence.

[0008] To address these challenges, traditional data processing methods and platforms are no longer sufficient to meet the needs of the Industrial Internet. Therefore, there is an urgent need for an efficient data processing platform and methodology for the Industrial Internet to solve the aforementioned problems and fully unlock the value of industrial data. Summary of the Invention

[0009] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides an efficient data processing platform and method for the Industrial Internet, solving the problems mentioned in the background section.

[0010] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: a high-efficiency data processing platform and method for the industrial internet, the platform including a data acquisition module, a data preprocessing module, a data storage module, a data processing module, and a data visualization module; The data acquisition module is used to collect industrial data in real time through sensors and controller devices in the Industrial Internet. The data preprocessing module is used to perform cleaning, noise reduction, and normalization preprocessing operations on the collected data. The data storage module is used to store the preprocessed data in a distributed data storage system. The data processing module is used to process data stored in the distributed data storage system in parallel using a distributed computing framework. The data visualization module is used to visualize the processed data in the form of charts and reports.

[0011] Preferably, the data acquisition module further includes: a wireless sensor network interface for real-time acquisition of operating status data of industrial equipment via a wireless sensor network; the data preprocessing module further includes: an outlier detection and removal module for outlier detection and removal of the acquired data; the data storage module further includes: a distributed file system for storing the preprocessed data to achieve high availability and fault tolerance; the data processing module further includes: a machine learning algorithm module for intelligent analysis and prediction of the data stored in the distributed data storage system; and the data visualization module further includes: a web front-end display module for displaying the processed data in the form of interactive charts and reports using web front-end technology.

[0012] Preferably, the data storage module further includes: a distributed file system for storing preprocessed data to achieve high availability and fault tolerance; the data processing module further includes: a machine learning algorithm module for intelligent analysis and prediction of data stored in the distributed data storage system; and the data visualization module further includes: a web front-end display module for displaying the processed data in the form of interactive charts and reports using web front-end technology.

[0013] Preferably, the machine learning algorithm module in the data processing module includes: Feature selection submodule: used to automatically select the features that have the most impact on the prediction of the target variable from the preprocessed data. Feature selection methods include statistical feature selection, model-based feature selection, and embedded feature selection. Model Training Submodule: Used to train machine learning models by leveraging the correlation between selected features and target variables. Model types include linear regression models, support vector machine models, decision tree models, random forest models, and neural network models. Model Evaluation Submodule: Used to evaluate the performance of the trained machine learning model. Evaluation metrics include mean squared error, coefficient of determination, accuracy, recall, and F1 score. Model optimization submodule: Used to optimize machine learning models based on model evaluation results. Optimization methods include, but are not limited to, hyperparameter tuning, feature engineering, and model fusion.

[0014] Preferably, the platform further includes: Edge computing nodes: Deployed near the data source, they are used to perform preliminary cleaning, filtering, and feature extraction on the collected raw data; they work in conjunction with the data processing module to preprocess the data according to preset rules or models, and send the processed data to the distributed data storage system; they run lightweight machine learning models to perform real-time analysis and anomaly detection on the data, and send the analysis results to the data processing module.

[0015] Preferably, the platform further includes: Data security and privacy protection module: Encrypts data stored in the distributed data storage system to ensure data security during storage and transmission; implements access control policies to restrict unauthorized users or devices from accessing the data; and employs differential privacy and federated learning privacy protection technologies to perform data analysis and model training while ensuring data privacy.

[0016] Preferably, the platform further includes: Intelligent Operation and Maintenance and Monitoring Module: Real-time monitoring of the operational status of each component of the data processing platform, including hardware resource utilization and software service status; employing anomaly detection algorithms to provide early warnings and diagnose abnormal situations during platform operation; ensuring the stability and reliability of the platform through automated fault recovery and resource scheduling operations; and providing a visual operation and maintenance interface for easy management and maintenance by operation and maintenance personnel.

[0017] Preferably, this efficient data processing method includes: Step 1: Data Acquisition: Real-time industrial data is acquired through sensors and controllers in the Industrial Internet, and the acquired data is transmitted to the data processing platform. Step 2: Data Preprocessing: The collected data undergoes cleaning, noise reduction, and normalization preprocessing to improve data quality. The formula for data normalization is as follows: in The original data, For the normalized data, and These are the minimum and maximum values ​​of the data, respectively. Step 3: Data storage: Store the preprocessed data in a distributed data storage system to achieve fast data access and efficient management; Step 4: Data Processing: A distributed computing framework is used to process the data stored in the distributed data storage system in parallel. The data processing formula is as follows: Linear regression formula: Y = aX + b, where X is the independent variable, Y is the dependent variable, and a and b are regression coefficients; Time series forecasting formula: Y(t)=f(Y(t-1),Y(t-2),...,Y(tn)), where Y(t) is the predicted value at the current time, Y(t-1),Y(t-2),...,Y(tn) are the values ​​at historical times, and f is the forecast function; Step 5: Data Visualization: Visualize the processed data in the form of charts and reports so that users can intuitively understand the data information and operating status in the Industrial Internet.

[0018] Preferably, the data acquisition in step one further includes: collecting the operating status data of industrial equipment in real time through the wireless sensor network in the Industrial Internet, and transmitting the collected data to the data processing platform via wireless transmission; the data preprocessing in step two further includes: detecting and removing outliers from the collected data to eliminate the impact of outliers on subsequent data processing.

[0019] Preferably, the data storage in step three further includes: storing the preprocessed data using a distributed file system to achieve high availability and fault tolerance of the data; the data processing in step four further includes: using machine learning algorithms to intelligently analyze and predict the data stored in the distributed data storage system to mine the potential value in the data.

[0020] Preferably, the data visualization in step five further includes: displaying the processed data in the form of interactive charts and reports using web front-end technology, so that users can easily query and analyze the data; the data analysis steps include: analyzing and mining the data using a machine learning algorithm library; and quickly analyzing and processing real-time data using a real-time data analysis engine.

[0021] (III) Beneficial Effects Compared with existing technologies, this invention provides a high-efficiency data processing platform and method for the Industrial Internet, which has the following beneficial effects: 1. Improved real-time performance and accuracy: The data acquisition module provided by this invention can collect data from various industrial equipment in real time and accurately, ensuring the timeliness and accuracy of the data and providing a reliable data foundation for subsequent data processing and analysis.

[0022] 2. Significantly improved data processing efficiency: The data preprocessing module used in this invention can efficiently clean, deduplicatize, and format the collected data, greatly reducing data noise and redundancy, improving data quality, and providing high-quality data input for subsequent data analysis and mining.

[0023] 3. Strong scalability of storage management: This invention uses a distributed database for data storage, which can flexibly expand storage resources according to the growth of data volume, effectively meet the storage needs of massive data in the industrial Internet, and ensure the scalability and stability of the system.

[0024] 4. Enhanced Data Analysis Capabilities: The data analysis module provided by this invention integrates a machine learning algorithm library and a real-time data analysis engine, enabling in-depth analysis and mining of data stored in the database, quickly extracting valuable information and knowledge, and providing strong support for industrial decision-making.

[0025] 5. Visualization effect is intuitive and easy to use: The visualization module provided by this invention can intuitively display the analysis results in the form of charts, reports and other forms, and provide user interaction through the visualization interface, so that users can more easily understand and use the analysis results, thereby improving the efficiency of data utilization.

[0026] 6. Reduced overall costs: Through the efficient data processing platform and method of this invention, enterprises can optimize production processes, improve equipment utilization, reduce maintenance costs, and promote product innovation, thereby reducing overall costs and enhancing market competitiveness.

[0027] 7. Enhanced intelligence level: This invention combines advanced technologies such as cloud computing, big data, and artificial intelligence to achieve automatic data cleaning, preprocessing, and intelligent analysis, thereby enhancing the intelligence level of the industrial internet and promoting the transformation and upgrading of industrial automation to industrial intelligence. Attached Figure Description

[0028] Figure 1 This is a schematic diagram of the overall system architecture of the present invention; Figure 2 This is a graph showing experimental data for evaluating the predictive maintenance effectiveness of the present invention. Detailed Implementation

[0029] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0030] This efficient data processing platform and method for the Industrial Internet includes a data acquisition module, a data preprocessing module, a data storage module, a data processing module, and a data visualization module. Data acquisition module: Used to acquire industrial data in real time through sensors and controller devices in the Industrial Internet; Data preprocessing module: used to perform cleaning, noise reduction, and normalization preprocessing operations on the collected data; Data storage module: Used to store preprocessed data in a distributed data storage system; Data processing module: Used to perform parallel processing of data stored in a distributed data storage system using a distributed computing framework; Data visualization module: used to visualize processed data in the form of charts, reports, etc. The steps of this method are as follows: Step 1: Data Acquisition: Real-time industrial data is acquired through sensors and controllers in the Industrial Internet, and the acquired data is transmitted to the data processing platform. Step 2: Data Preprocessing: The collected data undergoes cleaning, noise reduction, and normalization preprocessing to improve data quality. The formula for data normalization is as follows: in The original data, For the normalized data, and These are the minimum and maximum values ​​of the data, respectively. Step 3: Data storage: Store the preprocessed data in a distributed data storage system to achieve fast data access and efficient management; Step 4: Data Processing: A distributed computing framework is used to process the data stored in the distributed data storage system in parallel. The data processing formula is as follows: The linear regression formula is: Y = aX + b, where X is the independent variable, Y is the dependent variable, and a and b are the regression coefficients.

[0031] Time series forecasting formula: Y(t)=f(Y(t-1),Y(t-2),...,Y(tn)), where Y(t) is the predicted value at the current time, Y(t-1),Y(t-2),...,Y(tn) are the values ​​at historical times, and f is the forecast function; Step 5: Data Visualization: Visualize the processed data in the form of charts and reports so that users can intuitively understand the data information and operating status in the Industrial Internet. Specific Implementation Example 1

[0032] In this embodiment, we designed and implemented a high-efficiency data processing platform for the Industrial Internet, which includes a data acquisition module, a data preprocessing module, a data storage module, a data analysis module, and a data visualization module.

[0033] 1. Data Acquisition Module: This module is responsible for acquiring data from industrial equipment in real time. Specifically, sensors can be installed at various key locations within the industrial equipment to collect operational status data, such as temperature, pressure, and vibration. These sensors transmit the collected data to the data acquisition module via wireless networks (such as Wi-Fi, LoRa, Zigbee, etc.) or wired networks (such as Ethernet). The data acquisition module performs preliminary filtering and buffering of the received data and then sends it to the data preprocessing module.

[0034] 2. Data Preprocessing Module: This module cleans, deduplicates, and formats the received data. Specifically, data cleaning algorithms remove noise and outliers, deduplication algorithms remove duplicate records, and formatting algorithms convert the data into a uniform format. The preprocessed data is then stored in the data storage module.

[0035] 3. Data Storage Module: This module stores the preprocessed data in a distributed database. In this embodiment, we use the Hadoop Distributed File System (HDFS) to store massive amounts of data and HBase as the distributed database to manage the data. HBase is a Hadoop-based distributed database that provides highly reliable, high-performance, and scalable data storage services.

[0036] 4. Data Analysis Module: This module is used to analyze and process the data stored in the database. Specifically, we can use machine learning algorithm libraries (such as Scikit-learn, TensorFlow, etc.) to analyze and mine the data, and use real-time data analysis engines (such as Apache Spark) to quickly analyze and process real-time data. Through data analysis, we can extract valuable information and knowledge to support industrial decision-making.

[0037] 5. Data Visualization Module: This module is used to visually display analysis results in the form of charts, reports, etc., and provides user interaction through a visual interface. In this embodiment, we used the ECharts chart library to generate various types of charts, the JasperReports report generator to generate reports, and a web-based visualization interface to display the charts and reports. Users can interact with the data through the visualization interface, such as querying, filtering, and sorting.

[0038] Through the above specific embodiments, we can see that the efficient data processing platform and method provided by the present invention can effectively solve the data processing problems in the Industrial Internet and provide strong support for industrial decision-making. Of course, the present invention is not limited to the above embodiments and can be appropriately modified and extended according to actual needs. Specific Implementation Example 2

[0039] In this embodiment, we will focus on the missing value handling and outlier handling methods in the data preprocessing module, as well as the predictive maintenance algorithm in the data analysis module.

[0040] Data preprocessing module: a. Missing Value Handling: During industrial data acquisition, data may be missing due to sensor malfunctions, network interruptions, etc. This embodiment uses a combination of mean imputation and median imputation to handle missing values. The specific steps are as follows: I. First, perform missing value detection on each feature and count the number and proportion of missing values.

[0041] II. For features with a missing value ratio less than a certain threshold (e.g., 10%), the mean imputation method is used. That is, the average of the non-missing values ​​of the feature is calculated, and the missing values ​​are imputed with the average value.

[0042] III. For features with a missing value ratio greater than or equal to a certain threshold (e.g., 10%), the median imputation method is used. That is, the median of the non-missing values ​​of the feature is calculated, and the missing values ​​are imputed using the median.

[0043] b. Outlier Handling: Outliers are values ​​that deviate significantly from the majority of data points, possibly due to measurement errors, equipment malfunctions, or other reasons. This embodiment uses an IQR (interquartile range)-based method to handle outliers. The specific steps are as follows: I. First, perform statistical description for each feature, calculate the first quartile (Q1) and the third quartile (Q3), as well as the interquartile range IQR (IQR = Q3 - Q1).

[0044] II. Next, determine the range of outliers. Typically, outliers are defined as values ​​less than Q1 - 1.5IQR or greater than Q3 + 1.5IQR.

[0045] III. Finally, process the data that falls within the outlier range. You can choose to delete outliers or replace them with the median or mean of that feature.

[0046] Data Analysis Module: a. Predictive Maintenance Algorithm: Predictive maintenance is an important application in the Industrial Internet. It analyzes equipment operating data to predict the probability of future equipment failures, thereby enabling proactive maintenance and avoiding losses caused by equipment malfunctions. This embodiment employs a predictive maintenance method based on machine learning algorithms. The specific steps are as follows: I. First, collect historical operating data of the equipment, including equipment status data, maintenance records, fault records, etc.

[0047] II. Then, the data is preprocessed, including handling missing values, handling outliers, and feature engineering.

[0048] III. Next, select a suitable machine learning algorithm, such as Support Vector Machine (SVM) or Random Forest, to build a prediction model.

[0049] IV. Finally, predictive models are used to forecast the future operating status of the equipment, and maintenance plans are developed based on the forecast results.

[0050] Through the above specific embodiment 2, we can see that the efficient data processing platform and method provided by this invention have unique advantages in data preprocessing and data analysis. Effective methods for handling missing and outlier values ​​can improve data quality, providing a reliable data foundation for subsequent data analysis. Furthermore, predictive maintenance algorithms based on machine learning can help enterprises achieve intelligent equipment management, reduce maintenance costs, and improve equipment utilization. Specific Implementation Example 3

[0051] In this embodiment, we will describe in detail the distributed database design in the data storage module and the interactive visualization interface design in the data visualization module.

[0052] Data storage module: a. Distributed Database Design: To meet the storage needs of massive amounts of data in the Industrial Internet, this embodiment uses distributed database technology to store the preprocessed data. The specific steps are as follows: I. First, based on the characteristics and access frequency of the data, the data is divided into three categories: hot data, warm data, and cold data. Hot data refers to data that is accessed frequently, such as real-time production data; warm data refers to data that is accessed less frequently, such as historical production data; and cold data refers to data that is rarely accessed, such as backup data.

[0053] II. Next, select the appropriate storage media for different types of data. Hot data is stored on high-performance solid-state drives (SSDs) to provide fast data access speeds; warm data is stored on hard disk drives (HDDs) to balance cost and performance; cold data is stored in lower-cost archive storage systems.

[0054] III. Next, design the table structure of the distributed database, including data tables, index tables, and metadata tables. Data tables are used to store the actual data records, index tables are used to improve data query efficiency, and metadata tables are used to store descriptive information about the data, such as data dictionaries and data flow information.

[0055] IV. Finally, implement the partitioning and backup strategies for the distributed database. The partitioning strategy distributes data across different database nodes based on the data's key values ​​to achieve load balancing and parallel processing; the backup strategy performs periodic backups to prevent data loss.

[0056] Data visualization module: a. Interactive Visual Interface Design: To provide a better user experience, this embodiment designs an interactive visual interface through which users can interact with data, such as querying, filtering, and sorting. The specific steps are as follows: I. First, design the layout of the visual interface, including the menu bar, toolbar, data display area, and control panel. The menu bar provides the system's function menu, the toolbar provides commonly used tool buttons, the data display area is used to display charts and reports, and the control panel is used to provide data filtering and sorting operations.

[0057] II. Then, use chart libraries (such as D3.js, ECharts, etc.) to implement various types of charts, such as line charts, bar charts, pie charts, scatter plots, etc. Users can choose different chart types to display data as needed.

[0058] III. Next, implement the dynamic loading and updating function for data. When users perform data filtering or sorting operations, the visualization interface can update charts and reports in real time to reflect the latest data status.

[0059] IV. Finally, a data export function is provided, allowing users to export charts and reports as images or PDF files for offline viewing and sharing.

[0060] Through the above specific embodiment 3, we can see that the high-efficiency data processing platform and method provided by the present invention have unique advantages in data storage and visualization. Through distributed database design, massive amounts of data can be effectively stored and managed, improving data access efficiency; through interactive visualization interface design, a better user experience can be provided, helping users to better understand and analyze data.

[0061] Specific Implementation Example 4: Industrial Internet Data Processing Platform Based on Edge Computing and Federated Learning Based on the above claims, this embodiment will detail a specific implementation of an industrial internet data processing platform that combines edge computing and federated learning.

[0062] The industrial internet data processing platform in this embodiment includes the following main components: Edge computing nodes: Deployed near data sources such as factory workshops and production lines, they are responsible for collecting data from on-site equipment and performing preliminary cleaning, filtering, and feature extraction. Each edge computing node runs a lightweight machine learning model for real-time data analysis and anomaly detection.

[0063] Edge server: Responsible for aggregating data sent from multiple edge computing nodes and performing the following operations: Further feature engineering can be performed on the data, such as feature selection and dimensionality reduction.

[0064] By using a federated learning framework, various edge computing nodes can be coordinated to jointly train a global machine learning model without uploading data to a central server, thereby protecting data privacy.

[0065] The trained global model is distributed to various edge computing nodes for real-time prediction and decision-making.

[0066] Central cloud server: Responsible for storing and managing all data, and performing the following operations: Long-term storage and analysis of data sent from edge servers.

[0067] Run complex machine learning models to perform deeper data mining and prediction.

[0068] It provides data visualization and management functions, making it easy for users to access and analyze data.

[0069] Workflow The workflow of the industrial internet data processing platform in this embodiment is as follows: 1. Data Acquisition: Edge computing nodes collect field data, such as temperature, pressure, and vibration, through devices such as sensors and PLCs.

[0070] 2. Edge preprocessing: Edge computing nodes perform preprocessing operations such as cleaning and denoising on the collected data, and extract key features.

[0071] 3. Edge Analytics: Edge computing nodes utilize locally running lightweight machine learning models to perform real-time data analysis and anomaly detection. For example, they can detect whether a device is showing signs of malfunction.

[0072] 4. Data Upload: Edge computing nodes send the processed data to the edge server.

[0073] 5. Federated Learning: Edge servers utilize a federated learning framework to coordinate various edge computing nodes to jointly train a global machine learning model. For example, a model can be trained to predict the remaining lifespan of a device.

[0074] 6. Model distribution: The edge server distributes the trained global model to various edge computing nodes.

[0075] 7. Real-time prediction: Edge computing nodes use a global model to make real-time predictions on the collected data and apply the prediction results to production control or equipment maintenance.

[0076] 8. Data storage and analysis: Edge servers send data to the central cloud server for long-term storage and analysis.

[0077] 9. Data Visualization and Management: Users can access and analyze data through the visual interface provided by the central cloud server, and manage and configure the platform.

[0078] Advantages The industrial internet data processing platform in this embodiment has the following advantages: Low latency: Real-time data processing and analysis via edge computing nodes reduce data transmission latency and improve response speed.

[0079] High reliability: Edge computing nodes can operate independently, ensuring basic real-time data processing functions even if the network connection is interrupted.

[0080] Data privacy protection: Federated learning technology can train more accurate machine learning models without compromising data privacy.

[0081] Scalability: The platform architecture is flexible and can add or remove edge computing nodes and edge servers according to actual needs.

[0082] This embodiment details a specific implementation of an industrial internet data processing platform based on edge computing and federated learning, demonstrating how the technical features of the aforementioned claims can be applied to real-world scenarios to achieve efficient, secure, and reliable data processing. In summary, the efficient data processing platform and method provided by this invention have significant beneficial effects in terms of data processing efficiency, accuracy, scalability, and intelligence level. They can effectively solve the data processing problems in the Industrial Internet and create greater value for enterprises.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An efficient data processing platform for industrial internet, characterized in that: The platform comprises a data acquisition module, a data preprocessing module, a data storage module, a data processing module and a data visualization module; The data acquisition module is configured to acquire industrial data in real time through sensors and controller devices in the industrial internet; The data preprocessing module is configured to perform cleaning, denoising and normalization preprocessing operations on the acquired data; The data storage module is configured to store the preprocessed data in a distributed data storage system; The data processing module is configured to perform parallel processing on the data stored in the distributed data storage system using a distributed computing framework; The data visualization module is configured to visualize the processed data in the form of charts and reports.

2. The high-efficiency data processing platform for industrial internet of claim 1, wherein: The data acquisition module further comprises a wireless sensor network interface configured to acquire running state data of industrial equipment in real time through a wireless sensor network; the data preprocessing module further comprises an outlier detection and elimination module configured to detect and eliminate outliers from the acquired data; the data storage module further comprises a distributed file system configured to store the preprocessed data to achieve high availability and fault tolerance of the data; the data processing module further comprises a machine learning algorithm module configured to intelligently analyze and predict the data stored in the distributed data storage system; and the data visualization module further comprises a Web front-end display module configured to display the processed data in the form of interactive charts and reports through Web front-end technology. 3.The industrial internet of things oriented high-efficiency data processing platform according to claim 1, characterized in that: The machine learning algorithm module in the data processing module comprises: a feature selection sub-module configured to automatically select features that have the greatest impact on the prediction of a target variable from the preprocessed data, the feature selection methods including statistical-based feature selection, model-based feature selection and embedded-based feature selection; a model training sub-module configured to train a machine learning model using the correlation between the selected features and the target variable, the model types including linear regression model, support vector machine model, decision tree model, random forest model and neural network model; a model evaluation sub-module configured to evaluate the performance of the trained machine learning model, the evaluation indicators including mean square error, determination coefficient, accuracy, recall rate and F1 score; a model optimization sub-module configured to optimize the machine learning model according to the model evaluation results, the optimization methods including but not limited to hyperparameter adjustment, feature engineering and model fusion.

4. The high-efficiency data processing platform for industrial internet of claim 1, wherein: The platform further comprises: an edge computing node deployed near the data source, configured to perform preliminary cleaning, filtering and feature extraction on the acquired raw data; cooperates with the data processing module to preprocess the data according to preset rules or models, and sends the processed data to the distributed data storage system; performs real-time analysis and anomaly detection on the data by running a lightweight machine learning model, and sends the analysis results to the data processing module.

5. The high-efficiency data processing platform for industrial internet according to claim 1, characterized in that: The platform further comprises: Data security and privacy protection module: encrypt the data stored in the distributed data storage system to ensure the security of the data during storage and transmission; implement access control policies to limit unauthorized user or device access to data; use differential privacy, federated learning privacy protection technology to analyze data and train models while ensuring data privacy. 6.The industrial internet of things oriented high-efficiency data processing platform according to claim 1, characterized in that: The platform further comprises: Intelligent operation and maintenance and monitoring module: real-time monitoring of the running state of each component of the data processing platform, including hardware resource utilization, software service state; use anomaly detection algorithms to warn and diagnose abnormal conditions during platform operation; through automated execution of fault recovery, resource scheduling and operation and maintenance operations, ensure the stability and reliability of the platform; provide a visual operation and maintenance interface to facilitate the management and maintenance of the platform by operation and maintenance personnel.

7. An efficient data processing method for industrial internet, characterized in that: The efficient data processing method comprises: Step one, data collection: real-time collection of industrial data through sensors and controller devices in the industrial internet, and transmission of the collected data to the data processing platform; Step two, data preprocessing: cleaning, denoising and normalization preprocessing operations are performed on the collected data to improve data quality, and the data normalization calculation formula is as follows: wherein is the original data, is the normalized data, and are the minimum and maximum values of the data, respectively. Step three, data storage: store the preprocessed data in a distributed data storage system to realize fast access and efficient management of data; Step four, data processing: use a distributed computing framework to perform parallel processing on the data stored in the distributed data storage system, and the data processing formula is as follows: Linear regression formula: Y=aX+b, where X is the independent variable, Y is the dependent variable, and a and b are regression coefficients; Time series prediction formula: Y(t)=f(Y(t-1),Y(t-2),...,Y(t-n)), where Y(t) is the current prediction value, Y(t-1),Y(t-2),...,Y(t-n) are historical values, and f is the prediction function; Step five, data visualization: visualize the processed data in the form of charts and reports to allow users to intuitively understand the data information and running status in the industrial internet.

8. The high-efficiency data processing method for the industrial internet of things according to claim 4, characterized in that: The data collection in step one further comprises: real-time collection of industrial equipment running state data through a wireless sensor network in the industrial internet, and transmission of the collected data to the data processing platform through wireless transmission; the data preprocessing in step two further comprises: performing outlier detection and elimination on the collected data to eliminate the influence of outliers in the data on subsequent data processing.

9. The high-efficiency data processing method for the industrial internet of things according to claim 4, characterized in that: The data storage in step three further comprises: using a distributed file system to store the preprocessed data to realize high availability and fault tolerance of the data; the data processing in step four further comprises: using machine learning algorithms to perform intelligent analysis and prediction on the data stored in the distributed data storage system to mine the potential value of the data.

10. The high-efficiency data processing method for the industrial internet of things according to claim 4, characterized in that: The data visualization in the fifth step further includes: through a web front-end technology, the processed data is displayed in the form of interactive charts and reports, so that the user can conveniently query and analyze the data;The data analysis step includes: using a machine learning algorithm library to analyze and mine data;Using a real-time data analysis engine to quickly analyze and process real-time data.

Images