A data management system for the Internet of Things platform based on cloud technology

By performing intelligent data filtering and preprocessing at edge nodes, combined with distributed, multi-region redundant storage and intelligent routing algorithms, the problems of network congestion and data storage concentration in traditional IoT platform data management systems are solved, and the reliability of the system is improved and availability, reducing latency and packet loss rates.

CN118631873BActive Publication Date: 2024-11-15YUNSHUO INTELLIGENT TECH (NANJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411052576.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-11-15
Estimated Expiration
2044-08-02

AI Technical Summary

Technical Problem

The data management system of traditional IoT platforms is likely to cause network congestion, increase data transmission delay and packet loss rate, and the data storage is concentrated in a single data center, resulting in the system facing hardware failure or Data is unavailable or lost in the event of a network outage, reducing the reliability and availability of the system.

Method used

Adopt a data management system for IoT platform based on cloud technology, selectively transmit key data by performing intelligent data filtering and preprocessing at edge nodes, and adopting distributed and multi-region redundant storage in the cloud, combining intelligent routing algorithms and machine learning The model conducts data transmission and storage management, monitors network conditions in real time and dynamically adjusts transmission protocols and paths, predicts equipment failures and maintains them in advance.

Benefits of technology

It significantly reduces data transmission delay and packet loss rate, improves system reliability and availability, reduces equipment failure rate and maintenance costs, and enhances equipment management efficiency and data processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118631873B_ABST
    Figure CN118631873B_ABST
Patent Text Reader

Abstract

The present invention discloses a data management system for an Internet of Things platform based on cloud technology, and relates to the technical field of data management. The present invention performs preliminary data processing near a data source, uses an AI algorithm for intelligent data filtering, selectively transmits key data, reduces the transmission of useless data, and adopts distributed, multi-region redundant storage to distribute and store data in multiple geographical areas, thereby improving data availability and disaster recovery capabilities. By configuring multi-region instances and clusters, data sharding and distribution are implemented to ensure that even when a data center fails, data can still be restored through data copies in other areas, thereby improving system reliability and availability. A real-time processing unit is introduced, and a stream processing framework is used to efficiently process real-time data. By performing preliminary data processing at edge nodes, the amount of data transmitted to the cloud is reduced, thereby improving overall data processing efficiency.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data management, and in particular to a data management system for an Internet of Things platform based on cloud technology. Background Art

[0002] The Internet of Things is a system of relationships between computing devices, machinery, and digital machines. It has a universal unique identification code and the ability to transmit data through the network without the need for interaction between people or between people and devices. The Internet of Things digitizes the real world and has a wide range of applications. The Internet of Things can bring scattered data closer and integrate digital information between objects. The application areas of the Internet of Things mainly include the following: transportation and logistics, industrial manufacturing, health care, smart environment, personal and social fields, etc.

[0003] For example, the patent with publication number CN103957248B discloses a public real-time data management cloud service platform based on the Internet of Things technology, including an access interface layer, a support service layer and a data layer. The access interface layer provides an SDK, including a device management interface, a data acquisition interface and a data access interface; the support service layer provides the processing logic within the platform system, and provides the background implementation of the platform system for the access interface layer, including a Web-based device management service, a directory service and a memory-based real-time data management service; the data layer uses a relational database and a historical database to store data persistently, wherein the relational database stores relatively static information and the historical database stores historical data of device attributes. This invention supports the original data collection of Internet of Things devices in different industries, realizes the unified collection, storage, query and management of data, and can remotely manage and control various intelligent Internet of Things devices in the Internet of Things.

[0004] Although the above scheme has the above advantages, the traditional IoT platform data management system usually transmits all data directly to the cloud for processing. This method is prone to network congestion and increase data transmission delay and packet loss rate when the data volume is huge and the devices are widely distributed. Especially in real-time application scenarios, such as industrial automation and intelligent transportation systems, this delay and packet loss will directly affect the performance and security of the system. In addition, data storage is usually concentrated in a single data center, which makes the system prone to data unavailability or loss when facing hardware failure or network interruption, reducing the reliability and availability of the system. Therefore, there is an urgent need for a data management system for the IoT platform based on cloud technology to solve such problems. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a data management system for an Internet of Things platform based on cloud technology, which solves the problem that the prior art processing method of directly transmitting all data to the cloud easily leads to network congestion, increased data transmission delay and packet loss rate, and data storage is usually concentrated in a single data center, which makes the system prone to data unavailability or loss when facing hardware failure or network interruption.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] The present invention provides a data management system for an Internet of Things platform based on cloud technology, comprising:

[0008] The data acquisition module is used to obtain data from IoT devices; the data acquisition module includes:

[0009] Sensor interface, connect sensors and collect physical data, including temperature, humidity, and pressure;

[0010] Device management unit, used to manage connected devices, including device registration, status monitoring, and remote configuration;

[0011] The preprocessing unit is used to perform preliminary processing on the raw data, including denoising and formatting;

[0012] The data transmission module is used to transmit the collected data to the cloud, and use AI algorithms to perform intelligent data filtering at the edge nodes to selectively transmit key data; the data transmission module includes:

[0013] The protocol adaptation unit supports multiple communication protocols, including MQTT, CoAP, and HTTP. It selects the corresponding protocol according to the application scenario, monitors the data transmission quality in real time, and automatically retransmits and optimizes when problems are found.

[0014] A data encryption unit encrypts data during transmission;

[0015] The transmission optimization unit uses an intelligent routing algorithm to dynamically select the optimal transmission path based on real-time network conditions;

[0016] The data processing and storage module processes, stores and manages data in the cloud. The storage method adopts distributed, multi-region redundant storage, redundantly stores data in multiple geographic areas, and predicts equipment failures at edge nodes through machine learning models to perform maintenance in advance.

[0017] The data analysis and visualization module analyzes the stored data and presents the results in a visual way; it provides data accuracy and timeliness through environmental data collection; the equipment management unit enhances the controllability and management efficiency of equipment through equipment registration, status monitoring and remote configuration, and reduces equipment failure rate and maintenance costs; the pre-processing unit removes noise and formats the raw data to improve data quality and reduce data errors;

[0018] The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network conditions. The adjustment method is as follows:

[0019] Evaluate the network status, monitor the network status in real time, collect the current network bandwidth B, delay L, and packet loss rate P, use the sliding window algorithm to smooth the network status, and obtain the average network status: Where Bi, Li, and Pi are the bandwidth, delay, and packet loss rate at the i-th sampling moment, respectively, and n is the sliding window size;

[0020] Select the corresponding transmission protocol according to the real-time monitored network status. The judgment method is: Among them, Bthred, Lthred, and Pthred are the thresholds of bandwidth, delay, and packet loss rate respectively;

[0021] Dynamically adjust the transmission protocol parameters according to the real-time network conditions;

[0022] The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network conditions. The adjustment methods include:

[0023] For the MQTT protocol, select different QoS levels based on network conditions:

[0024]

[0025] For the CoAP protocol, adjust the number of retransmissions based on the packet loss rate: Nretrans represents the number of retransmissions. The data transmission module selects the transmission protocol according to the application scenario, optimizes the transmission efficiency and reliability, adapts to different network environments, and reduces transmission delay and packet loss rate. It ensures the security of data during transmission through data encryption, prevents data leakage and tampering, and protects user privacy.

[0026] The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network status. The adjustment methods also include:

[0027] Monitor the data transmission process in real time, collect data transmission rate R, transmission delay D, number of retransmissions Nretrans, and adjust transmission parameters through feedback mechanism;

[0028] According to the real-time monitoring data, the transmission parameters are adjusted using the feedback control algorithm, and the transmission parameters are optimized using the PID control algorithm: Among them, e(t) represents the difference between the target transmission rate and the actual transmission rate, Kp, Ki, and Kd are the proportional, integral, and differential coefficients respectively, and u(t) is the adjusted parameter value. The protocol adaptive unit monitors the network bandwidth, delay, and packet loss rate in real time, dynamically adjusts the transmission protocol parameters, and smoothly processes the network status through the sliding window algorithm to adjust the parameters for different protocols.

[0029] The transmission optimization unit dynamically selects the optimal transmission path based on the real-time network status. The adjustment method is as follows:

[0030] Based on the bandwidth B, delay L, and packet loss rate P of each path obtained by real-time monitoring of available transmission paths, the quality score of each path is calculated by weighting the bandwidth, delay, and packet loss rate, and the weight coefficients are set to represent the importance of bandwidth, delay, and packet loss rate respectively: Qi = α·Bpath,i-β·Lpath,i-γ·Ppath,i, where Bpath,i, Lpath,i, Ppath,i represent the bandwidth, delay, and packet loss rate of path i respectively, α, β, γ are positive numbers, and satisfy α+β+γ=1, Qi is the quality score of the i-th path; select the path with the highest quality score from all paths as the transmission path; recalculate the quality scores of all paths when a change in network conditions is detected, and dynamically adjust the transmission path;

[0031] Monitor the actual transmission process, collect the actual transmission rate R and actual transmission delay D, and compare them with the expected values. Where n is the number of sampling times in the monitoring cycle, and j is the current sampling point index. Here, the bandwidth, delay, and packet loss rate of the transmission path are evaluated in real time, and the path with the highest quality score is selected. When the network conditions change, the path quality score is recalculated, and the transmission path is dynamically adjusted to adapt to changes in the network environment and maintain continuous and stable data transmission.

[0032] The data processing and storage module includes,edge computing nodes that perform preliminary data processing near the data source;

[0033] Real-time processing unit, which uses stream processing framework to process real-time data;

[0034] Database unit, using distributed database for data storage;

[0035] The present invention is further configured as follows: the database in the database unit selects a database system that supports distributed and multi-region redundant storage; the multi-region storage configuration step includes:

[0036] Create multi-region instances and clusters in the database management console;

[0037] Data sharding and distribution: Hash sharding is used to split data into multiple shards, each of which is stored and managed independently.

[0038] Configure the data synchronization mechanism between database instances;

[0039] Select a data consistency model based on the application scenario and configure a conflict resolution strategy;

[0040] Perform preliminary data processing near the data source to reduce data transmission volume and latency, and improve data processing efficiency. Use the stream processing framework to efficiently process real-time data, ensure the timeliness and accuracy of data, and improve the response speed of the system. Ensure data distribution storage and management through the configuration of multi-region instances and clusters. Use hash sharding technology for data sharding and independent storage management to improve the scalability and performance of data storage.

[0041] The present invention is further configured as follows: in the data processing and storage module, the device failure prediction method is:

[0042] Collect operating data from sensors and equipment, including temperature, pressure, current, vibration, and usage time; collect and pre-process operating data from sensors and equipment to ensure data integrity and accuracy, extract key features, support subsequent predictive analysis, use machine learning models to predict equipment failures, identify potential failures in advance, and issue early warning signals;

[0043] Process missing values, noisy data and outliers, extract key features, generate new features, and the data after standardization is represented as follows: Where X is the original data, μ is the mean, and σ is the standard deviation;

[0044] Mark the time point and type of the fault, generate a training data set, and divide the data set into a training set, a validation set, and a test set;

[0045] The training set is represented as { (x1,y1),(x2,y2),...,(xn,yn) } ;

[0046] The validation set is represented as { (xn+1,yn+1),...,(xm,ym) } ;

[0047] The test set is represented as {(xm+1,ym+1),...,(xp,yp)}, where xn is the feature, yn is the label, n is the training set size, m is the validation set size, and p is the test set size;

[0048] Random forest, support vector machine SVM, and long short-term memory network LSTM are selected as machine learning models for predicting faults;

[0049] Use the training set to train the model and adjust the model parameters;

[0050] Then deploy the model to edge nodes, where device operation data is collected in real time;

[0051] Preprocess the real-time data to keep it consistent with the training data;

[0052] Use models deployed on edge nodes to predict real-time data and determine whether equipment is at risk of failure;

[0053] When the model predicts that the equipment is at risk of failure, it issues an early warning signal;

[0054] The present invention is further configured as follows: the data analysis and visualization module includes:

[0055] Data analysis unit, which uses machine learning algorithms for data analysis and prediction;

[0056] Report generation unit, which generates regular reports and provides system operation status and data analysis results;

[0057] Visualization unit, using visualization tools to present data analysis results;

[0058] The present invention is further configured as follows: the data analysis unit performs data analysis and prediction in the following manner:

[0059] Collect data from sensors, logs, and databases, and perform data cleaning;

[0060] Select features that have an important impact on the model, extract new features from the original data, and standardize the features;

[0061] Construct training and test data sets and divide the data into training set and test set;

[0062] Select linear regression, decision tree, support vector machine SVM, random forest, and neural network as learning models, and use training set data to train the models;

[0063] Use the validation set to evaluate model performance and adjust hyperparameters to optimize the model;

[0064] Deploy the trained model to the production environment, integrate it into the data analysis system, and use the deployed model to make real-time predictions on new data.

[0065] Compared with the prior art, the present invention has the following beneficial effects:

[0066] The present invention introduces edge computing nodes to perform preliminary data processing near the data source, uses AI algorithms for intelligent data filtering, selectively transmits key data, and reduces the transmission of useless data, which not only significantly reduces the amount of data transmission and network congestion, but also greatly reduces data transmission delay and packet loss rate. In addition, the transmission optimization unit uses an intelligent routing algorithm to dynamically select the optimal transmission path according to the real-time network status, further reducing delays and optimizing transmission efficiency.

[0067] The present invention adopts distributed, multi-region redundant storage to distribute and store data in multiple geographical areas, improve the availability and disaster recovery capabilities of data, and implement data sharding and distribution by configuring multi-region instances and clusters to ensure that even if a data center fails, data can still be restored through data copies in other areas, thereby improving the reliability and availability of the system;

[0068] The present invention introduces a real-time processing unit and uses a stream processing framework to efficiently process real-time data. By performing preliminary data processing at the edge node, the amount of data transmitted to the cloud is reduced, and the overall data processing efficiency is improved. In addition, the machine learning model is used to predict equipment failures at the edge node, and real-time monitoring and predictive maintenance of the equipment operating status are achieved. Equipment maintenance is performed in advance to avoid data loss and delay, further improving data processing efficiency.

[0069] The present invention uses real-time data for analysis and prediction, and achieves efficient analysis and accurate prediction of data through feature engineering, model training, verification and deployment. The trained model is deployed to the production environment and integrated into the data analysis system to perform real-time prediction. The prediction results and analysis results are displayed through visualization tools to improve the decision support capability of the system.

[0070] It solves the problem of the existing processing method of directly transmitting all data to the cloud, which easily leads to network congestion, increased data transmission delay and packet loss rate. At the same time, data storage is usually concentrated in a single data center, which makes the system prone to data unavailability or loss when facing hardware failure or network interruption. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is a framework diagram of the data management system of the Internet of Things platform based on cloud technology in the present invention. DETAILED DESCRIPTION

[0072] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0073] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0074] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0075] Example 1

[0076] See also Figure 1 The present invention provides a data management system for an Internet of Things platform based on cloud technology, comprising:

[0077] Data acquisition module, used to obtain data from IoT devices;

[0078] The data acquisition module includes:

[0079] Sensor interface, connect sensors and collect physical data, including temperature, humidity, and pressure;

[0080] Device management unit, used to manage connected devices, including device registration, status monitoring, and remote configuration;

[0081] The preprocessing unit is used to perform preliminary processing on the raw data, including denoising and formatting;

[0082] The data transmission module is used to transmit the collected data to the cloud, and use AI algorithms to perform intelligent data filtering at the edge nodes, selectively transmit key data, and reduce the transmission of useless data;

[0083] The data transmission module includes:

[0084] The protocol adaptation unit supports multiple communication protocols, including MQTT, CoAP, and HTTP. It selects the corresponding protocol according to the application scenario, monitors the data transmission quality in real time, and automatically retransmits and optimizes when problems are found.

[0085] A data encryption unit encrypts data during transmission;

[0086] The transmission optimization unit uses an intelligent routing algorithm to dynamically select the optimal transmission path based on real-time network conditions to reduce delays; optimize data transmission paths and improve transmission efficiency;

[0087] The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network conditions. The adjustment method is as follows:

[0088] Evaluate the network status, monitor the network status in real time, collect the current network bandwidth B, delay L, and packet loss rate P, use the sliding window algorithm to smooth the network status, and obtain the average network status: Where Bi, Li, and Pi are the bandwidth, delay, and packet loss rate at the i-th sampling moment, respectively, and n is the sliding window size;

[0089] Select the corresponding transmission protocol according to the real-time monitored network status. The judgment method is:

[0090]

[0091] Among them, Bthred, Lthred, and Pthred are the thresholds of bandwidth, delay, and packet loss rate respectively;

[0092] Dynamically adjust the transmission protocol parameters according to the real-time network conditions;

[0093] For the MQTT protocol, select different QoS levels based on network conditions:

[0094]

[0095] For the CoAP protocol, adjust the number of retransmissions based on the packet loss rate:

[0096] Where Nretrans represents the number of retransmissions;

[0097] Monitor the data transmission process in real time, collect data transmission rate R, transmission delay D, and number of retransmissions Nretrans,

[0098] Adjust transmission parameters through feedback mechanisms;

[0099] According to the real-time monitoring data, the transmission parameters are adjusted using the feedback control algorithm, and the transmission parameters are optimized using the PID control algorithm: Wherein, e(t) represents the difference between the target transmission rate and the actual transmission rate, Kp, Ki, and Kd are the proportional, integral, and differential coefficients, respectively, and u(t) is the adjusted parameter value;

[0100] In the data transmission module, the transmission protocol is dynamically selected based on the real-time network status to improve transmission efficiency and reliability. At the same time, the PID control algorithm is used to dynamically optimize the transmission parameters to dynamically adjust the transmission protocol parameters according to the network status, effectively dealing with the delay and packet loss in the large amount of data transmission generated by IoT devices;

[0101] The transmission optimization unit dynamically selects the optimal transmission path based on the real-time network status. The adjustment method is as follows:

[0102] Based on the bandwidth B, delay L, and packet loss rate P of each path obtained by real-time monitoring of available transmission paths, the quality score of each path is calculated by weighting the bandwidth, delay, and packet loss rate. The weight coefficients are set to represent the importance of bandwidth, delay, and packet loss rate, respectively: Qi = α·Bpath,i-β·Lpath,i-γ·Ppath,i, where Bpath,i, Lpath,i, and Ppath,i represent the bandwidth, delay, and packet loss rate of path i, respectively. α, β, and γ are positive numbers and satisfy α+β+γ=1. Qi is the quality score of the i-th path.

[0103] Select a path with the highest quality score from all paths as the transmission path;

[0104] When changes in network conditions are detected, the quality scores of all paths are recalculated and the transmission paths are dynamically adjusted.

[0105] Monitor the actual transmission process, collect the actual transmission rate R and actual transmission delay D, and compare them with the expected values. Where n is the number of sampling times in the monitoring period, and j is the index of the current sampling point;

[0106] The data processing and storage module processes, stores and manages data in the cloud. The storage method adopts distributed, multi-region redundant storage, and redundantly stores data in multiple geographical areas to improve data availability and disaster recovery capabilities. It also uses machine learning models to predict equipment failures at edge nodes, perform maintenance in advance, and avoid data loss and delays.

[0107] The data processing and storage modules include:

[0108] Edge computing nodes perform preliminary data processing near the data source to reduce transmission delays and data volume;

[0109] Real-time processing unit, which uses stream processing framework to process real-time data;

[0110] Database unit, using distributed database for data storage;

[0111] The data transmission module and the data processing and storage module effectively improve the intelligence level of the system through intelligent data filtering and predictive maintenance, reduce the occupation of useless data during the transmission process, and ensure the high availability and fault tolerance of data;

[0112] The database in the database unit selects a database system that supports distributed and multi-region redundant storage;

[0113] The steps to configure multi-region storage include:

[0114] Create multi-region instances and clusters in the database management console;

[0115] Data sharding and distribution: Hash sharding is used to split data into multiple shards, each of which is stored and managed independently.

[0116] Configure the data synchronization mechanism between database instances;

[0117] Select a data consistency model based on the application scenario and configure a conflict resolution strategy;

[0118] In the data processing and storage module, the equipment failure prediction method is:

[0119] Collect operational data from sensors and equipment, including temperature, pressure, current, vibration, and usage time;

[0120] Process missing values, noisy data and outliers, extract key features, generate new features, and the data after standardization is represented as follows: Where X is the original data, μ is the mean, and σ is the standard deviation;

[0121] Mark the time point and type of the fault, generate a training data set, and divide the data set into a training set, a validation set, and a test set;

[0122] The training set is represented as { (x1,y1),(x2,y2),...,(xn,yn) } ;

[0123] The validation set is represented as { (xn+1,yn+1),...,(xm,ym) } ;

[0124] The test set is represented as {(xm+1,ym+1),...,(xp,yp)}, where xn is the feature, yn is the label, n is the training set size, m is the validation set size, and p is the test set size;

[0125] Random forest, support vector machine SVM, and long short-term memory network LSTM are selected as machine learning models for predicting faults;

[0126] Use the training set to train the model and adjust the model parameters;

[0127] Then deploy the model to edge nodes, where device operation data is collected in real time;

[0128] Preprocess the real-time data to keep it consistent with the training data;

[0129] Use models deployed on edge nodes to predict real-time data and determine whether equipment is at risk of failure;

[0130] When the model predicts that the equipment is at risk of failure, it issues an early warning signal;

[0131] Develop maintenance plans based on early warning signals and perform equipment maintenance in advance to avoid data loss and delays caused by failures;

[0132] Data analysis and visualization module, which analyzes the stored data and presents the results in a visual way;

[0133] Data analysis and visualization modules include:

[0134] Data analysis unit, which uses machine learning algorithms for data analysis and prediction;

[0135] Report generation unit, which generates regular reports and provides system operation status and data analysis results;

[0136] Visualization unit, using visualization tools to present data analysis results;

[0137] The data analysis unit performs data analysis and prediction in the following ways:

[0138] Collect data from sensors, logs, and databases, and perform data cleaning;

[0139] Select features that have an important impact on the model, extract new features from the original data, and standardize the features;

[0140] Construct training and test data sets and divide the data into training set and test set;

[0141] Select linear regression, decision tree, support vector machine SVM, random forest, and neural network as learning models, and use training set data to train the models;

[0142] Use the validation set to evaluate model performance and adjust hyperparameters to optimize the model;

[0143] Deploy the trained model to the production environment, integrate it into the data analysis system, and use the deployed model to make real-time predictions on new data.

[0144] The present invention obtains data from IoT devices and performs preliminary processing on the original data. In the process of transmitting the collected data to the cloud, the AI ​​algorithm is used to perform intelligent data filtering at the edge node, selectively transmit key data, and reduce the transmission amount of useless data. The protocol adaptive unit supports multiple communication protocols, selects appropriate protocols according to application scenarios, and monitors the data transmission quality in real time, and automatically retransmits and optimizes when problems are found; through real-time network status evaluation, the bandwidth, delay and packet loss rate of the current network are collected, and the network status is smoothed using a sliding window algorithm to obtain the average network status. According to the real-time monitored network status, the corresponding transmission protocol is selected, and the judgment standard is the preset bandwidth, delay and packet loss rate threshold; for the MQTT protocol, different service quality QoS levels are selected according to the network status; for the CoAP protocol, the number of retransmissions is adjusted according to the packet loss rate;

[0145] At the same time, an intelligent routing algorithm is used to dynamically select the optimal transmission path according to the real-time network status, and the quality score of each path is calculated based on the weighted bandwidth, delay and packet loss rate of the path, and the transmission path is adjusted dynamically. In addition, data is processed, stored and managed in the cloud, and distributed, multi-region redundant storage is used to improve data availability and disaster recovery capabilities. Machine learning models are used to predict equipment failures at edge nodes, and maintenance is performed in advance to avoid data loss and delays. Edge computing nodes perform preliminary data processing near the data source to reduce transmission delays and data volume; the real-time processing unit uses a stream processing framework to process real-time data to ensure the timeliness and accuracy of the data. The database unit uses a database system that supports distributed and multi-region redundant storage, creates multi-region instances or clusters in the database management console, performs data sharding and distribution, and configures data synchronization mechanisms between database instances;

[0146] In terms of equipment failure prediction, we collect operation data from sensors and equipment, process missing values, noise data, and outliers, extract key features and standardize them, mark the time point and type of failure, generate training data sets, and divide them into training sets, validation sets, and test sets. We use machine learning models such as random forests, support vector machines (SVMs), and long short-term memory networks (LSTMs) for training, adjust model parameters, and then deploy them to edge nodes. We collect equipment operation data in real time at edge nodes, preprocess the real-time data, and use models to make real-time predictions to determine whether the equipment has a failure risk. When a failure risk is predicted, we issue a warning signal, formulate a maintenance plan based on the warning signal, and perform equipment maintenance in advance to avoid data loss and delays caused by failures.

[0147] The data analysis and visualization module analyzes the stored data and presents the results in a visual way. The module includes a data analysis unit, a report generation unit, and a visualization unit. The data analysis unit uses machine learning algorithms to analyze and predict data, collect data from sensors, logs, and databases, clean the data, select and extract important features, and perform standardization. It builds training and test data sets, trains models using training set data, evaluates model performance, and adjusts hyperparameters to optimize the model; it deploys the trained model to the production environment, integrates it into the data analysis system, performs real-time predictions, and displays the prediction and analysis results through visualization tools.

[0148] The above contents are only for explaining the technical idea of ​​the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims

1. A data management system for an Internet of Things platform based on cloud technology, characterized in that: include: Data acquisition module, used to obtain data from IoT devices; The data acquisition module includes: Sensor interface, connect sensors and collect physical data, including temperature, humidity, and pressure; Device management unit, used to manage connected devices, including device registration, status monitoring, and remote configuration; The preprocessing unit is used to perform preliminary processing on the raw data, including denoising and formatting; The data transmission module is used to transmit the collected data to the cloud, and use AI algorithms to perform intelligent data filtering at the edge nodes to selectively transmit key data; the data transmission module includes: The protocol adaptation unit supports multiple communication protocols, including MQTT, CoAP, and HTTP. It selects the corresponding protocol according to the application scenario, monitors the data transmission quality in real time, and automatically retransmits and optimizes when problems are found. A data encryption unit encrypts data during transmission; The transmission optimization unit uses an intelligent routing algorithm to dynamically select the optimal transmission path based on real-time network conditions; The data processing and storage module processes, stores and manages data in the cloud. The storage method adopts distributed, multi-region redundant storage, redundantly stores data in multiple geographic areas, and predicts equipment failures at edge nodes through machine learning models to perform maintenance in advance. Data analysis and visualization module, which analyzes the stored data and presents the results in a visual way; The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network conditions. The adjustment method is as follows: Evaluate the network status, monitor the network status in real time, collect the current network bandwidth B, delay L, and packet loss rate P, use the sliding window algorithm to smooth the network status, and obtain the average network status: in, , , are the bandwidth, delay and packet loss rate at the i-th sampling moment, respectively, and n is the sliding window size; Select the corresponding transmission protocol according to the real-time monitored network status. The judgment method is: in, , , are the thresholds for bandwidth, delay, and packet loss rate, respectively; Dynamically adjust the transmission protocol parameters according to the real-time network conditions; The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network conditions. The adjustment methods include: For the MQTT protocol, select different QoS levels based on network conditions: For the CoAP protocol, adjust the number of retransmissions based on the packet loss rate: in Indicates the number of retransmissions; The protocol adaptation unit dynamically adjusts the transmission protocol parameters according to the network status. The adjustment methods also include: Monitor the data transmission process in real time, collect data transmission rate R, transmission delay D, and number of retransmissions , adjust the transmission parameters through feedback mechanism; According to the real-time monitoring data, the transmission parameters are adjusted using the feedback control algorithm, and the transmission parameters are optimized using the PID control algorithm: Where e(t) represents the difference between the target transmission rate and the actual transmission rate. , , are the proportional, integral and differential coefficients respectively, and u(t) is the adjusted parameter value; The transmission optimization unit dynamically selects the optimal transmission path based on the real-time network status. The adjustment method is as follows: Based on the bandwidth B, delay L, and packet loss rate P of each path obtained by real-time monitoring of available transmission paths, the quality score of each path is calculated by weighting bandwidth, delay, and packet loss rate. The weight coefficients are set to represent the importance of bandwidth, delay, and packet loss rate respectively: ,in , , They represent the bandwidth, delay and packet loss rate of path i respectively. α, β, γ are positive numbers and satisfy α+β+γ=1. is the quality score of the i-th path; select the path with the highest quality score from all paths as the transmission path; recalculate the quality scores of all paths when a change in network conditions is detected, and dynamically adjust the transmission path; Monitor the actual transmission process, collect the actual transmission rate R and actual transmission delay D, and compare them with the expected values. Where n is the number of sampling times in the monitoring period, and j is the index of the current sampling point.

2. According to claim 1, a data management system for an Internet of Things platform based on cloud technology is characterized in that: The data processing and storage module includes,edge computing nodes that perform preliminary data processing near the data source; Real-time processing unit, which uses stream processing framework to process real-time data; The database unit uses a distributed database for data storage.

3. According to claim 2, a data management system for an Internet of Things platform based on cloud technology is characterized in that: The database in the database unit selects a database system that supports distributed and multi-region redundant storage; the steps for configuring multi-region storage include: Create multi-region instances and clusters in the database management console; Data sharding and distribution: Hash sharding is used to split data into multiple shards, each of which is stored and managed independently. Configure the data synchronization mechanism between database instances; Select a data consistency model based on the application scenario and configure a conflict resolution strategy.

4. According to claim 3, a data management system for an Internet of Things platform based on cloud technology is characterized in that: In the data processing and storage module, the equipment failure prediction method is: Collect operational data from sensors and equipment, including temperature, pressure, current, vibration, and usage time; Process missing values, noisy data and outliers, extract key features, generate new features, and the data after standardization is represented as follows: Where X is the original data, μ is the mean, and σ is the standard deviation; Mark the time point and type of the fault, generate a training data set, and divide the data set into a training set, a validation set, and a test set; The training set is represented as ; The validation set is represented as ; The test set is represented as ,in, Characterized by is the label, n is the training set size, m is the validation set size, and p is the test set size; Random forest, support vector machine SVM, and long short-term memory network LSTM are selected as machine learning models for predicting faults; Use the training set to train the model and adjust the model parameters; Then deploy the model to edge nodes, where device operation data is collected in real time; Preprocess the real-time data to keep it consistent with the training data; Use models deployed on edge nodes to predict real-time data and determine whether equipment is at risk of failure; When the model predicts that the equipment is at risk of failure, an early warning signal is issued.

5. The data management system of the Internet of Things platform based on cloud technology according to claim 4 is characterized in that: Data analysis and visualization modules include: Data analysis unit, which uses machine learning algorithms for data analysis and prediction; Report generation unit, which generates regular reports and provides system operation status and data analysis results; Visualization unit, uses visualization tools to present data analysis results.

6. The data management system of the Internet of Things platform based on cloud technology according to claim 5 is characterized in that: The data analysis unit performs data analysis and prediction in the following ways: Collect data from sensors, logs, and databases, and perform data cleaning; Select features that have an important impact on the model, extract new features from the original data, and standardize the features; Construct training and test data sets and divide the data into training set and test set; Select linear regression, decision tree, support vector machine SVM, random forest, and neural network as learning models, and use training set data to train the models; Use the validation set to evaluate model performance and adjust hyperparameters to optimize the model; Deploy the trained model to the production environment, integrate it into the data analysis system, and use the deployed model to make real-time predictions on new data.