Internet of Things data processing method, system and terminal based on data service

Abstract

The invention discloses an Internet of Things data processing method and system based on data service, and a terminal. The method comprises the following steps: collecting initial equipment data of a specific type or a specific purpose of target Internet equipment in a preset time period; performing data cleaning and data induction on the initial equipment data to obtain target equipment data; performing real-time processing on the target equipment data to generate a data real-time analysis result and decision support; and carrying out deep analysis processing on the target equipment data to generate a deep analysis result, and providing a targeted optimization suggestion according to the deep analysis result. Real-time data cleaning and analysis functions are provided, the data quality and accuracy are improved, a large amount of data is rapidly processed and analyzed according to the real-time requirement under the Internet of Things scene, data in the Internet of Things environment can be rapidly processed in real time, and accurate analysis results and decision support are provided.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things data processing, and in particular to an Internet of Things data processing method, system, terminal and computer-readable storage medium based on data service. Background Art

[0002] In the IoT environment, a large number of devices generate a large amount of data, but these data may have noise, missing or redundant problems, which makes data analysis difficult. In the IoT environment, multiple devices generate a large amount of data. The purpose of IoT data analysis is to extract useful information from the massive device data, but the problems with the data make it difficult to quickly and accurately analyze the data. In addition, traditional data processing methods cannot effectively handle real-time data and rapidly changing environmental requirements.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] The main purpose of the present invention is to provide an Internet of Things data processing method, system, terminal and computer-readable storage medium based on data services, aiming to solve the problems in the prior art that a large amount of data has defects, it is difficult to quickly and accurately analyze the data, and it is not possible to effectively process real-time data and rapidly changing environmental requirements.

[0005] To achieve the above object, the present invention provides an Internet of Things data processing method based on data service, and the Internet of Things data processing method based on data service comprises the following steps:

[0006] Collect initial device data of a specific type or purpose from the target Internet device within a preset time period;

[0007] Performing data cleaning and data summarization on the initial device data to obtain target device data;

[0008] Process the target device data in real time to generate real-time data analysis results and decision support;

[0009] The target device data is subjected to in-depth analysis and processing to generate in-depth analysis results, and targeted optimization suggestions are provided based on the in-depth analysis results.

[0010] Optionally, in the IoT data processing method based on data service, the step of collecting initial device data of a specific type or for a specific purpose of a target Internet device within a preset time period specifically includes:

[0011] Collecting initial device data of a specific type or specific purpose of a target Internet device within a preset time period according to preliminary screening rules;

[0012] Collect initial device data related to specific analysis objectives or business needs from target Internet devices within a preset time period based on preliminary screening rules.

[0013] Optionally, in the data service-based IoT data processing method, the data cleaning includes: data identification, invalid data removal, missing data filling, duplicate data removal and data standardization; the data summarization includes: data classification, data aggregation and feature extraction.

[0014] Optionally, the IoT data processing method based on data service, wherein the data cleaning and data summarization of the initial device data to obtain the target device data specifically includes:

[0015] Performing data identification on the initial device data to determine the data source and format;

[0016] Delete the data in the initial device data that is incomplete, erroneous in format, or obviously exceeds the normal range;

[0017] Fill in missing data in the initial device data using the mean or median of the previous and subsequent data or using an interpolation method;

[0018] Checking and deleting duplicate data records in the initial device data;

[0019] Converting the initial device data into a uniform format or scale;

[0020] After the initial device data completes data identification, invalid data removal, missing data filling, duplicate data removal and data standardization, intermediate device data is obtained;

[0021] Classifying the intermediate device data according to certain standards;

[0022] Aggregating the intermediate device data;

[0023] extracting features from the middleware data that are useful for analysis;

[0024] After the intermediate device data completes data classification, data aggregation and feature extraction, the target device data is obtained.

[0025] Optionally, in the data service-based IoT data processing method, the real-time processing includes: data stream processing, data filtering, data aggregation, data conversion, complex event processing and real-time prediction;

[0026] The real-time processing of the target device data to generate real-time data analysis results and decision support specifically includes:

[0027] Using a data stream management system to ingest, process and analyze the target device data in real time;

[0028] According to preset rules or thresholds, filter out key data or abnormal data in the target device data;

[0029] Summarizing the target device data in real time;

[0030] Performing format conversion, normalization or encoding conversion on the target device data;

[0031] Recognizing and processing complex event patterns in the target device data;

[0032] A machine learning model is used to make real-time predictions on the target device data.

[0033] Optionally, in the IoT data processing method based on data services, the in-depth analysis processing includes: data preprocessing, feature engineering processing, model training and verification processing, data analysis and insight processing, and result interpretation and reporting processing;

[0034] The in-depth analysis and processing of the target device data to generate in-depth analysis results, and providing targeted optimization suggestions based on the in-depth analysis results, specifically includes:

[0035] Merging the cleaned target device data with other relevant data sources, and converting the merged data into a format suitable for analysis;

[0036] Extract features that are helpful for analysis from the target device data, select the features that are most useful for model prediction, exclude irrelevant or redundant features, and create new features;

[0037] Select appropriate machine learning algorithms based on the analysis objectives, train models using historical data, optimize model performance by adjusting parameters, and evaluate model accuracy and generalization capabilities through cross-validation techniques;

[0038] Analyze the target device data to identify hidden patterns, associations, and anomalies, use models to predict future trends or results, and propose improvement measures or optimization plans based on model outputs;

[0039] Convert analysis results into an easy-to-understand format and create detailed analysis reports.

[0040] Optionally, in the data service-based IoT data processing method, the analysis report includes key findings, data support, recommendations and follow-up action plans.

[0041] In addition, to achieve the above-mentioned purpose, the present invention further provides an Internet of Things data processing system based on data services, wherein the Internet of Things data processing system based on data services includes:

[0042] A data collection module, used to collect initial device data of a specific type or specific purpose of a target Internet device within a preset time period;

[0043] A data processing module, used for cleaning and summarizing the initial device data to obtain target device data;

[0044] A real-time processing module, used to process the target device data in real time and generate real-time data analysis results and decision support;

[0045] The intelligent analysis module is used to perform in-depth analysis and processing on the target device data, generate in-depth analysis results, and provide targeted optimization suggestions based on the in-depth analysis results.

[0046] In addition, to achieve the above-mentioned purpose, the present invention also provides a terminal, wherein the terminal includes: a memory, a processor, and an Internet of Things data processing program based on data services stored in the memory and executable on the processor, and when the Internet of Things data processing program based on data services is executed by the processor, the steps of the Internet of Things data processing method based on data services as described above are implemented.

[0047] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an Internet of Things data processing program based on data services, and when the Internet of Things data processing program based on data services is executed by a processor, the steps of the Internet of Things data processing method based on data services as described above are implemented.

[0048] In the present invention, initial device data of a specific type or specific purpose of a target Internet device within a preset time period is collected; data cleaning and data induction are performed on the initial device data to obtain target device data; the target device data is processed in real time to generate real-time data analysis results and decision support; the target device data is deeply analyzed and processed to generate in-depth analysis results, and targeted optimization suggestions are provided based on the in-depth analysis results. The present invention provides real-time data cleaning and analysis functions, improves data quality and accuracy, and quickly processes and analyzes a large amount of data in response to the real-time requirements in the Internet of Things scenario. It can quickly process data in the Internet of Things environment in real time and provide accurate analysis results and decision support. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of a preferred embodiment of the method for processing Internet of Things data based on data service of the present invention;

[0050] Figure 2 It is a structural diagram of a preferred embodiment of the Internet of Things data processing system based on data services of the present invention;

[0051] Figure 3 It is a structural diagram of a preferred embodiment of the terminal of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0053] In the present invention, in the IoT environment, a variety of devices will generate a large amount of data. The following are some typical devices and the types of data they generate:

[0054] (1) Sensor equipment:

[0055] Temperature sensor: generates ambient temperature data.

[0056] Humidity sensor: generates ambient humidity data.

[0057] Light sensor: generates light intensity data.

[0058] Pressure sensor: generates pressure data.

[0059] Displacement sensor: generates position change data.

[0060] (2) Camera: Video stream data: used for monitoring and image recognition.

[0061] (3) Intelligent Instruments:

[0062] Electricity consumption data: record electricity consumption.

[0063] Water meter data: records water consumption.

[0064] (4) RFID tags and readers: Item tracking data: records the location and movement of items.

[0065] (5) Smart terminal devices: User behavior data: records user operations and preferences.

[0066] Reasons for analyzing this data include:

[0067] (1) Optimizing resource usage: By analyzing sensor data, energy usage can be optimized and waste can be reduced, such as in applications in smart grids.

[0068] (2) Improve security: Analysis of video stream data can help timely detect security risks, such as intrusion detection.

[0069] (3) Enhance user experience: Analysis of user behavior data can be used to personalize services and improve user satisfaction.

[0070] (4) Predictive maintenance: By analyzing equipment operation data, equipment failure can be predicted and maintenance can be performed in advance.

[0071] (5) Improve products and services: Analyze the collected data and use it for continuous improvement of products and services.

[0072] The purposes of data analysis are to:

[0073] (1) Information extraction: extracting valuable information from large amounts of data.

[0074] (2) Decision support: Provide data-based decision support to management.

[0075] (3) Trend forecasting: predicting future trends and patterns so as to make corresponding preparations.

[0076] (4) Anomaly detection: timely detect anomalies in data and take corresponding measures.

[0077] (5) Efficiency improvement: Optimize processes through data analysis to improve work efficiency and effectiveness.

[0078] In short, the purpose of IoT data analysis is to extract useful information from massive amounts of device data to support better decision-making, optimize business processes, improve operational efficiency, enhance user experience, and ultimately drive business growth and innovation.

[0079] In the present invention, real-time data and rapidly changing environmental requirements: Real-time data refers to data that is processed and used almost immediately after it is generated. In the IoT environment, real-time data usually comes from various sensors and devices, and these data need to be captured, processed and analyzed quickly in order to make timely responses and decisions. Rapidly changing environmental requirements refer to the dynamic environmental conditions faced by the IoT system, which may include changes in device status, changes in user behavior, changes in the external environment, etc. These requirements require the system to be able to adapt to changes quickly and provide corresponding services or responses.

[0080] The reasons why existing technologies cannot effectively handle real-time data and rapidly changing environmental requirements:

[0081] (1) Processing power limitations: Traditional data processing systems may not have sufficient computing power to handle large amounts of real-time data streams.

[0082] (2) Delay problem: Traditional data processing methods may introduce too much delay, resulting in failure to meet real-time requirements.

[0083] (3) Inflexible system architecture: Traditional data processing systems may not be flexible enough to adapt to rapidly changing environmental requirements.

[0084] (4) Complexity of data analysis: Real-time data analysis usually requires complex algorithms and models, and traditional systems may not have such analytical capabilities.

[0085] (5) Data quality and preprocessing: Real-time data may contain a lot of noise and outliers, and traditional systems may not be efficient in data cleaning and preprocessing.

[0086] Existing technologies for handling real-time data and rapidly changing environmental requirements usually include:

[0087] (1) Batch processing: Data is collected for a period of time and then processed in batches. This method is not suitable for scenarios that require real-time response.

[0088] (2) Simple data filtering: Use basic rules or thresholds to filter data, but this method may not be able to handle complex data cleaning tasks.

[0089] (3) Fixed-pattern analysis: Use preset analysis models to process data, but these models may not adapt to environmental changes.

[0090] (4) Manual intervention: Introducing manual intervention in the data analysis process is inefficient and unsustainable when faced with large amounts of data.

[0091] To overcome these limitations, IoT data cleaning and analysis systems based on data services have adopted more advanced technologies and methods, such as distributed computing, stream processing, machine learning, and adaptive algorithms, to better handle real-time data and rapidly changing environmental requirements.

[0092] The Internet of Things data processing method based on data service described in the preferred embodiment of the present invention is as follows: Figure 1 As shown, the Internet of Things data processing method based on data service includes the following steps:

[0093] Step S10: Collect initial device data of a specific type or specific purpose of the target Internet device within a preset time period.

[0094] Specifically, when the present invention collects data generated by IoT devices, the data collection generally follows the following scope or requirements:

[0095] (1) Device-specific data: The system may focus on collecting data from IoT devices of a specific type or for a specific purpose. For example, only collecting data from temperature sensors or surveillance cameras.

[0096] (2) Relevant data: Only collect data that is relevant to a specific analysis goal or business need. For example, if the goal is to optimize energy use, only data related to energy consumption may be collected.

[0097] (3) Time range: Data collection may be based on a specific time range, such as real-time data, data from the past hour, or daily summary data.

[0098] (4) Data sampling: In some cases, data sampling may be used to collect only representative samples of data rather than all the data.

[0099] (5) Data screening: Preliminary screening rules are applied during the data collection phase to transmit only data that meets specific conditions.

[0100] Specific data collection scope or requirements include:

[0101] (1) Importance: Collect data that is critical to business decision making, such as key performance indicators (KPIs).

[0102] (2) Frequency: The frequency of data collection is determined based on the importance of the data and the system processing capacity. For example, critical data may need to be collected in real time, while some non-critical data may only need to be collected periodically.

[0103] (3) Data type: Determine the type of data that needs to be collected based on analysis requirements, such as numbers, text, images, or audio and video.

[0104] (4) Data volume: Set an upper limit on the data volume to avoid exceeding the system's processing capacity.

[0105] (5) Privacy and security: Ensure that the collected data complies with relevant laws and regulations on privacy protection and data security.

[0106] Therefore, when collecting data, it is important to ensure that the collected data is both valuable and manageable, while meeting the needs of real-time and analysis.

[0107] Step S20: clean and summarize the initial device data to obtain target device data.

[0108] Specifically, the data cleaning includes: data identification, invalid data removal, missing data filling, duplicate data removal and data standardization; data identification is performed on the initial device data to determine the data source and format; incomplete, formatted incorrectly or data that is obviously beyond the normal range in the initial device data is deleted; the missing data in the initial device data is filled with the average value or median of the previous and next data or an interpolation method is used; duplicate data records in the initial device data are checked and deleted; the initial device data is converted into a unified format or scale; when the initial device data completes data identification, invalid data removal, missing data filling, duplicate data removal and data standardization, the intermediate device data is obtained; the data induction includes: data classification, data aggregation and feature extraction; the intermediate device data is classified according to certain standards; the intermediate device data is summarized; features that are helpful for analysis are extracted from the intermediate device data; when the intermediate device data completes data classification, data aggregation and feature extraction, the target device data is obtained.

[0109] In the present invention, the process of cleaning and summarizing data can be divided into the following specific steps, which will be explained below in conjunction with hypothetical IoT sensor data:

[0110] Specific steps for data cleaning:

[0111] (1) Data identification: Determine the data source and format. For example, if the data collected is from a temperature sensor, the data format may be CSV or JSON.

[0112] (2) Remove invalid data: Delete data that is incomplete, erroneous in format, or clearly outside the normal range. For example, if the data from a temperature sensor should be within a reasonable temperature range (such as -50°C to +50°C), then any data outside this range may be considered invalid and deleted.

[0113] (3) Filling in missing data: For missing data, there are many methods that can be used to fill in the missing data, such as using the average or median of the previous and next data or using interpolation methods. For example, if the temperature data at a certain time point is missing, it can be filled in with the average of the temperature data at the previous and next two time points.

[0114] (4) Remove duplicate data: Check and delete duplicate data records. For example, if data at the same time point is recorded multiple times due to a system error, the duplicates need to be deleted.

[0115] (5) Data standardization: Convert data to a uniform format or scale. For example, convert temperature data from Celsius to Fahrenheit (if necessary).

[0116] Specific steps for data summarization:

[0117] (1) Data classification: classify data according to certain standards. For example, classify temperature data according to different sensor locations or functional areas.

[0118] (2) Data aggregation: Summarize the data, such as calculating the average temperature, maximum temperature, and minimum temperature of each area.

[0119] (3) Feature extraction: Extract features from raw data that are useful for analysis. For example, trend, periodicity, and seasonality features can be extracted from temperature time series data.

[0120] For example, suppose we have the following temperature sensor data:

[0121] Time point, sensor ID, temperature (C);

[0122] 2023-09-27 12:00:00, sensor_001, 23;

[0123] 2023-09-27 12:00:00, sensor_002, 25;

[0124] 2023-09-27 12:05:00, sensor_001;

[0125] 2023-09-27 12:05:00, sensor_002, 26;

[0126] 2023-09-27 12:10:00, sensor_001, 24;

[0127] 2023-09-27 12:10:00, sensor_002, 501 (invalid data);

[0128] Data cleaning:

[0129] (1) Delete the last line because the temperature 501°C is obviously invalid.

[0130] (2) Use the average of the temperature data at 12:00 and 12:10 to fill the missing data of sensor_001 at 12:05.

[0131] Data summary:

[0132] (1) Classification: Classify the data according to the sensor ID.

[0133] (2) Aggregation: Calculate the average temperature of each sensor in the time period from 12:00 to 12:10.

[0134] (3) Feature extraction: Extract the trend of temperature change from the data, for example, the temperature of sensor_002 is rising.

[0135] Through these steps, cleaned and summarized data can be obtained, laying a good foundation for subsequent real-time processing and intelligent analysis.

[0136] Step S30: Process the target device data in real time to generate real-time data analysis results and decision support.

[0137] Specifically, the real-time processing includes: data stream processing, data filtering, data aggregation, data conversion, complex event processing and real-time prediction; using a data stream management system to ingest, process and analyze the target device data in real time; screening out key data or abnormal data in the target device data according to preset rules or thresholds; real-time aggregation of the target device data; format conversion, normalization or encoding conversion of the target device data; identifying and processing complex event patterns in the target device data; and using a machine learning model to predict the target device data in real time.

[0138] The "real-time requirements" of the present invention refer to the time constraints that the system must meet when processing and analyzing data, specifically including the following aspects:

[0139] (1) Low latency: The system needs to complete processing and analysis tasks within a very short time after the data is generated to ensure that the information is up to date and can support quick decision-making.

[0140] (2) Fast response: The system needs to be able to respond quickly to queries or events sent by IoT devices, which is especially important for monitoring and control applications.

[0141] (3) Continuous processing: The system is able to process data streams continuously, rather than processing batches of data only at specific points in time.

[0142] (4) Time sensitivity: In some application scenarios, the value of data decreases rapidly over time, so the system needs to process the data when its value is maximized.

[0143] Specific to the IoT data cleaning and analysis system, the following are some specific real-time requirements:

[0144] (1) Real-time data collection: The system can collect data from various IoT devices in real time without being limited by the amount of data.

[0145] (2) Real-time data cleaning: Before the data is further analyzed, the system can clean the data in real time to remove invalid, erroneous or irrelevant data.

[0146] (3) Real-time data analysis: The system can analyze the cleaned data in real time, providing immediate insights and decision support.

[0147] (4) Event-driven processing: The system can respond to specific events (such as equipment failure, security alarm, etc.) in real time and trigger the corresponding processing flow.

[0148] (5) Real-time feedback loop: The system can adjust device behavior or trigger alarms in real time based on the analysis results, forming an effective feedback loop.

[0149] Meeting these real-time requirements is critical to improving the performance and efficiency of IoT applications, especially in scenarios that require instant decision-making and optimization, such as smart manufacturing, smart transportation, smart healthcare, and other fields.

[0150] The real-time data processing in this invention involves immediate operations on the collected data to support real-time monitoring and decision-making of the IoT system. The following is a detailed description of real-time data processing, including the processing method, purpose and function:

[0151] Real-time processing method:

[0152] (1) Data stream processing:

[0153] Method: Use data stream management systems (such as Apache Kafka, Apache Flink) to ingest, process, and analyze data in real time.

[0154] Purpose: To ensure that data can be processed continuously and quickly, suitable for scenarios that require immediate response.

[0155] (2) Data filtering:

[0156] Method: Filter out key data or abnormal data based on preset rules or thresholds.

[0157] Purpose: To reduce the amount of data to be subsequently processed and to quickly identify potential problems or opportunities.

[0158] (3) Data aggregation:

[0159] Method: Summarize data in real time, such as calculating average, sum, maximum, minimum, etc.

[0160] Function: Provide real-time statistical information to help users quickly understand the system status.

[0161] (4) Data conversion:

[0162] Method: Perform format conversion, normalization or encoding conversion on the data.

[0163] Purpose: To ensure compatibility and consistency of data during transmission and analysis.

[0164] (5) Complex Event Processing (CEP):

[0165] Methods: Identify and process complex event patterns in data streams.

[0166] Function: Monitor the sequence of events in the Internet of Things and respond quickly to complex events.

[0167] (6) Real-time prediction:

[0168] Methods: Use machine learning models to make real-time predictions on data.

[0169] Purpose: To predict future data trends or events and support forward-looking decision making.

[0170] Specific real-time processing examples:

[0171] (1) Temperature monitoring:

[0172] Processing: Monitor the data stream of the temperature sensor in real time and trigger an alarm immediately once the temperature exceeds the set threshold.

[0173] Purpose: To prevent equipment overheating or abnormal ambient temperature.

[0174] (2) Equipment status monitoring:

[0175] Processing: Real-time analysis of the equipment's operating status data, such as current, voltage, speed, etc.

[0176] Function: Monitor the health of equipment in real time and prevent failures.

[0177] (3) Traffic flow management:

[0178] Processing: Aggregates data from multiple traffic sensors in real time to calculate average speed and volume on the road.

[0179] Purpose: Dynamically adjust traffic lights to optimize traffic flow.

[0180] (4) Anomaly Detection

[0181] Processing: Identify unusual patterns or behaviors through real-time data stream analysis.

[0182] Function: Timely detection of security threats or system anomalies, such as intrusion detection.

[0183] Through these real-time processing methods, the system can analyze and respond to data as soon as it is generated, thereby supporting the efficient operation and intelligent decision-making of the IoT system. Real-time data processing is a crucial part of IoT applications, which ensures the real-time and reliability of the system and provides users with timely and accurate information and decision-making support.

[0184] Step S40: perform in-depth analysis and processing on the target device data to generate in-depth analysis results, and provide targeted optimization suggestions based on the in-depth analysis results.

[0185] Specifically, the in-depth analysis processing includes: data preprocessing, feature engineering processing, model training and verification processing, data analysis and insight processing, and result interpretation and reporting processing; merging the cleaned target device data with other relevant data sources, and converting the merged data into a format suitable for analysis; extracting features that are helpful for analysis from the target device data, selecting the features that are most useful for model prediction, excluding irrelevant or redundant features, and creating new features; selecting a suitable machine learning algorithm based on the analysis objectives, training the model using historical data, optimizing model performance by adjusting parameters, and evaluating the accuracy and generalization ability of the model through cross-validation technology; analyzing the target device data to identify hidden patterns, associations, and anomalies, using the model to predict future trends or results, and proposing improvement measures or optimization plans based on the model output; converting the analysis results into an easy-to-understand format and creating a detailed analysis report; the analysis report includes key findings, data support, recommendations, and follow-up action plans.

[0186] In the present invention, the process of in-depth analysis of data involves using advanced data analysis techniques to mine patterns and insights in the data. The following are the specific processes and methods of in-depth analysis:

[0187] In-depth analysis process:

[0188] (1) Data preprocessing:

[0189] Data integration: Merge cleaned data with other relevant data sources to enrich the analysis dimension.

[0190] Data transformation: Converting data into a format suitable for analysis, such as encoding categorical data into numerical data.

[0191] (2) Feature Engineering:

[0192] Feature extraction: Extract features from raw data that are helpful for analysis, such as extracting trends, periodicity, and other features from time series data.

[0193] Feature selection: Select the features that are most useful for model prediction and exclude irrelevant or redundant features.

[0194] Feature construction: creating new features, such as generating new indicators by combining existing features.

[0195] (3) Model training and verification:

[0196] Select algorithm: Select appropriate machine learning algorithm according to the analysis goal, such as regression, classification, clustering, etc.

[0197] Train the model: Use historical data to train the model and adjust parameters to optimize model performance.

[0198] Model validation: Evaluate the accuracy and generalization ability of the model through techniques such as cross-validation.

[0199] (4) Data analysis and insights:

[0200] Pattern Recognition: Analyzing data to identify hidden patterns, associations, and anomalies.

[0201] Predictive analytics: Using models to predict future trends or outcomes.

[0202] Optimization analysis: Propose improvement measures or optimization plans based on model output.

[0203] (5) Interpretation and reporting of results:

[0204] Interpretation of results: Convert the analysis results into an understandable form, such as a visual chart or text description.

[0205] Report Generation: Create detailed analysis reports including key findings, supporting data, recommendations, and follow-up action plans.

[0206] Specific examples of in-depth analysis:

[0207] Example, energy consumption analysis of smart grid:

[0208] (1) Data preprocessing: Integrate energy consumption data from different power plants and unify timestamps and data formats.

[0209] (2) Feature engineering: extracting the time characteristics (such as season and time period) and weather characteristics (such as temperature and humidity) of energy consumption data.

[0210] (3) Model training and validation: Use historical energy consumption data to train a time series prediction model, such as a long short-term memory network (LSTM).

[0211] (4) Data analysis and insights: Analyze the relationship between energy consumption and factors such as weather and season, and identify periods of high energy consumption and potential energy-saving opportunities.

[0212] (5) Result interpretation and reporting: Generate a report to explain the future energy consumption trend predicted by the model and provide energy-saving suggestions.

[0213] Through this process, in-depth data insights can be provided for IoT applications, helping users make more accurate decisions and optimize resource allocation.

[0214] Here are some specific optimization suggestions for different situations:

[0215] Optimization suggestions for smart grid:

[0216] (1) Energy consumption optimization:

[0217] Recommendation: Adjust grid load and optimize energy distribution.

[0218] Specific operations: When peak electricity consumption is predicted, it is recommended to start backup power generation facilities in advance or adjust demand response strategies to balance supply and demand.

[0219] (2) Preventive maintenance:

[0220] Recommendation: Predict potential failures based on equipment operating data.

[0221] Specific actions: For equipment that is predicted to fail, it is recommended to perform maintenance inspection or replacement in advance.

[0222] Optimization suggestions for smart manufacturing:

[0223] (1) Improved production efficiency:

[0224] Suggestion: Optimize the production line operation process.

[0225] Specific operations: By analyzing production data, identifying bottlenecks, and recommending adjustments to the production line layout or the introduction of automated equipment.

[0226] (2) Quality Control:

[0227] Suggestion: Improve product quality inspection process.

[0228] Specific operations: Use machine learning models to analyze product quality data and provide a basis for improving testing standards or methods.

[0229] Optimization suggestions for intelligent transportation systems:

[0230] (1) Traffic flow management:

[0231] Suggestion: Dynamically adjust traffic lights.

[0232] Specific operations: Based on real-time traffic flow data, it is recommended to adjust the traffic light cycle to reduce traffic congestion.

[0233] (2) Vehicle maintenance:

[0234] Recommendation: Perform predictive maintenance based on vehicle operation data.

[0235] Specific actions: For vehicle parts that may be worn as shown by the analysis results, it is recommended to inspect and replace them in advance.

[0236] Optimization suggestions for smart cities:

[0237] (1) Environmental monitoring:

[0238] Recommendation: Improve urban environmental planning.

[0239] Specific actions: By analyzing air quality data, it is recommended to increase green belts or limit emissions from pollution sources.

[0240] (2) Public safety:

[0241] Recommendation: Optimize emergency response paths.

[0242] What it does: Based on historical emergency response data, it recommends adjustments to emergency vehicle routes to reduce response times.

[0243] Optimization suggestions for smart home:

[0244] (1) Energy management:

[0245] Recommendation: Optimize home energy use.

[0246] Specific operations: By analyzing household energy consumption data, it is recommended to adjust the working mode of the air conditioning or lighting system to achieve energy saving.

[0247] (2) Security monitoring:

[0248] Recommendation: Enhance home security protection.

[0249] Specific actions: If abnormal activities are detected, it is recommended to adjust the monitoring range of security cameras or add a security alarm system.

[0250] These optimization recommendations are based on in-depth analysis of the data generated by IoT devices and are designed to help users or system operators improve efficiency, reduce costs, and enhance safety and reliability.

[0251] The present invention combines the functions of data cleaning, real-time processing and intelligent analysis, and can process data in the Internet of Things environment in real time and provide accurate analysis results and decision support; clean, format and summarize the data generated by the Internet of Things to remove noise and ensure the accuracy of the data; can process data efficiently and provide real-time data analysis and decision support according to real-time requirements; use machine learning and data mining and other technologies to analyze historical and real-time data and provide predictions and optimization suggestions.

[0252] Beneficial effects of the present invention:

[0253] (1) Provide real-time data cleaning and analysis functions to improve data quality and accuracy.

[0254] (2) Rapidly process and analyze large amounts of sensor data in response to real-time requirements in IoT scenarios.

[0255] (3) Provide deeper data mining and decision support to help optimize the application effects of IoT.

[0256] Furthermore, if Figure 2 As shown, based on the above-mentioned Internet of Things data processing method based on data service, the present invention also provides an Internet of Things data processing system based on data service, wherein the Internet of Things data processing system based on data service includes:

[0257] The data collection module 51 is used to collect initial device data of a specific type or specific purpose of a target Internet device within a preset time period;

[0258] The data processing module 52 is used to clean and summarize the initial device data to obtain target device data;

[0259] A real-time processing module 53 is used to process the target device data in real time and generate real-time data analysis results and decision support;

[0260] The intelligent analysis module 54 is used to perform in-depth analysis and processing on the target device data, generate in-depth analysis results, and provide targeted optimization suggestions based on the in-depth analysis results.

[0261] Furthermore, if Figure 3 As shown, based on the above-mentioned Internet of Things data processing method and system based on data service, the present invention also provides a terminal accordingly, and the terminal includes a processor 10, a memory 20 and a display 30. Figure 3 Only some components of the terminal are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0262] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal. In other embodiments, the memory 20 may also be an external storage device of the terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the memory 20 may also include both an internal storage unit of the terminal and an external storage device. The memory 20 is used to store application software and various types of data installed on the terminal, such as the program code of the installation terminal. The memory 20 may also be used to temporarily store data that has been output or is to be output. In one embodiment, the memory 20 stores an Internet of Things data processing program 40 based on data services, and the Internet of Things data processing program 40 based on data services can be executed by the processor 10, thereby realizing the Internet of Things data processing method based on data services in this application.

[0263] In some embodiments, the processor 10 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run the program code or process data stored in the memory 20, such as executing the IoT data processing method based on data services.

[0264] In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 30 is used to display information on the terminal and to display a visual user interface. The components 10-30 of the terminal communicate with each other via a system bus.

[0265] In one embodiment, when the processor 10 executes the data service-based Internet of Things data processing program 40 in the memory 20, the steps of the data service-based Internet of Things data processing method as described above are implemented.

[0266] The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores an Internet of Things data processing program based on data services, and when the Internet of Things data processing program based on data services is executed by a processor, the steps of the Internet of Things data processing method based on data services as described above are implemented.

[0267] In summary, the present invention provides an IoT data processing method, system, terminal and computer-readable storage medium based on data services, the method comprising: collecting initial device data of a specific type or specific purpose of a target Internet device within a preset time period; performing data cleaning and data summarization on the initial device data to obtain target device data; performing real-time processing on the target device data to generate real-time data analysis results and decision support; performing in-depth analysis and processing on the target device data to generate in-depth analysis results, and providing targeted optimization suggestions based on the in-depth analysis results. The present invention provides real-time data cleaning and analysis functions, improves data quality and accuracy, and can quickly process and analyze large amounts of data in response to real-time requirements in IoT scenarios, and can quickly process data in IoT environments in real time, and provide accurate analysis results and decision support.

[0268] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or terminal including the element.

[0269] Of course, those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer-readable storage medium that can be read by a computer, and the program can include the processes of the above-mentioned method embodiments when executed. The computer-readable storage medium can be a memory, a disk, an optical disk, etc.

[0270] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A data processing method for Internet of Things based on data service, characterized in that: The Internet of Things data processing method based on data service includes: Collect initial device data of a specific type or purpose from the target Internet device within a preset time period; Performing data cleaning and data summarization on the initial device data to obtain target device data; Process the target device data in real time to generate real-time data analysis results and decision support; The target device data is subjected to in-depth analysis and processing to generate in-depth analysis results, and targeted optimization suggestions are provided based on the in-depth analysis results.

2. The method for processing Internet of Things data based on data service according to claim 1, characterized in that: The collecting of initial device data of a specific type or specific purpose of the target Internet device within a preset time period specifically includes: Collecting initial device data of a specific type or specific purpose of a target Internet device within a preset time period according to preliminary screening rules; Collect initial device data related to specific analysis objectives or business needs from target Internet devices within a preset time period based on preliminary screening rules.

3. The method for processing Internet of Things data based on data service according to claim 1, characterized in that: The data cleaning includes: data identification, invalid data removal, missing data filling, duplicate data removal and data standardization; the data summarization includes: data classification, data aggregation and feature extraction.

4. The method for processing Internet of Things data based on data service according to claim 3, characterized in that: The data cleaning and data summarizing of the initial device data to obtain the target device data specifically includes: Performing data identification on the initial device data to determine the data source and format; Delete the data in the initial device data that is incomplete, erroneous in format, or obviously exceeds the normal range; Fill in missing data in the initial device data using the mean or median of the previous and subsequent data or using an interpolation method; Checking and deleting duplicate data records in the initial device data; Converting the initial device data into a uniform format or scale; After the initial device data completes data identification, invalid data removal, missing data filling, duplicate data removal and data standardization, intermediate device data is obtained; Classifying the intermediate device data according to certain standards; Aggregating the intermediate device data; extracting features from the middleware data that are useful for analysis; After the intermediate device data completes data classification, data aggregation and feature extraction, the target device data is obtained.

5. The method for processing Internet of Things data based on data service according to claim 1, characterized in that: The real-time processing includes: data stream processing, data filtering, data aggregation, data conversion, complex event processing and real-time prediction; The real-time processing of the target device data to generate real-time data analysis results and decision support specifically includes: Using a data stream management system to ingest, process and analyze the target device data in real time; According to preset rules or thresholds, filter out key data or abnormal data in the target device data; Summarizing the target device data in real time; Performing format conversion, normalization or encoding conversion on the target device data; Recognizing and processing complex event patterns in the target device data; A machine learning model is used to make real-time predictions on the target device data.

6. The method for processing Internet of Things data based on data service according to claim 1, characterized in that: The in-depth analysis process includes: data preprocessing, feature engineering processing, model training and verification processing, data analysis and insight processing, and result interpretation and reporting processing; The in-depth analysis and processing of the target device data to generate in-depth analysis results, and providing targeted optimization suggestions based on the in-depth analysis results, specifically includes: Merging the cleaned target device data with other relevant data sources, and converting the merged data into a format suitable for analysis; Extract features that are helpful for analysis from the target device data, select the features that are most useful for model prediction, exclude irrelevant or redundant features, and create new features; Select appropriate machine learning algorithms based on the analysis objectives, train models using historical data, optimize model performance by adjusting parameters, and evaluate model accuracy and generalization capabilities through cross-validation techniques; Analyze the target device data to identify hidden patterns, associations, and anomalies, use models to predict future trends or results, and propose improvement measures or optimization plans based on model outputs; Convert analysis results into an easy-to-understand format and create detailed analysis reports.

7. The method for processing Internet of Things data based on data service according to claim 6, characterized in that: The analysis report includes key findings, supporting data, recommendations and follow-up action plans.

8. An Internet of Things data processing system based on data services, characterized in that: The Internet of Things data processing system based on data services includes: A data collection module, used to collect initial device data of a specific type or specific purpose of a target Internet device within a preset time period; A data processing module, used for cleaning and summarizing the initial device data to obtain target device data; A real-time processing module, used to process the target device data in real time and generate real-time data analysis results and decision support; The intelligent analysis module is used to perform in-depth analysis and processing on the target device data, generate in-depth analysis results, and provide targeted optimization suggestions based on the in-depth analysis results.

9. A terminal, characterized in that: The terminal includes: a memory, a processor, and an Internet of Things data processing program based on data services stored in the memory and executable on the processor. When the Internet of Things data processing program based on data services is executed by the processor, the steps of the Internet of Things data processing method based on data services as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an Internet of Things data processing program based on a data service, and when the Internet of Things data processing program based on a data service is executed by a processor, the steps of the Internet of Things data processing method based on a data service as described in any one of claims 1 to 7 are implemented.

Images