Sensing device data processing method and processing system based on AIoT

Abstract

The invention discloses a sensing device data processing method and processing system based on AIoT, and provides a set of complete solution for the challenge of sensing data processing in the Internet of Things. The system obtains data of various types of sensors in real time, and the data processing efficiency is improved through independent task design and a buffer area technology. And meanwhile, the data fusion module realizes integration and sharing of different sensor data, so that the data utilization rate is increased. And the data analysis module deeply mines data features by using an advanced algorithm to improve the analysis accuracy. And the feedback module is combined with a deep learning algorithm to realize intelligent processing and decision suggestion. In addition, the system supports flexible data storage and management modes, and diversified requirements are met. Generally speaking, the system effectively solves key problems in sensing data processing, improves processing efficiency, utilization rate and analysis accuracy, promotes development of intelligent application in the field of Internet of Things, and provides powerful technical support and solutions for related fields.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a sensor device data processing method and processing system based on AIoT. Background Art

[0002] With the rapid development of IoT technology, more and more devices and systems are connected to the network, forming a huge data system. Among these data, the data generated by sensors occupies an important position. As an important part of the perception layer of IoT, sensors can monitor and collect various environmental parameters and device status in real time, providing rich data support for upper-layer applications.

[0003] However, faced with massive amounts of sensor data, how to efficiently process, store and analyze this data has become an urgent problem to be solved. Traditional data processing methods often have drawbacks such as slow processing speed, low data utilization, and inaccurate analysis results. Especially when faced with heterogeneous data generated by multiple types of sensors, the difficulty and complexity of data processing are significantly increased.

[0004] In addition, with the continuous advancement of artificial intelligence technology, combining AI algorithms with IoT technology to achieve intelligent data processing and analysis has become an important trend in the development of IoT. However, how to effectively apply AI technology to sensor data processing and improve the level of intelligent data processing is also a technical problem that needs to be solved. Summary of the invention

[0005] 1. Technical issues to be resolved To solve the above problems, the present invention proposes a data processing method and system for a sensor device based on AIoT, aiming to solve the shortcomings of traditional data processing methods, such as slow processing speed, low data utilization, and inaccurate analysis results.

[0006] (II) Technical solution A sensor device data processing method based on AIoT of the present invention comprises: Independently acquire data collected by multiple sensor devices in real time, generate data streams based on the collected data, store the data streams in the area to be processed, temporarily store data and uniformly process the data streams of the same type, and process the data streams based on asynchronous tasks; Assigning a timestamp to each of the data streams, obtaining a first eigenvalue by calculating a statistical index of each timestamp, and integrating the different data streams into a data set based on the first eigenvalue; Optimizing the data set to obtain a second eigenvalue, and integrating the data in the plurality of specific files into one data set to form a specific data set based on the second eigenvalue, and obtaining an optimization decision; Analyze the data set in real time based on optimization decision combined with deep learning algorithm, record and evaluate production effect, generate feedback result, and adjust the second characteristic value based on the feedback result; The data sets are stored in specific files and databases.

[0007] In the present invention, the data stream is transmitted and stored in the area to be processed through a data channel to determine whether the area to be processed is fully loaded; If yes, merging the data streams into the data set; If not, the area to be processed is refreshed.

[0008] In the present invention, a first eigenvalue is obtained by analyzing statistical indicators of timestamps based on a time series model, and different data streams having the first eigenvalue are integrated together to form the data set, wherein the first eigenvalue is the linearity or hysteresis of the sensor device.

[0009] In the present invention, noise and outliers in the data set are removed, missing data are processed, and the accuracy and consistency of the data are ensured; The data is standardized according to actual production requirements to extract the second eigenvalue; The data from multiple specific files are integrated into one data set based on the second characteristic value for in-depth analysis and decision making, and based on the integrated data set and previous analysis results, optimization decisions are made to support generation development.

[0010] Another sensor device data processing system based on AIoT of the present invention is used to implement the sensor device data processing method described in the above technical solution, and the sensor device data processing system includes: At least two sensors of different types, A sensor data acquisition module, based on an independent task, acquires the data of the sensor in real time and obtains a data stream; A data storage and processing module, used for storing the data stream in a buffer, and performing data processing to obtain a first characteristic value, which is used for integrating the different data streams into a data set; A data fusion module, used to fuse data from different sensors into one data set and extract a second eigenvalue; The timestamp management module is used to assign a timestamp to each piece of data and manage the time sequence of the data; Data storage module, used to save processed data into CSV files and support data storage in databases; Data aggregation module, used to merge data from multiple CSV files or databases into one dataset; Data analysis module, used to analyze the features in the data set and generate statistical reports; The feedback module combines deep learning algorithms to analyze data in real time, record and evaluate on-site effects, and provide a feedback mechanism.

[0011] In the present invention, the sensor data acquisition module sends data to the data storage and processing module through a data channel, caches the data in a buffer, merges the data streams and writes them into a CSV file when the buffer is full, and refreshes the next CSV file after each write.

[0012] In the present invention, the timestamp management module can assign a timestamp to each data stream and calculate the statistical index of each timestamp under specific conditions, so as to obtain the first characteristic value in the data set.

[0013] (III) Beneficial effects Compared with the prior art, the present invention has the following beneficial effects: In the present invention, the independent task design of the sensor data acquisition module realizes the real-time acquisition and efficient transmission of data. At the same time, the buffer design of the data storage and processing module effectively copes with the suddenness and instability of the data flow and improves the efficiency of data processing.

[0014] In the present invention, the data from different sensors are fused into a unified data set through a data fusion module, thereby realizing data integration and sharing, which not only improves the utilization rate of data, but also provides a richer and more comprehensive data foundation for subsequent data analysis and mining. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a schematic diagram of the overall logical structure of the sensor device data processing method; Figure 2 A logic block diagram of a method for processing data from a sensor device; Figure 3 Schematic diagram of the functional flow for writing data to a specific file. DETAILED DESCRIPTION

[0017] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. Embodiment 1:

[0018] like Figure 1-Figure 3 This embodiment provides a sensor device data processing method based on AIoT (Artificial Intelligence Internet of Things), which aims to efficiently and accurately process data collected by multiple sensor devices and perform real-time analysis through deep learning algorithms to optimize production decisions. The following is a detailed description of the specific implementation steps of the method.

[0019] S100, independently acquiring collected data from multiple sensor devices in real time, generating a data stream based on the collected data, storing the data stream in a to-be-processed area, and processing the data stream based on an asynchronous task.

[0020] The collected data is acquired independently from multiple sensor devices in real time. These data may include various physical quantities such as temperature, humidity, pressure, speed, etc. The acquired data is organized into a data stream and temporarily stored in the processing area. This processing area is used as a temporary buffer for data to uniformly process the same type of data streams for subsequent asynchronous task processing. In this way, the system can efficiently manage and process a large number of concurrent data streams.

[0021] S200, assigning a timestamp to each of the data streams, obtaining a first eigenvalue by calculating a statistical index of each timestamp, and integrating the different data streams into a data set based on the first eigenvalue.

[0022] Next, a timestamp is assigned to each data stream to record the specific time point when the data was generated, which is convenient for subsequent time series analysis. By calculating the statistical indicators of each timestamp, such as the mean, standard deviation, and trend, the system can extract the first eigenvalue of the data stream. These first eigenvalues ​​may represent key performance parameters such as linearity and hysteresis of the sensor device. Based on these first eigenvalues, the system integrates different data streams into a data set for higher-level data analysis.

[0023] S300, optimizing the data set to obtain a second characteristic value, and integrating the data in the plurality of specific files into one data set to form a specific data set based on the second characteristic value, and obtaining an optimization decision.

[0024] After the data set is integrated, the system will optimize the data set. This step includes removing noise and outliers and processing missing data to ensure the accuracy and consistency of the data. Subsequently, the data is standardized according to actual production needs to extract the second eigenvalue. These second eigenvalues ​​may be closer to key indicators in the production process, such as production efficiency, energy consumption level, etc. Based on the second eigenvalue, the system can integrate the data from multiple specific files into one data set to form a specific data set, which provides a basis for in-depth analysis and decision making.

[0025] S400. Analyze the data set in real time based on optimization decision combined with deep learning algorithm, record and evaluate production effect, generate feedback result, and adjust the second characteristic value based on the feedback result.

[0026] Next, the system uses deep learning algorithms to analyze specific data sets in real time. By training and testing deep learning models, the system is able to identify and predict various patterns in the production process, record and evaluate production results. Based on the feedback results generated by the analysis, the system can dynamically adjust the second eigenvalue to optimize production decisions. This feedback mechanism enables the system to continuously learn and adapt to changes in the production environment, thereby improving production efficiency and product quality.

[0027] The combination of optimization decision and deep learning algorithm is a key step. In specific implementation, first select a suitable deep learning model, such as convolutional neural network (CNN) or recurrent neural network (RNN), and select the optimal model according to the data characteristics. For the extraction of the second eigenvalue, the feature extraction layer in the model is used to automatically learn and extract key features from the data set. These features can more comprehensively reflect the inherent laws of the data set.

[0028] When analyzing the data set in real time, the extracted second eigenvalue is input into the deep learning model, and the model will make predictions and classifications based on these features to generate an evaluation of the production effect. Then, based on the output of the model, that is, the feedback result, it can be judged whether the current production status meets the expectations. If not, the second eigenvalue is adjusted according to the model's suggestions. This adjustment process may be to modify the feature extraction method or adjust the model parameters to make subsequent data processing more accurate.

[0029] This process will continue to iterate, and the deep learning model will be able to analyze the data set more accurately and provide more precise feedback results through continuous learning and optimization, thereby guiding the adjustment of the second eigenvalue and forming a closed-loop optimization process. Ultimately, this optimization decision-making method combined with the deep learning algorithm can significantly improve the effect of data processing and provide strong data support for production.

[0030] S500: Store the data set in a specific file and database.

[0031] Finally, the data set after the optimized decision is stored in a specific file and database. This not only ensures the security and reliability of the data, but also facilitates subsequent data query and analysis. By building such a data processing flow, the system can achieve efficient processing and analysis of sensor device data, providing strong support for the company's production decision-making.

[0032] In summary, the sensor device data processing method based on AIoT provided in this embodiment realizes efficient management and utilization of sensor device data through the steps of real-time acquisition and processing of sensor device data, extraction of feature values, optimization of data sets, analysis and decision-making using deep learning algorithms, etc. This method not only improves the accuracy and efficiency of data processing, but also provides strong support for production optimization and decision-making of enterprises. Embodiment 2:

[0033] The figure shows a sensor device data processing system based on AIoT, which can efficiently process data from different types of sensors and realize data collection, storage, processing, fusion, analysis and feedback through a series of modules. The following is a detailed description of the components of the system and how they work.

[0034] First, the system contains at least two different types of sensors, which can be temperature sensors, humidity sensors, pressure sensors, etc., for real-time monitoring of different environmental parameters. The sensor data acquisition module acquires the data of these sensors in real time based on independent tasks to form a continuous data stream. This module ensures the real-time and accuracy of the data and provides a reliable basis for subsequent processing.

[0035] After data acquisition, the sensor data acquisition module sends the data to the data storage and processing module through the data channel. The module first caches the data in the buffer to cope with the suddenness and instability of the data stream. When the buffer is full, the data storage and processing module will fuse the data stream and write it to the CSV file. After each write, the next CSV file will be refreshed and prepared to ensure the convenience of continuous storage and management of data.

[0036] In the data storage and processing module, it is also responsible for preliminary processing of the data and extracting the first eigenvalue. The first eigenvalue is obtained by counting and analyzing the data in the buffer, and can include statistical indicators such as average value, standard deviation, maximum value, minimum value, etc. These eigenvalues ​​are helpful for subsequent data analysis and pattern recognition. At the same time, the data storage and processing module can also integrate different data streams into data sets, which facilitates subsequent data fusion.

[0037] Next, the data fusion module fuses the data from different sensors into a unified data set. During this process, the module will consider factors such as the timestamp, data format, and eigenvalue of each sensor data to ensure the accuracy and consistency of the data. After data fusion, the module will further extract the second eigenvalue, which may involve higher-level data processing results such as correlation analysis and trend prediction of multiple sensor data.

[0038] The timestamp management module plays a vital role in the entire system. It assigns a unique timestamp to each piece of data to ensure the time order and synchronization of the data. In addition, the timestamp management module can also calculate the statistical indicators of each timestamp under certain conditions, such as time interval, data change rate, etc., to further enrich the first feature value in the data set.

[0039] The processed data is saved in CSV files through the data storage module, and also supports data storage in the database. This makes data management and query more flexible and efficient. The data aggregation module is responsible for merging data from multiple CSV files or databases into one data set, making large-scale data analysis and mining possible.

[0040] The data analysis module uses advanced algorithms and models to conduct in-depth analysis of the characteristics of the data set. In the form of statistical reports, it displays the distribution patterns, anomalies, trend forecasts and other information of the data. These reports provide strong data support for decision makers.

[0041] Finally, the feedback module combines deep learning algorithms to analyze data in real time, record and evaluate on-site effects. Through continuous learning and optimization, the feedback module can provide more accurate predictions and decision-making suggestions. At the same time, it also has a feedback mechanism that can adjust data processing and analysis strategies based on actual results to ensure continuous improvement and optimization of the system.

[0042] In summary, the AIoT-based sensor device data processing system provided in this embodiment can achieve comprehensive data collection, efficient storage, accurate processing and intelligent analysis. Through the collaborative work of various modules, the system can provide users with rich and accurate data support, and provide strong guarantee for intelligent applications in the field of Internet of Things.

[0043] The above-described embodiments are merely descriptions of preferred implementations of the present invention, and are not intended to limit the concept and scope of the present invention. Without departing from the design concept of the present invention, various modifications and improvements made by ordinary persons in the art to the technical solution of the present invention should fall within the protection scope of the present invention, and the technical contents for which protection is sought in the present invention have been fully recorded in the claims.

Claims

1. A sensor device data processing method based on AIoT, characterized in that: The sensor device data processing method comprises: Independently acquire data collected by multiple sensor devices in real time, generate data streams based on the collected data, store the data streams in the area to be processed, temporarily store data and uniformly process the data streams of the same type, and process the data streams based on asynchronous tasks; Assigning a timestamp to each of the data streams, obtaining a first eigenvalue by calculating a statistical index of each timestamp, and integrating the different data streams into a data set based on the first eigenvalue; Optimizing the data set to obtain a second eigenvalue, and integrating the data in the plurality of specific files into one data set to form a specific data set based on the second eigenvalue, and obtaining an optimization decision; Analyze the data set in real time based on optimization decision combined with deep learning algorithm, record and evaluate production effect, generate feedback result, and adjust the second characteristic value based on the feedback result; The data sets are stored in specific files and databases.

2. The AIoT-based sensor device data processing method according to claim 1, characterized in that: The data stream is transmitted and stored in the area to be processed through a data channel, and it is determined whether the area to be processed is fully loaded; If yes, merging the data streams into the data set; If not, the area to be processed is refreshed.

3. The AIoT-based sensor device data processing method according to claim 2, characterized in that: A first eigenvalue is obtained by analyzing statistical indicators of the timestamp based on a time series model, and different data streams having the first eigenvalue are integrated together to form the data set, wherein the first eigenvalue is the linearity or hysteresis of the sensor device.

4. The AIoT-based sensor device data processing method according to claim 3, characterized in that: Remove noise and outliers from the dataset, handle missing data, and ensure data accuracy and consistency; The data is standardized according to actual production requirements to extract the second eigenvalue; The data from multiple specific files are integrated into one data set based on the second characteristic value for in-depth analysis and decision making, and based on the integrated data set and previous analysis results, optimization decisions are made to support generation development.

5. A sensor device data processing system based on AIoT, characterized in that: The sensor device data processing system is used to implement the sensor device data processing method according to any one of claims 1 to 5, and the sensor device data processing system comprises: At least two sensors of different types, A sensor data acquisition module, based on an independent task, acquires the data of the sensor in real time and obtains a data stream; A data storage and processing module, used for storing the data stream in a buffer, and performing data processing to obtain a first characteristic value, which is used for integrating the different data streams into a data set; A data fusion module, used to fuse data from different sensors into one data set and extract a second eigenvalue; The timestamp management module is used to assign a timestamp to each piece of data and manage the time sequence of the data; Data storage module, used to save processed data into CSV files and support data storage in databases; Data aggregation module, used to merge data from multiple CSV files or databases into one dataset; Data analysis module, used to analyze the features in the data set and generate statistical reports; The feedback module combines deep learning algorithms to analyze data in real time, record and evaluate on-site effects, and provide a feedback mechanism.

6. The AIoT-based sensor device data processing system according to claim 5, characterized in that: The sensor data acquisition module sends the data to the data storage and processing module through the data channel, caches the data in the buffer, merges the data streams and writes them into a CSV file when the buffer is full, and refreshes the next CSV file after each writing.

7. The AIoT-based sensor device data processing system according to claim 6, characterized in that: The timestamp management module may assign a timestamp to each data stream and calculate statistical indicators of each timestamp under specific conditions, thereby obtaining a first characteristic value in the data set.

Images