Internet of Things data storage method and system

Abstract

The present invention discloses a method and system for storing data of the Internet of Things, the method comprising: the edge end obtains the Internet of Things data, and classifies and processes the Internet of Things data to obtain first priority data and second priority data; the edge end stores the first priority data in itself, and processes the second priority data and sends it to the middle end; the middle end processes the second priority data, obtains the long-term storage data in the second priority data, and processes the long-term storage data and sends it to the cloud end; when the middle end or the cloud end receives a data query instruction, the data query processing is performed according to the data query instruction to obtain a target query result. The present invention effectively improves the storage efficiency of the Internet of Things data by hierarchically setting the edge end, the middle end and the cloud end, and performing hierarchical storage according to the priority of the Internet of Things data, and can well meet the growing data storage needs of Internet of Things applications.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a method and system for storing data in the Internet of Things. Background Art

[0002] With the rapid development of IoT technology, a large number of devices and sensors continue to generate massive amounts of data. At present, common IoT data storage methods include centralized storage and distributed storage, but when IoT data appears on a large scale and with high concurrency, these storage methods cannot meet the actual storage needs, resulting in low efficiency of IoT data storage.

[0003] Therefore, the prior art still needs to be improved and developed. Summary of the invention

[0004] The main purpose of the present invention is to provide an Internet of Things data storage method and system, aiming to solve the problem in the prior art that when Internet of Things data appears on a large scale and with high concurrency, centralized storage and distributed storage cannot meet actual storage needs, thus resulting in low Internet of Things data storage efficiency.

[0005] To achieve the above object, the present invention provides an Internet of Things data storage method, which comprises the following steps:

[0006] The edge terminal obtains the IoT data, and classifies and processes the IoT data to obtain first priority data and second priority data;

[0007] The edge end stores the first priority data in itself, and processes the second priority data and sends it to the middle end;

[0008] The middle end processes the second priority data, obtains the long-term storage data in the second priority data, and processes the long-term storage data and sends it to the cloud;

[0009] When the intermediate end receives the data query instruction, it performs data query processing on the data stored in the intermediate end according to the data query instruction, or when the cloud receives the data query instruction, it performs data query processing on the long-term storage data stored in the cloud according to the data query instruction to obtain the target query result.

[0010] Optionally, the IoT data storage method, wherein the edge terminal obtains IoT data and classifies the IoT data to obtain first priority data and second priority data, specifically includes:

[0011] The edge terminal obtains IoT data generated by the IoT device and determines preset classification requirements, wherein the preset classification requirements include data importance, data timeliness, and data access frequency;

[0012] The edge terminal classifies the IoT data according to the preset classification requirements to obtain a first classification result and a second classification result;

[0013] The edge end obtains priority attributes of the first classification result and the second classification result, and sets attribute labels for the first classification result and the second classification result according to the priority attributes to obtain first priority data and second priority data.

[0014] Optionally, the IoT data storage method, wherein the edge end stores the first priority data in itself and processes the second priority data and sends it to the intermediate end, specifically includes:

[0015] The edge terminal stores the first priority data in its own edge storage area;

[0016] The edge end determines a preset compression algorithm and a preset encryption algorithm, and the edge end compresses the second priority data according to the preset compression algorithm to obtain compressed second priority data;

[0017] The edge end encrypts the compressed second priority data according to the preset encryption algorithm to obtain encrypted second priority data, and sends the encrypted second priority data to the intermediate end.

[0018] Optionally, in the IoT data storage method, the edge terminal stores the first priority data in its own edge storage area, and then further comprises:

[0019] The edge end performs real-time data monitoring on the first priority data. When the data volume corresponding to the first priority data exceeds the preset data volume, the excess data in the first priority data that exceeds the preset data volume is extracted and sent to the middle end.

[0020] Optionally, in the IoT data storage method, the priority attribute includes a first priority attribute corresponding to the first priority data and a second priority attribute corresponding to the second priority data;

[0021] The edge end stores the first priority data in its own edge storage area, and then further includes:

[0022] When the first priority attribute of the first priority data is changed to the second priority attribute corresponding to the second priority data, the first priority data is sent to the intermediate end.

[0023] Optionally, the IoT data storage method, wherein the intermediate end processes the second priority data, obtains long-term storage data in the second priority data, and processes the long-term storage data and sends it to the cloud, specifically includes:

[0024] When the intermediate end receives the encrypted second priority data, it decrypts and decompresses the encrypted second priority data to obtain the second priority data;

[0025] The intermediate end performs data processing on the second priority data and obtains long-term storage data in the second priority data, wherein the data processing includes abnormal value processing, redundant data processing, format conversion processing and data conversion processing;

[0026] The intermediate end performs the compression processing and the encryption processing on the long-term storage data to obtain encrypted long-term storage data, and sends the encrypted long-term storage data to the cloud.

[0027] Optionally, in the IoT data storage method, the intermediate end processes the second priority data and obtains long-term storage data in the second priority data, and then further comprises:

[0028] The intermediate end performs data analysis processing on the long-term stored data to obtain data analysis results, wherein the data analysis processing includes data trend analysis processing and data statistical analysis processing.

[0029] Optionally, the IoT data storage method, wherein the intermediate end processes the second priority data, obtains long-term storage data in the second priority data, and processes the long-term storage data and sends it to the cloud, further comprising:

[0030] When the cloud receives the encrypted long-term storage data, it archives the encrypted long-term storage data to obtain archived long-term storage data.

[0031] Optionally, the IoT data storage method, wherein when the intermediate end receives a data query instruction, data query processing is performed on the data stored in the intermediate end according to the data query instruction, or when the cloud receives the data query instruction, data query processing is performed on the long-term storage data stored in the cloud according to the data query instruction to obtain a target query result, specifically includes:

[0032] Establishing a distributed index mechanism according to preset information, wherein the preset information includes a timestamp, an IoT device ID, and a data type, and the distributed index mechanism includes an inverted index and a hash index;

[0033] When the intermediate end receives the first data query instruction sent by the user, the distributed index mechanism is used to perform data query processing on the intermediate end storage area according to the first data query instruction to obtain a first target query result;

[0034] Alternatively, when the cloud receives the second data query instruction sent by the user, the distributed index mechanism is used to perform data query processing on the cloud storage area according to the second data query instruction to obtain a second target query result.

[0035] In addition, to achieve the above-mentioned purpose, the present invention also provides an Internet of Things data storage system, wherein the Internet of Things data storage system comprises: an edge end, an intermediate end and a cloud end; the intermediate end is connected to the edge end and the cloud end respectively;

[0036] The edge terminal is used to obtain IoT data and classify the IoT data to obtain first priority data and second priority data;

[0037] The edge end is further used to store the first priority data in itself, and send the second priority data to the middle end after processing;

[0038] The intermediate end is used to process the second priority data, obtain the long-term storage data in the second priority data, and send the long-term storage data to the cloud after processing;

[0039] When the intermediate end or the cloud end receives a data query instruction, it performs data query processing according to the data query instruction to obtain a target query result.

[0040] In the present invention, the edge end obtains IoT data, and classifies and processes the IoT data to obtain first priority data and second priority data; the edge end stores the first priority data in itself, and processes the second priority data and sends it to the middle end; the middle end processes the second priority data, obtains the long-term storage data in the second priority data, and processes the long-term storage data and sends it to the cloud end; when the middle end receives a data query instruction, it performs data query processing on the data stored in the middle end according to the data query instruction, or when the cloud end receives the data query instruction, it performs data query processing on the long-term storage data stored in the cloud end according to the data query instruction to obtain the target query result. The present invention hierarchically sets the edge end, the middle end and the cloud end, and performs hierarchical storage according to the priority of the IoT data, so that the IoT data with high priority can be stored in the edge end for easy use, and the data that needs long-term storage can be stored in the cloud end, which reduces the storage pressure of the database, effectively improves the storage efficiency of the IoT data, and can well meet the growing data storage needs of IoT applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of a preferred embodiment of the method for storing data in the Internet of Things of the present invention;

[0042] Figure 2 It is a schematic diagram of a hierarchical storage architecture of a preferred embodiment of the Internet of Things data storage method of the present invention;

[0043] Figure 3 It is a schematic diagram of intelligent data management of a preferred embodiment of the Internet of Things data storage method of the present invention;

[0044] Figure 4 It is a schematic diagram of the data compression and encryption process of a preferred embodiment of the Internet of Things data storage method of the present invention;

[0045] Figure 5 It is a schematic diagram of the distributed index and query architecture of a preferred embodiment of the Internet of Things data storage method of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] With the rapid development of IoT technology, a large number of devices and sensors continue to generate massive amounts of data. Traditional IoT data storage methods face many challenges, such as limited storage capacity, low data query efficiency, low data security, and difficulty adapting to the dynamically changing IoT environment. At present, common IoT data storage methods include centralized storage and distributed storage, but these methods often cannot meet actual needs when faced with large-scale, high-concurrency IoT data.

[0048] The method for storing data in the Internet of Things according to a preferred embodiment of the present invention is as follows: Figure 1 As shown, the IoT data storage method comprises the following steps:

[0049] Step S10: The edge terminal obtains IoT data, and classifies the IoT data to obtain first priority data and second priority data.

[0050] like Figure 2 As shown, a layered IoT data storage architecture is set up in the present invention, including edge layer storage (first layer, i.e., edge end in the present invention), middle layer storage (second layer, i.e., middle end in the present invention) and cloud storage (third layer, i.e., cloud in the present invention).

[0051] When the edge obtains the IoT data, it will be classified according to the corresponding priority of the IoT data. At the same time, high-priority data will be stored in the edge and low-priority data will be stored in the cloud, so as to achieve hierarchical storage of IoT data.

[0052] Specifically, the edge end obtains the IoT data generated by the IoT device and determines preset classification requirements, wherein the preset classification requirements include data importance, data timeliness, and data access frequency; the edge end classifies the IoT data according to the preset classification requirements to obtain a first classification result and a second classification result; the edge end obtains priority attributes of the first classification result and the second classification result, and sets attribute labels for the first classification result and the second classification result according to the priority attributes to obtain first priority data and second priority data.

[0053] For the deployment of edge storage nodes: According to the distribution of IoT devices, edge storage nodes are deployed in appropriate locations. Gateways, routers or dedicated storage devices close to the devices can be selected as edge storage nodes. Edge storage nodes and IoT devices can be connected through wireless communication technologies such as Bluetooth, ZigBee, and Wi-Fi.

[0054] It is understandable that the present invention is arranged to deploy edge storage nodes (i.e., edge terminals in the present invention) near IoT devices for temporarily storing real-time data generated by the devices. Among them, edge storage nodes can use low-power, high-capacity storage devices, such as solid-state drives or flash memory cards. Edge layer storage can quickly respond to data storage requests from devices, reduce data transmission delays, and reduce the demand for network bandwidth.

[0055] When the edge is deployed, the acquired IoT data needs to be classified and labeled: different attribute labels are assigned to IoT data according to factors such as the importance, timeliness, and access frequency of IoT data. For example, data with high real-time requirements can be marked as "high priority" (i.e., the first priority data in the present invention), and data that needs to be stored for a long time can be marked as "low priority" (i.e., the second priority data in the present invention).

[0056] Step S20: The edge end stores the first priority data in itself, and processes the second priority data and sends it to the middle end.

[0057] like Figure 3 As shown, the present invention has developed an intelligent data management system for dynamically managing IoT data. According to factors such as the importance, timeliness, and access frequency of IoT data, data can be automatically migrated between different storage levels. For example, data with high real-time requirements can be stored in the edge layer; data that needs to be stored for a long time can be migrated to cloud storage. At the same time, the intelligent data management system can also regularly clean and archive data to free up storage space.

[0058] Data migration can be understood as the selection of technology for separating hot and cold data. For example, IoT log data is generally not accessed and is only queried when a problem occurs. This part of the data is cold data and can be stored in the cloud. For data that needs to be applied, such as device temperature and humidity, data for some time periods can be stored at the edge, such as storing temperature data for a week (the current device has a temperature history query function). For most statistical data in the middle layer, for example, the average temperature for the current week and the temperature year-on-year curve for the previous week, etc.

[0059] Specifically, the edge end stores the first priority data in its own edge storage area; the edge end determines a preset compression algorithm and a preset encryption algorithm, and the edge end compresses the second priority data according to the preset compression algorithm to obtain compressed second priority data; the edge end encrypts the compressed second priority data according to the preset encryption algorithm to obtain encrypted second priority data, and sends the encrypted second priority data to the intermediate end.

[0060] Regarding the data migration strategy set in the present invention: formulate a data migration strategy, and automatically migrate IoT data between different storage levels according to the attribute labels of the data and the characteristics of the storage level. For example, for high-priority data, it can be stored in the edge layer first; when the storage capacity of the edge layer is insufficient, part of the data can be migrated to the middle layer storage; for low-priority data, it can be regularly migrated to cloud storage for long-term preservation.

[0061] For data compression and encryption, the edge layer, middle layer, and cloud layer can be configured as a compression and encryption model, and configured according to the preset compression method and preset encryption method.

[0062] like Figure 4 As shown, in order to reduce data storage capacity and improve data transmission efficiency, the IoT data needs to be compressed. The present invention adopts an efficient compression algorithm (i.e., the preset compression algorithm in the present invention), such as LZ77, LZ78, etc., to compress the data losslessly. At the same time, in order to ensure the security of IoT data, it is also necessary to encrypt the compressed data (i.e., the preset encryption algorithm in the present invention), and a symmetric encryption algorithm (such as AES) or an asymmetric encryption algorithm (such as RSA) can be used to encrypt the data to ensure that the data is not stolen or tampered with during storage and transmission.

[0063] For the implementation of data compression and encryption:

[0064] 1. Data compression: By deploying data compression software on edge storage nodes and middle-tier storage servers, IoT data can be compressed in real time. The compression algorithm can be selected according to the type and characteristics of the data to achieve the best compression effect. For example: In an indoor environment monitoring system, the temperature sensor collects indoor temperature data at regular intervals. If the indoor temperature changes slowly, Delta coding can calculate the difference between two adjacent collected temperature values ​​and encode these differences. For another example, the indoor temperature rises slowly from 20 degrees Celsius to 22 degrees Celsius over a period of time, and the adjacent temperature differences are small. Delta coding can use fewer bits to represent these differences instead of directly storing each temperature value, thereby reducing the amount of data storage. For another example: For text files (such as log files, text storage data, etc.), compression algorithms based on dictionary models, such as LZ77 and LZ78, can usually be used. Taking the sensor's operation log text data as an example, there will be a large number of repeated words, phrases, and sentence structures. The LZ77 algorithm searches for the longest string in the processed text that is the same as the current part to be compressed, and replaces the repeated part with a pointer (pointing to the location where the same string appeared before) and the length of the string to achieve compression. For example, if "The current time collection data is: {}" appears multiple times in the log, the algorithm will identify this repeated phrase and encode it accordingly, effectively reducing the storage space of the repeated text.

[0065] Regarding the data encryption set in the present invention: after the IoT data is compressed, the IoT data is encrypted using encryption software, and the encryption key can be managed by the user himself or by using the key management service provided by the cloud service provider. During the data transmission and storage process, the data is always kept encrypted to ensure the security of the data.

[0066] The implementation process of data encryption is as follows: 1. Data preparation: Convert the data to be encrypted (such as sensor data collected by IoT devices) into byte array form to ensure that the data format meets the requirements of the encryption algorithm. 2. Initialize the encryptor: Use the generated symmetric key to create an AES encryptor instance. For example, in Java, you can use the getInstance("AES") method of the javax.crypto.Cipher class to obtain the AES encryptor and initialize it with the key. 3. Perform encryption operations: Input the prepared data byte array into the encryptor, which will encrypt the data according to the AES algorithm and generate an encrypted byte array (the encryption key is the device ID). 4. Store the results.

[0067] Furthermore, the edge end performs real-time data monitoring on the first priority data. When the data volume corresponding to the first priority data exceeds the preset data volume, the excess data in the first priority data that exceeds the preset data volume is extracted and the excess data is sent to the middle end.

[0068] Further, when the first priority attribute of the first priority data is changed to the second priority attribute corresponding to the second priority data, the first priority data is sent to the intermediate end.

[0069] Step S30: the middle end processes the second priority data, obtains the long-term storage data in the second priority data, and sends the processed long-term storage data to the cloud.

[0070] For mid-tier storage: The mid-tier is located between the edge layer and the cloud and consists of a group of high-performance servers. The mid-tier storage is responsible for collecting and integrating data from multiple edge storage nodes and performing preliminary data processing and analysis. The mid-tier storage can use a distributed file system or database to improve data reliability and availability.

[0071] Configuration of the middle-tier storage server: You can choose a high-performance server as the middle-tier storage node and configure a distributed file system or database software. The middle-tier storage server can be connected to the edge storage node and cloud storage via a wired or wireless network.

[0072] For cloud storage: Utilize the large-scale storage resources of cloud service providers to perform long-term storage and backup of IoT data. Cloud storage can provide highly reliable and highly available data storage services while supporting large-scale data query and analysis.

[0073] Specifically, when the intermediate end receives the encrypted second priority data, it decrypts and decompresses the encrypted second priority data to obtain the second priority data; the intermediate end processes the second priority data and obtains the long-term storage data in the second priority data, wherein the data processing includes outlier processing, redundant data processing, format conversion processing and data conversion processing; the intermediate end performs the compression processing and the encryption processing on the long-term storage data to obtain encrypted long-term storage data, and sends the encrypted long-term storage data to the cloud.

[0074] After the middle end receives the data uploaded by the edge end, since the edge end has compressed and encrypted the data during transmission, the middle end needs to decrypt and decode the data before processing it, so as to realize the subsequent data processing of the second priority data.

[0075] The data processing content of the middle end includes the following aspects: 1. Outlier processing: Since IoT devices operate in complex environments, the collected data may contain outliers caused by factors such as device failure and signal interference. The middle-tier storage can identify and correct or delete these outliers by setting a reasonable threshold range or using statistical methods (such as calculating the mean and standard deviation, and treating data that exceeds a certain multiple of the standard deviation range as outliers) to ensure the accuracy of the data for subsequent analysis and application. 2. Duplicate data removal: During the data collection and transmission process, duplicate data records may appear. The middle-tier storage can compare the key features of the data (such as timestamp, device identifier, data content, etc.), identify and delete duplicate data, reduce data redundancy, and improve data storage and processing efficiency. 3. Format unification: Different types of IoT devices may use different data formats for data collection. For example, some devices may output temperature data in Fahrenheit, while other devices may output Celsius. The middle-tier storage needs to convert these data in different formats into a unified format for unified analysis and processing to avoid analysis errors caused by inconsistent data formats. 4. Data type conversion: According to actual needs, the collected data is converted into a data type suitable for further analysis. For example, the string type digital data collected by the sensor is converted into a numeric type for mathematical calculation and statistical analysis.

[0076] Furthermore, the intermediate end performs data analysis processing on the long-term stored data to obtain data analysis results, wherein the data analysis processing includes data trend analysis processing and data statistical analysis processing.

[0077] Data analysis and processing include the following two aspects: 1. Trend analysis: Identify the trend of data changes over time by analyzing historical data. For example, analyze the change trend of the operating parameters (such as temperature, pressure, etc.) of a factory equipment over a period of time to predict whether the equipment may fail or degrade in performance, so as to take maintenance measures in advance and improve the reliability and production efficiency of the equipment. 2. Statistical analysis: Statistic a certain type of data by hour, minute, day, etc.

[0078] Furthermore, when the cloud receives the encrypted long-term storage data, it archives the encrypted long-term storage data to obtain archived long-term storage data.

[0079] The middle-tier storage can initially optimize and refine IoT data, providing cloud storage with more valuable data that is easier to manage and analyze, while also providing upper-layer applications with more efficient and accurate data services, thereby improving the performance and practicality of the entire IoT data storage solution.

[0080] Step S40: When the intermediate end receives a data query instruction, it performs data query processing on the data stored in the intermediate end according to the data query instruction, or when the cloud receives the data query instruction, it performs data query processing on the long-term storage data stored in the cloud according to the data query instruction to obtain a target query result.

[0081] Specifically, a distributed index mechanism is established according to preset information, wherein the preset information includes a timestamp, an IoT device ID, and a data type, and the distributed index mechanism includes an inverted index and a hash index; when the middle end receives a first data query instruction issued by a user, the distributed index mechanism is used to perform data query processing on the middle end storage area according to the first data query instruction to obtain a first target query result; or when the cloud end receives a second data query instruction issued by the user, the distributed index mechanism is used to perform data query processing on the cloud end storage area according to the second data query instruction to obtain a second target query result.

[0082] Choice of cloud storage: Choose a reliable cloud service provider to store IoT data in the cloud. You can use the API interface of the cloud storage service to upload, download and manage data.

[0083] like Figure 5 As shown, for the distributed index and query set in the present invention: In order to improve the efficiency of data query, a distributed index mechanism is established. The present invention is set to establish indexes in the middle layer storage and cloud storage respectively, so as to quickly locate the required data. The index can adopt data structures such as inverted index and hash index to support efficient query operations. At the same time, a distributed query engine is developed, which can perform data query in multiple storage levels at the same time, improving the speed and accuracy of the query.

[0084] Among them, the generation and query of the inverted index are implemented using the middleware ElaticSearch. First, determine the data to be indexed (such as documents, sensor data records, etc.), and extract the key information (such as words in the text, categories of sensor data, etc.). Then, create a list for each key information, and record the identifiers of all data containing the key information (such as document ID, data record number, etc.). For example, for a set of environmental monitoring data collected by IoT devices, if the monitoring location is the key information, then the inverted index list corresponding to the location "park" will contain the numbers of all data records collected in the park, and the inverted index is created.

[0085] The user sends a query request to the distributed query engine. The query request contains the key information to be queried (for example, to query the temperature data of a certain area within a specific time period, the "specific time period" and "certain area" in the query request are the key information). After that, according to the parsed key information, the corresponding list is found in the inverted index. For example, if you need to query the temperature data in the "park", find the list of "park" in the inverted index, obtain all the data record numbers that may be related, and use the obtained data identifier to actually obtain the corresponding data content from the middle-tier storage or cloud storage. For example, read the corresponding temperature data record from the storage according to the data record number.

[0086] It is understandable that the implementation of distributed indexing and querying includes the following two aspects:

[0087] 1. Index establishment: Establish indexes in the middle-tier storage and cloud storage respectively. The index can be designed according to the attributes of the data and the query requirements. For example, you can establish an index based on attributes such as timestamp, device ID, data type, etc. to quickly locate the required data.

[0088] 2. Query engine development: Develop a distributed query engine that can query data in multiple storage levels at the same time. The query engine can receive user query requests, quickly locate the required data based on index information, and return the query results to the user. The query engine can use parallel computing technology to improve the speed and efficiency of queries.

[0089] Furthermore, the present invention also provides data cleaning and archiving: regularly cleaning and archiving the data in the storage hierarchy, deleting expired or useless data, and releasing storage space. Data cleaning strategies can be formulated based on factors such as data storage time and access frequency. At the same time, important data is archived and backed up to prevent data loss.

[0090] In addition, data cleaning and archiving can be configured according to actual conditions. For example, if the maximum storage time of the current device is configured to be 3 months, there is a global scheduled task that will delete the data that exceeds the configuration every day based on the configuration and storage time. Another scheduled task is to perform scheduled cleaning based on the configured access frequency, for example, deleting data that has not been accessed for a week.

[0091] In summary, the present invention can provide an efficient, secure and scalable IoT data storage solution to solve the problems existing in the prior art and meet the growing data storage needs of IoT applications.

[0092] Technical effects of the present invention:

[0093] 1. Improve storage efficiency: Through the layered storage architecture and data compression technology, data storage capacity can be effectively reduced and storage efficiency can be improved. At the same time, the intelligent data management system can automatically migrate data between different storage levels according to the characteristics and needs of the data, make full use of various storage resources, and improve storage flexibility and scalability.

[0094] 2. Enhanced data security: The use of data encryption technology can ensure the security of IoT data during storage and transmission, and prevent data from being stolen or tampered with. At the same time, the distributed index and query mechanism can reduce the risk of data exposure and improve data security.

[0095] 3. Improve query efficiency: Establishing a distributed index and query engine can quickly locate the required data and improve the speed and accuracy of the query. At the same time, the application of parallel computing technology can further improve the efficiency of the query and meet the real-time requirements of IoT applications for data query.

[0096] 4. Adapt to the dynamically changing IoT environment: The IoT data storage solution of the present invention can adapt to the dynamically changing IoT environment. With the increase of IoT devices and the growth of data volume, it can automatically expand storage capacity and query capabilities to meet the ever-changing business needs.

[0097] Furthermore, based on the above-mentioned Internet of Things data storage method, the present invention also provides an Internet of Things data storage system accordingly, wherein the Internet of Things data storage system comprises:

[0098] An edge end, an intermediate end and a cloud end; the intermediate end is connected to the edge end and the cloud end respectively;

[0099] The edge terminal is used to obtain IoT data and classify the IoT data to obtain first priority data and second priority data;

[0100] The edge end is further used to store the first priority data in itself, and send the second priority data to the middle end after processing;

[0101] The intermediate end is used to process the second priority data, obtain the long-term storage data in the second priority data, and send the long-term storage data to the cloud after processing;

[0102] When the intermediate end or the cloud end receives a data query instruction, it performs data query processing according to the data query instruction to obtain a target query result.

[0103] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A method for storing data in the Internet of Things, characterized in that: The IoT data storage method is applied to an IoT data storage system, which includes an edge end, an intermediate end, and a cloud end; the intermediate end is connected to the edge end and the cloud end respectively; The Internet of Things data storage method comprises: The edge terminal obtains the IoT data, and classifies and processes the IoT data to obtain first priority data and second priority data; The edge end stores the first priority data in itself, and processes the second priority data and sends it to the middle end; The middle end processes the second priority data, obtains the long-term storage data in the second priority data, and sends the long-term storage data to the cloud after processing; When the intermediate end receives the data query instruction, it performs data query processing on the data stored in the intermediate end according to the data query instruction, or when the cloud receives the data query instruction, it performs data query processing on the long-term storage data stored in the cloud according to the data query instruction to obtain the target query result.

2. The IoT data storage method according to claim 1, characterized in that: The edge terminal obtains the IoT data, and classifies and processes the IoT data to obtain first priority data and second priority data, specifically including: The edge terminal obtains IoT data generated by the IoT device and determines preset classification requirements, wherein the preset classification requirements include data importance, data timeliness, and data access frequency; The edge terminal classifies the IoT data according to the preset classification requirements to obtain a first classification result and a second classification result; The edge end obtains priority attributes of the first classification result and the second classification result, and sets attribute labels for the first classification result and the second classification result according to the priority attributes to obtain first priority data and second priority data.

3. The method for storing IoT data according to claim 2, characterized in that: The edge end stores the first priority data in itself, and processes the second priority data and sends it to the middle end, specifically including: The edge terminal stores the first priority data in its own edge storage area; The edge end determines a preset compression algorithm and a preset encryption algorithm, and the edge end compresses the second priority data according to the preset compression algorithm to obtain compressed second priority data; The edge end encrypts the compressed second priority data according to the preset encryption algorithm to obtain encrypted second priority data, and sends the encrypted second priority data to the intermediate end.

4. The method for storing IoT data according to claim 3, characterized in that: The edge end stores the first priority data in its own edge storage area, and then further includes: The edge end performs real-time data monitoring on the first priority data. When the data volume corresponding to the first priority data exceeds the preset data volume, the excess data in the first priority data that exceeds the preset data volume is extracted and sent to the middle end.

5. The method for storing IoT data according to claim 3, characterized in that: The priority attributes include a first priority attribute corresponding to the first priority data and a second priority attribute corresponding to the second priority data; The edge end stores the first priority data in its own edge storage area, and then further includes: When the first priority attribute of the first priority data is changed to the second priority attribute corresponding to the second priority data, the first priority data is sent to the intermediate end.

6. The method for storing IoT data according to claim 3, characterized in that: The middle end processes the second priority data, obtains the long-term storage data in the second priority data, and processes the long-term storage data and sends it to the cloud, specifically including: When the intermediate end receives the encrypted second priority data, it decrypts and decompresses the encrypted second priority data to obtain the second priority data; The intermediate end performs data processing on the second priority data and obtains long-term storage data in the second priority data, wherein the data processing includes abnormal value processing, redundant data processing, format conversion processing and data conversion processing; The intermediate end performs the compression processing and the encryption processing on the long-term storage data to obtain encrypted long-term storage data, and sends the encrypted long-term storage data to the cloud.

7. The method for storing IoT data according to claim 6, characterized in that: The intermediate end processes the second priority data and obtains the long-term storage data in the second priority data, and then further includes: The intermediate end performs data analysis processing on the long-term stored data to obtain data analysis results, wherein the data analysis processing includes data trend analysis processing and data statistical analysis processing.

8. The method for storing IoT data according to claim 6, characterized in that: The intermediate end processes the second priority data, obtains the long-term storage data in the second priority data, and sends the processed long-term storage data to the cloud, and then further includes: When the cloud receives the encrypted long-term storage data, it archives the encrypted long-term storage data to obtain archived long-term storage data.

9. The method for storing IoT data according to claim 1, characterized in that: When the intermediate end receives the data query instruction, the data stored in the intermediate end is subjected to data query processing according to the data query instruction, or when the cloud receives the data query instruction, the data query processing is subjected to data query processing according to the data query instruction on the long-term storage data stored in the cloud to obtain the target query result, which specifically includes: Establishing a distributed index mechanism according to preset information, wherein the preset information includes a timestamp, an IoT device ID, and a data type, and the distributed index mechanism includes an inverted index and a hash index; When the intermediate end receives the first data query instruction sent by the user, the distributed index mechanism is used to perform data query processing on the intermediate end storage area according to the first data query instruction to obtain a first target query result; Alternatively, when the cloud receives the second data query instruction sent by the user, the distributed index mechanism is used to perform data query processing on the cloud storage area according to the second data query instruction to obtain a second target query result.

10. An Internet of Things data storage system, characterized in that: The IoT data storage system comprises: an edge terminal, an intermediate terminal and a cloud terminal; the intermediate terminal is connected to the edge terminal and the cloud terminal respectively; The edge terminal is used to obtain IoT data and classify the IoT data to obtain first priority data and second priority data; The edge end is further used to store the first priority data in itself, and send the second priority data to the middle end after processing; The intermediate end is used to process the second priority data, obtain the long-term storage data in the second priority data, and send the long-term storage data to the cloud after processing; When the intermediate end receives a data query instruction, it performs data query processing on the data stored in the intermediate end according to the data query instruction, or when the cloud end receives the data query instruction, it performs data query processing on the long-term storage data stored in the cloud end according to the data query instruction to obtain a target query result.

Images