An adaptive IoT data management method and related equipment

By adopting an adaptive IoT data management approach, which distinguishes between major and minor versions of data based on data volume and time sequence, and stores the data in different databases, the problem of resource waste in existing technologies is solved, and efficient IoT data management is achieved.

CN117453789BActive Publication Date: 2026-11-17CETC BIGDATA RES INST CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311595453.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-27
Publication Date
2026-11-17
Estimated Expiration
2043-11-27

AI Technical Summary

Technical Problem

Existing IoT data management methods cannot effectively meet the characteristics of multi-source heterogeneity, high time sequence, and data complexity, resulting in resource waste and failing to fully meet the management needs of IoT data.

Method used

Through an adaptive IoT data management approach, different time-series databases, sensing databases, and data warehouses are used for management based on the size and time sequence of the data, including OpenTSDB, HBase, Hive, InfluxDB, MongoDB, and PostgreSQL, and major and minor version data are stored separately.

Benefits of technology

Effectively reduce resource waste, meet the management needs of IoT data, and improve data management efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117453789B_ABST
    Figure CN117453789B_ABST
Patent Text Reader

Abstract

This application discloses an adaptive IoT data management method and related equipment, which can achieve unified management of IoT data based on the amount of data accessed and on different time-series databases, sensing databases, and data warehouses, effectively reducing resource waste and thus meeting the management needs of IoT data. This application includes: acquiring target IoT data, which includes IoT data source information and real-time IoT data information; determining the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second; using Z-scores to calculate a first deviation score of the total data amount and a second deviation score of the rate at which the target IoT data is acquired per second; calculating a total deviation score based on the first and second deviation scores; and determining a target threshold based on the total deviation score.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an adaptive Internet of Things (IoT) data management method and related equipment. Background Technology

[0002] Governments and enterprises often face the challenge of managing IoT data when leveraging it to generate value. IoT data is characterized by its multi-source heterogeneity, high temporal sequence, and complexity. Common IoT data management methods include data access, data quality management, data governance, and data services, typically managing structured data. However, due to the multi-source heterogeneity, high temporal sequence, and complexity of IoT data, existing methods treat different volumes of data with the same approach, leading to resource waste and failing to fully meet the management needs of IoT data. Summary of the Invention

[0003] This application provides an adaptive IoT data management method and related equipment, which can achieve unified management of IoT data based on the amount of data accessed and on different time-series databases, sensing databases, and data warehouses, effectively reducing resource waste and thus meeting the management needs of IoT data.

[0004] The first aspect of this application provides an adaptive Internet of Things (IoT) data management method, comprising: Acquire target IoT data, which includes IoT data source information and real-time IoT data information; Determine the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second; The Z-score is used to calculate a first deviation score for the total amount of data and a second deviation score for the rate at which the target IoT data is acquired per second; Calculate the total deviation score based on the first deviation score and the second deviation score; The target threshold is determined based on the total deviation score; Determine whether the data threshold of the target IoT data is greater than the target threshold; If so, the target IoT data is determined to be large version data, and the large version data is transferred to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage; If not, the target IoT data is determined to be minor version data, and the minor version data is transferred to InfluxDB time-series database, MongoDB sensing database, and PostgreSQL data warehouse for management and storage.

[0005] Optionally, determining whether the data threshold of the target IoT data is greater than the target threshold includes: Determine whether the target IoT data is structured data or semi-structured data; If so, then determine whether the data threshold of the target IoT data is greater than the target threshold according to the first judgment rule; If not, then the second judgment rule is used to determine whether the data threshold of the target IoT data is greater than the target threshold.

[0006] Optionally, determining whether the data threshold of the target IoT data is greater than the target threshold according to the first determination rule includes: Obtain the magnitude and quantity of the target IoT data; Determine whether the magnitude of the target IoT data is greater than the target threshold. If not, proceed to determine that the target IoT data is a minor version data and transfer the minor version data to the InfluxDB time series database, MongoDB sensing database and PostgreSQL data warehouse for management and storage. If yes, determine whether the quantity of the target IoT data is greater than the target threshold. If it is determined that the amount of the target IoT data is greater than the target threshold, then the steps of determining that the target IoT data is large version data and transferring the large version data to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage are executed. If it is determined that the amount of the target IoT data is less than the target threshold, then the steps of determining that the target IoT data is minor version data and transferring the minor version data to the InfluxDB time series database, MongoDB sensing database and PostgreSQL data warehouse for management and storage are performed.

[0007] Optionally, determining whether the data threshold of the target IoT data is greater than the target threshold according to the second determination rule includes: If the magnitude of the target IoT data exceeds the target threshold, then the steps of determining that the target IoT data is a major version data and transferring the major version data to the OpenTSDB time-series database, HBase awareness database, and Hive data warehouse for management and storage are executed; otherwise, the steps of determining that the target IoT data is a minor version data and transferring the minor version data to the InfluxDB time-series database, MongoDB awareness database, and PostgreSQL data warehouse for management and storage are executed.

[0008] Optionally, determining the total amount of data acquired from the target IoT per unit time includes: Determine the unit of time; Obtain the capacity data of each data record in the target IoT data acquired within the unit time period; Determine the total number of data points acquired from the target IoT within the unit time period; The total amount of data in the target IoT is calculated based on the capacity data of each data record in the target IoT data and the sum of the number of data records.

[0009] Optionally, determining the rate at which the target IoT data is acquired per second per unit time includes: Determine the unit of time; Obtain the timestamp attached to each target IoT data point within a unit of time. The rate at which the target IoT data is acquired per second is calculated based on the timestamp attached to each target IoT data within the unit time period.

[0010] Optionally, determining the target threshold based on the total deviation score includes: The target magnitude threshold and target quantity threshold are determined based on the total deviation score.

[0011] A second aspect of this application provides an adaptive Internet of Things (IoT) data management system, comprising: An acquisition unit is used to acquire target IoT data, which includes IoT data source information and real-time IoT data information. The first determining unit is used to determine the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second. The first calculation unit is used to calculate a first deviation score of the total amount of data and a second deviation score of the rate at which the target IoT data is acquired per second using Z-scores; The second calculation unit is used to calculate the total deviation score based on the first deviation score and the second deviation score; The second determining unit is used to determine the target threshold based on the total deviation score; A judgment unit is used to determine whether the data threshold of the target IoT data is greater than the target threshold. The first transmission unit is configured to determine that the target IoT data is large version data when the judgment unit determines that the data threshold of the target IoT data is greater than the target threshold, and transmit the large version data to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage. The second transmission unit is used to determine that the target IoT data is minor version data when the judgment unit determines that the data threshold of the target IoT data is less than the target threshold, and to transmit the minor version data to the InfluxDB time series database, MongoDB sensing database and PostgreSQL data warehouse for management and storage.

[0012] A third aspect of this application provides an adaptive Internet of Things (IoT) data management device, comprising: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, which the processor invokes to execute the adaptive Internet of Things data management method as described in the first aspect and any one of the first aspects.

[0013] A fourth aspect of this application provides a computer-readable storage medium storing a program that, when executed on a computer, performs the method described in the first aspect and any one thereof.

[0014] As can be seen from the above technical solutions, this application has the following advantages: The adaptive IoT data management method of this application first acquires target IoT data, which includes IoT data source information and real-time IoT data information. Then, it determines the total amount of target IoT data acquired per unit time and the rate at which target IoT data is acquired per second. It uses Z-scores to calculate a first deviation score for the total data amount and a second deviation score for the rate at which target IoT data is acquired per second. A total deviation score is calculated based on the first and second deviation scores. Next, a target threshold is determined based on the total deviation score. It further determines whether the target IoT data threshold is greater than the target threshold. If so, the target IoT data is determined to be large-version data and the large-version data is transferred to an OpenTSDB time-series database, an HBase awareness database, and a Hive data warehouse for management and storage. If not, the target IoT data is determined to be small-version data and the small-version data is transferred to an InfluxDB time-series database, a MongoDB awareness database, and a PostgreSQL data warehouse for management and storage.

[0015] Therefore, after acquiring the target IoT data, this application calculates the deviation score based on the total amount of target IoT data acquired per unit time and the rate at which target IoT data is acquired per second. Based on the total deviation score, a target threshold is determined. Then, the total amount of target IoT data is compared with the target threshold. The target IoT data can be further divided into small version data or large version data. Based on different time-series databases, sensing databases, and data warehouses, unified management of small version data and large version data can be achieved, effectively reducing resource waste and thus meeting the management needs of IoT data. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of an embodiment of the adaptive IoT data management method of this application; Figure 2-1 and Figure 2-2 This is a schematic diagram of another embodiment of the adaptive IoT data management method of this application; Figure 3 This is a schematic diagram of an embodiment of the adaptive IoT data management system of this application; Figure 4 This is a schematic diagram of one embodiment of the adaptive IoT data management device of this application. Detailed Implementation

[0018] This application provides an adaptive IoT data management method and related equipment, which can achieve unified management of IoT data based on the amount of data accessed and on different time-series databases, sensing databases, and data warehouses, effectively reducing resource waste and thus meeting the management needs of IoT data.

[0019] It should be noted that the adaptive IoT data management method provided in this application can be applied to terminals as well as servers. For example, the terminal can be a smart desk lamp, depth camera, smartphone, computer, tablet, smart TV, smartwatch, portable computer terminal, or a desktop computer, etc. For ease of explanation, this application uses the terminal as the implementing entity for illustration.

[0020] Please see Figure 1 The first aspect of this application provides a schematic diagram of an embodiment of an adaptive Internet of Things (IoT) data management method, including: 101. The terminal acquires target IoT data, which includes IoT data source information and real-time IoT data information; In this embodiment of the application, the external system is connected to the terminal and accesses the external system data into the terminal. The data obtained by the terminal includes IoT data source information and real-time IoT data information. The real-time IoT data information is obtained through the distributed message queue Kafka. The terminal supports, but is not limited to, the following types of data access: MySQL, PostgreSQL, MongoDB, Kafka and API. After obtaining the IoT data source information and real-time IoT data information, the terminal executes step 102.

[0021] 102. The terminal determines the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second; In this embodiment, after acquiring the target IoT data, the terminal further determines the total amount of target IoT data acquired per unit time and the rate at which it acquires target IoT data per second. For example, if the terminal acquires target IoT data 24 hours a day, and a unit time of 3 hours is selected, the total amount of target IoT data acquired by the terminal within these 3 hours, and the rate at which the terminal acquires target IoT data per second within these 3 hours, are calculated. It should be noted that the time frame can also be set to 4 hours, 5 hours, or 6 hours; no specific limitation is made here, and the setting can be made according to the actual situation.

[0022] 103. The terminal uses Z-scores to calculate a first deviation score for the total amount of data and a second deviation score for the rate at which the target IoT data is acquired per second; In this embodiment, after the terminal acquires the total amount of data from the target IoT within a unit time and the rate at which it acquires the target IoT data per second, it further calculates the corresponding first deviation score and second deviation score using Z-scores. The Z-score, also called the standard score, measures the distance of a raw score from the mean using the standard deviation as a yardstick. The z-score is determined by the number of standard deviations in this distance, thus determining the position of this data within the entire dataset. Therefore, calculating the first deviation score requires calculating the mean and standard deviation of the total data volume within a unit time. Similarly, calculating the second deviation score requires calculating the mean and standard deviation of the rate at which the target IoT data is acquired per second within a unit time. After calculating the first and second deviation scores, step 104 is executed.

[0023] 104. The terminal calculates the total deviation score based on the first deviation score and the second deviation score; In this embodiment of the application, after obtaining the first deviation score and the second deviation score, the first deviation score and the second deviation score are summed or averaged to determine the total deviation score. Since the total deviation score is calculated per unit time, it can reflect the total amount of target IoT data and the rate of acquiring target IoT data per second to a certain extent.

[0024] 105. The terminal determines the target threshold based on the total deviation score; In this application, after obtaining the total deviation score, the terminal further determines the target threshold for the data. The target threshold includes a target magnitude threshold and a target quantity threshold. The total deviation score can, to a certain extent, indicate the total amount of target IoT data obtained by the terminal. The target threshold is determined based on the total deviation score. For example, when the total deviation score is large, it means that the total amount of target IoT data obtained per unit time is larger, and the corresponding target threshold number is larger. When the total deviation score is small, it means that the total amount of target IoT data obtained per unit time is smaller, and the corresponding target threshold number is smaller. The establishment of the target threshold needs to be determined based on the total deviation score, which facilitates the classification and processing of the data. This helps to determine whether the terminal can divide the total target IoT data into large version data and small version data for processing after obtaining the total data.

[0025] The specific values ​​for the target magnitude threshold and the target number threshold can be set according to the actual situation. For example, if the target IoT data magnitude is G and the target IoT data number is millions, then it can be considered to have reached the magnitude of big data; otherwise, it can be considered to have reached the magnitude of small data. After determining the target threshold for the data, proceed to step 106.

[0026] 106. The terminal determines whether the data threshold of the target IoT data is greater than the target threshold; After acquiring the target IoT data and determining the target threshold of the data, the terminal needs to compare the data threshold of the target IoT data with the target threshold. This comparison is used to determine whether the target IoT data is a big data version or a small data version, which is used for subsequent data processing. Specifically, when the terminal determines that the data threshold of the target IoT data is greater than the target threshold, step 104 is executed; when the terminal determines that the data threshold of the target IoT data is less than the target threshold, step 105 is executed.

[0027] 107. The terminal determines that the target IoT data is large version data, and transmits the large version data to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage; In this embodiment, when the terminal determines that the data threshold of the target IoT is greater than the target threshold, it determines that the target IoT data is large version data and transmits the large version data to the large version database for processing. Specifically, the large version data is transmitted to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage.

[0028] 108. The terminal determines that the target IoT data is minor version data, and transmits the minor version data to the InfluxDB time series database, MongoDB sensing database and PostgreSQL data warehouse for management and storage.

[0029] In this embodiment, when the terminal determines that the data threshold of the target IoT is less than the target threshold, it determines that the target IoT data is minor version data and transmits the minor version data to the minor version database for processing. Specifically, the minor version data is transmitted to the InfluxDB time series database, the MongoDB sensing database, and the PostgreSQL data warehouse for management and storage.

[0030] Based on this, after acquiring the target IoT data, this application calculates the deviation score based on the total amount of target IoT data acquired per unit time and the rate at which target IoT data is acquired per second. The target threshold is then determined based on the total deviation score. The total amount of target IoT data is then compared with the target threshold. The target IoT data can be further divided into small version data or large version data. The small version data and large version data can be managed in a unified manner based on different time-series databases, sensing databases, and data warehouses, effectively reducing resource waste and thus meeting the management needs of IoT data.

[0031] Please see Figure 2-1 and Figure 2-2 The first aspect of this application provides another embodiment of an adaptive Internet of Things (IoT) data management method, including: 201. The terminal acquires target IoT data, which includes IoT data source information and real-time IoT data information; In this embodiment of the application, step 201 is similar to the aforementioned step 101, and will not be described again here.

[0032] 202. The terminal determines the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second; In this embodiment, the terminal first determines the unit time, then obtains the capacity data of each data record in the target IoT data obtained within the unit time, and further determines the total number of data records of the target IoT obtained within the unit time; finally, it calculates the total amount of data of the target IoT by multiplying the capacity data of each data record in the target IoT data by the total number of data records.

[0033] Specifically, for the rate of acquiring the target IoT data per second within a unit time, the unit time is determined, and then the timestamp attached to each target IoT data within the unit time is acquired; the rate of acquiring the target IoT data per second is calculated based on the timestamp attached to each target IoT data within the unit time.

[0034] 203. The terminal uses Z-scores to calculate a first deviation score for the total amount of data and a second deviation score for the rate at which the target IoT data is acquired per second; 204. The terminal calculates the total deviation score based on the first deviation score and the second deviation score; 205. The terminal determines the target threshold based on the total deviation score; In the embodiments of this application, steps 202 to 205 are similar to the aforementioned steps 102 and 105, and will not be described again here.

[0035] 206. The terminal determines whether the target IoT data is structured data or semi-structured data; In this embodiment of the application, after the terminal obtains the target IoT data, it needs to further judge the target IoT data. The data structure includes structured data, semi-structured data and unstructured data. Different processing methods will be used for different data structures. Specifically, when the terminal determines that the target IoT data is structured data or semi-structured data, step 207 is executed. When the terminal determines that the target IoT data is not structured data or semi-structured data, but unstructured data, step 209 is executed.

[0036] 207. The magnitude and quantity of the target IoT data acquired by the terminal; In this application, when the terminal determines that the target IoT data is structured data or semi-structured data, the terminal will obtain the magnitude and quantity information of the target IoT data. The terminal will automatically read and judge the magnitude and quantity information of the target IoT data. After obtaining the magnitude and quantity of the target IoT data, step 208 will be executed.

[0037] 208. The terminal determines whether the magnitude of the target IoT data is greater than the target threshold; In this embodiment, the terminal first determines the magnitude of the target IoT data. It determines whether the magnitude of the target IoT data is greater than the target magnitude threshold. If not, the terminal automatically determines that the target IoT data is a minor version of the data and transmits the minor version data to the InfluxDB time-series database, MongoDB sensing database, and PostgreSQL data warehouse for management and storage. If yes, step 210 is executed: determining whether the quantity of the target IoT data is greater than the target quantity threshold. If the terminal determines that the quantity of the target IoT data is greater than the target data threshold, step 210 is executed. If the terminal determines that the quantity of the target IoT data is less than the target data threshold, step 211 is executed.

[0038] 209. The terminal determines whether the magnitude of the target Internet of Things is greater than the target threshold; When the terminal determines that the target IoT data is unstructured data rather than structured or semi-structured data, the terminal will further determine whether the magnitude of the target IoT data exceeds a target magnitude threshold. If yes, proceed to step 210; otherwise, proceed to step 211.

[0039] 210. The terminal determines that the target IoT data is large version data, and transmits the large version data to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage; 211. The terminal determines that the target IoT data is minor version data, and transmits the minor version data to InfluxDB time series database, MongoDB sensing database and PostgreSQL data warehouse for management and storage.

[0040] Please see Figure 3 The second aspect of this application provides a schematic diagram of an embodiment of an adaptive Internet of Things (IoT) data management system, including: The acquisition unit 301 is used to acquire target IoT data, which includes IoT data source information and real-time IoT data information. The first determining unit 302 is used to determine the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second. The first calculation unit 303 is used to calculate a first deviation score of the total amount of data and a second deviation score of the rate at which the target IoT data is acquired per second using Z-scores; The second calculation unit 304 is used to calculate the total deviation score based on the first deviation score and the second deviation score; The second determining unit 305 is used to determine the target threshold based on the total deviation score; The judgment unit 306 is used to determine whether the data threshold of the target IoT data is greater than the target threshold; The first transmission unit 307 is used to determine that the target IoT data is large version data when the judgment unit determines that the data threshold of the target IoT data is greater than the target threshold, and to transmit the large version data to the OpenTSDB time series database, HBase perception database and Hive data warehouse for management and storage. The second transmission unit 308 is used to determine that the target IoT data is minor version data when the judgment unit determines that the data threshold of the target IoT data is less than the target threshold, and to transmit the minor version data to the InfluxDB time series database, MongoDB sensing database and PostgreSQL data warehouse for management and storage.

[0041] Please see Figure 4 This application, in a third aspect, provides an adaptive Internet of Things (IoT) data management device, comprising: Processor 401, memory 402, input / output unit 403, and bus 404; The processor 401 is connected to the memory 402, the input / output unit 403 and the bus 404; The memory 402 stores a program, which the processor 401 calls to execute the adaptive Internet of Things data management method as described in the first aspect and any one of the first aspects.

[0042] A fourth aspect of this application provides a computer-readable storage medium on which a program is stored, which, when executed on a computer, performs the method as described in the first aspect and any one of the first aspects.

[0043] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0044] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0045] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0046] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0047] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An adaptive Internet of Things (IoT) data management method, characterized in that, include: Acquire target IoT data, which includes IoT data source information and real-time IoT data information; Determine the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second; The Z-score is used to calculate a first deviation score for the total amount of data and a second deviation score for the rate at which the target IoT data is acquired per second; Calculate the total deviation score based on the first deviation score and the second deviation score; The target magnitude threshold and target quantity threshold for the data are determined based on the total deviation score. Determine whether the target IoT data is structured data or semi-structured data; If so, obtain the magnitude and quantity of the target IoT data; determine whether the magnitude of the target IoT data is greater than the target magnitude threshold; if not, determine that the target IoT data is a minor version data and transfer the minor version data to InfluxDB time-series database, MongoDB awareness database, and PostgreSQL data warehouse for management and storage; if so, determine whether the quantity of the target IoT data is greater than the target quantity threshold; if the quantity of the target IoT data is greater than the target quantity threshold, determine that the target IoT data is a major version data and transfer the major version data to OpenTSDB time-series database, HBase awareness database, and Hive data warehouse for management and storage; if the quantity of the target IoT data is less than the target quantity threshold, determine that the target IoT data is a minor version data and transfer the minor version data to InfluxDB time-series database, MongoDB awareness database, and PostgreSQL data warehouse for management and storage. If not, determine whether the magnitude of the target IoT data is greater than the target magnitude threshold. If yes, determine that the target IoT data is major version data and transfer the major version data to the OpenTSDB time series database, HBase sensing database, and Hive data warehouse for management and storage. If not, determine that the target IoT data is minor version data and transfer the minor version data to the InfluxDB time series database, MongoDB sensing database, and PostgreSQL data warehouse for management and storage.

2. The adaptive IoT data management method according to claim 1, characterized in that, The determination of the total amount of data acquired by the target Internet of Things per unit time includes: Determine the unit of time; Obtain the capacity data of each data record in the target IoT data acquired within the unit time period; Determine the total number of data points acquired from the target IoT within the unit time period; The total amount of data in the target IoT is calculated based on the capacity data of each data record in the target IoT data and the sum of the number of data records.

3. The adaptive IoT data management method according to claim 1, characterized in that, The determination of the rate at which the target IoT data is acquired per second per unit time includes: Determine the unit of time; Obtain the timestamp attached to each target IoT data point within a unit of time. The rate at which the target IoT data is acquired per second is calculated based on the timestamp attached to each target IoT data within the unit time period.

4. An adaptive Internet of Things (IoT) data management system, characterized in that, include: An acquisition unit is used to acquire target IoT data, which includes IoT data source information and real-time IoT data information. The first determining unit is used to determine the total amount of target IoT data acquired per unit time and the rate at which the target IoT data is acquired per second. The first calculation unit is used to calculate a first deviation score of the total amount of data and a second deviation score of the rate at which the target IoT data is acquired per second using Z-scores; The second calculation unit is used to calculate the total deviation score based on the first deviation score and the second deviation score; The second determining unit is used to determine the target magnitude threshold and the target quantity threshold of the data based on the total deviation score; The judgment unit is used to determine whether the target IoT data is structured data or semi-structured data; The first transmission unit is configured to, when the determination unit determines that the target IoT data is structured data or semi-structured data, acquire the magnitude and quantity of the target IoT data; determine whether the magnitude of the target IoT data is greater than the target magnitude threshold; if not, determine that the target IoT data is a minor version data and transmit the minor version data to an InfluxDB time-series database, a MongoDB awareness database, and a PostgreSQL data warehouse for management and storage; if yes, determine whether the quantity of the target IoT data is greater than the target quantity threshold; if the quantity of the target IoT data is greater than the target quantity threshold, determine that the target IoT data is a major version data and transmit the major version data to an OpenTSDB time-series database, an HBase awareness database, and a Hive data warehouse for management and storage; if the quantity of the target IoT data is less than the target quantity threshold, determine that the target IoT data is a minor version data and transmit the minor version data to an InfluxDB time-series database, a MongoDB awareness database, and a PostgreSQL data warehouse for management and storage. The second transmission unit is configured to, when the determination unit determines that the target IoT data is not structured or semi-structured data, determine whether the magnitude of the target IoT data is greater than the target magnitude threshold. If so, it determines that the target IoT data is a major version data and transmits the major version data to the OpenTSDB time-series database, HBase awareness database, and Hive data warehouse for management and storage. If not, it determines that the target IoT data is a minor version data and transmits the minor version data to the InfluxDB time-series database, MongoDB awareness database, and PostgreSQL data warehouse for management and storage.

5. An adaptive Internet of Things (IoT) data management device, characterized in that, include: Processor, memory, input / output units, and bus; The processor is connected to the memory, the input / output unit, and the bus; The memory stores a program, which the processor invokes to execute the adaptive IoT data management method as described in any one of claims 1-3.

6. A computer-readable storage medium having a program stored thereon, the program performing the method as described in any one of claims 1 to 3 when executed on a computer.

Citation Information

Patent Citations

  • HVS-based novel video quality evaluation method

    CN104023226A

  • Bin-lake integrated data management system and method

    CN116303818A